Starter engagement while running

I assume that the starter was not doing this prior to its replacement? If that is the case the there is a chance you have a faulty starter out of the box.

https://www.2carpros.com/articles/how-to-replace-a-starter-motor

We can confirm this by checking voltage at the starter on the control wire when this happens. You should only have voltage on this wire if the key is in the start position. If this is engaging and you do not have voltage on this wire then the solenoid is shorted inside the starter.

Here is the wiring diagram and a guide that will help with doing this:

https://www.2carpros.com/articles/how-to-check-wiring

Thanks