starter and horn are working together

when I start the vehicle the starter works and the horn blows, while running if I try to honk the horn the starter will engage. I have removed the starter relay to prevent damage to the starter. I removed the signal wire to the starter and when I try to start the engine the horn still sounds.