Pickups starter just clicks

I have a 1986 toyota sr5 pickup and there is something wrong with the starting system. About a week ago it stalled in my yard, it seemed like the battery had died. I tried to start it and it just clicked. I hooked up the battery charger turned it on heard a loud pop, then I noticed the last person to put the battery cables on had put the red as negative and black as positive. I pushed it in my shop and replaced the big 80 amp fuse that protects the system. I connected everything and it just clicked, so I put in a different battery and it easily started. I turned it off and tried to start again an hour later and just clicks. I replaced the leads on the cables and still nothing. I pulled the starter and bench tested it, it worked but I when I put it back in it still clicked. I thought maybe there was something internal that made it not have any torque, so I replaced the starter and it still just clicks. I double checked and there is 13 volts going into the starter. I cleaned the connections again and tried again. This time it clicked twice then wouldn't click at all, even with a booster. I thought maybe it was possible it the engine was seized, but it easily turned with a breaker bar. Everything else electrical works. Ideas?