having trouble starting my 1995 toyota avalon xl

Get the battery and alternator tested and make sure the connections are clean and tight before we get into the starting circuit then comeback. Check 5amp starter fuse at instrument panel junction block left side of dash. At the starter black and red small wire put key in start position do you have power on it-if so replace the starter. If not find the starter relay in the engine room junction box and test as describe below. Further study the EWD below how the starting system is wired in the system.

I had the same symtoms.
after considerable research I found advice that 99% of the time the starter solenoid contacts have worn out. These vehicles and a lot of Japanese makes use a Denso brand starter and this is the first thing to go. Its very easy to remove starter from the vehicle as it is mounted up high on the right side of the block. The only other component I had to get out of the way was the battery and a couple of hoses. Its a bit tight to mount a socket on the mounting bolts -there are two of them. Once out of the vehicle and on the bench, the solenoid cap (the end where the electrical cables attach, comes off also with two mounting bolts. The part you will need are the plunger and the two heavy copper contacts. You should be able to get these parts from a Starter/Alternator shop for between $10 and $20. When I was buying the parts at my local indie starter shop, I asked and the mechanic confirmed that this is exactly when they do to 'rebuild' these starters. Saved me at least $150.
Good luck. :D