car won't start sometimes, now more often then not

I have replaced the ignition switch, and the starter. I hear my fuel pump come on and there are lights on my dash, but when I turn the keys, it does absolutely nothing. No noise, no clicking sound....just nothing. Then later I can to start it, and sometimes it works...but again..it is becoming a routine of not starting. Do you know what may be the problem?