Rattle/clicking noise at cold start up

I've got a 2006 Pontiac GTO, with 6-speed manual. Every time I start the engine from a COLD start, I get a rattle(like a paint can rattle)or more like clicking sound from the front of the engine. This only happens for the first 2-3 minutes and then the noise goes completely away when the engine gets warm. To me it seems like its coming from the power steering pump. Once the engine warms up and the noise goes away, if I turn the steering wheel left or right the noise will come back a little bit and then it goes away. Could this be the power steering pump and if so, is it a sign that its going bad? It started doing this a few months ago.