2000 Chevy S-10 Sometimes Brakes drag

I've noticed sometimes when I take off I hear the moan of what I believe is the back rear barkes dragging. I recently got to work 11miles away the last week and smelled brakes. I touched the rear hup and it was warm on the right and cold on the left. But, I've been checking for 3-4 days and the they're both cold. So, this does not happen all the time. With gas prices high I need all the mileage I can get. Any ideas why sometimes my rear brakes grab? I check the emergency brake constantly making sure I didn't bump it w/mt foot.
Thanks very much TerryR