Oil residue in throttle body

I have noticed rough idle and poor performance, I detached air intake hose to throttle body and I am noticing oil residue in plastic intake tube and evening throttle body. Once I clean it with throttle body cleaner car runs better for a while but the oil slowly returns. I am going to do valve cover gasket this weekend cause of small leak, but I am wondering what else I should be looking for while I am changing gasket? No check engine light.