2005 Saturn Vue driver door locked

The interior handle and exterior handle are in working order. It seems to be latched shut at the door actuator. We can see the actuator but are unable to latch the latch to get the door opened. We have checked and the door is not in the locked position. When you pull the handle up from the outside you can see a yellow plastic piece inside the door moving up and down but the latch remains closed. I'm sure there is something simple we are missing to pop the latch. Have tried pushing the yellow plastic with a rod but door remains locked