in the higher gears I can get take it out of gear without pushing in the clutch is that ok

My Peugeot 107 can be taken out of 3rd gear or higher with output putting the clutch down but it has to be pushed down to go back in to the gear. I know this is not normal and wondered if you could tell me what you think may be wrong? This does only happen in 3rd 4th and 5th. It doesn't happen by its self it does have to be taken out of the gear.