Didn't have a problem with mine; it just popped out. Not sure if clips had been broken by previous owner. Seems to fit back in pretty snug, though.

Old car: '98 SVT T-Red K&N air filter; Ford Focus RS shift knob. Off to be auctioned off somewhere. New car: 2003 Honda Accord DX Sedan, 5-speed.