Get a knife or flathead screwdriver and work at it slow. It's a pain in the arse but it will come out eventually. To prevent damage don't pull on the board very hard to get the connector up.
Make sure you seal the IMRC 100% after you are done and close it up. Moisture inside the IMRC will corrode the board.