YOU ASKED!!! Gotta get the facts so ya don't get the wrong diagnosis! :p
It is possible that it could be some type of weathering, but usually that would result in faded paint. I just thought of this, could be that the clearcoat was pulled off around the edges of the adhesive when you removed them... I highly doubt that, but it could be a possiblity and that's why it's grabbing onto the rubbing compound.
