I have the same problem on my CSVT. It comes and goes, but is usually at its worst after the car has been sitting for a few hours - especially in the morning.
I've tried squirting some DuraLube onto the pivot point, but it didn't provide any noticeable improvement.
So, rather than fixing it, I've just tried to condition myself to the problem by how I work the pedal. It is pretty annoying, though.