I've got it in both doors, same spot as your describing.
It just gets worse by adding speakers with better bass response. I've got some 'Bahn 6.5" mid-bass drivers, and they can make the doors sound like beehives.
I'm planning an entire system tear-down with rewiring and sound deadening this summer. Should hopefully solve this very annoying problem.