I believe you just have to remove the surround that covers the drivers switches, there's a screw behind a plastic cover behind the inner door handle. Then gently with a flat head or something you can pull and pry if off. The switch is then accessable. You can pull it out and the plug to it will come out too (a little ways)
I doubt it's a connection, but I could be wrong. Sounds more like one of your switches on the drivers control is bad. They are pretty hard if not impossible to repair (but I'm no expert)
My advice is to go to the salvage yard and get a new one, they are all alike. Plug it in and I'll bet your problem will dissapear.
Remember, the salvage yard is salvation!
