Replace the idle air control valve (IAC) or see if you have a large vacuum leak.
The IAC is responsible for maintaining idle speed and dashpot control.
Cleaning it sometimes helps. If not, replace it.
As your idles speed drops below the normal minimum, alternator output also drops off. That is what is causing the lights to dim.
Steve