1999 SVT 2.5L duratech v6

when i accelerate in the low rpm, there is an annoying rattle or grinding type noise...it makes the car sound like crap...however it doesn't do it when you rev it in neutral...anyone know what might cause this?