if you let go of the wheel and the car does go straight it is the alignment, now don't forget most roads are crowned and you car should drift off as it is just following the road
check the tie rods, bushings, and bal joints and replace as needed and get an alignment done and that should take care of it