As stated.. tie-rods (for bends, etc) or possibly the control arms.
A bent control arm (or a control arm bushing that is missing) can cause that same drifting "with a mind of its own" and can make it very unnerving... rear control arms (being bent) tend to make taking curves, or corners more of an... ahem... experience... shall we say.
Ray
