I had an Impreza the first 90 or so races. I switched to a Skyline around then and finished with it.
Yeah, drag races were easier to win, and there was some loss of high-speed handling ability, but it wasn't nearly what there should have been; and once I got the last of the mods, there's virtually no difference between the 2 if I switch back and forth.
One thing I was really impressed with as far as the physics went was how your car could tip over if you turned hard enough with the e-brake pulled. I can't recall another game that even attempts to model that at all.