As an '06 owner; I really enjoy the car. It has everything I wanted on my next car. It doesn't handle as well as the CSVT did, however it has ALOT more power. The manual shifter does suck in a major way; but there are options to improve it. I personally love how quiet the car is, with just a hint of exhaust noise and engine rumble coming into the passanger compartment. Just enough to please your ears, but not too much to be distracting.
The back actually has a good deal of room. I can/have (only one time) sat in the back of mine as was reasonably comfortable (I'm about 6'10"). Getting in and out of the back seat however will require chiropractor visits.
The GTO does have it's quirks, but it is alot of fun, and great bang for the buck. Mods are relatively limited, although more available than those for the Contour, and can be pricey as most of them come from Australia. But overall, it's a really good car, and you should be able to get an '04 and a pretty good price.