It's always been my experience building cars for many years, that rasp was always a problem when the exhaust diameter was too small. 2 1/4" pipes on V-8 cars for example always were raspy. Then if you went to 3" exhaust the rasp was prettymuch gone. Another factor that always was important (w/ dual exhaust anyway) was it would rasp a lot until you welded in a crossover tube between the headers and the mufflers to connect both bank's exhaust pipes (so the opposing side's sound waves would cancel out each other since the cylinder's firing pulses are closer together), but that isn't an issue with single exh. Contours though since all 6 cylinders are firing into one pipe. But if you just increase the diameter of the pipe all the way from front to rear, the rasp should go away, since you will have much less airspeed in the pipe at higher RPMs.
Also, the reason why it is more raspy when cold is because IR energy (heat) absorbs soundwaves, which is why a hot exhaust system is always quieter than when cold
