To keep it short and sweet, the big concern is in the event of a rollover. The roof of the car is not designed to absorb impact loads without crushing the top of the car. If you look closely at how the factory belts work, you have the ability to tilt at the waist toward the center of the car. In a rollover, with the roof collapsing, your head has to go somewhere. With the factory belts you can pivot or tilt at the waist which allows your head to lower (with respect to the height of the car). In a secured harness and seat, the belt is designed to keep you rigid in your seat. If you roll over and the roof collapses the only part on your body that can pivot or tilt is your neck - i.e. broken neck. I have listed some websites that I gathered my information from. Ultimately it is up to you if you wish to do this (there are plenty of people who do this today).
one two three four five six seven eight nine This is by no means a complete list of available information - just several that I thought were relevant. I have also spoken to a few people that are serious racers and have also advised against it.