The two cats located in the exhaust are called "pre-cats." If you have high miles on your CSVT, those pre-cats start to clog. Many people have gutted their precats, search new and old forum postings and how to's for gutted pre-cats and "mystery mod". While this clears out the blockage, you're still left with a gutted stock exhaust manifold, but this is easy and virtually free to do (unlike headers).
Dropping the main cat doesn't do much either unless it's clogged, you have other exhaust system components around it which still compress your exhaust flow down to the same inlet/outlet diameter as that main cat.
Optimized your y pipe yet? That would actually give you some gain.