I you don't have an amp then it's the HU. It's getting hot and with heat, electronics don't perform as well. Computers, stereos, amps etc... they don't like heat.
Did you install yourself? On my cheap Sony I can pull the whole radio/front cover assembly out without unplugging. Maybe in your case you can do that and leave it on for awhile. Does the performance degrade when it's not enclosed in the dash still? That MIGHT be a good way to test it.
Also I would immediately get rid of the stock speakers. They are cheap 20 watt units......you know that could be a big part of the problem there too. You may be pumping too many watts to those cheap "Premium" speakers.
Good luck