Ok, well after reading all the problems that you've had and all the things you've tried to fix it, I would go along with everyone else on here and say, yes its the hard drive. I have to deal with this problem day after day with crappy HP's. Unfortunately like everyone said, newer HP's do not include any recovery disks. It is all on the hard drive.
If you change the hard drive, then you will need to get your hands on a Windows XP Home CD. As long as the computer has the COA (Certificate of Authenticity) sticker on it, then you are an owner of Windows XP Home and can use another persons CD to install it. But realize that, if you are not familar with installing Windows than you may have trouble. Not only will you have to manually install it instead of using recovery, but you will need to find drivers for all hardware that XP does not include drivers for, additional software such as CDRW software, DVD software, etc.
Not sure if I read it properly but I think it was mentioned earlier about taking the hard drive from the other computer and booting from it to see how it runs or vice-versa. Know that its not as simple as that. Niether computer will boot if you try to boot from the other computers hard drive. Anything Windows 2000 and higher requires that the hard drive be reloaded before being able to boot to it. A Windows 98 hard drive can be run on another computer with some work. It will take alot of software configuration but is possible (not worth it at all just for you to try and test it).
And oh yeah, I did see someone mention about taking your HP's hard drive and hooking it up into the older computer to test it out and see if the other computer has trouble reading it. This will not be possible because Windows 98 does not recognize Windows NT File System (NTFS), which is what your HP's hard drive is running on.
Oh, by the way, do not buy a Dell!!! Not worth the money at all. Lately i've been fixing more and more newer Dell's and usually turn out to be a bad hard drive. Dell likes to use crappy Maxtor brand hard drives in alot of there systems which is also what some HP's use. They have a tendency to last half as long as other well known brands such as Western Digital, Seagate, etc.
When you do decide to get a new hard drive, do not just chose any brand that has the best price. Look for Western Digital and Seagate first. I like to think that those are the best brands out there. (which is why my computer has both brands inside it right now

)
Good luck with everything, and if you have any questions, let me know. I'm very familar with HP.