The best option performance-wise, needless to say, would be to forget about HP and get something custom built. Buying a brand new system today that uses PC2100 RAM and the 266MHz FSB is not the best way to achieve longetivity. If you're new to building computers perhaps you should know that you can build a powerful new system (the box itself, without monitor, printer, etc.) for around $500 that would blow all but the $1500 HP's out of the water. If you would like to consider that option then I'm sure plenty of fellow posters will be glad to recommend configurations.
If you're set on the HP, though, you'll undoubtedly need another 256MB stick of RAM ($40 to $50) and a good AGP video card (ranging from ~$80 to ~$450 depending on how much performance you need in games).