Certainly a very mild system like that shouldn't be getting too hot unless there is some fairly straightforward problem. Make sure all the fans are spinning and it's located in an area where it has a reasonable cool air intake.
An SSD is a good option for speeding up general use. Do you game, and at what resolution? What kind of power supply does the system come with?
Exactly what RaistlinZ said:
SSDs are aggressively priced right now, the 256 GB Crucial MX100 is regularly ~$100 from amazon/newegg, and that's a great general purpose upgrade. GPU upgrade depends on budget and PSU, but even a fairly anemic psu should be able to support the GTX 750/750 Ti (as it requires no external power connectors) and those should be a huge step up from the HD6670.