I think you may have a serious problem. Check the bios under power management or similar heading to be sure that the system is set to shut down completely during an off cycle rather than to sleep or some other low power mode. With an atx supply, voltage is always available on the +5v standby wire, but the psu should NOT be "really hot" at all when the system is in sleep or off.
Seems to me that you likely have a bad psu, that it's drawing a lot of power when it should be off. Might even be dangerous.