The whole idea of protection against ESD is to keep the charge in your body from damaging a device. That is done by keeping your body at the same potential as the device (not necessarily ground).
You do that by wearing a wrist strap and connecting it to the ground of the PC (the power supply case is connected to the motherboard ground if the power supply is connected to the motherboard).
In fact, if you ground yourself but the PC is not grounded, you may still damage the device. But, if you connect the wrist strap to the ground of your PC, your body and the motherboard will be at the same potential. There will be no charge dumped into the motherboard. There will be no damage. You could be at 2000V! As long as the motherboard is also at 2000V, there will be no damage. This last sentence was to get a point across. I am not suggesting that it is safe to be at 2000V! The point is that voltage is a relative thing.
So, as long as you connect the strap to the motherboard ground, you can have the PC unplugged and there will be no damage.