In general, all you need to do is keep yourself at the same potential as the computer chassis so that there will be no surge of static through a device. That is why you will see the advice to touch the frame of the computer before you handle the electronic components to discharge any static. The strap keeps you in constant contact if you clip it to the computer frame. If you are concerned about large potential buildups, you might want to keep the whole system grounded to a true ground but that generally isn't necessary.
To do this cheaply, just get an old grounded (three prong) power cord. Cut the cord open and cut out and insulate the power leads, leaving only the ground wire. Then (assuming you have a modern grounded electrical system) you can just plug in the power supply using this modified cord and have the system grounded. (You still need the ground strap, of course.)