As long as you don't do any drastic voltage mods, you will reach your instability long before you cause any damage. Don't quote me, but I believe Athlons are rated to 1.85v and over 70c so stay with those numbers and you will be fine. Of course, the cooler the better. V's suggestion of under 60c is excellent advice.
I'm sure you know that pci devices are supposed to work at 33mhz. As a safe bet I would try to keep your pci bus to no more than 37mhz. I think you will find that as a conservative number and I have seen pci devices operate normally up to a 41bus. Don't put anything important on your harddrive until you have finished tinkering with overclocking and have your system stable.
I use Prime95 as my stability test. It will point out any potential overclocking problems in your system. If you do very important work or very cpu demanding work (i.e. 3d rendering or video encoding) I would pay alot of attention to Prime95. It is very picky though and for daily use you may never notice any instability even though Prime95 says there might be a problem.
Basically, just have fun with o/c. As long as you do all your overclocking adjustments from the bios, there is a very very very small chance of damaging anything in your system.