As long as you have decent cooling, I don't think overclocking will do much harm, as long as your voltages stay reasonable.
On the other hand, some people feel that the CPU gets too warm with the stock cooler, even without overclocking. So I think that you should strongly consider getting an aftermarket heatsink, whether or not you are overclocking.
I think if voltages stay reasonable (within Intel's specs) then temps really don't matter (assuming they're within Intel's specs as well). Not trying to contradict you here, just expressing my understanding. When you venture outside Intel's voltage recommendations then you need to start compensating with better cooling (water, in my opinion. Why? Well, 5c less from a better air cooler isn't going to necessarily overcome the damage brought from the extra 0.05v you just added-- damage being from, well, keep reading-- I'm not simply saying "higher voltage is going to lead to higher temps, so you need a better cooler")
The main problem is when an electron tunnels halfway into the dielectric between the gate and channel of the transistor and lodges there permanently, thereby decreasing the insulation between the gate and channel, affecting the operation of the transistor in the chip. This can happen when either a). the chip gets [really] hot or b). very high voltage or c). chip gets hot with marginally high voltage [somewhere close out of Intel's specs].
Most of us are in c) territory; water cooling lets us get rid of the a). issue and lets us play exclusively in b).
Liquid Nitrogen, for example, greatly lowers the resting state of the electrons [and, particularly, stabilizes them], allowing us to apply a much higher voltage, while minimizing the number of electrons that jump into the conduction band and turn a 1 to a 0 or a 0 to a 1. It also "calms" them in that they are much less likely play any quantum-tunneling games and implant themselves into the Si02/HfO2 (AMD/Intel), thereby permanently damaging the chip.
Aside from this, the only other damage that would occur from voltage (which is not preventable with lower temps) is electromigration. As process node shrinks, this becomes a greater and greater problem because we're not talking about 1/100 of the atoms getting dislodged; we're talking about, perhaps 1/10-- of course we're not really making much that is 10 atoms wide, yet, but I think you get the point.
2nd thought; since silicon is a lattice; I wonder if it's more resilient to electromigration.