If you are overclocking then temperature is your enemy. As the temperature increases so too must the voltage if you want to remain stable, of course raising voltage then raises the temperature even more...it is a vicious upwards spiral.
Even if you don't intend to OC higher, anything you can do to lower your operating temperatures will dramatically increase the lifetime reliability of the processor. (significantly reduces the rate of degradation)
Here is a real-world example of how much I have to increase the Vcore on my 3770k to keep the CPU stable as its operating temperature increases (a temperature controlled study):
So even if I didn't want to increase the clockspeed, just being able to operate at lower temperatures because I delidded enabled me to further lower the Vcore - all of which lowered my power consumption and significantly increased the expected operating lifetime of my processor (the ability for it to continue operating at 4.6GHz before enough degradation finally accumulates such that it is no longer stable at 4.6GHz).
Here is another example showing how the temperature effected the lowest voltage I could use at any given temperature while keeping the clockspeed the same.
Delidding isn't done just to get higher overclocks, it is also done to lessen the damage to your CPU from overclocking (TJmax is only a safe operating temperature if you are NOT overclocking) as well as to lower power consumption, heat in your room, and noise from your cooling setup.