I did this on several older AMD chips including an Opteron 170. This consistently lowered my load temperatures by around 10-13C. It makes a huge difference. The 2.0 Ghz, dual-core opteron 170 I was able to get stable at 3.1 thanks to the IHS removal. My load temps were 42c on water.
Lapping makes a difference but it's minute. Removing the IHS is a much greater difference because you are removing two layers from the heat transfer process (the core->TIM heat transfer and the TIM->IHS heat transfer) instead of simply improving the IHS->heatsink transfer.
The downsides are that it is easy to damage the IC's around the core with the razor. Make sure the razor doesn't go any further in than it needs to. Also, you have to be careful mountaing your heatsink on a naked core because uneven pressure across it can crack the edges.
If you have enough experience modding/overclocking, it ends up being pretty trivial. Just make sure that the heatsink you use will mount properly to a core without an IHS (a few mm lower than usual).
@ SickBeast Idk why you didn't lose any temperature from removing the IHS as the improvement numbers I saw (>10C under load) are representative of what I noticed from others. Though there is some inherent variability because sometimes the core makes better contact through AMD's TIM to the IHS than other times... Sometimes the contact under the IHS is quite poor though... This is why some processors are unusually hot... And obviously they lose the most temp when you cut the IHS off.
I remember some people dropping 20c under load when running without IHS. Of course these are outliers representing CPU's with very poor internal contact.
Edit: my experience relates to older AMD's and not sure exactly if the newer Phenoms are the same but I would assume so. Looks the same.
