As long as you don't disable any of the various power management stuff in the bios, the cpu will still atuo-adjust clock speeds based on load, lowering power and heat usage/output. It's nothing to worry about, the chip will automatically clock up to default levels when loaded.
As for added heat, yes it does make a difference, but in a well-thought out setup, it's not a huge problem. I generally recommend cases with 120mm fans front and rear so that a lot of air can be moved through the case with push/pull without a ton of noise. Personally, I'm more annoyed with extra noise than a bit of extra heat. As a personal rule, I like the ambient internal case temps to be no higher that ~90f, and preferably a good bit lower.
As for speeds, if you are primarily gaming/watching video/etc, then I would personally stay with stock voltage, and probably at this time no more than 4ghz would be useful. If you are doing distributed computing or hd video encoding, that's where the higher clock speeds would see improvements in further overclocking. So 4400mhz vs. 4000mhz would theoretically give up to a 10% improvement in those tasks, though in reality the other elements in your system would lower that gain somewhat.
My absolute best advice remains though, when you get a new setup, there's nothing wrong with finding the limits, though I don't recommend going to very high voltages. Once you find the stable limits for Prime/etc, backing off a few steps is a prudent measure. I never like to run right at the limit of stability for a 24/7 personal system. It's a bit different when you have a number of rigs or you are intending to reach the absolute limit for competitive/enthusiast computing. Gaming is almost solely limited by the video card these days as long as your CPU isn't ancient. By that I mean that a lowly Athlon X4 @ 3ghz w/GTX580 will run circles around a 5ghz SB i7 w/GTX560 in pretty much any game, even though the X4 is hugely outgunned by the SB proc. You have the basis for a great system for a long time for gaming, you just may have to upgrade the GPU in a couple of years to keep pace with the latest games, or if you upgrade to say a 2560x1440 monitor or so.
Also, you can take anything Mark has to say straight to the bank, he's run a private army of overclocked systems for as long as I've been here, and is a valuable contributor to any thread. I never run more than 1 or 2 systems personally, but have built many thousands over the past couple of decades. Of those number, more than half have just been average business-use desktops. Of the enthusiast machines I have made, I offer the best information to my knowledge. As time has gone on, I haven't had as much time or desire to continually push the limits of new stuff.