I have tried very hard to understand x264 encoder settings and how to best use them for optimal results (without overdoing something, like doubling encoding time for virtually indistinguishable quality gains). I don't pretend to fully understand it (lol) but I think I found a decent middleground.
I use the DXVA-HD-Balanced profile in MeGUI; for calculating bitrate, I usually do Vres x Hres x fps x .25 (bits/pixel). I remember reading lots of guides once upon a time and most of them stressed on .25 bits/pixel for superior results. For instance, 1280 x 720 x 23.976 x .25 = 5,524,070.4 bps or 5,394 kbps or 5.27Mbps
Having said that, I've seen lots of videos that seem to have just as good results even with something like .15 bits/pixel (resulting in 3.16 Mbps for a 1280x720 file). I tried to find the sweet spot, drove myself crazy and stopped. If you use another front-end for encoding and don't have a base profile to start from, try and duplicate these settings:
1 2 3
And yeah, mousing over each setting in MeGUI gives you a brief description with recommended settings. You can try a few test encodes and check what works best for you.