. . . . .
Yes, airflow and fans are really important but ultimately it makes the most sense to test the coolers with the included fans at differing fan speed%. What do you mean by throwing noise considerations out the window? If that means running the included fans/pump at max RPM then sure I agree that's a good test to do. As they are AIO units I don't think it's relevant to start swapping out the fans though.
Chances are, given my obsession for just "building computers," I'm going to pick different fans for a cooler regardless of those bundled with it. They don't attend to acoustic modification of the overall system in those reviews as I do. I've constantly read assessments of forum members that radiator fans are noisy, and it only stands to reason that any contact between the hard surface of a fan and radiator metal will transmit vibration and therefore sound through the cooler.
This is not likely to be as noticeable unless one swaps in some fans with high static pressure and high airflow potential, even though that potential is either never realized, or not realized as often. You can isolate the hard surfaces with rubber fittings, and they may need to be "invented" or custom-made -- the sort of thing for which I'll take pains and others will not incline themselves. But inserting a radiator into the case-configuration arrangement changes the way of dealing with airflow.
Take for instance my CoolerMaster "Crossflow" barrel fan -- once a mark of distinction for the Stacker 830. The only way to use it requires the tiny sheet-metal screws bundled with the fan, and barring other considerations, it involves metal-to-metal contact possible over greater area of the surfaces than other fan installations. How do you build "shock absorbers" for such a fan? I found myself fiddling tediously with a fan that was already mounted in a "tight squeeze." Then I was testing the difference of "before and after" with a stethoscope. It was a lot of trouble, but it paid off in a considerable reduction in noise.
So for cooler-related fans, I'll want to blasts them past the point where CFM has passed a point of no difference for temperature reduction. I'll pick those fans carefully, but they will be real Blasters for CFM. I want to know what the cooler is capable of doing, and I'll realize that a great range of usage situations aren't going to spin those fans up to 100%. For instance, I've got two Nidec-Servo Gentle Typhoon AP-30 fans I use in two different computers that spin up to a top-end of 4,000 RPM with 145CFM. I'm fine for using them in the final rendition of the computer-build! So I'd want to get that information from a cooler-comparison review.
. . Anyway I agree with you that air is a good contender, I just disagreed about how much of a temp disadvantage an air cooler has to an AIO in many instances, and the case for air gets even better when taking price into consideration.
I'm always looking for "last grains of rice." So small differences are also important, if one doesn't have to expend tremendous attention and materials in the one-time investment of energy constructing the Whole. At some point-- you make compromises. I had my system "arranged" and prepared for a 280mm radiator assembly, and essentially that "arrangement" still exists, because the space for such an assembly is covered by two fans in an almost identical position. Since I'd chosen to use a certain ICY-DOCK "2x2.5"+ODD" 5.25" bay device and add laptop drives into a hot-swap mix of things, I didn't bother reclaiming the bays lost with those fans.
Ultimately -- you're right about the price factor. When you're dealing with small differences, a 30% to 50% difference in price matters. But XavierMace will tell you I have violated that precept in other ways -- just because I was curious, and didn't pursue some inferential logic to a fine-tuned hilt.
With the Silicon Lottery $50 delid and CLU preparation, I figured I'd conquered the system's main cooling problem enough that taking it further with water didn't matter much. Given the way I test these recent builds with AVX-enabled LinX or LinPack where others say "no more!" -- I think I could limit my testing to milder choices and clock this puppy to the 4.8 Ghz for which it was binned. I just don't like the voltage requirement for that. At the extreme, 1.424V is too much a difference over 1.40V. Otherwise, I'd be one of AigoMorla's biggest followers.