Spin Off: AT's Testing Methods & Uber Mode

[krumme]

Where can I buy a 290 which defaults to a 34% fan limit, as tested here by Anandtech - http://www.anandtech.com/show/7481/the-amd-radeon-r9-290-review/15?

If there's now an "only test at defaults" policy it hasn't been around for that long.

Rofl.

 

[Atreidin]

I'm not defending AMD's cooler but yes, the cooler and the different modes are entirely different issues. There's nothing stopping a 290 card with a better cooler taking advantage of an easy switch like that.

Let's say AMD released an awesome cooler that was much quieter in uber mode, and left the switch set to uber mode from the factory, however the quiet mode instead becomes nearly silent. Then they marketed the quiet mode as an alternative for when you want silent operation at the expensbye of some speed. So it goes from a default of no throttling, but acceptably low db, to lower performance but virtually silent. It's just an optional switch you can use if you feel like it. It might not be used by enthusiasts who want max performance all the time, but when someone wants to use the card in a quiet computer, maybe sitting in the family room, it would be nice to have the option.

Reviewers would test the "silent option" just out of curiosity. People would want to know how fast a high end card could be while being silent.

(Just to make it clear, I get it, the cooler sucks, it's the fifth horseman of the apocalypse, I agree, blah blah blah.)

 

[Abwx]

http://www.hardware.fr/articles/912-4/bruit-temperatures.html

 

[KingFatty]

Then again, the entire situation could have been prevented entirely - why do we even need uber vs silent mode? Just make a better cooler and the problem is solved without a BIOS switch.

I don't understand why the mode switch is condemned? Can that be an example of personal bias interfering with objective judgment? Couldn't a reasonable person view the mode switch as an additional feature, providing more choice?

The mode switch gives users a choice. The mode switch enables users to make up their own minds and literally switch between different compromises of noise/performance.

Yet, how can someone unilaterally condemn that as always a bad thing? Different users will value things differently. Yet the quote above seems to scoff and question this feature choice/tradeoff option.

This "always bad" mentality is the crux of this thread, how noise is always bad, that it's irrelevant to consider cost vs performance vs noise because noisy. So it's ironic, in that by trying to defend the problems of the review, you've seem to have fallen victim to the very same problem - failing to recognize when you've condemned something as nothing but bad, failing to recognize your own subjective feelings clouding things over, when reality is it's a compromise with tradeoffs where people value things differently.

If you love peanut butter, can't you see that it's ridiculous to listen to the rantings of someone with a peanut allergy telling you to never buy peanut butter? Here, we are dealing with someone who has a deadly allergy to noise, and expecting everyone else to have that same sensitivity. It's simply not reality.

Some people are happy to accept noise to save money, and happy to have a noise/performance switch choice. Are they wrong? The review seems to give that message. And the point of the subject matter of this thread seems to ask if it's valid to disregard performance numbers because noisy. I disagree.

Also, I can't help but think of CPUs and how the entire industry expects cheap noisy coolers to be included in the CPU box. But think about this - would you dismiss the merits of a CPU, just because it was bundled with a cheap noisy cooler? Would you use that same line over and over and say "but Intel could have avoided this whole problem by bundling an expensive fancy CPU cooler!" Maybe AMD is following the CPU path of putting a cheap cooler to bring down the price? The point is for me personally, I like the choice of a lower initial price, because I'm going to use my own cooler anyway.

 

[SiliconWars]

The important parts are what Terry wrote about on page 2/3 and flynnsk on page 5. Anandtech tests memory and changes it from default "out the box" settings - this has to be done with just about all memory that you buy, and they also test Titan with the non-default DP driver switch. How can they then go and say they won't do it with AMD's cards because it's not out the box? Sorry, that's too much BS for me to handle.

 

[Imouto]

PSUs have been providing for years a switch to run passive at lower usage or just spin the fan slow.

Really dunno how anyone in his right mind would ask for that switch removal when it's nice to have such a nice extra depending on your concerns.

 

[krumme]

http://www.hardware.fr/articles/912-4/bruit-temperatures.html

The meassuring of the 290 and 780ti actually show nearly the same difference of 5db between the cards in this review, as Ryan measuring of 5.5db in his.

As 3db constitutes what is perciwed as a "difference" and 10db as a "doubling". Now go read the 290 noise rant.

Noise is highly personal. But what is certainly clear is that what is meassured have no connection with the words Ryan used.

Thats bias.

 

[Abwx]

The meassuring of the 290 and 780ti actually show nearly the same difference of 5db between the cards in this review, as Ryan measuring of 5.5db in his.

As 3db constitutes what is perciwed as a "difference" and 10db as a "doubling". Now go read the 290 noise rant.

Noise is highly personal. But what is certainly clear is that what is meassured have no connection with the words Ryan used.

Thats bias.

Here it is measured case closed at 0.6 metre distance ,
at 1.2 metre sound pressure will be 6dB lower , looks
like the noise rants are far more noisy than the card.

 

[Erenhardt]

Where can I buy a 290 which defaults to a 34% fan limit, as tested here by Anandtech - http://www.anandtech.com/show/7481/the-amd-radeon-r9-290-review/15?

If there's now an "only test at defaults" policy it hasn't been around for that long.

Not only that (34%), but the review has 290X on "quiet mode" only. Graphs are missing "uber mode". Looks like we have both in one review: out of the box 290X (no uber mode) and not so much out of the box 34% fan speed 290.

 

[TerryMathews]

The important parts are what Terry wrote about on page 2/3 and flynnsk on page 5. Anandtech tests memory and changes it from default "out the box" settings - this has to be done with just about all memory that you buy, and they also test Titan with the non-default DP driver switch. How can they then go and say they won't do it with AMD's cards because it's not out the box? Sorry, that's too much BS for me to handle.

Turbo mode was a good point too. There are a number of motherboards where the default setting is off. Per AnandTech's policy, doesn't that mean those boards should not have it enabled?

 

[Teizo]

I don't understand why the mode switch is condemned? Can that be an example of personal bias interfering with objective judgment? Couldn't a reasonable person view the mode switch as an additional feature, providing more choice?

I wouldn't call it that, and I think that is unfair. The mode shipped from the factory is quiet mode. You have to deliberately switch to uber mode. It's the same with drivers. Default settings in the drivers with equal settings to make sure each card is running with the same optimizations enabled/disabled for the most fair comparison. If people then want to enable/disable certain optimizations to get more performance...then that is up to them.

There is no bias, except by AMD fans who just don't want to admit that AMD can do any wrong and think somehow is being targeted unfairly when it is their own design team that should actually be coming under fire by their users....not the reviewers. Calling out Ryan, who has been does this for years, and making out like he has some sort of bias towards Nvidia because he gave the Nvidia card a more favorable review is really shameless.

All of this is a result of people just taking this whole AMD vs Nvidia thing too seriously. Or, this is some attempt by AMD insiders/marketers trying to create problems for Ryan and AnandTech because he had the courage to tell it like it is when so many try to beat around the bush trying not to hurt people's feelings by skirting the truth.

 

[TerryMathews]

I wouldn't call it that, and I think that is unfair. The mode shipped from the factory is quiet mode. You have to deliberately switch to uber mode. It's the same with drivers. Default settings in the drivers with equal settings to make sure each card is running with the same optimizations enabled/disabled for the most fair comparison. If people then want to enable/disable certain optimizations to get more performance...then that is up to them.

There is no bias, except by AMD fans who just don't want to admit that AMD can do any wrong and think somehow is being targeted unfairly when it is their own design team that should actually be coming under fire by their users....not the reviewers. Calling out Ryan, who has been does this for years, and making out like he has some sort of bias towards Nvidia because he gave the Nvidia card a more favorable review is really shameless.

All of this is a result of people just taking this whole AMD vs Nvidia thing too seriously. Or, this is some attempt by AMD insiders/marketers trying to create problems for Ryan and AnandTech because he had the courage to tell it like it is when so many try to beat around the bush trying not to hurt people's feelings by skirting the truth.

Curious how you deliberately ignore every counter example provided in this thread...

" The lady doth protest too much, methinks."

 

[Carfax83]

I wouldn't call it that, and I think that is unfair. The mode shipped from the factory is quiet mode. You have to deliberately switch to uber mode. It's the same with drivers. Default settings in the drivers with equal settings to make sure each card is running with the same optimizations enabled/disabled for the most fair comparison. If people then want to enable/disable certain optimizations to get more performance...then that is up to them.

That's the perfect analogy. Drivers typically come with three modes:

1) Default

2) Performance

3) Quality

Nvidia has a high quality mode as well which completely disables all optimizations. Anyway, the point is that testing with Uber mode is comparable to testing with driver performance enhancements turned on, while leaving the competitor in quality mode.

AMD took some flak for doing just that years ago, when their default driver settings contained some questionable performance enhancements that could increase performance as much as 10% over quality mode. As a result, several review websites now test AMD video cards with the driver setting on quality.

 

[MrK6]

Not that I consider Anandtech an enthusiast board anymore, but this is getting ridiculous. Anyone who condemns extra options on a product is an idiot or biased (or both, as this thread is seeming to prove). Adding more flexibility to the end user for how their card operates, especially in the way of making it easier to choose how it operates, is a step in the right direction. The morons who are condemning it our the biggest bunch of hypocrites I have yet see; you are not technology enthusiasts and are a disgrace to the community.

When I read hardware sites, especially reviews, I want information and furthermore expert opinion. I want up-to-date driver comparisons, head-to-head performance match-ups, and a detailed understanding of what I'm getting with the card. At AT, at least in their video card reviews, I'm getting none of this. Cutting corners to pump-out some biased, half-assed review is an insult to your readership. Find someone who isn't lazy to do your video card reviews, Anand.

 

[Abwx]

That's the perfect analogy. Drivers typically come with three modes:

1) Default

2) Performance

3) Quality

Nvidia has a high quality mode as well which completely disables all optimizations. Anyway, the point is that testing with Uber mode is comparable to testing with driver performance enhancements turned on, while leaving the competitor in quality mode.

AMD took some flak for doing just that years ago, when their default driver settings contained some questionable performance enhancements that could increase performance as much as 10% over quality mode. As a result, several review websites now test AMD video cards with the driver setting on quality.

Surely that testing the 780ti fully ocked was still a default setting
that required only a switch....heck , it s on the graphs..

 

[KingFatty]

There is no bias, except by AMD fans who just don't want to admit that AMD can do any wrong and think somehow is being targeted unfairly when it is their own design team that should actually be coming under fire by their users....not the reviewers.

... trying to create problems for Ryan and AnandTech because he had the courage to tell it like it is when so many try to beat around the bush trying not to hurt people's feelings by skirting the truth.

I think we agree then, that additional driver settings/features/modes/choices are good to have.

Now, I still don't understand how you can disregard all the points raised about the inconsistencies and odd 'judgment'-style non-objectiveness that have come to light in the review. It's not about defending AMD, it's about trying to understand these points.

A good question would be, why do you think you can erase all the valid points raised, just by using the tired line of "It's AMD's fault!" That doesn't somehow explain the odd inconsistencies or other points made regarding the review. Why should someone's personal subjective feelings about noise be forced as what everyone else should accept as the standard, instead of simply pointing out the pros/cons/tradeoffs?

Also, saying that it's AMD's fault doesn't explain the seeming contradictions raised about the review, like how can the review cherry pick just the bad results, by using uber mode to criticize the noise, then turning around and using quiet mode to judge the performance? There are other points raised better than can do it here, but I just still can't see how one can justify the "AMDs fault!" as somehow explaining the review issues.

Could it be that video cards aren't reviewed anymore, instead they are just evaluated in a competition to see who wins? So, instead of telling us what to expect from owning a card, the reviews should just dismiss all other cards that "lose" and just declare one winner?

I simply don't see it that way. I think they should talk about the features of a card, you know, review that card. If I want to buy an more affordable card, I don't want to read a review of that card where the review spends more time talking about this other more expensive card and how the more affordable card is not like this other more expensive card, and how it loses to that more expensive card.

I think Uber mode and regular mode should be evaluated in a review, because they are features I want to learn about regarding a card. To omit them is just falling into the black-and-white, winner-take-all thinking that destroys the point of a review. Sure, you can have your horse races and try to set a fair set of ground rules for competition.

But reviews aren't competitions. They are ... reviews!

 

[AnandThenMan]

..snip..

Tell us how you really feel. Being serious, I have to agree with you. I'm at a loss to understand why a reviewer would refuse to test even basic features, heck it doesn't get anymore basic than flipping a switch, but somehow that's not to be touched because it violates some phantom "out of the box" rule?

Now if I'm AMD, I simply send Anandtech a card with the switch in uber mode. Sure people won't be informed about quiet mode because can't touch that switch, but at least we'll see the full potential of the card in the review.

 

[GaiaHunter]

That's the perfect analogy. Drivers typically come with three modes:

1) Default

2) Performance

3) Quality

Nvidia has a high quality mode as well which completely disables all optimizations. Anyway, the point is that testing with Uber mode is comparable to testing with driver performance enhancements turned on, while leaving the competitor in quality mode.

AMD took some flak for doing just that years ago, when their default driver settings contained some questionable performance enhancements that could increase performance as much as 10% over quality mode. As a result, several review websites now test AMD video cards with the driver setting on quality.

Lol.

Uber performance is true performance.

Not that optimizations are bad - they are only bad when they compromise visible IQ.

 

[Vesku]

That's the perfect analogy. Drivers typically come with three modes:

1) Default

2) Performance

3) Quality

Nvidia has a high quality mode as well which completely disables all optimizations. Anyway, the point is that testing with Uber mode is comparable to testing with driver performance enhancements turned on, while leaving the competitor in quality mode.

AMD took some flak for doing just that years ago, when their default driver settings contained some questionable performance enhancements that could increase performance as much as 10% over quality mode. As a result, several review websites now test AMD video cards with the driver setting on quality.

Except for being a terrible analogy since the "uber" switch does not alter visuals or optimizations.

The closest analogy is if Nvidia decided to offer and promote a software turbo button in their drivers that upped boost clocks and perhaps fan speed and voltage beyond the shipped mode and ensured this was covered by warranty. Let's say getting this mode running was as simple as moving your mouse over to a desktop or taskbar Nvidia icon and clicking perhaps twice. Would you expect reviewers to not test that mode?

 

[AtenRa]

Just a simple question, why not having both measurements ?? one with default fan speed and one with uber mode ??? since both of the modes are present in the factory card, providing both settings results will make an even better review.

The job of the reviewer is to provide information about the product he/she reviews. By not providing performance measurements of Uber mode the review is not fully addressing the features/capabilities/performance of the product, thus is not complete.

That result in making a wrong conclusion at the end of the review.

 

[Carfax83]

Not that I consider Anandtech an enthusiast board anymore, but this is getting ridiculous. Anyone who condemns extra options on a product is an idiot or biased (or both, as this thread is seeming to prove). Adding more flexibility to the end user for how their card operates, especially in the way of making it easier to choose how it operates, is a step in the right direction. The morons who are condemning it our the biggest bunch of hypocrites I have yet see; you are not technology enthusiasts and are a disgrace to the community.

What a ridiculous rant. Nobody is condemning extra options.. Uber mode is fine, I have no problem with it as an extra option for consumers to increase performance.

The only problem is, how do you resolve it when it comes to reviews, which are supposed to be FAIR and impartial.

That's why reviews are conducted in default mode in the first place, to impose fairness. There is always extra performance to be gleaned for those that want it whether through overclocking or adjusting settings, but that should not be the focus of a baseline review, the purpose of which to showcase out of the box performance.

 

[tg2708]

Just a simple question, why not having both measurements ?? one with default fan speed and one with uber mode ??? since both of the modes are present in the factory card, providing both settings results will make an even better review.

The job of the reviewer is to provide information about the product he/she reviews. By not providing performance measurements of Uber mode the review is not fully addressing the features/capabilities/performance of the product, thus is not complete.

That result in making a wrong conclusion at the end of the review.

I wholeheartedly agree with you on this. When making any review all aspects of whatever product being reviewed should be covered especially in an unbiased manner. Half baking a review so to speak is showing favoritism in a case like this.

 

[AnandThenMan]

That's why reviews are conducted in default mode in the first place, to impose fairness.

There is no clear definition (and never will be) of what "default mode" actually is. Several examples have been cited of reviews here that were certainly not even close to representing what one would consider default or out of the box mode. What a review is supposed to be about is information, explore the features of the product and inform the reader.

 

[Erenhardt]

Except thats not entirely true.. See your Geforce Titan review.

The Double/Dual Precision function is NOT enabled by default ("out of the box"). You need to enable such in the drivers prior to testing.
Enabled:

Default (from Titan Review part 1)

Maybe it would have better worded
Our policy is to always compare out of the box performance when possible, except when we did it before. The inclusion of the uber mode numbers is essentially a one-time event., well other than when we did that before.. as well.

/shrug

Good catch.
What is wrong with this chart and why?

How will nv entry compute card look with your policy in place and why it doesn't look this way?

 

[Vesku]

When AnandTech reviews a 290X custom card will they OC it and only show default 780Ti in the graphs or will they include their 780Ti OC results. Worse yet will they not OC it at all even if it obviously has headroom?

This is the stuff we can't be sure about with the latest rounds of reviews. It's kind of ridiculous, glad there are other sources for GPU information out there.