Spin Off: AT's Testing Methods & Uber Mode

[Saylick]

What a ridiculous rant. Nobody is condemning extra options.. Uber mode is fine, I have no problem with it as an extra option for consumers to increase performance.

The only problem is, how do you resolve it when it comes to reviews, which are supposed to be FAIR and impartial.

That's why reviews are conducted in default mode in the first place, to impose fairness. There is always extra performance to be gleaned for those that want it whether through overclocking or adjusting settings, but that should not be the focus of a baseline review, the purpose of which to showcase out of the box performance.

You can highlight the baseline by testing the card in each preset that the card comes with when its shipped. If you, the customer, have no intention of running the card in its other settings then simply ignore the data that corresponds to those settings.

Out-of-the-box, the card comes with 2 presets. You can't say that "out-of-the-box" it only has one. The fair thing to do is to test both. The reviewer isn't manually running the product outside of spec here; it's simply that the card has two supported presets, both of which are covered under warranty. It is my belief that you test EVERYTHING the product has to offer so long as it's a standard feature. By not testing each feature, YOU, the reviewer, are CONSCIOUSLY and PERSONALLY making the decision to omit data.

If I was doing a review on a blender which has multiple settings for the different foods you can blend, as a reader, would you expect me to only use the generic "blend" button when I test it?

If I was doing a review on a hair dryer which clearly has a low speed and high speed setting, would you expect me to do a review of the hair dryer only in the low speed setting?

If I was doing a review on digital camera, which clearly has many photo presets for the different types of photos I can take, would you only expect me to do a review with the camera in it's Auto preset only?

 

[MrK6]

What a ridiculous rant. Nobody is condemning extra options.. Uber mode is fine, I have no problem with it as an extra option for consumers to increase performance.

The only problem is, how do you resolve it when it comes to reviews, which are supposed to be FAIR and impartial.

That's why reviews are conducted in default mode in the first place, to impose fairness. There is always extra performance to be gleaned for those that want it whether through overclocking or adjusting settings, but that should not be the focus of a baseline review, the purpose of which to showcase out of the box performance.

So then how is it fair and impartial to omit out-of-the-box options? Go reread the part about where I said people condemning extra options being hypocrites.

You can highlight the baseline by testing the card in each preset that the card comes with when its shipped. If you, the customer, have no intention of running the card in its other settings then simply ignore the data that corresponds to those settings.

Out-of-the-box, the card comes with 2 presets. You can't say that "out-of-the-box" it only has one. The fair thing to do is to test both. The reviewer isn't manually running the product outside of spec here; it's simply that the card has two supported presets, both of which are covered under warranty. It is my belief that you test EVERYTHING the product has to offer so long as it's a standard feature. By not testing each feature, YOU, the reviewer, are CONSCIOUSLY and PERSONALLY making the decision to omit data.

If I was doing a review on a blender which has multiple settings for the different foods you can blend, as a reader, would you expect me to only use the generic "blend" button when I test it?

If I was doing a review on a hair dryer which clearly has a low speed and high speed setting, would you expect me to do a review of the hair dryer only in the low speed setting?

If I was doing a review on digital camera, which clearly has many photo presets for the different types of photos I can take, would you only expect me to do a review with the camera in it's Auto preset only?

Exactly. Like I said, it just shows the reviewer's laziness at best, even worse, it shows bias.

 

[Janooo]

Not that I consider Anandtech an enthusiast board anymore, but this is getting ridiculous. Anyone who condemns extra options on a product is an idiot or biased (or both, as this thread is seeming to prove). Adding more flexibility to the end user for how their card operates, especially in the way of making it easier to choose how it operates, is a step in the right direction. The morons who are condemning it our the biggest bunch of hypocrites I have yet see; you are not technology enthusiasts and are a disgrace to the community.

When I read hardware sites, especially reviews, I want information and furthermore expert opinion. I want up-to-date driver comparisons, head-to-head performance match-ups, and a detailed understanding of what I'm getting with the card. At AT, at least in their video card reviews, I'm getting none of this. Cutting corners to pump-out some biased, half-assed review is an insult to your readership. Find someone who isn't lazy to do your video card reviews, Anand.

When I read your first paragraph I was not sure you were talking about Ryan but your last sentence cleared that up.
I'll give you +10 (8 fingers and 2 thumbs up )!!!

AT owes us an apology.

When Ryan says

...
Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

just a few days after he invented "Noise Equalization" at 34% fan speed that does not come out of the box it's a hypocrisy, almost a lie. Very shameful!

 

[wand3r3r]

Subjective information is one person's opinion. In a newspaper, the editorial section is the place for subjectivity. It can be based on fact, but it is one person's interpretation of that fact. In this way, subjective information is also analytical.

Student research papers are usually subjective, in that the writer formulates a thesis statement and uses sources that support that thesis. Bear in mind that there is usually another equally valid viewpoint that can be supported with other sources.
​

Objective information reviews many points of view. It is intended to be unbiased. News reporters are supposed to be objective and report the facts of an event. Encyclopedias and other reference materials provide objective information.

---
Clearly subjectivity has completely overridden the last bit of objectivity in the past couple reviews.


Now reviewing all of the blurbs and inconsistencies pointed out in this thread as well as the way the last two reviews were conducted, it was clear subjectivity outweighed objectivity.

Objective conclusion:
The price is $400 and the performance ties/beats the $100 more expensive 780.

In the end we are curious about the reviewers subjective opinion, however it should be based on the hard data, and left for reader interpretation.

Based on the performance alone, the card is XXX. However we feel that 1 db more than the 480/580 is no longer acceptable since the competitor released a $1000 card with a better cooler so we now have changed our standards bla bla bla.

 

[KCfromNC]

Also, saying that it's AMD's fault doesn't explain the seeming contradictions raised about the review, like how can the review cherry pick just the bad results, by using uber mode to criticize the noise, then turning around and using quiet mode to judge the performance?

Compare this approach to the one other reviewers are saying Nvidia used to try and sway reviewers :

http://forums.anandtech.com/showpost.php?p=35714780&postcount=54

Originally Posted by Kyle_Bennett
This sounds exactly like an NDA meeting I was in last week. The fan was referred to multiple times as "jet engine" etc., but all performance comparisons shown were with the card in Quiet Mode. Funny stuff when you are reaching for straws.

You'd have to admit that's quite a coincidence.

 

[krumme]

Ryan have a write up partly of what we discusses here.
http://www.anandtech.com/show/7501/amd-changes-290-series-fan-algorithms

 

[VulgarDisplay]

I completely disagree with him about the noise with open backed headphones. I use q701's and when gaming I can't hear my 7970 (same shroud) at 55% fan speed.

Is he using an open bench sitting on top of his head or something?

 

[Vesku]

Ryan have a write up partly of what we discusses here.
http://www.anandtech.com/show/7501/amd-changes-290-series-fan-algorithms

Guess we can close up the thread, Ryan was just concerned we might miss the delivery man or a phone call:

"When I game I use open back headphones so I can listen for phone calls or the door for shipments, and as a result acoustics do matter to me."

Will it turn out the resistance to "uber" mode was actually to save us from potentially breaking a nail flipping that switch?

 

[krumme]

I completely disagree with him about the noise with open backed headphones. I use q701's and when gaming I can't hear my 7970 (same shroud) at 55% fan speed.

Is he using an open bench sitting on top of his head or something?

Well it looks like Ryan have learned noise is personal. Now he just need to grasp it because he still thinks its somethink you can vote for.

So in one sentence he says its personal in the next that its loud.

He just cant escape the fact that what is clearly loud in his head is perhaps oposite in others.

We all think the same way. That what we perciewe is the same as what others perciewe. "Why cant you see it!"

It doesnt make sense to share opinions on what is noisy or not. Cold or warm. Especially if you insist your perception is the true reality.

But ot - most importantly he doesnt adress the inconsistencies discussed here. The unwillingness to use uber.

 

[AnandThenMan]

Am I misunderstanding here, or is the refusal to use uber mode a way of sending a message to AMD about noise?

 

[krumme]

Guess we can close up the thread, Ryan was just concerned we might miss the delivery man or a phone call:

"When I game I use open back headphones so I can listen for phone calls or the door for shipments, and as a result acoustics do matter to me."

Will it turn out the resistance to "uber" mode was actually to save us from potentially breaking a nail flipping that switch?

Well as you explained yourself you dont use the uber because you might miss the delivery man?

 

[Vesku]

Well as you explained yourself you dont use the uber because you might miss the delivery man?

No, that would be the reason for a tirade in the conclusions. But to not test it at all would mean he was just trying to save us from some other mild disappointment directly involving the "uber" switch. Which is why I guessed the nail breaking thing.

 

[krumme]

Am I misunderstanding here, or is the refusal to use uber mode a way of sending a message to AMD about noise?

Well he have send that signal prior. Its not like ryan is suddenly noise sensitive i will give him that.
But he might have develloped a sensitivity to switches also but thats another matter.

 

[Silverforce11]

"The Radeon R9 290 does offer a tremendous value, and if you’re a gamer that can isolate yourself from the card’s acoustics (or otherwise don’t care) it’s easily the best buy at $399. If acoustics are important to you, then you’re in a tougher position today."

AT, THIS IS WHAT YOU SHOULD HAVE STATED IN THE ORIGINAL REVIEW. CLEARLY you are too biased by your hatred of loud GPUs and have completely missed the fact many people game with their box isolated and further away then your test conditions, or the facts many gamers play with good headphones and as such, noise from their rig is an irrelevant concern.

I also hate noisy rigs, cannot stand them, I would NOT recommend any AMD reference cards (and a few NV ones too) to anyone who are also sensitive to noise, but I would definitely have to include this simple caveat: If you don't care much about noise, go for it, at $399 its a steal.

There's also another caveat: If you already have a water cooling setup, it becomes pure awesome of a card.

Not including these statements and simply rejecting the card outright because of your own sensitivity to noise is not being objective nor representative of your reader base. I had made several posts about it, but at the end of the day, if you do not cater to enthusiasts who find nothing difficult or "wrong" in flipping a switch (under warranty even!), then they shall definitely go elsewhere for reviews which do cater to them.

 

[krumme]

Guess we can close up the thread, Ryan was just concerned we might miss the delivery man or a phone call:

"When I game I use open back headphones so I can listen for phone calls or the door for shipments, and as a result acoustics do matter to me."

Will it turn out the resistance to "uber" mode was actually to save us from potentially breaking a nail flipping that switch?

Ahh now i understand your post. He is sending a signal to amd to take care of our nails. Is that it?

 

[Vesku]

"The Radeon R9 290 does offer a tremendous value, and if you’re a gamer that can isolate yourself from the card’s acoustics (or otherwise don’t care) it’s easily the best buy at $399. If acoustics are important to you, then you’re in a tougher position today."

AT, THIS IS WHAT YOU SHOULD HAVE STATED IN THE ORIGINAL REVIEW. CLEARLY you are too biased by your hatred of loud GPUs and have completely missed the fact many people game with their box isolated and further away then your test conditions, or the facts many gamers play with good headphones and as such, noise from their rig is an irrelevant concern.

I also hate noisy rigs, cannot stand them, I would NOT recommend any AMD reference cards (and a few NV ones too) to anyone who are also sensitive to noise, but I would definitely have to include this simple caveat: If you don't care much about noise, go for it, at $399 its untouchable.

There's also another caveat: If you already have a water cooling setup, it becomes pure awesome of a card.

Not including these statements and simply rejecting the card outright because of your own sensitivity to noise is not being objective nor representative of your reader base. I had made several posts about it, but at the end of the day, if you do not cater to enthusiasts who find nothing difficult or "wrong" in flipping a switch (under warranty even!), then they shall definitely go elsewhere for reviews which do cater to them.

Well put, you should post this directly in the comment section of Ryan's explanation if you haven't done so already.

This has been my main point from first reading his 290 conclusions, why did he not just say something along those lines. "I don't like the acoustics, I think it's heading back in the wrong direction in terms of default fan speed and the resultant noise. But if you can deal with the noise..." Maybe an additional "I'm disappointed AMD did not provide a quiet mode for the 290." Instead we get almost a declaration of war. Is the "uber" thing an extension of this personal battle or is it something else?

Ahh now i understand your post. He is sending a signal to amd to take care of our nails. Is that it?

Perhaps he wants the switch to be larger and more ergonomic so you can hook it with your finger. That or AMD should include a flip switching tool. If this 290 explanation is anything to go by I would not be shocked to find that is the explanation for resisting "uber" testing.

 

[AnandThenMan]

Clarification on why uber mode will no longer be used seems to be in order.

 

[Silverforce11]

Clarification on why uber mode will no longer be used seems to be in order.

Honestly, its because its TWICE the workload to bench the same card at two different conditions.

It's NOT a reviewer friendly option with these bios switches AMD is doing. In future, they need to drop it and go full reliance on powertune boost, that with a $10 more expensive reference cooler will solve all their issues.

So indeed, AMD needs to be sent a message, because from their prior history, they simply don't get it unless they are told. Its the same story over again, AMD, awesome engineers, horrible management. Who thought it was wise to put essentially the same 7970 reference heatsink on it? That person needs to be fired.

 

[AnandThenMan]

Honestly, its because its TWICE the workload to bench the same card at two different conditions.

True, but that didn't stop pretty much every other site from doing it. But again that was not the reason given by Ryan.

Agreed on the craptastic cooler AMD used, no idea why they would do such a thing. It would have been better to hold off releasing until custom versions were available.

 

[Erenhardt]

This update is probably not aimed at this thread. We do not discuss here his noise sensitivity.

I am all in for harsh and nit-picky review style. Ask for more and we will see better products in the feature. True that there was a lot (maybe even to much) of personal opinion in the reviews, but still, mostly valid criticism.

The issue is consistency. While Hawaii reviews were focused on the downsides of the cards, 780ti review avoided that approach. One could go as far as to say: 780ti review silently passed by every shortcoming of the card.

The policy to test product as it comes "out of the box" is not something I would expect from enthusiast PC parts review - more likely from console/mobile review. But not even that is the issue. The issue is once again consistency. If they refuse to use "uber mode", why do they show Titan DP performance if it is off by default. Why do they show SLI performance if it needs hardware ingratiation mod to make it work? Why in one review we have both, out of the box only for one part and modified for other parts?

I would like to see what nvidia demanded from reviewers, what instructions nv made them to fallow. I think that sharing all the backstage action with readers could let them prepare for what they will see in the review. How to approach the data, and how to form they own opinion based on presented point of view (which seems to be influenced by card manufacturers).

 

[blackened23]

Am I misunderstanding here, or is the refusal to use uber mode a way of sending a message to AMD about noise?

GOOD. This needs to be done. I've said it before but noise is only part of the problem, the other MUCH BIGGER problem is throttling that results from that crap cooler.

AMD needs to understand that this is not acceptable - too much noise is not acceptable, and throttling with quiet acoustics is REALLY NOT acceptable. I personally hope that other review websites send this message as well. The fact that the 290 and 290X cards require a 20% drop in performance for a quiet fan profile is downright RETARDED. Come on, are you kidding me? How can anyone defend this crap? AMD should have made a better shroud. Period. I hope more websites make this message loud and clear to AMD that they screwed up, people do care about reference blower quality because not everyone is able to use an aftermarket design.

Good LORD. You people need to stop acting like AMD is the bullied victim here. THIS IS THEIR FAULT. PERIOD. I praised the 7970 back in the day because I thought it was a great GPU but it also DID NOT require an insanely high fan profile for good overclocked performance. I didn't lose 20% performance at 40% fan - I could overclock to 1100 *WITH* 40% fan. The GPU landscape has changed. This is not and should not be acceptable, and only if consumers and review websites hammer that point repeatedly will AMD get the message.

 

[AnandThenMan]

....

100% agreed.

 

[Silverforce11]

True, but that didn't stop pretty much every other site from doing it. But again that was not the reason given by Ryan.

Agreed on the craptastic cooler AMD used, no idea why they would do such a thing. It would have been better to hold off releasing until custom versions were available.

Which is why I also posted recently, other good reputable review sites do both modes just fine, why is AT "chucking a spaz" over this.. its probably sending AMD a message, stop using the stupid bios switch to make everyone's lives harder than it has to be and go full Powertune software control.

 

[TerryMathews]

Ryan have a write up partly of what we discusses here.
http://www.anandtech.com/show/7501/amd-changes-290-series-fan-algorithms

Would have been nice to know what the algorithm change did to uber mode on the 290X.

Also, when I got to the part where he was sending a message to AMD over loudness, I got the feeling that Ryan fancies himself to be Dean Wormer from Faber College.

"It's time for someone to put their foot down, and that foot is me."

 

[AnandThenMan]

GOOD. This needs to be done.

You can't honestly be suggesting that review sites should refuse to use certain features because they want to send a company a message.