Spin Off: AT's Testing Methods & Uber Mode

[Erenhardt]

IMHO, "Out of the box" comparison is good for console review, not enthusiast PC parts.

AT is not obligated to test anything

Very true. Last reviews feel rushed and inconsistent.

Your tone is exactly why no one in their right mind would respond. It's accusatory, hostile, and altogether rude. It's a no win scenario. If he comes, he'll just get ganged up on and pummeled by trolls.

Welcome to the Internet. I bet AMD was reading 290x AT review with a wide smile on their faces. Criticism is what makes us do better next time.
I don't see anyone being hostile here. Rude - maybe, but pardon me if I'm rude. My English is not good enough and sometimes people are offended by the way I say things, which is caused by cultural differences between countries and translation between our languages.
(before I get infraction/vacation for callout again - it is not a callout. I'm responding to the user post in discussion, not a "mod on duty")

 

[ViRGE]

I don't see anyone being hostile here. Rude - maybe, but pardon me if I'm rude. My English is not good enough and sometimes people are offended by the way I say things, which is caused by cultural differences between countries and translation between our languages.
(before I get infraction/vacation for callout again - it is not a callout. I'm responding to the user post in discussion, not a "mod on duty")

Nah, you're cool for the moment.

I was just responding to Owl's accusation of bias. It's quite an extreme thing to accuse AT of, and it implies that the poster's mind is already made up.

 

[TerryMathews]

Nah, you're cool for the moment.

I was just responding to Owl's accusation of bias. It's quite an extreme thing to accuse AT of, and it implies that the poster's mind is already made up.

Well, I'll tell ya what ViRGE. No bias, no BS. Ive been here forever reading AnandTech.

I don't read the reviews much anymore. I did on occasion use Bench, but this attitude is making me question the value of that as well. If I have to double check and see if the database has booked a low ball score, I may as well just use a different data source.

I'm commenting because I want the quality of the reviews of old. I don't want this new attitude to be how it is because I do value the content here and this stance is going to damage that value.

 

[Meekers]

t only has the one mode, and was trashed based on that. 290X does have two modes, but that certainly didn't get trashed.

He trashed the 290 at 47% fan speed when uber on the 290x is 55%. It may not be fair to interpret his condemnation for the 290 as also against the 290x but it is an obvious connection.

 

[Owls]

Your tone is exactly why no one in their right mind would respond. It's accusatory, hostile, and altogether rude. It's a no win scenario. If he comes, he'll just get ganged up on and pummeled by trolls.

No one is being hostile or accusatory. Your disregard for the facts presented in the review and calling me "hostile, and rude" is quite frankly ridiculous and childish. There's no point in even arguing with you because you work for AT and your mind is made up. Moving on.

 

[Erenhardt]

Just a thought:
Next reviews will probably be 290/290X with coostom coolers from AMD's board partners (as always: sapphire). Not including "Uber mode" and showing 290X crippled performance in "quiet mode" will result in a huge performance jump compared to aftermarket cards. I fully expect him to be accused of AMD bias then for showing custom 290Xs in better light than they really are.

 

[Edgy]

quote "...Typically we’d only include results from the default mode in articles such as these, ...."

I'm not sure what the exact meaning of "articles such as these" is but the 290X review included data for both Uber and Quiet mode along with respective analysis for both modes.

Therefore it seems odd that for 780 Ti review the analysis portion regarding 290X comparison only covered Quiet mode only.

Additionally in the conclusion of 780 Ti review :

quote "...GTX 780 Ti still has other advantages – power and noise in particular..."

This seems to suggest the conclusion was based on comparison between 290X Uber mode vs 780 Ti instead of the 290X Quiet mode - since from what I can observe within the provided data, the power and noise levels between 290X Quiet mode vs 780 Ti were not "particular" enough to warrant such notable advantage toward 780 Ti.

This seems relevant enough to note since if one were to strictly keep the analysis focused on the "default" (quiet) mode for 290X, it should apply also to power and temp conclusion as well as any performance analysis.

As to the decision to only use "default mode" data/analysis by Anandtech - I think the only relevant issue there should be "what does the audience want/need."

I think it is safe to state that enough potential users who purchase 290X cards would use it in Uber mode no matter what the "default" mode is. Therefore if those readers are interested then Anandtech should include them lest they start losing those audiences to the next reputable review site who would include them by accurately predicting those needs.

 

[flynnsk]

We covered it in far more detail in the 290X launch article. For the GTX 780 Ti review we made the performance data available in our charts, but from a commentary perspective the scaling factors and the reasons behind them have not changed since the 290X article where we compared GTX 780 SLI to 290X CF.

Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

Except thats not entirely true.. See your Geforce Titan review.

The Double/Dual Precision function is NOT enabled by default ("out of the box"). You need to enable such in the drivers prior to testing.
Enabled:

Default (from Titan Review part 1)

Maybe it would have better worded
Our policy is to always compare out of the box performance when possible, except when we did it before. The inclusion of the uber mode numbers is essentially a one-time event., well other than when we did that before.. as well.

/shrug

 

[Janooo]

Well, I'll tell ya what ViRGE. No bias, no BS. Ive been here forever reading AnandTech.

I don't read the reviews much anymore. I did on occasion use Bench, but this attitude is making me question the value of that as well. If I have to double check and see if the database has booked a low ball score, I may as well just use a different data source.

I'm commenting because I want the quality of the reviews of old. I don't want this new attitude to be how it is because I do value the content here and this stance is going to damage that value.

You are not alone. When I read AT I have a bias filter on.

 

[TerryMathews]

Except thats not entirely true.. See your Geforce Titan review.

The Double/Dual Precision function is NOT enabled by default ("out of the box"). You need to enable such in the drivers prior to testing.

Another good example.

 

[Subyman]

quote "...Typically we’d only include results from the default mode in articles such as these, ...."

I'm not sure what the exact meaning of "articles such as these" is but the 290X review included data for both Uber and Quiet mode along with respective analysis for both modes.

Therefore it seems odd that for 780 Ti review the analysis portion regarding 290X comparison only covered Quiet mode only.

That is a very valid point. Perhaps that is why I found the review so strange. After reading the R290X review with an analysis of the "uber" mode in comparison to the Titan and 780, it was very disappointing to read that the 780TI would only be compared using "quiet mode." It is inconsistent for readers looking to compare the two models. In fact, the analysis for the 780TI is pretty much useless for a respective buyer looking to use either card to its full potential.

As for sticking strictly to a review policy, it can cause the reviewer to be blind to obvious features that a user---who doesn't stick to some mystical policy---would certainly want to be analyzed versus the card's respective rival.

 

[SiliconWars]

Except thats not entirely true.. See your Geforce Titan review.

The Double/Dual Precision function is NOT enabled by default ("out of the box"). You need to enable such in the drivers prior to testing.
Enabled:

Default (from Titan Review part 1)

Maybe it would have better worded
Our policy is to always compare out of the box performance when possible, except when we did it before. The inclusion of the uber mode numbers is essentially a one-time event., well other than when we did that before.. as well.

/shrug

The recent reviews still use the double precision switch for some of Titans compute benchmarks. Ok so a few compute benchmarks aren't the same as pages of gaming benchmarks, but the reason given was clear enough - "out of the box"

This is a little too close to double standards from my pov. Even if the Titan double precision benchmarks are dropped from now on, it'll be hollow as the card is EOL and has been superseded by the Ti. That would just smack of Anandtech rewriting the rules after they stopped to benefit Nvidia.

Many people were upset at the 290 review conclusion and with this follow up there is a real danger of the bias accusations getting out of control. Anandtech needs to look more closely at this and consider a change of decision before more harm is done.

 

[Vesku]

I think my original comment on the AT 290 review final words is relevant here.

From Anandtech GTX 480 review:

"Meanwhile let’s talk about the other factors: price, power, and noise. At $500 the GTX 480 is the world’s fastest single-GPU card, but it’s not a value proposition. The price gap between it and the Radeon 5870 is well above the current performance gap, but this has always been true about the high-end. Bigger than price though is the tradeoff for going with the GTX 480 and its much bigger GPU – it’s hotter, it’s noisier, and it’s more power hungry, all for 10-15% more performance. If you need the fastest thing you can get then the choice is clear, otherwise you’ll have some thinking to decide what you want and what you’re willing to live with in return."

http://www.anandtech.com/show/2977/...x-470-6-months-late-was-it-worth-the-wait-/20

The negativity in the Anandtech 290 review conclusions is way out of proportion. If they'd just said "AMD missed an opportunity by not following Nvidia's lead with a better blower", sure. But look at how many negative phrases are in the final words page:

"it’s admittedly not very often that we write a negative video card review"

"great deal of confusion and a tinge of sadness"

"At 57.2dB the 290 is a loud card. A very loud card. An unreasonably loud card."

"this is one of a handful of cards we’ve ever had to recommend against"

"point where a video card is simply too loud"

"an unreasonable level of noise."

"the reference 290 untenable as a purchase"

http://www.anandtech.com/show/7481/the-amd-radeon-r9-290-review/17

That is an amazing amount of repetitive criticism for a review conclusion.

Meanwhile, in the GTX 480 original review the 480 was actually 64.1dB to the 5870s 59.3dB and the entire tone of the conclusion is extremely neutral with a few positives. From the same reviewer.

I actually agree in a much milder manner, I did not buy a 480 and I'm not going to get a reference 290. It's just reading a conclusion that basically consists of "it's loud" "did I mention it's loud" "it's the greatest price/perf in the high end since the 5850/70 but hey IT IS LOUD, MAKES A LOT OF NOISE, THE FAN RUNS 7% FASTER THAN I WOULD HAVE ACCEPTED" is way overboard. Enough so to make me go back to that 480 review because I knew they didn't call it out much at all.

 

[Vesku]

Just a thought:
Next reviews will probably be 290/290X with coostom coolers from AMD's board partners (as always: sapphire). Not including "Uber mode" and showing 290X crippled performance in "quiet mode" will result in a huge performance jump compared to aftermarket cards. I fully expect him to be accused of AMD bias then for showing custom 290Xs in better light than they really are.

Yes, if this is a truly "one time only" thing it is going to be pretty silly when AnandTech does not show reference "quiet" and "uber" vs custom cards versions.

 

[SiliconWars]

Vesku, If you looked at the front page on review day they also chose "Loud" as the featured image caption for the 290 review (where the current 780 Ti review says "Featured Review").

 

[Attic]

Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

This comes across as rigorously adhering to the letter of the law without regard for it's intent. Anandtech has always done a great job of giving readers the information they need/want to get.

I'm not understanding why it's overly contentious to simply test the card and explain the different settings.

For an enthusiast card at this price level I personally feel that it can be argued that AMD made a mistake shipping the cards in quiet mode instead of Uber, but personally i'd expect Anandtech to cover the difficult/ease of making the performance switch and then quantify the performance differences. Pretty much what happened IMO. The writing regarding the performance differences appear to have taken on a more subjective tone than a lot of folks expected. Given the context of the task (extreme gaming) why not use and champion uber mode for comparing the heavyweights (290x vs 780ti)?

I get that AMD can have their feet held to the fire for the quiet/uber mode concerns, I just think that going that route is kind of outrageous given the ease of switching modes and their intended purpose to give users a choice.

Right or wrong I can understand folks feeling their is a bias. There have been numerous VC reviews where the stuff comes across that way, most will recall the 6850/6870 reviews when the inclusion of a very limited availability Super Duper High OC EVGA 460GTX was included (if I recall prior to it's availability in retail) to be shown against the reference AMD stuff. The shades of bias appear to favor nVidia methods/tactics, i'm not saying it to call out, just saying it because that's how this reader sees it. I have a hard time believing that nVidia didn't put out some considerations to reviewers to test the 290x in quiet mode vs the 780ti because of XYZ.

 

[Imouto]

http://forums.anandtech.com/showthread.php?p=35694414&highlight=#post35694414

"What I see is double standards everywhere. Forum folks, reviewers.

Lets see what Ryan Smith said at the GTX 580 launch (44.4db Idle - 57.1db Load).

Load

the GTX 580 is not whisper quiet, but at no point in our testing did it ever get “loud”.

Now lets see what he just said about the R9 290 (39.5db Idle - 57.2db Load).

Load

With the 290 AMD has thrown out any kind of reasonable noise parameters

Consistency, that is."

 

[Vesku]

Vesku, If you looked at the front page on review day they also chose "Loud" as the featured image caption for the 290 review (where the current 780 Ti review says "Featured Review").

Oh wow, that is pretty bad form if true.

Exactly, Imouto. Same thing with the 480 review but looks like it applies to the 580 review as well, which means the "times have changed argument" is going to have a hard time finding legitimacy.

 

[Attic]

Vesku, If you looked at the front page on review day they also chose "Loud" as the featured image caption for the 290 review (where the current 780 Ti review says "Featured Review").

My initial and second impression of that was that it was an error or joke. After reading the review I was a bit more perturbed. A $400 card that is battling with the $1k Titan at the time of review and the "Loud" heading was not a joke or error, rather it was the intended way to shade the entire review.

I get that it was loud, i'm just not sure if that was an appropriate way to center the review all else considered.

 

[Subyman]

http://forums.anandtech.com/showthread.php?p=35694414&highlight=#post35694414

"What I see is double standards everywhere. Forum folks, reviewers.

Lets see what Ryan Smith said at the GTX 580 launch (44.4db Idle - 57.1db Load).

Load
Now lets see what he just said about the R9 290 (39.5db Idle - 57.2db Load).

Load
Consistency, that is."

Wow, the 580 was tested at the same noise levels, but given a pass? Very sad. Hopefully we get some kind of follow up from the review staff.

 

[blackened23]

Christ, why are you guys digging for GPU data from 3 freaking years ago? Understand that GPU standards now are higher than they were in 2010.

The AMD cooler situation that may have been okay in 2010 is not acceptable now. The throttling in quiet mode is not acceptable now. The noise as compared to competing solutions (eg Kepler) is not acceptable now. Stop acting like a bullied victim, this is 100% AMD's fault. Not the reviewers fault. If you stop playing the victim card and start admitting that AMD screwed up, maybe AMD will actually work on creating a better user experience next time around. The cooler situation is less forgiving now precisely because you cannot realistically operate the 290 cards in a quiet mode unless you want to sacrifice performance. Did any prior card exhibit such behavior? No. Noise isn't the only reason for the cooler being unacceptable, the quiet mode throttling is also a reason. As well, 2010 standards != 2013 standards.

How much of this circus nonsense would have been avoided had AMD put some effort into user experience (eg better shroud design). Toms controversy? Would not have happened. Reviewer complaints of noise? Would not have happened. User complaints of noise? Would not have happened. Throttling in quiet mode? Would not have happened. What WOULD have happened would be uber performance with quiet mode noise levels - AMD could have done that. But didn't. Don't try to blame reviewers for this.

 

[Abwx]

What was acceptable then is not acceptable now.

Say who.??..

 

[blackened23]

Say who.??..

Oh I don't know. I just got done reading a controversy at tomshardware that was picked up by probably 30 other web news outlets that is based directly on the terrible cooler on the 290 series. And then, I see noise mentioned as a negative in 100% of every web review of the 290 series. And then practically every PC enthusiast forum has pages upon pages of people complaining about the noise and how they wouldn't consider a 290 series card because of noise. Says who? Indeed. Was all of this entirely, 100% preventable? Yes, it could have been prevented - No throttling, no noise complaints, no user experience complaints, no conspiracy theories about reviews at AT.. Preventable. Yes, yes it was. But wasn't.

But, let's forget all that. AMD is a victim here, right? I mean AMD didn't need to prevent all of this with a better design.

 

[Vesku]

Yes, let's base our judgements on what is linked on the internet. We shall all be amazingly muscled yet skinny super model billionaires in no time.

GTX 580 was basically just 1 GPU generation ago, actual gamers haven't suddenly decided that quiet is the most important attribute of a GPU. We don't see the market dominated by factory water blocked cards, do we?

The issue isn't that the conclusion was "the 290 is louder than Y and Z" it's that there was an actual tirade, as in extremely repetitive negative statements, against it. Top that off with now some questions about whether "Uber" mode results will even be included in the future and you can see it's not about trying to find a victim but about whether this attitude make sense in GPU reviews.

 

[cmdrdredd]

Christ, why are you guys digging for GPU data from 3 freaking years ago? Understand that GPU standards now are higher than they were in 2010.

The AMD cooler situation that may have been okay in 2010 is not acceptable now. The throttling in quiet mode is not acceptable now. The noise as compared to competing solutions (eg Kepler) is not acceptable now. Stop acting like a bullied victim, this is 100% AMD's fault. Not the reviewers fault. If you stop playing the victim card and start admitting that AMD screwed up, maybe AMD will actually work on creating a better user experience next time around. The cooler situation is less forgiving now precisely because you cannot realistically operate the 290 cards in a quiet mode unless you want to sacrifice performance. Did any prior card exhibit such behavior? No. Noise isn't the only reason for the cooler being unacceptable, the quiet mode throttling is also a reason. As well, 2010 standards != 2013 standards.

How much of this circus nonsense would have been avoided had AMD put some effort into user experience (eg better shroud design). Toms controversy? Would not have happened. Reviewer complaints of noise? Would not have happened. User complaints of noise? Would not have happened. Throttling in quiet mode? Would not have happened. What WOULD have happened would be uber performance with quiet mode noise levels - AMD could have done that. But didn't. Don't try to blame reviewers for this.

Bravo...well said and I agree wholeheartedly.

This whole conversation could have been avoided by a little extra engineering.