Spin Off: AT's Testing Methods & Uber Mode

[Subyman]

Why do you care if he uses quiet or uber mode in the review? You can argue semantics all day long here but as far as i've seen "out of box" has always been the testing policy here.

If you want to blame someone, don't blame the reviewer - blame AMD for putting a terrible cooler on the card which creates the uber and quiet mode performance differences. If AMD had created a shroud, that you know, didn't suck - we wouldn't have this issue. We could have quiet mode with uber performance. Sadly, AMD wanted to save 3$ on their reference blower and pass the buck to AIB makers. Think about this for a second. If AMD had created a good reference blower that theoretically were as versatile as the Titan shroud - this would be a complete non issue. There would be zero noise complaints, and there would be no issues with performance differences between "quiet" and "uber". EVERYTHING would be quiet mode with uber level performance. Think about this point. This isn't his fault - it is AMD's fault.

I think most people's problem is that he wrote his analysis based on a mode no person spending $550 on a card would use to game with. Quiet mode is probably great for streaming movies or playing on a HTPC, but any enthusiast will be using uber mode, regardless of where a toggle is set "out of the box."

I guess the lesson for AMD is to flip the switch when send cards to anandtech for review.

 

[TerryMathews]

This isn't an argument over the 290x switch. They tested and listed both. I don't really see there being an issue there.

The inclusion of the uber mode numbers is essentially a one-time event.

The point isn't that they tested the card configuration (as they should have), it's that Ryan passes it off as though it's some sort of one-time exception when in reality failing to test the card in both of it's supported operating modes would in fact be failing to complete a thorough review.

I completely concur that AnandTech should be testing hardware "as it comes". They shouldn't be modifying hardware - things like replacing the TIM on a Haswell chip, or replacing the HSF with a waterblock (unless that is the specific purpose of the article).

I simply do not agree that "out of the box configuration" should carry down to manufacturer-supported settings. It's not even a consistent position amongst their other reviews. I would argue that

Overall our overclock for the GTX 780 Ti is a bit on the low side compared to the other GTX 780 cards we’ve seen in the past, but not immensely so. With a GPU overclock of 150MHz, we’re able to push the base clock and maximum boost clocks ahead by 17% and 14% respectively, which should further extend NVIDIA’s performance lead by a similar amount.

Meanwhile the inability to unlock a higher boost bin through overvolting is somewhat disappointing, as this is the first time we’ve seen this happen. To be clear here GTX 780 Ti does support overvolting – our card offers up to another 75mV of voltage – however on closer examination our card doesn’t have a higher bin within reach; 75mV isn’t enough to reach the next validated bin. Apparently this is something that can happen with the way NVIDIA bins their chips and implements overvolting, though this the first time we’ve seen a card actually suffer from this. The end result is that it limits our ability to boost at the highest bins, as we’d normally have a bin or two unlocked to further increase the maximum boost clock.

from http://anandtech.com/show/7492/the-geforce-gtx-780-ti-review/16 is far less representative of the "out of the box configuration".

Again, uber mode is a manufactuer-supported and warrantied configuration.

 

[railven]

Ah, Ryan's comments here explains why the review felt so...off.

With the CFX 290X setup clearly besting the SLI 780 Ti setup, and the conclusion of "it's up in the air" I now understand.

He disregarded Uber mode. Kind of weird, but okay...I guess.

Not even trying to say it was a bias article (I still find Anandtech as one of the best sites) but at work just checking charts, the conclusion didn't line up and now I know why.

EDIT: Actually, if possible for Ryan to clarify - at what point does "out of the box" stop? Curious, for example if you got a 780 HOF with their Hyper Boost button, would you not use that feature in a review? Does the "switch" have to be a hardware switch or can it be a software switch ie, AMD Overdrive has a button for "Uber Mode"?

With most of the high end products now boasting some kind of "Clock Tweaker" it be a shame of AT doesn't review those features because of their "out of the box" policy.

 

[KingFatty]

Sadly, AMD wanted to save 3$ on their reference blower and pass the buck to AIB makers. Think about this for a second. If AMD had created a good reference blower that theoretically were as versatile as the Titan shroud - this would be a complete non issue.

I wonder sometimes, how expensive would a fancy cooler cost? Is it a complete mystery, or is it $3 like you suggest? Maybe we can look at prices for aftermarket coolers, or compare prices between similar cards with reference vs aftermarked coolers pre-installed, we could get a better idea.

Did AMD pass the savings onto the consumer here, or are they profiteering? Hmm, we can look at the speculation on what people thought the cost of the card would be, and turns out some people were surprised at how low it was, so maybe it does actually reduce the price? Maybe we'll compare the price of the reference cards to the aftermarket cards later and see which is lower priced?

But again, think of the people who remove the cooler entirely and put the card under water, surely they are celebrating that AMD saved them money, because they are entirely removing the cooler anyway?

And something for you to think about, have we reached the point now, where the amount of noise of a stock cooler is more important than price/performance of the entire video card? I could understand criticizing a noisy cooler, along the lines of the GTX480 how it caught some flack because the architecture itself was tied to producing more heat fundamentally, but to completely un-recommend a card because its cooler makes too much noise, I didn't know we are at that point, especially since it's a cheap cooler that helps bring down the price of the video card, that many people can easily replace by spending $40 on an aftermarket cooler. It's like with CPUs and their stock coolers - so many people replace it anyway, it would be a net harm to computer buyers if CPU makers forced a more expensive cooler on you that you were just going to replace anyway. It's good to save money and make do with a noisy cooler if you like it, then if not, you can choose to put a quieter cooler on there.

 

[Owls]

I had a lot of regard for Anands reviews until this one. A completely disingenuous review by any standard. Shameful

 

[tential]

If you don't like their review, don't read it? Read a different side then?

I had a lot of regard for Anands reviews until this one. A completely disingenuous review by any standard. Shameful

I feel you have to say WHY if you're going to post something like that.

 

[TerryMathews]

If you don't like their review, don't read it? Read a different side then?

So we shouldn't use the discussion forum to discuss the review? How TF does that even make sense?

 

[chimaxi83]

Out of the box performance doesn't include overclocking or adding voltage, but they still do it here. I mean, if you want to nitpick, the only thing they should review then, based on "inclusion of the uber mode numbers is essentially a one-time event", is the card after a driver install, and nothing else. After all, the card itself, out of the box, doesn't install Afterburner/Trixx/GPU Tweak/Precision, doesn't overclock itself (outside of boost, which is cool), doesn't decide it wants more voltage, etc.

How far down the hole do you want to go?

 

[Silverforce11]

AT doesn't need to do fancy modes, or go the extra-mile, it doesn't cater to that market.

We have plenty of other reviews to draw our conclusions from.

 

[Subyman]

AT doesn't need to do fancy modes, or go the extra-mile, it doesn't cater to that market.

We have plenty of other reviews to draw our conclusions from.

Which is sad, because Anand's personal articles are so incredibly in-depth with a ton of insight. His mobile articles are the top tier in the entire industry. That's the quality I come to Anandtech for. Analyzing the performance metrics of a card in "quiet mode" against the flagship of its rival is, well, disappointing.

There are plenty of other sites to go to, but I tend to love Anandtech articles, which is why I'm letting the community know my opinion.

 

[Owls]

If you don't like their review, don't read it? Read a different side then?



I feel you have to say WHY if you're going to post something like that.

Have you even read the review? He points out how the Ti is faster than the 290x when the 11% lead comes from the 290x being in quiet mode. It just goes to show how few people actually read the review fully except the "final words"

 

[Will Robinson]

From the first few sentences of the 780Ti review you could tell this was going to be a "we are SO non biased towards AMD" article.
780Ti turns up with not much of a performance gain over 290X,$150 more expensive,pretty high power use and noise levels and SLI drivers and scaling that are broken.
The article doesn't then go into a repetitive rave over these facts like the 290 article(OMG WE WONT EVEN RECOMMEND IT FOR SALE) but instead nicely glosses over it all for a feel good finish.
I think most NV fans would be mighty happy to have AT's review to hold up any time criticism of 780Ti surfaces.
[H] on the other hand sees it this way:

It is clear from this breakdown that the AMD Radeon R9 290X is the better value in terms of performance by a long shot. It is $150 cheaper than the GeForce GTX 780 Ti, but delivers the same gameplay experience and very near the same framerate performance. A performance difference that makes little or no real difference. The $150 price difference is a much larger difference than the performance difference.

For all intents and purposes, the GeForce GTX 780 Ti and the Radeon R9 290X at Uber Mode are identical in delivering the same gameplay experience.

 

[Silverforce11]

I have never been one to accuse reputable sites like AT or [H] of being biased, they have a long history of objectivity.

I do happen to agree with them, out of the box, I would not recommend reference R290/X due to heat/noise. Even though its cheaper, its something that I would not enjoy gaming next to. Now, I do wish they had mention its potential with a) aftermarket cooling or b) for those going on water or c) potential with upcoming AIB models, because thats the plain truth of the matter. However, they aren't entitled to do so.. because only true enthusiasts would do A and B. Thats where other sites focus on. Heaps of other reviews to read and draw conclusions from.

 

[krumme]

I agree with you. Ryan's review was strange. He based his analysis on "quiet mode" instead of the way anyone would run their 290X, uber mode. It's like Car and Driver auto crossing a Ferrari in Comfort Mode because the exhaust is too loud in Race Mode.

I dont know if this helps understand why the review feels strange to you:

Historically there have been a bad relationship between amd and anandtech at least on the gfx side. It takes money and competence from the amd side to build that relationship with eg prompt info, support and so on. Perhaps its remisents of that bad relationship that unintentional influeces the last 2 gfx review a little.

After all when you invest in pr and technical marketing you do it to get good roi on that investment. And nv is very good at that lets say support - because thats how its perciewed from reviewer side.

I dont mind reviewers are enthusiast themselves and have temper like eg kyle and ryan. And i dont care if they prefer a certain brand. And its fine with personal preferences eg for sound as long as its written its a personal opinion. And its fine fo me when ryan bashes amd for their lacking and late optimus response and noisy reference coolers.

But the last review was just plain bad. Its perfectly fine with temper but i dont know what hit ryan this time. Perhaps he is loaded with bs from amd and nv.

The 290 cf is very much faster than the sli counterpart. How that can be said otherwise is difficult to explain. My guess is when the dust settles, feelings is less we will see less of the last two review that was indeed very strange and return to normal.

When that return to normal happens perhaps we can get some answers to why its so much faster. My guess its very much hidden in the efficiency of the memory controller. Amd really needed to do something about it, because kaveri with ddr3 makes no sense imho with the technology and memory controller efficiency they have used prior. Remember the huge efficiency gain in mem controller from 580 to 680.

 

[Janooo]

You think everyone who has the kind of money actually does research? This forum and others like represent the minority.

Please, out of the box means no changes in image quality settings, they require a switch as well. Is it going to happen?

 

[railven]

AT doesn't need to do fancy modes, or go the extra-mile, it doesn't cater to that market.

We have plenty of other reviews to draw our conclusions from.

That's a terrible solution. You'd want to draw readers not drive them away. And please explain how using a feature on a product is "going the extra mile." Testing a product as advertised is a thorough review.

Again if AT got a HOF I'd be sad to see if they didn't test with the hyper boost button.

 

[cmdrdredd]

Please, out of the box means no changes in image quality settings, they require a switch as well. Is it going to happen?

Look I have been reading AT for over a decade. You want them to be like everyone else out there, they aren't and there is a reason I feel AT is better at technical reviews than some other sites.

They actually give you a lot of the technical bits that other sites leave off. Many sites show you the card and post a few benchmarks and say "hey this one is faster".

 

[TerryMathews]

Look I have been reading AT for over a decade. You want them to be like everyone else out there, they aren't and there is a reason I feel AT is better at technical reviews than some other sites.

You don't want to get into an epeen argument with me over who's been here longer...

Stating that normally you would not flip a switch in order to test both operating modes is ridiculous. It certainly isn't the review policy they had.

Manufacturers default settings is a good policy to have. This card has two default settings. Why is that so difficult to comprehend?

 

[cmdrdredd]

You don't want to get into an epeen argument with me over who's been here longer...

Stating that normally you would not flip a switch in order to test both operating modes is ridiculous. It certainly isn't the review policy they had.

Manufacturers default settings is a good policy to have. This card has two default settings. Why is that so difficult to comprehend?

It doesn't have two default. The quiet mode is normal mode. It is the standard mode of operation.

That's like saying you should have tested every hd6950 with the switch flipped to unlock the shaders to 6970 which of course wasn't done in reviews because the standard configuration is not with the switch flipped. It is fine to show the difference in initial reviews of the card but it is not how it performs on first boot up.

Blame AMD for making it this way.

 

[boxleitnerb]

Manufacturers default settings is a good policy to have. This card has two default settings. Why is that so difficult to comprehend?

Any device/program cannot have multiple default settings. Per definition, default is one (1) state.

 

[TerryMathews]

It doesn't have two default. The quiet mode is normal mode. It is the standard mode of operation.

That's like saying you should have tested every hd6950 with the switch flipped to unlock the shaders to 6970 which of course wasn't done in reviews because the standard configuration is not with the switch flipped.

A) I would see no issue with that if it was an advertised and manufacturer supported configuration. If you could RMA a card because the shaders didn't unlock, it should count.

B) What if a card shipps with the switch in Uber mode? Is AnandTech going to be comfortable looking like they are a bunch of fools or paid off because the EVGA Super skills extreme ultra OC 290XXX tests 25% faster because they ran it in uber mode and everything else in quiet?

Or are they going to (gasp!) modify it's default settings to be consistent with their review database?

This switch is like jumper settings back when you had to configure your FSB. Properly configuring your hardware according to the manufacturers directions should still be considered "default settings."

Finally, the card most certainly does have two default settings unless your are claiming there can only be one? I don't know how you square that with RAM, which can have 4+ default settings stored in their SPD.

What is your vested interest in making sure these cards are only tested in one configuration? Why would you NOT want to see both on the chart? That is what is really confusing me with your position, you're advocating for less data...

 

[TerryMathews]

Any device/program cannot have multiple default settings. Per definition, default is one (1) state.

Wrong. Explain SPD on RAM.

You can have multiple default settings. Manufacturers default settings just means, how they configured it. In this case, AMD gave you two settings to choose from.

 

[boxleitnerb]

We had this topic before and I thought it was dealt with:

http://en.wikipedia.org/wiki/Default_%28computer_science%29

A default, in computer science, refers to a setting or a value automatically assigned to a software application, computer program or device, outside of user intervention.

Quiet mode is the single default mode, there is nothing more to discuss.

Wrong. Explain SPD on RAM.

You can have multiple default settings. Manufacturers default settings just means, how they configured it. In this case, AMD gave you two settings to choose from.

You confuse default with preset. There may be multiple presets, but only one is active by default.
Nvidia also gives you multiple settings to choose from (temperature target ranging from 80C - default - to 95C). By your logic, you can justify changing every setting you please just because you have the ability to do so. If it's a physical switch or a software slider doesn't matter. Either you test all product like they come out of the box or you don't. But mix and match just doesn't constitute a fair comparison.

 

[TerryMathews]

We had this topic before and I thought it was dealt with:

http://en.wikipedia.org/wiki/Default_(computer_science)


Quiet mode is the single default mode, there is nothing more to discuss.

Nowhere in there does it state there can be only one default.

Again, explain SPD on RAM. Are you seriously arguing that only one of the 4+ timing configurations can be considered default settings?

ETA: Uber mode fits that definition. Your issue seems to be the configuration switch. So your position is that the EVGA Super Maxi Ultra 290XXX which ships with the switch on Uber mode would be "default" and quiet would not be tested?

Don't you see why this is a terrible review policy?

You confuse default with preset. There may be multiple presets, but only one is active by default.
Nvidia also gives you multiple settings to choose from (temperature target ranging from 80C - default - to 95C). By your logic, you can justify changing every setting you please just because you have the ability to. It doesn't work that way, sorry.

So which "preset" is "default" on a stick of RAM?

Did you look at the 780ti review I linked to? They had no issue modifying the very settings you are talking about. Much more in-depth that flipping a switch to the other manufacturer's default setting. Wonder why that is.

 

[Silverforce11]

That's a terrible solution. You'd want to draw readers not drive them away. And please explain how using a feature on a product is "going the extra mile." Testing a product as advertised is a thorough review.

Again if AT got a HOF I'd be sad to see if they didn't test with the hyper boost button.

Why would you be sad? Just go to TPU or a plethora of other sites that do these extra options..

AT caters to their audience, if their audience doesn't care about it, why should they spend extra resources to test settings which doesn't appeal to the readers?

Either way, as long as AT is upfront that they are benching the new series on the quiet mode for their results, why does it matter? Again, if you want to see the results on uber mode, go to another site...