Spin Off: AT's Testing Methods & Uber Mode

[Vesku]

I noticed that too, but Ryan didn't seem to comment much on multi-card performance. To me it seemed like a pretty big deal though especially for high end buyers. The crossfire scaling is incredible. In some of the benches, the 280X Crossfire is beating the 780 TI SLI.

No repetitive tirade in the 780Ti Anandtech review about SLI being subpar? Hmmm, I'd think things like being too expensive and needing to improve multi-gpu scaling would be at least as worthy of comment as +3 recorded decibels (see the "Final Words" tirade on noise in the Anandtech 290 review).

 

[krumme]

Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

I understand that policy and there is much good in it.
But is it a refection of user behavior with the 290x cards?

I doubt any enthusiast buying these cards will run them in quiet or for that matter without proper ventilation.

Why shouldnt the review results reflect more what performance - and noise - they will get using uber mode?

 

[KingFatty]

I noticed that too, but Ryan didn't seem to comment much on multi-card performance. To me it seemed like a pretty big deal though especially for high end buyers. The crossfire scaling is incredible. In some of the benches, the 280X Crossfire is beating the 780 TI SLI.

Agreed, the charts were noticeable enough to really pop out and demand an explanation, but the writing didn't seem to address it. I guess that's just rushed/sloppy writing, failing to have the insight/thoughtfulness to foresee what the reader would question/expect an explanation for.

But then again, perhaps it's even worse where the writing contradicts and just seems illogical in view of the charts? I dunno, it's just hard to figure out the Anandtech review writing, or maybe there is another explanation (personal bias?) to account for the oddness of it?

 

[Deaks2]

We covered it in far more detail in the 290X launch article. For the GTX 780 Ti review we made the performance data available in our charts, but from a commentary perspective the scaling factors and the reasons behind them have not changed since the 290X article where we compared GTX 780 SLI to 290X CF.

Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

So you plan to only bench video cards with launch drivers from now on too?

 

[cmdrdredd]

So you plan to only bench video cards with launch drivers from now on too?

Yeah cause that's what he said. Wait no he didn't.

Everyone is on a witch hunt here I guess.

 

[TerryMathews]

Yeah cause that's what he said. Wait no he didn't.

Everyone is on a witch hunt here I guess.

Well, they're acting like Uber mode is some sort of hack. Or overclocking.

How would you have reacted if they said they were benchmarking the GeForce 690 with SLI off per their guidelines? Its the same thing.

A review policy that causes you to not test an advertised feature is not a good policy IMO.

 

[Subyman]

What is worth mentioning are reviewers thoughts below graphs:
Metro:

In case someone didn't see graphs - SLI 780ti was 3 fps behind CF 290X(Uber) in best case scenario. So, topping charts but not.
Bioshock:

By "several frames per second ahead of 290X CF" they meant actually 2fps behind it.

BF3:

Chart topper (ahead in 1 of 3 charts)

And my favorite one in


88 for SLI 780Ti vs 98 CF 290X is a tie and at 4K SLI takes slight lead being 20% slower?!

I case someone want to jump on me. I see what they did there. Next time on anandtech: amazing race through firestrike benchmark on idle clocks! See which card idles faster!

I agree with you. Ryan's review was strange. He based his analysis on "quiet mode" instead of the way anyone would run their 290X, uber mode. It's like Car and Driver auto crossing a Ferrari in Comfort Mode because the exhaust is too loud in Race Mode.

 

[biostud]

Lets see when we start getting custom cards, and the drivers have fixed some of the bugs. Although the drivers for GTX 780 TI must be pretty optimized already, except for 4K apparently. A test with the same cooler on all cards would be great. Get an ASUS DCUII version of GTX 780, GTX 780TI, R290 and R290X (once they're released) and benchmark them against each other.

 

[Subyman]

Lets see when we start getting custom cards, and the drivers have fixed some of the bugs. Although the drivers for GTX 780 TI must be pretty optimized already, except for 4K apparently. A test with the same cooler on all cards would be great. Get an ASUS DCUII version of GTX 780, GTX 780TI, R290 and R290X (once they're released) and benchmark them against each other.

I'm looking forward to an MSI TF review myself.

 

[Janooo]

We covered it in far more detail in the 290X launch article. For the GTX 780 Ti review we made the performance data available in our charts, but from a commentary perspective the scaling factors and the reasons behind them have not changed since the 290X article where we compared GTX 780 SLI to 290X CF.

Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

What a joke.


I'm not laughing.


-Rvenger

 

[AnandThenMan]

Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

Make sure you use the drivers that come in the box as well.


Personal attacks are against the forum rules.

-Rvenger

 

[Subyman]

Make sure you use the drivers that come in the box as well.

Lol, can't take off the DVI/HDMI/DP protectors either.

 

[Erenhardt]

Erenhardt: From our testing notes on page 3:
Our policy is to always compare out of the box performance when possible. The inclusion of the uber mode numbers is essentially a one-time event.

So I guess putting 780ti SLI in the charts is wrong with your policy. You hard moded the cards - made hardware modification adding SLI connector to the setup. That is huge modification compared to pushing a switch.

The only thing that I don't like about those last reviews is consistency. It is as if two different people made those reviews. Yet both are signed by the same person.


Callouts will not be tolerated.

-Rvenger

 

[OCGuy]

1. Edit: OP changed the title to something that can actually be discussed
2. 780ti performance outside of the review thread



Yay forum pollution.



Your post is forum pollution. Please refrain from it next time.

-Rvenger

 

[chimaxi83]

1. Inflammatory thread title

2. 780ti performance outside of the review thread



Yay forum pollution.

1. I agree

2. So what? I gather he's comparing the performance of two different cards. If you go into the 780Ti thread talking about 290CF performance, I'm sure the brigade will report away.

3. Your post is thread pollution.

Report and get over it.

 

[cmdrdredd]

Well, they're acting like Uber mode is some sort of hack. Or overclocking.

How would you have reacted if they said they were benchmarking the GeForce 690 with SLI off per their guidelines? Its the same thing.

A review policy that causes you to not test an advertised feature is not a good policy IMO.

It's not even the same. Out of the box the 690 is an SLI card.

 

[TerryMathews]

It's not even the same. Out of the box the 690 is an SLI card.

And out of the box these cards support uber mode. Please explain how this is any different.

 

[cmdrdredd]

And out of the box these cards support uber mode. Please explain how this is any different.

As he said...out of the box. Once you plug the card in and install drivers how does it perform.

I don't understand the problem here. Are you that upset that AMD doesn't have the fastest card after a couple weeks to market that you need to start a witch hunt and want heads to roll because you disagree with a review and think that "uber mode" is somehow going to be the savior for the 290?

 

[GaiaHunter]

As he said...out of the box. Once you plug the card in and install drivers how does it perform.

I don't understand the problem here. Are you that upset that AMD doesn't have the fastest card after a couple weeks to market that you need to start a witch hunt and want heads to roll because you disagree with a review and think that "uber mode" is somehow going to be the savior for the 290?

I'm not entirely sure how flicking a switch in the card is any different than pluging the power cable.

It is part of the factory settings that are covered by the warranty and that are guaranteed to work.

I don't exactly see how having two bios holding two different default settings is somehow a con.

It would be fun if AMD started to send the cards with the switch on the uber position or half on each position.

 

[blackened23]

So I guess putting 780ti SLI in the charts is wrong with your policy. You hard moded the cards - made hardware modification adding SLI connector to the setup. That is huge modification compared to pushing a switch.

The only thing that I don't like about those last reviews is consistency. It is as if two different people made those reviews. Yet both are signed by the same person.

Why do you care if he uses quiet or uber mode in the review? You can argue semantics all day long here but as far as i've seen "out of box" has always been the testing policy here.

If you want to blame someone, don't blame the reviewer - blame AMD for putting a terrible cooler on the card which creates the uber and quiet mode performance differences. If AMD had created a shroud, that you know, didn't suck - we wouldn't have this issue. We could have quiet mode with uber performance. Sadly, AMD wanted to save 3$ on their reference blower and pass the buck to AIB makers. Think about this for a second. If AMD had created a good reference blower that theoretically were as versatile as the Titan shroud - this would be a complete non issue. There would be zero noise complaints, and there would be no issues with performance differences between "quiet" and "uber". EVERYTHING would be quiet mode with uber level performance. Think about this point. This isn't his fault - it is AMD's fault.

 

[GaiaHunter]

Why do you care if he uses quiet or uber mode in the review? You can argue semantics all day long here but as far as i've seen "out of box" has always been the testing policy here.

If you want to blame someone, don't blame the reviewer - blame AMD for putting a terrible cooler on the card which creates the uber and quiet mode performance differences. If AMD had created a shroud, that you know, didn't suck - we wouldn't have this issue. We could have quiet mode with uber performance. Sadly, AMD wanted to save 3$ on their reference blower and pass the buck to AIB makers. Think about this for a second. If AMD had created a good reference blower that theoretically were as versatile as the Titan shroud - this would be a complete non issue. There would be zero noise complaints, and there would be no issues with performance differences between "quiet" and "uber". EVERYTHING would be quiet mode with uber level performance. Think about this point. This isn't his fault - it is AMD's fault.

I'm pretty sure the 6990 Ausum mode was heavily criticized because it was a out of the box setting that wasn't under warranty.

From the 6990 AT review

From a usability standpoint, the benefit of using the BIOS switch for this is that it’s much more consistent across vendors and it doesn’t require any software interaction. Just flip the switch and you’re done.

Apparently that policy of just flip the switch and you are done has changed.

 

[Janooo]

As he said...out of the box. Once you plug the card in and install drivers how does it perform.

I don't understand the problem here. Are you that upset that AMD doesn't have the fastest card after a couple weeks to market that you need to start a witch hunt and want heads to roll because you disagree with a review and think that "uber mode" is somehow going to be the savior for the 290?

... and every OC chart is out of the box.
Right?
AT is losing credibility this way.

 

[cmdrdredd]

... and every OC chart is out of the box.
Right?
AT is losing credibility this way.

They show overclocking separately and they are trying to recreate an overclock that would be expected out of the box without BIOS messing and switch flipping.

You seem upset because they aren't showing the 780ti in a bad light. It's a good card and faster than the competition as many had predicted. Worth the price is another matter. However, there is no bias going on here. I think Ryan is doing a great job overall with these reviews. If all testing was done in uber mode you'd have some guy whining all day about how come his card is slower than the review said since he has to physically flip the switch and had no clue. Can't win

 

[Janooo]

They show overclocking separately and they are trying to recreate an overclock that would be expected out of the box without BIOS messing and switch flipping.

You seem upset because they aren't showing the 780ti in a bad light. It's a good card and faster than the competition as many had predicted. Worth the price is another matter. However, there is no bias going on here. I think Ryan is doing a great job overall with these reviews. If all testing was done in uber mode you'd have some guy whining all day about how come his card is slower than the review said since he has to physically flip the switch and had no clue. Can't win

You are insulting buyers of the high end cards.
If somebody knows how to check FPS it's most likely that person would know where is the switch.

 

[TerryMathews]

As he said...out of the box. Once you plug the card in and install drivers how does it perform.

I don't understand the problem here. Are you that upset that AMD doesn't have the fastest card after a couple weeks to market that you need to start a witch hunt and want heads to roll because you disagree with a review and think that "uber mode" is somehow going to be the savior for the 290?

But we aren't talking about modifying the card or running Afterburner. This is a standard configuration switch, one that documentation explains the purpose of and warranty covers.

I don't care which card is faster. I am just saying its a ridiculous policy to intentionally not test a piece of hardware in a supported configuration.

Under these testing standards, the review of the Abit BP6 would have been laughable.