Benchmarkers need to stop using biased games when testing GPUs.

[Lonyo]

There ya go.

See now that's just obvious trolling since you changed one of his statements to fit your replies. Be less obvious sometime?


Back to the original point:
If we take AT's benchmarks, some make sense, some don't.

Crysis: Warhead. Obvious, the most demanding game arguably.
5 DX11 games (Metro 2033, Civ 5, Stalker, Dirt 2, BF BC2) as well as 3 of them being popular games especially in their genres.
An Unreal Engine 3 game (ME2), being the most popular game engine.
An OpenGL game (Wolfenstein). The most recent OpenGL game on the major OGL engine (AFAIK?)

Then you have some which are outliers.
Battleforge DX10. As a DX11 test, it might make sense, as a DX11 test it doesn't really make sense.
HAWX. Again, doesn't make much sense, it has already been replaced by a DX11 sequel, HAWX2, which would make more sense for latest and greatest benchmarking. Both HAWX 1 and 2 seem to favour NV so in the grand scheme of things it doesn't make much difference. Possibly just there to represent another genre?


Xbitlabs has:
8 DX11 games
Crysis
Res Evil 5 (representing 3rd person shooter)
Far Cry (possibly not so useful these days, but still quite demanding)
Borderlands (Unreal Engine 3)
GTA 4 (big game, getting old now though)
Left 4 Dead 2 (Source representative)
Just Cause 2 (recent game, quite demanding)
Starcraft 2 (RTS, big game)
Mafia 2 ('big' game, quite demanding)

What other games would people like to see instead of FC2, Mafia 2 and Just Cause 2, which are the Xbitlabs outliers? (Bearing in mind DX11 games are pretty much all being tested).
Oh, and they are all NV games, and in Just Cause 2 and Mafia 2, AMD wins, and in FC2, AMD wins at 2560x1600 but loses at 1920x1200. So much for that bias.

I mean, if you are going to complain or talk about bias, you have to suggest what they should do instead.
Most sites try to cover a spectrum of games which is representative of the most demanding new games, and/or various different game engines, and feature sets (e.g. DX11). It's a sensible approach.
Hard for them to completely ignore games suggested by companies. If AMD says "bench Dirt 2/F1 2010" they are going to say "OK, we'll bench Dirt 2/F1 2010 because it's one of the handful of DX11 games on the market, and if we exclude it we're down a high percentage of DX11 games out there"

 

[TheMeanestGuest]

Sometimes I wonder if Wreckage actually believes what he posts.

Anyways, I'm going to agree with the general sentiment and say that its important to include games that might consistently perform better for one side or the other in reviews. I mean, that's the whole point. So we know where a card stands across a wide spectrum of games, and not just the ones that make that particular card look good.

 

[Teizo]

I personally believe that with properly coded drivers, AMD cards might even start beating the nV card Just my two cents...

Does Nvidia have AMD's driver team under some voodoo spell or something? Why can't AMD properly code drivers for their own hardware, as you say?

The differences between cards is very small (except in Hawx2 and LP), and not really noticeable from a gameplay point of view anyway, so I don't understand what the fuss is all about. Does the card perform well enough for you? Are you satisfied with it? Why did you buy it...because it was 2 fps faster than the other card compared to it? Or, the price was right and the performance was acceptable?

 

[Gloomy]

Does Nvidia have AMD's driver team under some voodoo spell or something? Why can't AMD properly code drivers for their own hardware, as you say?

I think he meant to say that the new architecture isn't optimized yet and should get better with driver updates. I'm inclined to agree, but I don't think it'll be anything special. Maybe a few % here and there, mostly minimums, but nothing earth-shattering.

The differences between cards is very small (except in Hawx2 and LP), and not really noticeable from a gameplay point of view anyway, so I don't understand what the fuss is all about. Does the card perform well enough for you? Are you satisfied with it? Why did you buy it...because it was 2 fps faster than the other card compared to it? Or?

I agree.

 

[Teizo]

I think he meant to say that the new architecture isn't optimized yet and should get better with driver updates. I'm inclined to agree, but I don't think it'll be anything special. Maybe a few % here and there, mostly minimums, but nothing earth-shattering.

It would be nice to see, but I doubt like you it's going to change things dramatically. The last time I bought into that story was when I purchased a 6800 Ultra when SM 3.0 was introduced to the market. Performance improvements were very minimal with respective driver increases, and nothing to really showed up much in gameplay unless I was using Fraps to track my fps.

Anyway, AMD has had the entire time this hardware was being designed to work on their drivers to go along with it. It's not like they designed a driver after the hardware was finished and hoped that it would work. If so, that might be part of the problem.

 

[Grooveriding]

i roll my eyes when i see biased and FUD-filled posts like these and everyone should stop reading them.
:thumbsdown:

A historically Nvidia game?
- it is on AMD's benching list; they bench it.

Ancient?
- it is two years old

A$100 card can run it?
- try 2560x1600 with 8xAA

A turd?
- it has a metacritic score of 85

Review sites are biased that use it?
- HardOCP praised the benchmark and they claim they had input into it's creation

Enough FUD about Far Cry 2

Sorry, I didn't mean to rub you the wrong way. The game is a turd.

 

[Grooveriding]

So if Crysis 2 gets benchmarked in future video card reviews then they are merely trying to help nVidia?

Expect to see 'extreme' tessellation in Crysis 2, due to NV involvement, but the game 100% does not include physx.

I asked the team themselves and received an answer on their official forums stating the game will use Crytek's own physics engine.

They're too good at making cutting edge games to pollute the game with garbage like physx that would just detract from the quality of the game.

 

[Piotrsama]

A turd?
- it has a metacritic score of 85

And a user score of:

5.5 out of 10
based on 652 Ratings

 

[shangshang]

460 1GB is not faster. It is cheaper. And there is a reason to the high HD 6850 price be right: You don't lower your price if you are selling very well.
470 is not on pair with the HD 5870, you may distort in any way you want, in real games the Radeon is faster.
HD 6950 is faster than the HD 5870, therefore, faster than the GTX 470, although it is more expensive.
HD 6970, well, here you are right, they are about the same speed, but the Radeon is more expensive :|

off topic, I find it very strange that a person would join a very popular tech site since 2009 and only has 23 posts (as of this writing) and suddenly he starting to post with the intent of following Wreckage (a controversial NV supporter around here).
carry on

 

[Wreckage]

off topic, I find it very strange that a person would join a very popular tech site since 2009 and only has 23 posts (as of this writing) and suddenly he starting to post with the intent of following Wreckage (a controversial NV supporter around here).
carry on

It happens more often than you would think. Cult of personality I guess.


Back on topic. So what is the next big game that should be benchmarked?

 

[SirPauly]

I mean, if you are going to complain or talk about bias, you have to suggest what they should do instead.

The key is how does each individual gauge their hardware when purchasing? What is important for them? Instead of attacking each other or offer bias finger pointing and agenda diatribes -- simply offer what is important and what they would like to see or what is important.

For me it is about targeting the capabilities of the GPU's in worse case areas. Heck, desire to know how the GPU's perform at their worse -- not their best. What this does is offer a gauge on potential bottlenecks the hardware may offer. It's not about making hardware looking good to sell but torture tests to investigate hardware and companies claims.

By doing this, was one of the first posters to discuss micro-stuttering before it was named, trying to figure out why dual GPU's differed from single GPU's. The heat I did get but stayed true on my own findings. Or, discussing the need for alpha test flexibility considering in worse case areas -- frame-rate would drop like rocks and yet rarely tested and investigated.

What I would like to see is more investigations.

 

[digitaldurandal]

There ya go.

Wreckage might be the first person on my ignore list, I think he might be a troll, or Rollo II. I am not trying to be rude - but I have a hard time understanding how his brain could possible think some of the things he says.

Dirt2 graphics worse than GT5.

460 1gb better than 6850.

There is so much more...


This manner of public pontification regarding your negative opinion of one of fellow forum colleagues is not acceptable.

You are making this needlessly personal. Please reconsider your approach. Restrict your posting to addressing the contents of the posts without making criticizing character assessments.

AnandTech Forum Guidelines

We want to give all our members as much freedom as possible while maintaining an environment that encourages productive discussion. It is our desire to encourage our members to share their knowledge and experiences in order to benefit the rest of the community, while also providing a place for people to come and just hang out.

We also intend to encourage respect and responsibility among members in order to maintain order and civility. Our social forums will have a relaxed atmosphere, but other forums will be expected to remain on-topic and posts should be helpful, relevant and professional.

We ask for respect and common decency towards your fellow forum members.

Your moderation staff joins your fellow forum members in asking you to please help us in keeping this forum civil, respectful, and open-minded towards ALL of its members.

If you have nothing good to say regarding any one of your fellow members then please refrain from publicly posting it. If you simply must express your negativity then do so by pm.

If you feel you have a valid concern or complaint regarding a post or poster then you are expected to report the post and restrict your communication of this concern to the comment box therein.

Moderator Idontcare

 

[Lonyo]

It happens more often than you would think. Cult of personality I guess.


Back on topic. So what is the next big game that should be benchmarked?

Crysis 2 is an obvious one, when it eventually arrives.
Something using the next gen iDTech (e.g. Rage).
Maybe DoW 2 Retribution.
Maybe Witcher 2 if it's demanding.
Test Drive Unlimited 2 possibly.
Deus Ex 3 if it's not Unreal Engine 3 or something else popular.
Shogun II if it's got a new/updated engine which is demanding.
Guild Wars 2?
Max Payne 3?

I assume Dragon Age 2, ME 3, Portal 2 and many other sequels will be on the same engines as their predecessors, so they won't be much use.
http://adrianwerner.wordpress.com/2011-games/

 

[RussianSensation]

I am not trying to be rude - but I have a hard time understanding how his brain could possible think some of the things he says.

460 1gb better than 6850.

Well it only takes 720mhz GTX460 to = HD6850. Considering you can get an 810mhz Galaxy GTX460 for $185 on Newegg, you get a card that's faster, cooler and quieter than the 6850. Alternatively, there are some 460s for $150-160. Once you consider overclocking, this makes the 460 better value regardless how you look at it vs. the 6850.

Also, car models in GT5 do look way better than they do in Dirt 2. In replay mode, the graphics are also much better than in Dirt 2.

So neither of those statements Wreckage made are controversial.

 

[MrK6]

Well it only takes 720mhz GTX460 to = HD6850. Considering you can get an 810mhz Galaxy GTX460 for $185 on Newegg, you get a card that's faster, cooler and quieter than the 6850. Alternatively, there are some 460s for $150-160. Once you consider overclocking, this makes the 460 better value regardless how you look at it vs. the 6850.

Also, car models in GT5 do look way better than they do in Dirt 2. In replay mode, the graphics are also much better than in Dirt 2.

So neither of those statements Wreckage made are controversial.

Buying pre-overclocked cards that offer nothing else is a poor value no matter how you look at it - I still have no idea why they're even recommended on an enthusiast's forum. Also, you do realize that a 6850 overclocks just as well, if not better, than a GTX 460, right? The GTX 460 1GB is cheaper than the 6850 for a reason - it's an inferior card.

In regards to the OP's comments; why would you want sites to include less information in their reviews? People buy video cards to play games, and want to see said games that they'll be playing in reviews so they can make an informed purchasing decision. If NVIDIA helped develop a game and therefore it plays much better on their hardware - that's important to know! Also note that just because NVIDIA helped develop it and got their name put in the intro credits doesn't mean that their hardware will run better on it - Crysis is a good example.

The problem is when people, the reviewer's themselves included, try to make/justify generalizations based on faulty logic either because they have no clue or are deliberately being misleading. There are several review sites I just skim over and take with a grain of salt because I've seen countless errors and even downright stupidity in their work ethic/presentation/scientific process. Even in review sites that I trust, I skim through conclusions because I've already come to my own after reading the charts and data. The best thing to do is review the data on performance and features and buy the product that works best for you.

Also, Far Cry 2 was an mediocre game at best; there's a reason it has a 5.5 user rating on Metacritic. Same as Call of Duty: MW2 - makes you wonder how much money is going into these game reviewer's pockets, and again is another example of why you should use your head when you read reviews.

 

[Dribble]

There is no such thing as a "bias" game, there are just games, we buy graphics cards to play them.

If one company can't manage to make a card that plays a game well then they loose sales. They have to fix it not cry about it. Most of us aren't interested in why their card lost - how it's not that companies fault, how nasty the opposition is, or any of that, we just want to play the games.

Enough of the "victim" mentality - just make it work better then the oppositions and we'll buy the cards, don't make it work and we won't - simples.

 

[WelshBloke]

off topic, I find it very strange that a person would join a very popular tech site since 2009 and only has 23 posts (as of this writing) and suddenly he starting to post with the intent of following Wreckage (a controversial NV supporter around here).
carry on

Dont attack people for their join date or post count. If you can attack their point fine, but we were all new at some point.

And to get sucked in to your argument, Wreckage obviously posts in a provocative manner to get a reaction, he shouldn't complain when he gets one.

 

[SirPauly]

Buying pre-overclocked cards that offer nothing else is a poor value no matter how you look at it - I still have no idea why they're even recommended on an enthusiast's forum. Also, you do realize that a 6850 overclocks just as well, if not better, than a GTX 460, right? The GTX 460 1GB is cheaper than the 6850 for a reason - it's an inferior card.

I don't know about that. Sometimes the OC editions have exotic cooling and considering the modest premiums bring differentiation to the AIB's over their default siblings. I'm all for OC editions from nVidia or AMD's partners.

To place things in perspective -- 369 MSRP OC edition 570 offers a difference in Performance when compared to the 369 MSRP 6970, according to Rage3d's findings:


http://www.rage3d.com/reviews/video/amd_hd6970_hd6950_launch_review/index.php?p=8

 

[MrK6]

I don't know about that. Sometimes the OC editions have exotic cooling and considering the modest premiums bring differentiation to the AIB's over their default siblings. I'm all for OC editions from nVidia or AMD.

Buying pre-overclocked cards that offer nothing else is a poor value no matter how you look at it...

Bolded the important part there. Buying a card for a better cooler is completely understandable. Buying higher clock speeds that you can easily do yourself isn't.

To place things in perspective -- 369 MSRP OC edition 570 offers a difference in Performance when compared to the 369 MSRP 6970, according to Rage3d's findings:

http://www.rage3d.com/reviews/video/amd_hd6970_hd6950_launch_review/index.php?p=8

And buying said GTX 570OC means you wasted $20 you could have spent elsewhere by not spending the 30 seconds it takes to overclock the card yourself.

 

[Wreckage]

Buying higher clock speeds that you can easily do yourself isn't.

A guaranteed overclock that's covered by a warranty sounds good to me. :thumbsup:

 

[SirPauly]

Bolded the important part there. Buying a card for a better cooler is completely understandable. Buying higher clock speeds that you can easily do yourself isn't.

And buying said GTX 570OC means you wasted $20 you could have spent elsewhere by not spending the 30 seconds it takes to overclock the card yourself.

That's not true at all. Because it is guaranteed over-clocking; a different AIB Sku if you will.

And many differ from exotic cooling to AIB PCB's with better components than default, which may allow even higher over-clocking potential with better acoustics and thermals.

 

[SirPauly]

To place this in perspective, this discussion of oc editions was raised about the GTX 460 -- look at the sheer amount of AIB editions that chip has and the differentiation - some up to 25 percent more performance than default, some with exotic cooling as well.

To blanket just default GTX 460 vs HD 6850 is a one-sided discussion; as is trying to dismiss or downplay OC editions.

 

[Lonyo]

Well I guess that's this thread successfully derailed.
Not that the original topic was worthwhile/made any sense at all.

 

[BFG10K]

Sorry, I didn't mean to rub you the wrong way. The game is a turd.

I can’t agree there; it’s one of the best games I’ve ever played. The graphics are amazing, there’s a ton of exploration available (I spent over 60 hours in the SP campaign), and the physics actually add meaningful gameplay elements while being done on the CPU.

 

[themodernlife]

I don't understand why you are so taken up with review sites benching biased games, most bench the better part of a dozen for you to look at. Besides you just spent money on a brand new graphics card, why aren't you playing it? I think integrated mobo graphics work for complaining on message boards, unless they used biased benchmarks for that too...