Nvidia accusses ATI/AMD of cheating - Benchmarks

[Lonyo]

What else can nVidia do? Simply accept using what was called for by the application and their competitors don't? Don't play by the same rules and don't bench apples-to-apples? I would wager that this less precision wasn't in the AMD reviewers guide to reviewers and AMD knew they were benching with less precision to nVidia's full precision. Kinda disingenuous.

Um, any website is free to disable Catalyst AI, which would also disable these optomisations.
I would assume that AMD's reviewer guide would suggest websites benchmark with Cat AI on, but at the end of the day, it's up to the website whether they keep these optimisations enabled or not.

AMD can only suggest.
This optimisation is not hidden, it's easy to disable, so it depends more on a website's discretion than anything AMD does.
If a website decides to leave Cat AI enabled, then they themselves are giving AMD the advantage, but like I said, it's up to the website.

If you have a problem with benchmark practices from websites, either complain to the website or don't visit it. It's up to the website to do Apples-to-Apples testing by disabling Cat AI if they see fit. The option is there and easily available with a single simple click.
If you have a problem with optimisations which can be disabled by anyone who wants to, then you're a little weird.

 

[happy medium]

Wait, what?? Do you really think AMD was so worried about the performance comparisons in a six-year old game that they sneakily threw some optimizations in there? 200fps just wasn't enough?

I honestly don't think Far Cry is really on anyone's radar anymore (even though it still is an awesome game). Are there any other games where the graphics took a hit?

I think you missed post #68

 

[Grooveriding]

Got to give the OP credit, finely executed post to incite trolls and open the door to another 20 page pile of crap. This is as useless a subject as the Mafia 2 physx on/off thread.

With some opinion piece from some site that reviews energy drinks as much as they do hardware

Keep spinning NV, I can't wait to see the CrapX and BloatVision(tm) threads springing up in the next few months and the fanboys howling about 'features' while AMD blows the doors off NV again for another half year stretch.

 

[VirtualLarry]

I already responded to that earlier.
What nVidia says is a fact: the dynamic range of R16G16B16A16 is the same as R11G11B10.
However, the *precision* is not.
Aside from that, nVidia's document was to advise the developer to make the best possible decision for their situation (in some cases it IS an acceptable alternative, but in others it is not). You cannot compare that with drivers that 'know better' than what the developers specifically request.

So AMD is being very misleading here.

According to the article, AMD does this for DX9 titles, which do not have access to the R11G11B10 surface formats via API call. Much like MSAA in certain cases, it looks like the drivers have to work around the limitations of DX9 API.

 

[badb0y]

Really? So now we argue about optional optimizations which can be turned off? Gimme a break

/lock please

 

[Scali]

According to the article, AMD does this for DX9 titles, which do not have access to the R11G11B10 surface formats via API call. Much like MSAA in certain cases, it looks like the drivers have to work around the limitations of DX9 API.

Well, not entirely...
Firstly, it IS possible to expose custom texture formats to Direct3D, and both nVidia and ATi have made heavy use of that through the years:
http://aras-p.info/texts/D3D9GPUHacks.html
So although R11G11B10 is not a standard DX9 format, it could easily be exposed to developers through regular means (and probably is, just find the right FOURCC code for it).

Secondly, where MSAA *improves* image quality above what is possible through DX9, this hack *degrades* image quality from what it normally would be.

 

[OCGuy]

Got to give the OP credit, finely executed post to incite trolls and open the door to another 20 page pile of crap. This is as useless a subject as the Mafia 2 physx on/off thread.

I'm with ya Groove! I'm with y.....

Keep spinning NV, I can't wait to see the CrapX and BloatVision(tm) threads springing up in the next few months and the fanboys howling about 'features' while AMD blows the doors off NV again for another half year stretch.

.....then you added to the pile of crap



Wasn't this old news, or did this article add something new?? (at work, no time to read the whole thing)

 

[OCGuy]

Thats the mobs opinion right now.
If this story swapped the company names , there would be 4x as many posts crucifying Nvidia, how they s***, always have, they cheat, all the workers are evil minions of a evil overlord, lol
Oh and lest I forget......about a dozen of this anecdote.
Thats why I won't ever buy company A , again !

Yes, but you have to understand that about this message board's slant. You will learn to ignore half the stuff that is posted in these flame threads.

Welcome to the forums

 

[brybir]

No.
They supplied a special tool to enable some hidden functionality in the 260 drivers to enable the same hack that AMD applied (silent R16G16B16A16->R11G11B10 downgrading).
With normal use, nVidia's drivers always just use R16G16B16A16 as requested.

Ofcourse nVidia may do some *other* things for certain games (shader replacement etc), and they may have done things like this in the past. But this specific example, no, they are not doing that.

Hi Scali,

It is my understanding that the variety of "optimizations" that go on at the driver level are in response to the various implementations, strengths and weaknesses of the various architectures. Since I think you are a software guy (judging from your posts), and since I am a hardware guy, it seems to me anyways, that when we talk about these optimizations, it is really just an attempt to leverage strengths and work around weaknesses.

For example, the card gets an API call for some pixels at some location with its various pieces of information. So, [hypothetically] the Nvidia driver sees this request for the pixel and says "hey wait, that pixel is going through the pipeline in way 1, but if it goes through in way 1, it is going to cause a pipeline stall because of a cache overflow (or whatever) and add latency to the rest of the pipeline. So, I am going to take that pixel and push it through the pipeline in way 2, such that some of the memory information is stripped to avoid the cache overflow and therefore help keep the pipeline filled". Take that same scenario on a [hypothetical] AMD card, and perhaps its drivers look at that same situation and say "no issues here, lets push it through the pipeline as requested by the API call, as the cache is sufficient to handle it without causing a pipeline stall". Repeat over and over.

My take is that this scenario above is essentially what happens constantly at the driver level, and in fact MUST occur given the significant difference in hardware being requested to execute a common codepath in order to provide consistent results across platforms, APIs, games etc. and so the real question becomes one of degree....rather than if it is occurring at all?

 

[Scali]

Hi Scali,

It is my understanding that the variety of "optimizations" that go on at the driver level are in response to the various implementations, strengths and weaknesses of the various architectures. Since I think you are a software guy (judging from your posts), and since I am a hardware guy, it seems to me anyways, that when we talk about these optimizations, it is really just an attempt to leverage strengths and work around weaknesses.

For example, the card gets an API call for some pixels at some location with its various pieces of information. So, [hypothetically] the Nvidia driver sees this request for the pixel and says "hey wait, that pixel is going through the pipeline in way 1, but if it goes through in way 1, it is going to cause a pipeline stall because of a cache overflow (or whatever) and add latency to the rest of the pipeline. So, I am going to take that pixel and push it through the pipeline in way 2, such that some of the memory information is stripped to avoid the cache overflow and therefore help keep the pipeline filled". Take that same scenario on a [hypothetical] AMD card, and perhaps its drivers look at that same situation and say "no issues here, lets push it through the pipeline as requested by the API call, as the cache is sufficient to handle it without causing a pipeline stall". Repeat over and over.

My take is that this scenario above is essentially what happens constantly at the driver level, and in fact MUST occur given the significant difference in hardware being requested to execute a common codepath in order to provide consistent results across platforms, APIs, games etc.

Well yes. What you are describing is pretty much what an optimizing compiler does.
Optimizing compilers are used for compiling all shaders (just like optimizing compilers are used for compiling x86 code, and they can often be targeted to a specific family of CPUs, eg Pentium 4, Core2 etc).
There's nothing wrong with that, since the source code merely describes WHAT needs to be done, not HOW it needs to be done. And the languages have pretty detailed specs, which determine the boundaries within which these compilers need to work (with integer math, you always get an EXACT result... with floating point, the order of operations, and even whether or not a value is stored in memory in between, can introduce slight rounding errors).

However, what we are talking about here, is of a completely different order.
The developer requests a render target with 16 bit floating point precision, and 4 components (RGBA).
Instead, what he gets is a render target with 3 components (RGB), where R and G are 11 bit floating point precision, and B is 10 bit. So one component is lost altogether, and the other 3 have significantly lower precision.
This means that the results will be truncated to the lower precision. This is NOT according to specification. You may 'get away' with it in certain games (but apparently not Far Cry, so the developer would not have chosen this format even if it were a standard DX9 format), but it's not what the driver is supposed to be doing. It's operating out-of-spec, and it yields unexpected results. Technically these are bugs/errors (which is why AI is disabled by default... they cannot pass WHQL testing when it is enabled).

 

[Keysplayr]

All this isn't really that big of a deal. I would like to see IQ comparisons in the future from review sites though. Both with and without tweaks from both sides if doable.

 

[GaiaHunter]

As long as the IQ is similar and the performance is better I keep Cat AI on, generally on Advanced even.

As long both companies offer the chance to turn it off, they are free to get whatever optimization in the drivers, imho.

If there is degration of IQ the reviewers can simply say so.

 

[RussianSensation]

performance does not equal better. If you all of a sudden wish to ignore everything that is not related to how fast you push a pixel, the HD5970 is still the highest performan piece out there.

Really, so you would recommend someone a 5670 for gaming over an HD4890? 5670 has DX11 (4890 only has DX10), 5670 has lower idle and power consumption, 5670 is quieter too. I guess you forgot the part where 4890 is 70%+ faster???

Enthusiasts buy parts for performance, first and foremost. What is the point of discrete graphics in the first place? If I didn't care about price/performance, I'd either get a passively cooled videocard with 0 noise, or the extreme being a 5970.

5970 is the highest performing card, when did I ever say it wasn't?

Let's summarize the facts shall we:

$105 GTS 450 that's faster than similarly priced 5750 - win.
$140 GTX460 768mb that's 30%+ faster than a $115-120 5770 1GB - win.
$200-230 GTX460 1GB that comes factory overclocked at 800+ and beats a $240 5850 - slight win.
$270 GTX470 1Gb that's within 10% of $360 HD5870 - win.
$400 GTX460 1GB SLI that's nearly as fast as HD5870 CF setup for $300 less - win.

At this point, ATI only has 1 card with no competition - 5970.

I am not a fanboy as I have recommended ATI cards since I joined the forums, and HD5000 series for 6+ months. At this time, it's nearly impossible to recommend anything but a 5770 and maybe a 5850. Of course, with HD6000, everything is about to reverse. :thumbsup:

I guess you won't ever recommend a 5870 designed for overclockers in mind because if its high power consumption??? Or maybe your judgment is clouded by the extra 150 Watts of power a 1000mhz 5850 consumes over a stock 5850? Well guess what, people who buy high end cards want performance and even overclock to get more of it. That's why we are enthusiasts, not Intel HD supporters.

BTW, $280 5850 overclocked to 5870 speeds was the best price/performance card in the last 6+ months. I don't remember many talking about the increased power consumption levels of OCed 5850s?? Do you know why? Because its price/performance delivered in spades!

As far as the optimizations are concerned, it would be more detrimental if image quality was reduced in modern games (as has been mentioned, probably the vast majority of users with modern cards aren't playing Far Cry 1 at this point). However, since optimizations can be disabled in the CCC, I don't see it as such a big deal. Reviewers should just test the cards without Catalyst A.I. on and with NV cards always set to maximum quality to remove unfair optimizations out of the equation. The differences in the article seem to have barely affected the image quality. So for a 17% performance improvement, I would probably leave the A.I. on anyways.

 

[Tempered81]

It's up to the website to do Apples-to-Apples testing by disabling Cat AI if they see fit. The option is there and easily available with a single simple click.

apples to apples testing would be with Cat AI on standard or maximum. Nvidia has numerous game specific optimizations in their drivers which do not have the option to toggle on/off. You can be sure that they are ALL ON at ALL times in the newest drivers. (this is aside from the profile options) Ati just gives you the option to disable any game optimzations all together.

 

[Flipped Gazelle]

Really, so you would recommend someone a 5670 for gaming over an HD4890? 5670 has DX11 (4890 only has DX10), 5670 has lower idle and power consumption, 5670 is quieter too. I guess you forgot the part where 4890 is 70%+ faster???

Enthusiasts buy parts for performance, first and foremost. What is the point of discrete graphics in the first place? If I didn't care about price/performance, I'd either get a passively cooled videocard with 0 noise, or the extreme being 5970.

5970 is the highest performing card, when did I ever say it wasn't?

Let's summarize the facts shall we:

$105 GTS 450 that's faster than similarly priced 5750 - win.
$140 GTX460 768mb that's 30%+ faster than a $115-120 5770 1GB - win.
$200-230 GTX460 1GB that come factory overclocked at 800+ and beat a $250 5850 - win.
$270 GTX470 1Gb that's within 10% of $360 HD5870 - win.
$400 GTX460 1GB SLI that's nearly as fast as HD5870 CF setup for $300 less - win.

At this point, ATI only 1 card with no competition - 5970.

I am not a fanboy as I have recommended ATI cards since I joined the forums, and HD5000 series for 6+ months. At this time, it's nearly impossible to recommend anything but a 5770 and maybe a 5850. Of course, with HD6000, everything is about to reverse. :thumbsup:

The problem w/the 1st 2 is that those prices are only in effect occasionally. Right now the cheapest I'm seeing a GTX 460 go for is $170.

5850's overclock also, and frequently beats the GTX 460, according to AT's own benchmarks.

 

[SirPauly]

apples to apples testing would be with Cat AI on standard or maximum. Nvidia has numerous game specific optimizations in their drivers which do not have the option to toggle on/off. You can be sure that they are ALL ON at ALL times in the newest drivers. (this is aside from the profile options) Ati just gives you the option to disable any game optimzations all together.

Apples to apples testing, sadly has to include a hack for reviewers to lower precision for nVidia GPU's.

 

[ZimZum]

Ofcourse nVidia may do some *other* things for certain games (shader replacement etc), and they may have done things like this in the past.

The irony is that NV is actually pointing out that ATI easily allows you to disable all driver optimizations. While NV does not.

 

[RussianSensation]

5850's overclock also, and frequently beats the GTX 460, according to AT's own benchmarks.

Why shouldn't a 5850 beat a 460? It costs $70 more than the 768mb version and $40-50 more than the 1GB version.

The irony is that NV is actually pointing out that ATI easily allows you to disable all driver optimizations. While NV does not.

So true.

 

[Flipped Gazelle]

Why shouldn't a 5850 beat a 460? It costs $70 more than the 768mb version and $40-50 more than the 1GB version.

Your own statement was that a factory OC'd GTX 460 beats a (presumably standard-clocked) 5850, and this is not necessarily true.

If you posit that the GTX 460 is a better value, then yeah, I'm on board with that.

 

[Outrage]

Apples to apples testing, sadly has to include a hack for reviewers to lower precision for nVidia GPU's.

Why ? just turn of cat ai and ati will use the same precision as nvidia. Problem is that you can't do an apples to apples anyway since nvidia dont offer the user an option to turn off there optimisations.

 

[SirPauly]

Why ? just turn of cat ai and ati will use the same precision as nvidia. Problem is that you can't do an apples to apples anyway since nvidia dont offer the user an option to turn off there optimisations.

NOTE: If you wish to test with Need for Speed: Shift or Dawn of War 2, we have enabled support for FP16 demotion - similar to AMD - in R260 drivers for these games. By default, FP16 demotion is off, but it can be toggled on/off with the AMDDemotionHack_OFF.exe and AMDDemotionHack_ON.exe files which can be found on the Press FTP.
For apples-to-apples comparisons with our hardware versus AMD, we ask that you run the AMDDemotionHack_ON.exe when performing your graphics testing with these games. In our own internal testing, speedups of up to 12% can be seen with our hardware with FP16 demotion enabled.

Sure you can, nVidia created an exe. to lower precision to have apples-to-apples. What else can they do? They don't do these kinds of optimizations by default. Maybe nVidia learned their lessons from the past.

nVidia allows flexibility with filtering from turning off anisotropy and trilinear optimizations and having a high quality setting, and allowing for a lod clamp.

 

[brybir]

Well yes. What you are describing is pretty much what an optimizing compiler does.
Optimizing compilers are used for compiling all shaders (just like optimizing compilers are used for compiling x86 code, and they can often be targeted to a specific family of CPUs, eg Pentium 4, Core2 etc).
There's nothing wrong with that, since the source code merely describes WHAT needs to be done, not HOW it needs to be done. And the languages have pretty detailed specs, which determine the boundaries within which these compilers need to work (with integer math, you always get an EXACT result... with floating point, the order of operations, and even whether or not a value is stored in memory in between, can introduce slight rounding errors).

However, what we are talking about here, is of a completely different order.
The developer requests a render target with 16 bit floating point precision, and 4 components (RGBA).
Instead, what he gets is a render target with 3 components (RGB), where R and G are 11 bit floating point precision, and B is 10 bit. So one component is lost altogether, and the other 3 have significantly lower precision.
This means that the results will be truncated to the lower precision. This is NOT according to specification. You may 'get away' with it in certain games (but apparently not Far Cry, so the developer would not have chosen this format even if it were a standard DX9 format), but it's not what the driver is supposed to be doing. It's operating out-of-spec, and it yields unexpected results. Technically these are bugs/errors (which is why AI is disabled by default... they cannot pass WHQL testing when it is enabled).

Okay I see what your saying now about this. Its one thing for a driver to be monitoring code to ensure that things work they way they need to work, but entirely different when it is overriding an otherwise functional execution that alters what the developers are intending.

So in my example before, it is the case then that the software is requesting 16bit precision, and the hardware can execute that 16bit precision just fine, but the driver is then dropping the precision just to inflate the performance numbers at a sacrifice of quality that neither the developer or the customer is necessarily aware of.

 

[brybir]

Sure you can, nVidia created an exe. to lower precision to have apples-to-apples. What else can they do? They don't do these kinds of optimizations by default. Maybe nVidia learned their lessons from the past.

nVidia allows flexibility with filtering from turning off anisotropy and trilinear optimizations and having a high quality setting, and allowing for a lod clamp.

I think that sites should just test like us average joes get to play. They download the latest driver and go to town.

If that means that in some games the AMD driver is doing some lower precision things, and that is the experience I would have as well, then that is what should be tested. Let *ME* decide if the image quality is worth a hit or not. Do performance and then do some side by side image comparisons.

Let the results speak for themselves.

 

[SirPauly]

Okay I see what your saying now about this. Its one thing for a driver to be monitoring code to ensure that things work they way they need to work, but entirely different when it is overriding an otherwise functional execution that alters what the developers are intending.

So in my example before, it is the case then that the software is requesting 16bit precision, and the hardware can execute that 16bit precision just fine, but the driver is then dropping the precision just to inflate the performance numbers at a sacrifice of quality that neither the developer or the customer is necessarily aware of.

That's how I see it and a bad precedent and a slippery slope.

 

[ShadowOfMyself]

Cant believe the usual 3 or so fanboys keep defending Nvidia and somehow think everyone else posting in this forum is wrong, Lol

Do you really think theres some conspiracy going on? Because its not just AT forum... Read the comments on the article itself, everyone agrees with the point we (as in the majority) are making, Nvidia is the stupid one for making a fuss about something that has been known (and done) for years

But the best part is this is just based on ancient DX9 games, which already get ridiculous fps with any recent card, it really is pathetic