• We should now be fully online following an overnight outage. Apologies for any inconvenience, we do not expect there to be any further issues.

FutureMark & Nvidia joint statement on 3DMark03; FutureMark tucks its tail between its legs.

Page 3 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

NFS4

No Lifer
Oct 9, 1999
72,636
47
91
Originally posted by: Evan Lieb

I don't get it, 3DMurk lowers performance, how is that a cheat?

See the first post in this thread (or just read the article;)):

http://forums.anandtech.com/messageview.cfm?catid=27&threadid=1065131
Basically renaming the 3DMark executable and running anisotropic filtering with it causes reduced scores as nVidia is reducing the level of anisotropy when it detects 3DMark is running with anisotropic filtering.

More dicussion (even the NVIDIA fans are up in arms;)):

http://www.nvnews.net/vbulletin/showthread.php?s=0d2133747a35a2db73fcf42f08273701&threadid=12944
http://www.rage3d.com/board/showthread.php?s=98459f647a42ce738c4d4cba5fcd1559&threadid=33690732
 

Gstanfor

Banned
Oct 19, 1999
3,307
0
0
Originally posted by: Evan Lieb

I don't get it, 3DMurk lowers performance, how is that a cheat?

Exactly.

It may be beneficial for people to actually compare the two images also. There is no visible IQ reduction that I can make out. Certainly not like when you compare ATi's QUAK cheat with a properly rendered image.

Lets not forget it's ATi who brought this "issue" to light, and this is happening with the patched version of 3dMark2003 - who is to say futuremark didn't lay a trap for nVidia to fall into?
 

NFS4

No Lifer
Oct 9, 1999
72,636
47
91
Originally posted by: Gstanfor
Originally posted by: Evan Lieb

I don't get it, 3DMurk lowers performance, how is that a cheat?

Exactly.

It may be beneficial for people to actually compare the two images also. There is no visible IQ reduction that I can make out. Certainly not like when you compare ATi's QUAK cheat with a properly rendered image.

Lets not forget it's ATi who brought this "issue" to light, and this is happening with the patched version of 3dMark2003 - who is to say futuremark didn't lay a trap for nVidia to fall into?

Did you READ the article?
In a funny twist of fate, I got a tip earlier this week about NVIDIA's Detonator FX drivers. The allegation: if you rename 3DMark03.exe to something else and run the benchmark with anisotropic filtering enabled in the drivers, test scores drop. In other words, NVIDIA appears to be using the same lame technique ATI did way back when: keying on the program's filename in order to trigger benchmark "optimizations." In this case, those optimizations appear to be a lower quality form of texture filtering than the anisotropic filtering method selected in the driver control panel. Many review sites like us benchmark cards with anisotropic filtering and edge antialiasing turned on, so these things do matter.
They are detecting the 3DMark executable in order to trigger a lower level of AF and thus give a higher 3DMark score as the end result. Renaming the executable file should have absolutely NO effect on the performance of any benchmark. But this is the case with NVIDIA's latest drivers.

It's the equivalent of a dealer selling a car labeled as a BMW 330i that has 330i badging on it, but only has a 2.5 liter inline-6 motor for the price of a 330i without telling the buyer the true nature of the purchase.

 

First

Lifer
Jun 3, 2002
10,518
271
136
Ah, I gotcha Brandon.

I'll have to run some AA tests with our 5900 Ultra here to see WTH is going on.
 

Gstanfor

Banned
Oct 19, 1999
3,307
0
0
Sorry NFS4, but I think this is an attempt to stitch nVidia up by futuremark, aided and abetted by ATi.

First of all, only the patched version of 3DMark2003 was tested. Strangely they didn't bother to test the original.

Secondly the IQ is effectively exactly the same between the two shots. In motion there is no way you could tell the two apart.

Thirdly the optimization is not blatent enough. When nVidia does optimize, it optimizes well. I would have expected far more evident image degradation if this was a true cheat.

What I think might possibly be happening is that the scene is rendered at the quality setting, but anisotropic filtering is being selectively applied - in some cases at the quality setting and in some cases at the performance setting. If this is what is happening it is a far more intelligent approach to anisotropic filtering than the approach ATi takes.
 

NFS4

No Lifer
Oct 9, 1999
72,636
47
91
Originally posted by: Gstanfor
Sorry NFS4, but I think this is an attempt to stitch nVidia up by futuremark, aided and abetted by ATi.

First of all, only the patched version of 3DMark2003 was tested. Strangely they didn't bother to test the original.

Secondly the IQ is effectively exactly the same between the two shots. In motion there is no way you could tell the two apart.

Thirdly the optimization is not blatent enough. When nVidia does optimize, it optimizes well. I would have expected far more evident image degradation if this was a true cheat.

What I think might possibly be happening is that the scene is rendered at the quality setting, but anisotropic filtering is being selectively applied - in some cases at the quality setting and in some cases at the performance setting. If this is what is happening it is a far more intelligent approach to anisotropic filtering than the approach ATi takes.

Leave the image quality out for a second and think about it on this level.

3DMark03.exe scores SHOULD BE EQUAL to 3DMurk03.exe no matter what FSAA or AF settings you use.

No ifs ands or buts. Bottom line. End of discussion:beer:
 

Gstanfor

Banned
Oct 19, 1999
3,307
0
0
I know the score drops when the .exe is renamed and that is why I am certain it is a futuremark stitchup.

I will almost guarantee nVidia's drivers will not be found to be the guilty party here.

Don't forget 44.03 was released BEFORE the futuremark patch was, this event happended AFTER futuremark accused nVidia of cheating.
 

Gstanfor

Banned
Oct 19, 1999
3,307
0
0
Looks at Gstanfor's sig...

**Off to bed for me**
Frankly NFS4, I'm amazed you would come out with something like that. I honestly thought you possessed more intelligence.

Never mind, if you want to see me as a nVidiot, I'm fine with that. I have had nothing but trouble free, pleasurable computing experiences with all the nVidia hardware I have ever owned.

nVidia products may not always win the performance race (GF-FX vs R3xx, TNT vs 3dfx, nForce1 vs VIA KT333), but you know, it isn't outright performance that counts. It can be a difficult thing to quantify, but nVidia products "feel smoother" in operation than their competition, and have superior drivers and support backing them up.

Yes, it is a fine signature, still as good as when I first adopted it. I'm quite proud of it, and stand by every word of it, thank you.
 

Agent004

Senior member
Mar 22, 2001
492
0
0
Originally posted by: Gstanfor
I know the score drops when the .exe is renamed and that is why I am certain it is a futuremark stitchup.

Care to explain why renaming would change the score which led you to believe this is a stitch-up?
 

NFS4

No Lifer
Oct 9, 1999
72,636
47
91
Originally posted by: Gstanfor
Looks at Gstanfor's sig...

**Off to bed for me**
Frankly NFS4, I'm amazed you would come out with something like that. I honestly thought you possessed more intelligence.

Never mind, if you want to see me as a nVidiot, I'm fine with that. I have had nothing but trouble free, pleasurable computing experiences with all the nVidia hardware I have ever owned.

nVidia products may not always win the performance race (GF-FX vs R3xx, TNT vs 3dfx, nForce1 vs VIA KT333), but you know, it isn't outright performance that counts. It can be a difficult thing to quantify, but nVidia products "feel smoother" in operation than their competition, and have superior drivers and support backing them up.

Yes, it is a fine signature, still as good as when I first adopted it. I'm quite proud of it, and stand by every word of it, thank you.

If this had been 6 months ago, I'd agree. But now, with NVIDIA's recent AF and FSAA image quality problems (documented by Anand and HardOCP, and recently fixed with the latest drivers), the 5800 Ultra (dustbuster with meager performance which even they made fun of with their recent video), bitching about 3DMark03 b/c their card wasn't shown to be the fastest, pulling out of the 3DMark beta test only to be caught cheating in 3DMark03 with the latest drivers, then pretty much sicking the lawyers on FutureMark, and now this?

Hell, even nV News people are starting to question NVIDIA now. Don't get me wrong, 90% of my graphics cards have been NVIDIA, but their actions recently have been in question.

I provided 3 links to back up the information I posted concerning the 3DMurk debate.

nV News
Rage3D
Beyond3D

You can discount Rage3D if you want since they are ATI based, but nV News is pretty much NVIDIA central. And Beyond3D is much more technically centered than Anandtech General Hardware Forums on most matters. I'd consider them THE place to do if you want deep discussions on drivers and hardware. They get so technical that it makes your head hurt sometimes;) Take it with a grain of salt if you want, but the truth is out there.
 

Gstanfor

Banned
Oct 19, 1999
3,307
0
0
I've already given reasons why.

Basically nVidia isn't stupid enough to be caught out that simply. ATi is/was (remember QUAK?). Guess who probably suggested this wonderful way to get even with nVidia to futuremark in the first place?

The drivers 44.03 predate the futuremark patch.

Check this link out for a much better look at the actual differences between the two shots (tech report don't have a clue when it comes to diff).
 

Agent004

Senior member
Mar 22, 2001
492
0
0
Thanks for the pic confirming there is different filtering method used, depending on the name of the file(or specific reference ). How is this different to ATI's quack? All they did was detect the filename and there it comes the 'optimization'

You still yet to answer why the filename changes the score.
 

Gstanfor

Banned
Oct 19, 1999
3,307
0
0
Why does the score change when the .exe is renamed? IMO because futuremark wanted to imply nVidia was employing driver cheats similar to the QUAK cheat.

Unfortunately futuremark (and probably ATi also) appear to have forgotten that it was nVidia who pointed out to the hardware websites exactly how the QUAK cheat operated in the first place...

And you are trying to tell me that nVidia is then stupid enough to use the exact same cheat method in their own drivers? And the source of this revelation is ATi??? How big a fool do you think I am?
 

Agent004

Senior member
Mar 22, 2001
492
0
0
Originally posted by: Gstanfor
Why does the score change when the .exe is renamed? IMO because futuremark wanted to imply nVidia was employing driver cheats similar to the QUAK cheat.

Unfortunately futuremark (and probably ATi also) appear to have forgotten that it was nVidia who pointed out to the hardware websites exactly how the QUAK cheat operated in the first place...

And you are trying to tell me that nVidia is then stupid enough to use the exact same cheat method in their own drivers? And the source of this revelation is ATi??? How big a fool do you think I am?


But how does it change? You have to ask yourself. When the filename is changed, 3dmarks' internal codes does not change. The output would be the same regardless of the filename when the file is executed (since all the code does is detects nVidia card/driver but those two stayed the same through the tests. Any nVidia penalties would be constant for those) . Hence no changes should occur.

Further, your pic does indeed showing a different, does it not? If the driver does render as instructed without 'specific reference', the output will be identical (Don't believe me? Run 3dmark a few times without any changes at all).

Which are the exact same steps taken in ATI's quacks...

I also believe it's not right for me to judge how much of a fool is someone, hence I won't be answering that.
 

Megatomic

Lifer
Nov 9, 2000
20,127
6
81
Originally posted by: NFS4
Looks at Gstanfor's sig...

**Off to bed for me**
Looks at NFS4's rig...

Yep, the pot calling the kettle black.
rolleye.gif