Futuremark confirms NVIDIA "cheating" with latest driver...releases audit report

[Mem]

Did anyone actually read the PDF? They expalin exactly HOW nVidia cheated. There is no doubt

I`m still not 100% sure.

The ones who were getting "screwed over" are us, the consumer, not nVidia.

Really?Bottom line is most consumers including myself buy video cards for gaming not 3Dmark03 and 99% of them just want their games to work fine,besides you`ve the choice of ATi and other companies ,remember games are the important thing not 3DMark03.

 

[SexyK]

I did read that, thanks. The last three of those eight "cheats" actaully make the score go down. That's a little odd in my book. I just don't see how you can indict them for achieving higher performace by including "alternative more efficient" shaders that produce "a similar looking rendering" (again, a bit vague for my tastes). If nVidia can optimize FM's program (which a lot of major players in the computer world have already derided as a truly bad DX9 benchmark) by developing more efficient shader routines and delivering them to the user, then what's the problem? FM told nVidia they had to pay an exorbinant fee to get access to the program, nVidia thought the price was too high, so they looked at the program on their own and made it as efficient as possible on their cards.

 

[oldfart]

I am one of the least fans of 3Dmark and dont even have it installed. I could also care less about the scores. I dont care that it was 3DMark that they cheated on. What bothers me is they deliberately cheated. Period. This was to intentionally inflate scores to defraud consumers into thinking they are buying a better product than they are. These little graphs will be on the back of the box. Not everyone is as up to speed on this stuff as we are.

Who's to say what other benchmarks weren't "optimized".

If the info in that PDF is not plain enough, I don't know what else to say. They cheated. No doubt about it.

 

[SexyK]

Originally posted by: oldfart
I am one of the least fans of 3Dmark and dont even have it installed. I could also care less about the scores. I dont care that it was 3DMark that they cheated on. What bothers me is they deliberately cheated. Period. This was to intentionally inflate scores to defraud consumers into thinking they are buying a better product than they are. These little graphs will be on the back of the box. Not everyone is as up to speed on this stuff as we are.

Who's to say what other benchmarks weren't "optimized".

If the info in that PDF is not plain enough, I don't know what else to say. They cheated. No doubt about it.

Of course they wanted the highest score possible. But how could they know the difference between "cheating" and "optimizing" if they were not given access to the details of the benchmark and its operation?

 

[Pariah]

This was to intentionally inflate scores to defraud consumers into thinking they are buying a better product than they are.

Any consumer buying a video card based on 3DMark scores deserves to be deceived. I hope more companies blatantly cheat on these synthetic benchmarks so that we can get rid of them or at least make them more representative of real world gaming.

 

[oldfart]

Nothing else I can say here. They cheated. The PDF doc shows exactly how they did it. Static clipping planes that dont belong in the game dont just happen by accident. They were added to cheat. If you choose to ignore the evidence, thats fine. I don't.

I've owned GF2, GF3, GF4 cards (still have a GF3 in one rig). I like nVidia cards but am really disappointed in them for stooping this low. They have been under the gun since they lost the performance crown to the ATi 9700P. Then the nV30 fiasco. Now they need to cheat? Come on nVidia get with it already.

Here is another point. Check out the in game UT 2K3 screenshots from [ H ]:

FX 5900U 8xAA/8xAF UT2K3 ---37 FPS
9800P 6xAA/16xAF UT2K3 ---95 FPS

FX5900U 8xAA/8xAF UT2K3 ---30 FPS
9800P 6xAA/16xAF UT2K3 ---69 FPS

Why so crummy in a real game? The 9800P has better IQ and WAY better framerates. The 5900U doesn't look so hot in high res Hi IQ gaming.

 

[oldfart]

Originally posted by: Pariah

This was to intentionally inflate scores to defraud consumers into thinking they are buying a better product than they are.

Any consumer buying a video card based on 3DMark scores deserves to be deceived. I hope more companies blatantly cheat on these synthetic benchmarks so that we can get rid of them or at least make them more representative of real world gaming.

So if mommy (who doesn't spend time at Anandtech) spends her hard earned money for a video card for little Johnny for Christmas because the picture on the box said it was the fastest video card on the market, that is OK? She deserves to be cheated?

Just because we all hate 3DMark does not justify cheating. Its wrong, and dishonest. No one has morals anymore?

 

[ElFenix]

Originally posted by: SexyK
I did read that, thanks. The last three of those eight "cheats" actaully make the score go down. That's a little odd in my book. I just don't see how you can indict them for achieving higher performace by including "alternative more efficient" shaders that produce "a similar looking rendering" (again, a bit vague for my tastes). If nVidia can optimize FM's program (which a lot of major players in the computer world have already derided as a truly bad DX9 benchmark) by developing more efficient shader routines and delivering them to the user, then what's the problem? FM told nVidia they had to pay an exorbinant fee to get access to the program, nVidia thought the price was too high, so they looked at the program on their own and made it as efficient as possible on their cards.

using a more efficient shader that results in the same image would be fine. adding a clipping plane where the game doesn't specify one isn't. the way you talk nvidia thought this was good engineering practice, when it clearly isn't. they stopped liking FM as soon as 3dmark exposed how poor the performance of their latest cards newest features are, and are taking conscious steps to destroy its credibility as a benchmark. it'd be like if they did took a common quake timedemo and added clipping planes in to reduce overdraw. no one might have caught on cuz, last i checked, you can't halt and freely operate the camera in a quake timedemo. sure, it'd make their scores go up in that specific quake timedemo, but you wouldn't see a similar increase running a different quake timedemo or just playing the game. if they were taking the op that was specified and inserting a more efficient op that gave the same results you'd at least have a performance increase in the whole game.

of course another bit of 3dmark tests something that there really isn't a game for atm, dx9. how else do you benchmark dx9 effects than with a dx9 benchmark? without 3dmark you'd have no way of knowing whether card A could do dx9 stuff as well as card B or not.

 

[ElFenix]

Originally posted by: Pariah

This was to intentionally inflate scores to defraud consumers into thinking they are buying a better product than they are.

Any consumer buying a video card based on 3DMark scores deserves to be deceived. I hope more companies blatantly cheat on these synthetic benchmarks so that we can get rid of them or at least make them more representative of real world gaming.

thats real good, what happens when they start cheating on common timdemos of actual games? the amount of benches that people pay attention to are a pretty limited amount compared to how many games are out there

 

[SexyK]

Originally posted by: oldfart
Nothing else I can say here. They cheated. The PDF doc shows exactly how they did it. Static clipping planes that dont belong in the game dont just happen by accident. They were added to cheat. If you choose to ignore the evidence, thats fine. I don't.

I've owned GF2, GF3, GF4 cards (still have a GF3 in one rig). I like nVidia cards but am really disappointed in them for stooping this low. They have been under the gun since they lost the performance crown to the ATi 9700P. Then the nV30 fiasco. Now they need to cheat? Come on nVidia get with it already.

Here is another point. Check out the in game UT 2K3 screenshots from [ H ]:

FX 5900U 8xAA/8xAF UT2K3 ---37 FPS
9800P 6xAA/16xAF UT2K3 ---95 FPS

FX5900U 8xAA/8xAF UT2K3 ---30 FPS
9800P 6xAA/16xAF UT2K3 ---69 FPS

Why so crummy in a real game? The 9800P has better IQ and WAY better framerates. The 5900U doesn't look so hot in high res Hi IQ gaming.

Well, this isn't a discussion of whether we should all be running out to buy 5900Ultras or not. It's about the accusations of cheating on 3dmark03. I guess there is nothing left to say. I just hope that you remember that the PDF you are refering to comes from Futuremark: a company that needs to make money, a company whose purpose nVidia called into question not to long ago, and a company that nVidia refused to do business with costing them hundreds of thousands of dollars. Obviously, there are plenty of reasons for FM to be gunning for nVidia, and I can't believe that you would take their word for gospel, especially considering their checkered past.

 

[Pariah]

"So if mommy (who doesn't spend time at Anandtech) spends her hard earned money for a video card for little Johnny for Christmas because the picture on the box said it was the fastest video card on the market, that is OK?"

If mommy is going to the store and shelling out $300-400 on a video card for Jr without having any idea what she is buying, tough luck, she deserves it.

"thats real good, what happens when they start cheating on common timdemos of actual games?"

There's a fine line between cheating and optimizing. Anything that makes a real game perform better without noticably hurting visuals is optimizing, and benficial to the consumer.

"the amount of benches that people pay attention to are a pretty limited amount compared to how many games are out there "

And how many of those games are framerate limited by a video card and actually need to be benchmarked?

 

[oldfart]

Originally posted by: SexyK ............I can't believe that you would take their word for gospel, especially considering their checkered past.

And I cant believe you insist on ignoring the presented evidence. You think FM could put this kind of accusation out in a press release if it were false? Don't you think nVidia's lawyers might have a slight problems with that? Do you think nVidia has enough resources to bury FM it they made false accusations? Open you eyes (take off the nVidia 3D glasses first) 😉

 

[ElFenix]

Originally posted by: Pariah
"So if mommy (who doesn't spend time at Anandtech) spends her hard earned money for a video card for little Johnny for Christmas because the picture on the box said it was the fastest video card on the market, that is OK?"

If mommy is going to the store and shelling out $300-400 on a video card for Jr without having any idea what she is buying, tough luck, she deserves it.

"thats real good, what happens when they start cheating on common timdemos of actual games?"

There's a fine line between cheating and optimizing. Anything that makes a real game perform better without noticably hurting visuals is optimizing, and benficial to the consumer.

"the amount of benches that people pay attention to are a pretty limited amount compared to how many games are out there "

And how many of those games are framerate limited by a video card and actually need to be benchmarked?

yes, there is a fine line between cheating and optimizing. an optimization i would say is something that would improve performance in every day game play with that game and games using that engine. a cheat would be something that improves performance on that specific benchmark only, and not in the game in general or even that benchmark altered slightly. adding a clipping plane where one isn't supposed to be just when a benchmark is detected is clearly a cheat, by that definition. they can do it in a real game as easily as they could 3dmark, i'd gander.

as for what games, they don't even really need to be different games. how many people are using a UT2k3 bench that isn't one of the common two? how hard would it be to start cheating by not clearing buffers or adding clipping planes to those two demos?

 

[ElFenix]

Originally posted by: SexyK
Well, this isn't a discussion of whether we should all be running out to buy 5900Ultras or not. It's about the accusations of cheating on 3dmark03. I guess there is nothing left to say. I just hope that you remember that the PDF you are refering to comes from Futuremark: a company that needs to make money, a company whose purpose nVidia called into question not to long ago, and a company that nVidia refused to do business with costing them hundreds of thousands of dollars. Obviously, there are plenty of reasons for FM to be gunning for nVidia, and I can't believe that you would take their word for gospel, especially considering their checkered past.

slander and libel were illegal, last i checked

 

[oldfart]

Originally posted by: Pariah
"So if mommy (who doesn't spend time at Anandtech) spends her hard earned money for a video card for little Johnny for Christmas because the picture on the box said it was the fastest video card on the market, that is OK?"

If mommy is going to the store and shelling out $300-400 on a video card for Jr without having any idea what she is buying, tough luck, she deserves it.

No she doesn't. No one "deserves" to be lied to and cheated. I hope you don't really feel that way. Maybe someday someone will rip you off with deceptive advertising tactics. You may feel differently then.

 

[Shamrock]

hmm,

I find it funny that Anandtech didnt use 3dMark...why is that? Because he doesnt have the confidence in the program itself?

Also, everyone of you guys that own a Pentium 4...praise them for their "SSE2" OPTIMIZATION extensions while the athlon gets a close enough score with RAW horsepower. Before you scoff at Nvidia for cheating, should you not also take a look at Intel for optimizing with software? If you feel differently, then you're a hypocrite. Intel "optimizes" to include the SSE2 extensions, while AMD does not...let's DISABLE the SSE2 instructions, and see how fast the P4 goes then. I've eseen the SSE2 instructions disabled on Adobe Photoshop 7, and it was slower than the Macintosh! WHile the Athlon cruised.

and I also have a question...about these new drivers, the 44.03

Does this "optimization" affect ALL cards, or just the GFFX 5900 series? Because I have the new drivers in my GF3 Ti200, and I find NO FLAW in ANY game I play, and that's over 15 games on my harddrive. I have not seen a glitch, nor a bug, nor slowdown, nor crashes from these new drivers...however, I HAVE seen performance gains in ALL my games,with no problems whatsoever. So...

3dMark can stick their little benchmark, I play games, and that what counts..REAL world benchmarks...I tend to like the benchmarks Anandtech had. Oh yeah, they were also impressive...I will be buying a GFFX 5900 😀

Another thing....Why did Anandtech get a bug where ATI cards have that window shining through the head of Sam on Splinter Cell? is ATI cheating because of that? hmm...maybe they are manipulating their own drivers...because Splinter Cell IS a DX9 game (it says it on my box)

 

[SexyK]

Originally posted by: ElFenix

Originally posted by: SexyK
Well, this isn't a discussion of whether we should all be running out to buy 5900Ultras or not. It's about the accusations of cheating on 3dmark03. I guess there is nothing left to say. I just hope that you remember that the PDF you are refering to comes from Futuremark: a company that needs to make money, a company whose purpose nVidia called into question not to long ago, and a company that nVidia refused to do business with costing them hundreds of thousands of dollars. Obviously, there are plenty of reasons for FM to be gunning for nVidia, and I can't believe that you would take their word for gospel, especially considering their checkered past.

slander and libel were illegal, last i checked

Obviously FM has the right to make the accusation, it's their product and they know how it works. Slander and libel are illegal, and I'm sure nVidia's lawyers are hard at work right now, figuring out how to handle this situation. Anyway, all i'm saying is that you have to open your eyes to the fact that they are not an objective third party in this argument, and there are plenty of motivations for them to release this other than for the sanctity of the benchmark. Whether or not nVidia was being dishonest, this is a play in the market by FM.

 

[oldfart]

Originally posted by: SexyKyou have to open your eyes to the fact that they are not an objective third party in this argument, and there are plenty of motivations for them to release this other than for the sanctity of the benchmark. Whether or not nVidia was being dishonest, this is a play in the market by FM.

And nVidia is an objective party in this? Who has more to lose? Please list what those "plenty of motivations" would be.

 

[SexyK]

Originally posted by: oldfart

Originally posted by: SexyKyou have to open your eyes to the fact that they are not an objective third party in this argument, and there are plenty of motivations for them to release this other than for the sanctity of the benchmark. Whether or not nVidia was being dishonest, this is a play in the market by FM.

And nVidia is an objective party in this? Who has more to lose? Please list what those "plenty of motivations" would be.

Please go back and read my posts. I never said that nVidia was an objective third party, I only said that relying on the information provided by Futuremark is a mistake.

I also already listed Futuremark's reasons for attacking nVidia, but here's a recap:

1)FM tells nVidia that they have to pay hundreds of thousands of dollars to join the "beta program" for 3dmark03
2)nVidia decides that 3dmark03 isn't a realistic DX9 benchmark (a fact that many in the industry agree on)
3)As fallout to these decisions, the validity of Futuremark's most important product is brought into question
4)A further result is Futuremark takes a direct loss of hundreds of thousands of dollars when nVidia refuses to join the program.

nVidia's decision severly impacted the reputation and bottom line of FM. If you think Futuremark wasn't extremely upset at nVidia after that turn of events, you're mistaken.

 

[oldfart]

Well, since FM is falsely accusing nV of cheating I guess we will be seeing a legal response from nVidia soon. Please post the news once it comes out. There is no way that nVidia would not take legal action...unless they actually did cheat.

 

[ElFenix]

Originally posted by: Shamrock
hmm,

I find it funny that Anandtech didnt use 3dMark...why is that? Because he doesnt have the confidence in the program itself?

Also, everyone of you guys that own a Pentium 4...praise them for their "SSE2" OPTIMIZATION extensions while the athlon gets a close enough score with RAW horsepower. Before you scoff at Nvidia for cheating, should you not also take a look at Intel for optimizing with software? If you feel differently, then you're a hypocrite. Intel "optimizes" to include the SSE2 extensions, while AMD does not...let's DISABLE the SSE2 instructions, and see how fast the P4 goes then. I've eseen the SSE2 instructions disabled on Adobe Photoshop 7, and it was slower than the Macintosh! WHile the Athlon cruised.

i clearly explained what a cheat is and what isn't. you can simply ignore that and spout off inane crap if you like.

 

[ElFenix]

Originally posted by: SexyK
2)nVidia decides that 3dmark03 isn't a realistic DX9 benchmark (a fact that many in the industry agree on)
.

name 3.

 

[Chobits]

Originally posted by: SexyK
I did read that, thanks. The last three of those eight "cheats" actaully make the score go down. That's a little odd in my book. I just don't see how you can indict them for achieving higher performace by including "alternative more efficient" shaders that produce "a similar looking rendering" (again, a bit vague for my tastes). If nVidia can optimize FM's program (which a lot of major players in the computer world have already derided as a truly bad DX9 benchmark) by developing more efficient shader routines and delivering them to the user, then what's the problem? FM told nVidia they had to pay an exorbinant fee to get access to the program, nVidia thought the price was too high, so they looked at the program on their own and made it as efficient as possible on their cards.

Because then the program isn't 3dmark03. The prupose of this is to provide a bunch of standard tests to compare the videocards witho one another using the EXACT same program.

By using their own pixel shader and refusing to use the one that is standard for 3dmark03 it is not the same standard program that everyone is comparing.




Also we maybe smart enough not to care too much about 3dmark03 but MOST people will buy a card based on this. I also post in places with less pc-afficionados and more teenagers and you them screaming when their 3dmark score is 100 points under a rig that is the same and they want a higher score.

And no I'm not ATI fanboy or 3mdark fanboy (would it even run on my Geforce4mx420?) but what Nvidia did was cheap and I see it as something cheaper than ATI Quack

 

[Chobits]

Splinter Cell is a DirectX 8 game...unless the Xbox secretly has a Geforce FX inside instead of a Geforce3

 

[galperi1]

3dmark03 DOES matter

maybe not to gamers, but it sure as hell does to big OEM. How do you think Dell, Compaq, Sony, and HP decide which graphics card to use. Easy, the one that scores the best 3dmarks. Nvidia knew this and that is ultimately the reason they went to such great lengths to cheat.

They could give a s**t less about the "gamers" because you know what??? They make ALL their money loot on OEM wins (low end - high margin products) and guess what.... those are based on 3dmarks.

It's as simple as that. It seems that Nvidia has inherited the worst attributes of a dying 3dfx company