GTaudiophile
Lifer
By Simon Nash: Wednesday 28 May 2003, 16:55
LAST WEEK, Futuremark, creator of the popular 3DMark-series of graphical benchmarking software, issued a patch for its latest flagship suite 3DMark03 along with a verbal blast aimed firmly at Nvidia. Futuremark basically accused Nvidia of using surreptitious means to artificially inflate its score.
At issue were Nvidia?s latest Detonator reference drivers, version 44.03. The tech enthusiast site ExtremeTech had previously brought the issue to light with an article published on 14th May. It concluded that Nvidia was cutting corners in its drivers, essentially making its cards do less work in 3DMark03 to boost its score, and from this Futuremark initiated its own investigation, with the results and 3DMark03 patch published on the 23rd May.
Among Futuremark?s findings were:
Several pixel and vertex shader effects in 3DMark03 are discarded by Nvidia?s drivers and replaced by its own (more efficient and heavily-optimised for their GPUs) shaders.
Sometimes an instruction by 3DMark03 to clear the back buffer was ignored at the driver level, specifically to increase performance.
Custom clip planes were used to reduce the workload on their GPUs, thereby again artificially inflating the score.
Bear in mind here that what we are not talking about is optimising the driver for maximum performance. As Futuremark stated in a press release, optimisation is a routine and regularly-used method to increase performance at no cost of image quality. What Nvidia was doing here was deliberate manipulation of 3DMark03?s instructions at the driver level, reducing image quality and reducing the workload on its cards to increase the scores. While the output image was similar to what it was meant to be, it was not identical.
Once Futuremark patched 3DMark03 and re-benchmarked using a reference Nvidia GeForce FX 5900 Ultra, the score dropped from 5806 to 4679, a difference of almost 25%. However, ATi did not come out of Futuremark?s audit report with completely flying colours either. Futuremark noted that with the new build of 3DMark, the score on a reference system based on a Radeon 9800 Pro also dropped, but by a much smaller margin of 1.9%.
The interesting thing here is both company?s reactions. ATi came out and publicly stated that it had optimised Game Test 4 of 3DMark03 and shuffled the instructions to better suit its architecture. Note that the scene was still rendered exactly how Futuremark had intended, the instructions were just moved around to suit their GPU and give them the slight boost which it had. However, despite it being a genuine optimisation, ATi still announced that it would remove this in the next release of their Catalyst drivers.
Nvidia, on the other hand, has to date done an excellent impression of a child got caught stealing from the cookie jar and sent into the corner of the room all the while yelling "Na na na na, I can?t hear you". It instead chose to attack Futuremark over the issue, accusing them of a deliberate campaign of attempting to make their cards look bad in 3DMark03. Since Nvidia left Futuremark?s beta program at the end of last year under a cloud, it no longer has official access to the so-called developer version of 3DMark03. This allows the user to pause and fully rotate the camera in any given part of the test, which is how ExtremeTech originally discovered the problem in the first place.
The pro-Nvidia and pro-ATi camps - labelled as "nvidiots" and "fanATics" - by the opposing sides, have also been at each other?s throats. Respected news sites such as nV News (pro-Nvidia) and Rage3D (pro-ATi) have had their forums flooded with one side attacking the other, both seeing the other as cheats.
It seems clear to me that Nvidia is the guilty party here. There is no way that reducing the workload on their GPUs, replacing shader code and using custom clip planes to artificially increase their scores is a legitimate tactic, and it should be ashamed of its actions. Even such industry heavyweights as Tim Sweeney, one of the lead developers on the Unreal series; and John Carmack, Doom III mastermind, have weighed in on the subject, again reiterating that an optimisation is not considered valid unless the end result is exactly as it was intended after optimisation. In Nvidia?s case the end result was similar, but not the same, but in ATi?s case the end result was exactly as it was meant to be.
Fingers have been pointed everywhere, mostly by nvidiots, in an attempt to get Nvidia "off the hook". Blame has been assigned to Futuremark (of course, they forced Nvidia to use such driver cheats) and to ExtremeTech. I?m surprised that Al-Quaeda, the Loch Ness monster and Elvis haven?t been implicated too, though that can only be a matter of time.
NV30 was a disaster for Nvidia, but trying to claw back ground this way is not giving it any more credibility. Early figures from NV35 show that it may well be a much better performer and could indeed regain the performance crown that ATi has held for many months, but what use is performance if your image has been tarnished to such an extent that no-one wishes to buy your products anymore? Nvidia has handled itself badly over this, continuing the slide that it has been in for the past six months. If it wishes to reclaim the popular support it once enjoyed, stunts like this will need to become a thing of yesterday, lest the gaming public vote with their wallets and put an end to Nvidia no matter how good the hardware. µ
The author of this opinion piece wishes us to point out that he owns products from both ATI and Nvidia
LAST WEEK, Futuremark, creator of the popular 3DMark-series of graphical benchmarking software, issued a patch for its latest flagship suite 3DMark03 along with a verbal blast aimed firmly at Nvidia. Futuremark basically accused Nvidia of using surreptitious means to artificially inflate its score.
At issue were Nvidia?s latest Detonator reference drivers, version 44.03. The tech enthusiast site ExtremeTech had previously brought the issue to light with an article published on 14th May. It concluded that Nvidia was cutting corners in its drivers, essentially making its cards do less work in 3DMark03 to boost its score, and from this Futuremark initiated its own investigation, with the results and 3DMark03 patch published on the 23rd May.
Among Futuremark?s findings were:
Several pixel and vertex shader effects in 3DMark03 are discarded by Nvidia?s drivers and replaced by its own (more efficient and heavily-optimised for their GPUs) shaders.
Sometimes an instruction by 3DMark03 to clear the back buffer was ignored at the driver level, specifically to increase performance.
Custom clip planes were used to reduce the workload on their GPUs, thereby again artificially inflating the score.
Bear in mind here that what we are not talking about is optimising the driver for maximum performance. As Futuremark stated in a press release, optimisation is a routine and regularly-used method to increase performance at no cost of image quality. What Nvidia was doing here was deliberate manipulation of 3DMark03?s instructions at the driver level, reducing image quality and reducing the workload on its cards to increase the scores. While the output image was similar to what it was meant to be, it was not identical.
Once Futuremark patched 3DMark03 and re-benchmarked using a reference Nvidia GeForce FX 5900 Ultra, the score dropped from 5806 to 4679, a difference of almost 25%. However, ATi did not come out of Futuremark?s audit report with completely flying colours either. Futuremark noted that with the new build of 3DMark, the score on a reference system based on a Radeon 9800 Pro also dropped, but by a much smaller margin of 1.9%.
The interesting thing here is both company?s reactions. ATi came out and publicly stated that it had optimised Game Test 4 of 3DMark03 and shuffled the instructions to better suit its architecture. Note that the scene was still rendered exactly how Futuremark had intended, the instructions were just moved around to suit their GPU and give them the slight boost which it had. However, despite it being a genuine optimisation, ATi still announced that it would remove this in the next release of their Catalyst drivers.
Nvidia, on the other hand, has to date done an excellent impression of a child got caught stealing from the cookie jar and sent into the corner of the room all the while yelling "Na na na na, I can?t hear you". It instead chose to attack Futuremark over the issue, accusing them of a deliberate campaign of attempting to make their cards look bad in 3DMark03. Since Nvidia left Futuremark?s beta program at the end of last year under a cloud, it no longer has official access to the so-called developer version of 3DMark03. This allows the user to pause and fully rotate the camera in any given part of the test, which is how ExtremeTech originally discovered the problem in the first place.
The pro-Nvidia and pro-ATi camps - labelled as "nvidiots" and "fanATics" - by the opposing sides, have also been at each other?s throats. Respected news sites such as nV News (pro-Nvidia) and Rage3D (pro-ATi) have had their forums flooded with one side attacking the other, both seeing the other as cheats.
It seems clear to me that Nvidia is the guilty party here. There is no way that reducing the workload on their GPUs, replacing shader code and using custom clip planes to artificially increase their scores is a legitimate tactic, and it should be ashamed of its actions. Even such industry heavyweights as Tim Sweeney, one of the lead developers on the Unreal series; and John Carmack, Doom III mastermind, have weighed in on the subject, again reiterating that an optimisation is not considered valid unless the end result is exactly as it was intended after optimisation. In Nvidia?s case the end result was similar, but not the same, but in ATi?s case the end result was exactly as it was meant to be.
Fingers have been pointed everywhere, mostly by nvidiots, in an attempt to get Nvidia "off the hook". Blame has been assigned to Futuremark (of course, they forced Nvidia to use such driver cheats) and to ExtremeTech. I?m surprised that Al-Quaeda, the Loch Ness monster and Elvis haven?t been implicated too, though that can only be a matter of time.
NV30 was a disaster for Nvidia, but trying to claw back ground this way is not giving it any more credibility. Early figures from NV35 show that it may well be a much better performer and could indeed regain the performance crown that ATi has held for many months, but what use is performance if your image has been tarnished to such an extent that no-one wishes to buy your products anymore? Nvidia has handled itself badly over this, continuing the slide that it has been in for the past six months. If it wishes to reclaim the popular support it once enjoyed, stunts like this will need to become a thing of yesterday, lest the gaming public vote with their wallets and put an end to Nvidia no matter how good the hardware. µ
The author of this opinion piece wishes us to point out that he owns products from both ATI and Nvidia
