Intel graphics drivers employ questionable 3DMark Vantage optimizations

MODEL3

Senior member
Jul 22, 2009
528
0
0
http://www.techreport.com/articles.x/17732

In the early days of GPUs, application-specification performance optimizations in graphics drivers were viewed by many as cheating. Accusations were hurled with regularity, and in some cases, there was real cheating going on. Some optimizations surreptitiously degraded image quality in order to boost performance, which obviously isn't kosher.
...
Futuremark's popular 3DMark benchmark has been the target of several questionable optimizations over the years.
...
With the exception of configuring the correct rendering mode on multi-GPU systems, it is prohibited for the driver to detect the launch of 3DMark Vantage executable and to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection. Optimizations in the driver that utilize empirical data of 3DMark Vantage workloads are prohibited.
No ambiguity there, then: Vantage-specific optimizations aren't allowed.

Intel may not be playing fair, though. We recently learned AMD has notified Futuremark that Intel's 15.15.4.1872 Graphics Media Accelerator drivers for Windows 7 incorporate performance optimizations that specifically target the benchmark, so we decided to investigate.
...
We first ran the benchmark normally. Then, we renamed the 3DMark executable from "3DMarkVantage.exe" to "3DMarkVintage.exe". And?wouldn't you know it??there was a substantial performance difference between the two.
...
Our system's overall score climbs by 37% when the graphics driver knows it's running Vantage. That's not all. Check out the CPU and GPU components of the overall score:
...
The GPU score jumps by a whopping 46% thanks to Intel's apparent Vantage optimization. At the same time, the CPU score falls by nearly 10%. Curious.
...
Intel appears to be offloading some of the work associated with the GPU tests onto the CPU in order to improve 3DMark scores.When asked for comment, Intel replied with the following:
"We have engineered intelligence into our 4 series graphics driver such that when a workload saturates graphics engine with pixel and vertex processing, the CPU can assist with DX10 geometry processing to enhance overall performance...."
...
Here's the tricky part: the very same 785G system managed 30 frames per second in Crysis: Warhead, which is twice the frame rate of the G41 with all its vertex offloading mojo in action. The G41's new-found dominance in 3DMark doesn't translate to superior gaming performance, even in this game targeted by the same optimization.

Larrabee's TWIMTBB also?
lol

jk
 

v8envy

Platinum Member
Sep 7, 2002
2,720
0
0
Wonder what happens if you rename Crysis Warhead (or any game) executable to 3DMarkVantage.exe. Could be quite spectacular as the workload is "offloaded" and "assisted."

If this was done as Intel claims then it wouldn't necessarily be a bad thing for the user. Intel on-board graphics can use all the help they can get, even from the CPU. But more likely it's a benchmark targeted cheat/optimization.
 

MODEL3

Senior member
Jul 22, 2009
528
0
0
From Techreport:

One of the games on the list for detection, Crysis Warhead, should have no problem saturating an integrated graphics chipset, to say the least. We tested it with the executable under its original name and then renamed to Crisis.exe, using FRAPS to collect real-world frame rate data with the game running at 800x600 and minimum detail levels.

http://www.techreport.com/r.x/gma-vantage/warhead.gif

Intel's software-based vertex processing scheme improves in-game frame rates by nearly 50% when Crysis.exe is detected, at least in the first level of the game we used for testing. However, even 15 FPS is a long way from what we'd consider a playable frame rate. The game doesn't exactly look like Crysis Warhead when running at such low detail levels, either.

Our Warhead results do prove that Intel's optimization can improve performance in actual games, though?if only in this game and perhaps the handful of others identified in the driver INF file.

"3DMarkVantage is one of those workloads, as are Call of Juarez, Crysis, Lost Planet: Extreme Conditions, and Company of Heroes."

 

MarcVenice

Moderator Emeritus <br>
Apr 2, 2007
5,664
0
0
Isn't this like, great fucking news? Maybe some shit games can now be played by owners of Intel igp-chipsets, if they have a decent cpu?
 

veri745

Golden Member
Oct 11, 2007
1,163
4
81
Originally posted by: MarcVenice
Isn't this like, great fucking news? Maybe some shit games can now be played by owners of Intel igp-chipsets, if they have a decent cpu?

It might be great news if the list of supported games didn't include like 4-5 of the most commonly used games for benchmarking graphics and nothing else...

Looks to me like Intel is trying to cheat quite explicitly.
 

Sylvanas

Diamond Member
Jan 20, 2004
3,752
0
0
I thought this came out a while back that when you changed vendor ID to Intel, on an AMD system your score jumped markedly.
 

YBS1

Golden Member
May 14, 2000
1,945
129
106
Actually, so long as the rendering quality remains the same I don't see this as a cheat at all and could prove very beneficial to the end user down the road. I'm guessing no one here gives two craps about Vantage performance on Intel's integrated graphics solution and I'm sure Intel doesn't believe any consumers who are using such a solution do either....I'm wondering if this isn't a test run for things to come. That would be a massive advantage and nice feature to have if somewhere down the road Intel's coming discrete cards ended up being truly competitive. CPU assist in GPU limited games, and who knows maybe they could find a way to reverse that, GPU assist in CPU limited games.
 

Ben90

Platinum Member
Jun 14, 2009
2,866
3
0
Maybe i can rename my 4890 to intels IGP and use their driver so my 4ghz i7 can finally start doing something

too bad it doesnt work like that

*edit*

If u put benchmarking aside, just think if ATI/Nvidia could deploy this as a global setting. Im sure most of us would love the extra fps at the expense of warming the cpu up a little bit
 

Majic 7

Senior member
Mar 27, 2008
668
0
0
Exactly who is this supposed to fool? Gamers, not hardly. Systems builders, not unless they follow turnip trucks to hire their system designers. Grandmas, wouldn't even know what Vantage is. Who knows for sure if this was done specifically done for Vantage or is just a byproduct of optimizing drivers for a chipset that can't run modern games anyway? Seems like a total non-issue to me.
 

alcoholbob

Diamond Member
May 24, 2005
6,380
448
126
Originally posted by: Sylvanas
I thought this came out a while back that when you changed vendor ID to Intel, on an AMD system your score jumped markedly.

I think it was a system benchmark like either PC Mark or Sisoft Sandra I don't remember but it was definitely shady.
 

tcsenter

Lifer
Sep 7, 2001
18,892
543
126
Originally posted by: VirtualLarry
all this proves is that intel is just as shady as other graphics vendors.
Not really. Intel has long disclosed that its newer GMA products will off-load to the CPU (e.g. software vertex) when some driver algorithm detects or determines that doing so would yield substantially better results.

The only thing "shady" going on is that Intel doesn't consider Futuremark to be some different kind of 3D application than any other 3D application where Intel's GMA will selectively off-load.
 

braddeicide

Junior Member
Oct 13, 2009
1
0
0
Originally posted by: Sylvanas
I thought this came out a while back that when you changed vendor ID to Intel, on an AMD system your score jumped markedly.

That was with their compiler.

"If the program detects a "Genuine Intel" microprocessor, it executes a fully optimized code path and operates with the maximum efficiency. However, if the program detects an "Authentic AMD" microprocessor, it executes a different code path that will degrade the program's performance or cause it to crash." - http://yro.slashdot.org/yro/05/07/12/1320202.shtml

People made programs that did nothing but remove this detection and programs sped up significantly. So yeh, no news here really, Intel is dodgy.
 

CurseTheSky

Diamond Member
Oct 21, 2006
5,401
2
0
Originally posted by: Majic 7
Exactly who is this supposed to fool? Gamers, not hardly. Systems builders, not unless they follow turnip trucks to hire their system designers. Grandmas, wouldn't even know what Vantage is. Who knows for sure if this was done specifically done for Vantage or is just a byproduct of optimizing drivers for a chipset that can't run modern games anyway? Seems like a total non-issue to me.

Person A is an "average" non-tech-savvy computer user who still likes to play a few games here and there.

Person B is a computer enthusiast with thousands of dollars worth of hardware and a huge library of games.

Person A: "Dude, I'm going to buy a new computer. I found a QUAD CORE rig on Dell.com - I bet that thing will run any program I want. Intel Inside, baby!"
Person B: "Uh, is this the one you mean? It has integrated Intel graphics, which suck. What are you going to use it for?"
Person A: "Oh, you know, surfing the web, doing homework, playing games on Facebook... I might resub my WoW account to check out the next expansion, and if that works well, I should be able to play Crysis too!"
Person B: "Let's... forget about Crysis. Anyway, let me build you a decent system. We could go with a dual core AMD to save a bunch of cash, and their newest 785G chipset's integrated graphics should be more than plenty for you. We can add a discrete card later if you want to do more serious gaming."
Person A: "AMD? Ewww. I don't want anything they make, it'll be too slow. Remember back in 2004? The processor you bought was only 2.4 GHz, but mine was 3.0!"
Person B: "They have quad cores that will be more than you'll need for years to come, and they're cheaper than anything Intel has out there right now. Don't even get me started on Netburst and clock speed."
Person A: "Wait, what? Now you're talking about blowing up my Internet connection? Hey, look at this - Intel's graphics are way better than AMD's graphics! See, it's right here on one of those charts you spend all your time looking over."
Person B: "Yeah, that's something Intel purposely did to get higher benchmark scores in Vantage. Ignore it - the real world performance of AMD's IGP is significantly better than Intel's."
Person A: "Blasphemy! You just hate Intel. Admit it!"
Person B: (rolling eyes) "...you know what, on second thought, just go ahead and buy the Dell. I'm sure you'll be happier."

Obviously fictitious, but I've had very similar conversations with different people on several occasions now. Basically, this targets the just-tech-savvy-enough crowd to be wowed by the marketing stickers you see all over systems at Best Buy and whatnot.
 

Dribble

Platinum Member
Aug 9, 2005
2,076
611
136
All graphics card makers optimise their drivers for every game now. Intel are only doing what nvidia and ati do for both 3d mark, and most games. We don't care as long as it works.
 

MODEL3

Senior member
Jul 22, 2009
528
0
0
Originally posted by: CurseTheSky
Originally posted by: Majic 7
Exactly who is this supposed to fool? Gamers, not hardly. Systems builders, not unless they follow turnip trucks to hire their system designers. Grandmas, wouldn't even know what Vantage is. Who knows for sure if this was done specifically done for Vantage or is just a byproduct of optimizing drivers for a chipset that can't run modern games anyway? Seems like a total non-issue to me.

Person A is an "average" non-tech-savvy computer user who still likes to play a few games here and there.

Person B is a computer enthusiast with thousands of dollars worth of hardware and a huge library of games.

Person A: "Dude, I'm going to buy a new computer. I found a QUAD CORE rig on Dell.com - I bet that thing will run any program I want. Intel Inside, baby!"
Person B: "Uh, is this the one you mean? It has integrated Intel graphics, which suck. What are you going to use it for?"
Person A: "Oh, you know, surfing the web, doing homework, playing games on Facebook... I might resub my WoW account to check out the next expansion, and if that works well, I should be able to play Crysis too!"
Person B: "Let's... forget about Crysis. Anyway, let me build you a decent system. We could go with a dual core AMD to save a bunch of cash, and their newest 785G chipset's integrated graphics should be more than plenty for you. We can add a discrete card later if you want to do more serious gaming."
Person A: "AMD? Ewww. I don't want anything they make, it'll be too slow. Remember back in 2004? The processor you bought was only 2.4 GHz, but mine was 3.0!"
Person B: "They have quad cores that will be more than you'll need for years to come, and they're cheaper than anything Intel has out there right now. Don't even get me started on Netburst and clock speed."
Person A: "Wait, what? Now you're talking about blowing up my Internet connection? Hey, look at this - Intel's graphics are way better than AMD's graphics! See, it's right here on one of those charts you spend all your time looking over."
Person B: "Yeah, that's something Intel purposely did to get higher benchmark scores in Vantage. Ignore it - the real world performance of AMD's IGP is significantly better than Intel's."
Person A: "Blasphemy! You just hate Intel. Admit it!"
Person B: (rolling eyes) "...you know what, on second thought, just go ahead and buy the Dell. I'm sure you'll be happier."

Obviously fictitious, but I've had very similar conversations with different people on several occasions now. Basically, this targets the just-tech-savvy-enough crowd to be wowed by the marketing stickers you see all over systems at Best Buy and whatnot.

:thumbsup:
 

Majic 7

Senior member
Mar 27, 2008
668
0
0
Still don't see how this useless factoid gets to the consumer. Intel uses their CPU to bolster their IGP that needs all the help it can get and this is a problem? I visit Dell or HP and I see no mention of this performance enhancement. All I see is a statement that if you want to game you need to get a better video card. Any review I see reviewing systems with IGPs of any make state this also. Sorry you know so many stupid people, I know some of them myself. Tell them the machine they are looking at won't play anything newer than 3 years old, if that, and I get that "but it's a computer and it should play computer games" look. The only danger to the consumer is a salesperson, and I doubt they know about Vantage either.
 

tynopik

Diamond Member
Aug 10, 2004
5,245
500
126
> Intel uses their CPU to bolster their IGP that needs all the help it can get and this is a problem?

the problem is that it's not generic

if it worked on all games fine, but these optimizations target only a select handful of games

and there is absolutely NO REASON for one of those targetted games to be 3dmark, because there is no 'gaming experience' to improve because it's NOT A GAME. The ONLY purpose is to deceive people into thinking the igp is faster than it really is
 

CurseTheSky

Diamond Member
Oct 21, 2006
5,401
2
0
Originally posted by: tynopik
and there is absolutely NO REASON for one of those targetted games to be 3dmark, because there is no 'gaming experience' to improve because it's NOT A GAME. The ONLY purpose is to deceive people into thinking the igp is faster than it really is

Exactly. This would be similar to if a car dealership offered you to take their latest high-performance car out to a track to test it before buying, and secretly slapped on several performance parts before you got there. Then, after you brought the car back, impressed with the performance, they removed all of the aftermarket performance parts and put it back to the stock configuration. If they left the performance parts on, it wouldn't be a problem. However, since they remove them, it's just false advertising.

Intel should just use this technology in ALL 3D environments (with an option to disable it if desired, and perhaps and option for it to kick in if FPS drops below a threshold) and market it as a (slight) advantage to their chipsets with onboard graphics.
 

Ben90

Platinum Member
Jun 14, 2009
2,866
3
0
I think someone out there who has one of these IGPs should rename a game thats not on their list to crysis.exe and see if it does anything different; to see if it could be used globally
 

tcsenter

Lifer
Sep 7, 2001
18,892
543
126
Originally posted by: tynopik
if it worked on all games fine, but these optimizations target only a select handful of games
Ehh...the point of Intel's software rendering is to improve 3D application performance when off-loading is justified, not to off-load everything just because its Monday. The "select" games are those that Intel's testing has verified would benefit from off-loading.

and there is absolutely NO REASON for one of those targetted games to be 3dmark, because there is no 'gaming experience' to improve because it's NOT A GAME.
Nor does Intel limit its off-loading scheme only to "3D games", but any 3D application (again, where it is proven justified, not because its Monday). 3DMark is a tool used to gauge 3D application performance of the tested SYSTEM, not strictly GPU performance. Ergo, for end-users wanting to know how their SYSTEM will perform compared to other solutions, its entirely valid.

For reviewers wanting to explore just the capabilities of Intel's GMA alone, it wouldn't be valid. Too bad 3DMark is not a tool exclusively targeted or intended for such limited purposes.

The real issue is disclosure. AMD and NVIDIA have never claimed to do this kind of off-loading and have only touted the capabilities of its hardware rendering. Intel has been disclosing freely for over two years now that its GMA 3000 and later IGPs are designed to selectively perform certain 3D rendering workloads in software where justified.
 

tynopik

Diamond Member
Aug 10, 2004
5,245
500
126
> 3DMark is a tool used to gauge 3D application performance of the tested SYSTEM, not strictly GPU performance. Ergo, for end-users wanting to know how their SYSTEM will perform compared to other solutions, its entirely valid

BULL and SHIT

there are hundreds of games out there, basing your comparison on an 'optimization' that helps only FIVE of those hundreds is LYING

95% of the time, users will NOT see the performance gains represented in 3dmark

If this actually helped half the games out there, you might have a point, but it's nowhere close that and never will be close to that

> Intel has been disclosing freely for over two years now that its GMA 3000 and later IGPs are designed to selectively perform certain 3D rendering workloads in software where justified

if all they have to show after 2 years is FIVE games, one of which happens to be THE premier video card benchmarking tool, i think it's pretty clear how sleazy intel is being
 

tcsenter

Lifer
Sep 7, 2001
18,892
543
126
Originally posted by: tynopik
there are hundreds of games out there, basing your comparison on an 'optimization' that helps only FIVE of those hundreds is LYING
Right, because it makes sense that Intel would spend several million dollars and enlist several dozen employees during a period of massive lay-offs and economic downturn, in order to test several hundred games just to find the two dozen most popular titles that would in fact benefit from off-loading. :roll:

If this actually helped half the games out there, you might have a point, but it's nowhere close that and never will be close to that
For technical reasons you will never understand, not all games are going to benefit. Intel never claimed otherwise.

if all they have to show after 2 years is FIVE games, one of which happens to be THE premier video card benchmarking tool, i think it's pretty clear how sleazy intel is being
Try to get your facts straight:

http://www.intel.com/support/graphics/sb/CS-028231.htm

Intel has tested several games and applications and determined that some provide higher performance when using software vertex processing. This is an initial list of games and applications and may change at any time.

The applications for Windows* XP include:

* 3DMark*06
* 3DMark*05
* 3DMark*03
* PCMark*05
* Serious Sam* 2
* Grand Theft Auto* San Andreas
* F.E.A.R*
* Final Fantasy* XI Bench 3
* World of Warcraft*
* Company of Heroes*
* Age of Empires* 3
* Madden* 07


The applications for Windows Vista* include:

* 3DMark*06
* 3DMark*05
* 3DMark*03
* PCMark*05
* Serious Sam* 2
* Grand Theft Auto* San Andreas
* F.E.A.R*
* Final Fantasy* XI Bench 3
* World of Warcraft*
* Company of Heroes*
* Age of Empires* 3
* Madden* 07
* Stalker*
* Unreal Tournament* 2004
* Guild Wars*


Date Created: 16-Aug-2007
Last Modified: 18-Sep-2007
From the latest driver INF for Vista/7:

[Enable3DContexts_AddSwSettings]

HKR,, ~3DMark03.exe
HKR,, ~3DMark05.exe
HKR,, ~3DMark06.exe
HKR,, ~age3.exe
HKR,, ~BF2.exe
HKR,, ~BF2142.exe
HKR,, ~cod2sp_s.exe
HKR,, ~cod2mp_s.exe
HKR,, ~dreamfall.exe
HKR,, ~farcry.exe
HKR,, ~FEAR.exe
HKR,, ~GRAW.exe
HKR,, ~gta_sa.exe
HKR,, ~H5_GAME.exe
HKR,, ~hl2.exe
HKR,, ~IW3SP.exe
HKR,, ~MAINAPP.exe
HKR,, ~medieval2.exe
HKR,, ~PCMark05.exe
HKR,, ~RelicCOH.exe
HKR,, ~Sam2.exe
HKR,, ~UT2004.exe
HKR,, ~WoW.exe
HKR,, ~XR_3DA.exe
HKR,, _HPTV.exe


[Enable3DContexts_BLB_AddSwSettings]

HKR,, ~BF2142.exe
HKR,, ~CoD2MP_s.exe
HKR,, ~CoD2SP_s.exe
HKR,, ~dragonshard.exe
HKR,, ~FarCry.exe
HKR,, ~FSW2.exe
HKR,, ~GTR2.exe
HKR,, ~hl2.exe
HKR,, ~JustCause.exe
HKR,, ~legostarwars.exe
HKR,, ~medieval2.exe
HKR,, ~Oblivion.exe
HKR,, ~OblivionLauncher.exe
HKR,, ~ShatteredUnion.exe
 

tynopik

Diamond Member
Aug 10, 2004
5,245
500
126
Originally posted by: tcsenter
Originally posted by: tynopik
there are hundreds of games out there, basing your comparison on an 'optimization' that helps only FIVE of those hundreds is LYING
Right, because it makes sense that Intel would spend several million dollars and enlist several dozen employees during a period of massive lay-offs and economic downturn, in order to test several hundred games just to find the two dozen most popular titles that would in fact benefit from off-loading. :roll:

1. that's exactly the point! such optimizations will only ever help a small handful of games and are not representative of the overall gaming experience

2. if they were so limited in resources, you would think they would spend most of their effort trying to actually help consumers

instead what do we see?

3DMark03.exe
3DMark05.exe
3DMark06.exe
3dMarkVantage.exe
PCMark05.exe

they sure did a thorough job of covering all the benchmark software

i'm sure that was all for the benefit of the consumer </sarcasm>

Originally posted by: tcsenter
If this actually helped half the games out there, you might have a point, but it's nowhere close that and never will be close to that
For technical reasons you will never understand, not all games are going to benefit. Intel never claimed otherwise.

and again i repeat myself, if not all games are going to benefit (or even most games), whether it's technical issues or lack of resources, 'optimizing' for benchmarks is simply a LIE

if all they have to show after 2 years is FIVE games, one of which happens to be THE premier video card benchmarking tool, i think it's pretty clear how sleazy intel is being
Try to get your facts straight:

http://www.intel.com/support/graphics/sb/CS-028231.htm

ooh, well it's 20 games instead of 5

the point still stands

it is NOT representative of the general gaming experience