AMD Comments on GPU Stuttering, Offers Driver Roadmap & Perspective on Benchmarking

[blackened23]

BFG perhaps you missed the part of the article which discusses how FRAPs measures latency before the GPU driver even touches anything. It measures data before the render pipeline, which is WHERE the driver begins doing work.

This is a point which nvidia agrees upon, FRAPs is flawed as a testing method for frame latency. Pre-render pipeline data is pretty worthless from what I gather in the article. So nvidia agrees its flawed, but a couple of posters here don't agree. Hmm.

 

[sontin]

The point that the article was making is that FRAPS doesn't measure frames output to the screen, which is completely different from saying FRAPS is useless. FRAPS is still useful for three reasons:

Pcper.com said that Fraps showing a much better picture for crossfire then their DVI capture card.

How can AMD blame Fraps for their problems when it is not even able to show the real picture?

AMD is still blaming every other entity on this planet instead of themself.

 

[blackened23]

Hmmmmmmmmmmmmmm

The problem here is not in using FRAPS to measure average framerates over the run of a benchmark, but rather when it comes to using FRAPS to measure individual frames. FRAPS is at the very start of the rendering pipeline; it’s before the GPU, it’s before the drivers, it’s even before Direct3D and the context queue. As such FRAPS can tell you all about what goes into the rendering pipeline, but FRAPS cannot tell you what comes out of the rendering pipeline.

Adding weight to the whole matter is the fact that FRAPS is one of the few things both AMD and NVIDIA can agree on. In our talks with NVIDIA and in past statements made to the press, NVIDIA dislikes FRAPS being used in this manner for roughly the same reason. The fact that it’s measuring Present calls instead of the time a frame is actually shown to the user impacts them just as well, and muddles the picture when it comes to trying to differentiate themselves from AMD. Again, not to say that NVIDIA thinks FRAPS is a bad tool, but there seems to be a general agreement with AMD’s stance that beyond a certain point it’s the wrong tool for measuring stuttering.

 

[Leadbox]

Pcper.com said that Fraps showing a much better picture for crossfire then their DVI capture card.

How can AMD blame Fraps for their problems when it is not even able to show the real picture?

AMD is still blaming every other entity on this planet instead of themself.

Where is Pcper's follow up to this? Omitting Xfire numbers from their benchmark was a pretty significant move
They also said this:

This is just a preview of what we have planned for our new Frame Rating Capture performance testing method. We have gone through many games with this and the results can vary from looking better than FRAPS to looking much, much worse.

Its been over a month now, lets have at it Pcper!

 

[sontin]

Hmmmmmmmmmmmmmm

nVidia only said that Fraps is not showing their efforts to reduce AFR microstuttering.

They never said that people should not use Fraps to show frames latency.

AMD made it very clear:

AMD’s second problem then is that even when FRAPS is being used to measure frame intervals, due to the issued we’ve mentioned earlier it’s simply not an accurate representation of what the user is seeing. Not only can FRAPS sometimes encounter anomalies that don’t translate to the end of the rendering pipeline, but FRAPS is going to see stuttering that the user cannot. It’s this last bit that is of particular concern to AMD. If FRAPS is saying that AMD cards are having more stuttering – even if the user cannot see it – then are AMD cards worse?

"To much Fraps" - I guess.
Blaming Fraps for showing a problem with their hardware and software is so cute.

That is the exact same argumentation they used for Tessellation:
"Tessellation done right" was nothing else than "less Tessellation".

"Fraps done right" means "don't use Fraps". Problem solved,

 

[Lonyo]

Did they address pcper's CF results, which were based on capturing the output at the display rather than within the render pipeline?

I know AMD acknowledged the frame time issue awhile ago, and I saw improvements pretty quickly which is why I took the leap for the 7950, but I'm a dual+ card kind of guy... What was said about missing/runt frame output for CF?

http://www.anandtech.com/show/6857/amd-stuttering-issues-driver-roadmap-fraps/6

Delaying frames (to prevent runt frames, but adding latency) would be added as a choice in the future.

 

[WhoBeDaPlaya]

generally pointless and inconclusive article. Anandtech stopped using fraps and frametime, but continues to use outdated "average" frame rate as a metric.
Other sites show min, and average fps, or frametimes. Or an fps over time graph for the length of the playthrough / demo (hardocp).
This just smells like a 'business decision'.

wtfwybywb?

 

[sontin]

Where is Pcper's follow up to this? Omitting Xfire numbers from their benchmark was a pretty significant move
They also said this:

Its been over a month now, lets have at it Pcper!

I guess someone heard you:
http://www.pcper.com/reviews/Graphi...ils-Capture-based-Graphics-Performance-Testin

But it looks like that nVidia was involved with it:
Anandtech has the same story:
http://www.anandtech.com/show/6862/fcat-the-evolution-of-frame-interval-benchmarking-part-1

 

[blackened23]

nVidia only said that Fraps is not showing their efforts to reduce AFR microstuttering.

Excellent way to side step the fact that FRAPS measures before anything even hits the render pipeline, before the driver does anything.

 

[Olikan]

nVidia only said that Fraps is not showing their efforts to reduce AFR microstuttering.

They never said that people should not use Fraps to show frames latency.

so.... FRAPS don't show reality afterall...

AMD made it very clear:
"To much Fraps" - I guess.
Blaming Fraps for showing a problem with their hardware and software is so cute.

"Fraps done right" means "don't use Fraps". Problem solved,

LOL, they didn't said that...
they said, Fraps is a shitty tool, but we do have latency problems, and we are working on it...

oh, just in time....anandtech will use FCAT, why? because FRAPS IS CRAP! http://www.anandtech.com/show/6862/fcat-the-evolution-of-frame-interval-benchmarking-part-1

 

[Leadbox]

I guess someone heard you:
http://www.pcper.com/reviews/Graphi...ils-Capture-based-Graphics-Performance-Testin

But it looks like that nVidia was involved with it:
Anandtech has the same story:
http://www.anandtech.com/show/6862/fcat-the-evolution-of-frame-interval-benchmarking-part-1

Thx

 

[sontin]

We know that Fraps is not showing the whole picture.
But if you don't see the difference between that and "don't use Fraps, because it's showing problems with our products" then is guess

"Keep up the strong work there, [blackened23]."

 

[Imouto]

That guy over PCPer is incredibly stupid.

"Enabling V-Sync solves most of the problems on Crossfire setups"
"I don't frigging know how V-Sync works so I'm going to run this bench jumping above and under 60 FPS and blame the cards for time variances between 16ms and 33ms frame times"

 

[Rikard]

We know that Fraps is not showing the whole picture.
But if you don't see the difference between that and "don't use Fraps, because it's showing problems with our products" then is guess

"Keep up the strong work there, [blackened23]."

That is not the message, so do not pretend that it is. Basically they said that despite all its flaws, the problem was so big that even Fraps could detect it. However to insist on that Fraps frame latencies should be used as a measure of perceived user experience is like building a machine where you measure distances with the width of your index finger. We need new tools for that, and not only measure frame intervals, but also the latency.

 

[sontin]

That is not the message, so do not pretend that it is. Basically they said that despite all its flaws, the problem was so big that even Fraps could detect it. However to insist on that Fraps frame latencies should be used as a measure of perceived user experience is like building a machine where you measure distances with the width of your index finger. We need new tools for that, and not only measure frame intervals, but also the latency.

And how you explain that Fraps showing a much better picture for Crossfire than the other method?

Take a look at some of our graphs on those game pages and compare the FRAPS FPS result to the Observed FPS result that calculates an average frame rate per second after removing runts and drops. Clearly the performance of the dual-card configuration is only barely faster than the single card, removing the “scaling” of CrossFire.

http://www.pcper.com/reviews/Graphi...ils-Capture-based-Graphics-Performance-Tes-12

Or maybe AMD means exact that: The "real gaming experience" is much more worse than Fraps is showing. :hmm:

 

[GlacierFreeze]

delete

 

[ICDP]

And how you explain that Fraps showing a much better picture for Crossfire than the other method?

http://www.pcper.com/reviews/Graphi...ils-Capture-based-Graphics-Performance-Tes-12

Or maybe AMD means exact that: The "real gaming experience" is much more worse than Fraps is showing. :hmm:

At no point in reading the AT article did I read that AMD said "don't use FRAPS", all I read was that they felt it didn't give the full accurate picture. Ironically the fact that the PCPER article shows FRAPS in some cases giving a better picture for CF than the newer FCAT method proves this.

If FRAPS was ideal for showing this problem why did Nvidia spend 2+ years developing what they feel to be a more accurate method? FRAPS does show there are problems, so it isn't a useless tool. It just isn't perfect, [H] stated this and AT stated this, they both felt a better method was required and now it's here.

PCPER have shown that in single GPU usage the experience is perfectly fine for AMD. It is CF that has been plagued by stutter problems out of the box. This is something most of us have known and acknowledged for a while now, so it's taking the piss a bit for AMD to say they only notice it now.

Another thing that gives us some solice is that it isn't every game where CF is a sttuery worthless mess out of the box. So at least we know it can be fixed. In most cases using vsync/fps caps and triple buffering does alleviate or even eliminate the issues for CF stutter, but that does not change the fact that out of the box CF is in most cases a stuttery mess. About time AMD acknowledged this. Now we will see if they can fix the problem.

 

[Imouto]

At the end of the day looks like [H] will be the only one to take full advantage of all this fuss. I just hope Ryan Smith won't be dumb enough to run V-Sync disabled tests over 60 FPS or V-Sync enabled below 60 FPS.

 

[KingFatty]

For a car analogy, think of FRAPs for cars, and that FRAPs estimates your speed by looking at how hard to press the gas pedal.

That may work most of the time, but what if you have a headwind, or go downhill, or drive over a dog? All of those will create changes to your speed (could help, could hurt), but your foot may never move from the pedal. Also, you could momentarily wiggle your foot but your speed would still remain fairly constant. So looking at your foot position isn't the best way to determine your car's speed.

FRAPs similarly just looks at the rendering pipeline from the initial part, and doesn't even know what's going on in later stages. This is actually described in the article, and there are even times when you'll get a nice constant output but FRAPS will not tell you that, misleading you into thinking you have stuttering. Other times you can have the reverse effect, where FRAPs will tell you everything is smooth sailing, missing the problems later in the pipeline that you can see but FRAPs misses.

Anyway, wouldn't it be better to use a speed trap or radar gun to check the actual car's speed, instead of looking at the gas pedal's input position? Similarly, it's better to use a different tool than FRAPs to get a better idea of what's going on in the whole rendering pipeline, not just the render input.

 

[railven]

That guy over PCPer is incredibly stupid.

"Enabling V-Sync solves most of the problems on Crossfire setups"
"I don't frigging know how V-Sync works so I'm going to run this bench jumping above and under 60 FPS and blame the cards for time variances between 16ms and 33ms frame times"

Haha, nice!

Welps, glad I use v-sync with my setup, hopefully I don't get the ugly Microstutter yo-hardware-worthless monsta!

 

[lagokc]

By looking at Present calls it’s completely blind to how the GPU is doing any frame pacing, which means it’s currently difficult to see the impact of frame pacing short of a high-speed camera. As further tools are developed that let us analyze the end of the rendering chain, this will allow us to more properly analyze how frame pacing works and what its true impact on the user is.

http://techreport.com/review/24051/geforce-versus-radeon-captured-on-high-speed-video

Looks like someone's had a high speed camera for a few months now. Now all that's left to do is for someone to write software to take the video from the camera and turn it into statistical data.

 

[Rikard]

And how you explain that Fraps showing a much better picture for Crossfire than the other method?

The answer to your question is actually contained in the text you quoted:

Basically they said that despite all its flaws, the problem was so big that even Fraps could detect it.

Or maybe AMD means exact that: The "real gaming experience" is much more worse than Fraps is showing. :hmm:

It could be. It could also not be. That is the point I wish you would understand. Fraps is a blunt instrument, but even blunt instruments have their uses.

 

[SirPauly]

Fraps may not be an ideal tool but still was a good tool to offer some insight and data instead of just offering subjective findings. Fraps was a foundation to build more awareness to create more ideal methods.

 

[BallaTheFeared]

Hmmmmmmmmmmmmmm

And yet it was a driver fix by AMD that reduced the stuttering and cleaned up fraps charts.

Dot > Dot = Conclusion





You have to love AMDs bravado, FRAPS sucks, but FRAPS schooled them and showed they had a problem. They used much better tools which did not show evidently that they had a problem compared to Nvidia.

The logic and reasoning here is similar to some other people I know.

 

[Rikard]

Fraps may not be an ideal tool but still was a good tool to offer some insight and data instead of just offering subjective findings. Fraps was a foundation to build more awareness to create more ideal methods.

+1