FCAT: The Evolution of Frame Interval Benchmarking

[bystander36]

And no one is saying Nvidia didn't get involved purely for science, only that it is not inaccurate.

 

[Grooveriding]

Sorry, but you are just looking at the most recent data.

I followed the link you offered that led you to think that there was no involvement and explained how you could see there already was. Do you have another link from earlier where there is no use being made of nvidia's tools ? I know many sites have used fraps frametime measurements before all this. But that is something completely different.

Either way there is really nothing to debate here. Obviously good research protocols are not going to matter to everyone and no one is saying the data is worthless. Not everyone is going to have the same view on the value of context for a researchers aims. I'm sure it can feel like the data is being attacked, I suppose that is the whole nature of the value of transparency when conducting research with value to the public, your results are more impervious to doubt when you don't appear to have taken a side with any of the samples you are working on. More importantly the weight of your opinions and subjective debatable conclusions benefit even more so from good practices.

 

[Lonbjerg]

See...even when AMD has confirmed they have issues, some people will try and muddy the Waters:

http://www.anandtech.com/show/6857/amd-stuttering-issues-driver-roadmap-fraps

Oddly...AMD dosn't point the finger at NVIDIA ^^

 

[VulgarDisplay]

See...even when AMD has confirmed they have issues, some people will try and muddy the Waters:

http://www.anandtech.com/show/6857/amd-stuttering-issues-driver-roadmap-fraps

Oddly...AMD dosn't point the finger at NVIDIA ^^

My god your posts are so relevant to what is actually being discussed. Teach me to post please.

 

[chimaxi83]

See...even when AMD has confirmed they have issues, some people will try and muddy the Waters:

http://www.anandtech.com/show/6857/amd-stuttering-issues-driver-roadmap-fraps

Oddly...AMD dosn't point the finger at NVIDIA ^^

Nice fallacy wrapped up in a red herring :thumbup:

 

[BallaTheFeared]

Is that the new thing, just use any combination of those words even if it makes no sense?

 

[omeds]

red herring, falacy, shill, placebo, fanboy, and mountains out of molehills

 

[BallaTheFeared]

My repertoire has been expanded :thumbsup:

 

[Jaydip]

lol

 

[willomz]

Again incorrect. That colour overlay software was from nvidia, also, the vernacular 'runt frames' is nvidia terminology directly from them to describe a shorter frame.

OK, but PCPer discovered 'runt frames' in the very first article, before they had the overlay or any other FCAT/Nvidia involvement. They might not have called them 'runt frames' but they certainly spotted their existence.

 

[willomz]

you realise that 100 FPS with 40 runts is better then 60 FPS with 0 runts. The reason is all the simulation is still tied to the renderer so you get the simulation run more often.

AMD have stated that they think this is "better" then pacing frames so they come out at an even interval, but in future drivers(june/july) they are going to provide a slider in CCC to change the behaviour.

But now we are getting back to the FRAPS issue.

If you recall AMD said that while FRAPS was showing jittery Present() calls this didn't mean that the display was jittery. But Andrew Lauritzen said even if the spikes in the FRAPS data don't turn into display spikes that "spikes [still] matter because they affect the simulation/backpressure".

"what FRAPS measures is also what the game simulation measures and uses to update the motion. Even if you don't starve the GPU and still end up delivering frames smoothly to the display, you will still notice obvious jitters as the simulation is driven by the underlying signal that FRAPS is measuring. Watch Scott's videos for the example... stuff jumps, then "slows down" to realign with the wall clock. Very distracting."

AMD are caught between a rock and a hard place, they can claim that FRAPS doesn't matter, or they can claim that FCAT doesn't matter, but they can't claim that BOTH don't matter.

 

[GaiaHunter]

OK, but PCPer discovered 'runt frames' in the very first article, before they had the overlay or any other FCAT/Nvidia involvement. They might not have called them 'runt frames' but they certainly spotted their existence.

http://hardforum.com/showpost.php?p=1039741816&postcount=32

Again this comes down to interpretation of the data and what is being compared in any review. If CrossFire was no better than a comparable single GPU card, then real world gaming testing of highest attainable settings and resolutions would expose this easily. Again, it is why we were first in the industry to start this tremendously resource intensive testing years ago.

We have been talking to NVIDIA about frametime testing and collection for a long time now and there is good information back from inside the NVIDIA organization that HardOCP GPU reviews was the catalyst for this coming about. We had the opportunity to help develop the program tools with NVIDIA but chose not to. PCPer has put an incredible about of time and money into this program that we were simply not comfortable with spending. PCPer has done a great deal of needed work on this with NVIDIA, which is commendable, but I am not sure data collection on this front will prove to be the end all be all in GPU reviews. It all still comes down to evaluating the end user gaming experience and how well the hardware allows you to achieve you wants and needs on this front. Frame time data collection will never be something that any users can use at home easily so it will never be more than a review data point. Focus on the user experience will still have the most impact on video card sales making sure the end user gets what he wants and needs.

The NVIDIA involvement might go a bit further back.
__________________

 

[willomz]

We're not really seeing much evidence that CF is worthless, but that it doesn't live up to expectations:

http://techreport.com/review/24553/inside-the-second-with-nvidia-frame-capture-tools/10

Techreport find that CF does increase FPS by 50%, but not the doubling that FRAPS suggests. This ties in with HardOCP findings that CF gives higher frame rates but SLI is smoother.

 

[BrightCandle]

Its kind of annoying actually the AMD cards are clearly better. If you look at the single test results AMD clearly has a smoother delivery of frames and is pretty much always faster even if its not by much. Without a doubt their hardware is better, more memory bandwidth and size means it works a lot better at higher resolutions and higher AA and more raw computation power. AMD always seems to get less out of more hardware. Yet again AMD seems to have shot themselves in the foot with their drivers and general quality issues.

I think what disappoints me most about this whole affair is that AMD seems blind sided by this whole report. This shouldn't be news to them. I would have expected that AMD would use very sophisticated mechanisms to test their cards to confirm that frames were being scanned out correctly and reliably etc. This should never ever have been out in the market for over a year without them even noticing there was a problem. Many people and reviews over the last year have said there was a problem. FCAT isn't really very useful beyond showing up the runt frames issue.

Right now the reviewers are working on the other piece of the puzzle, the latency from beginning to end of the pipeline and working out how to test that. I think its fair to say AMD is going to do worse there as well. Because I doubt they test for it because its become apparent that AMD's quality control really isn't very good. Its a shame because they seem to sell more hardware for less money. AMD's problems have made NVidia very wealthy as they could sell the 104 mid range chips at high end money.

 

[railven]

Thanks for the info on your Crossfire setup. Are you using dynamic vsync in Radeon Pro? If so, that isn't going to reduce tearing (or stuttering) when you drop below 60fps. On the flipside, if you just use standard vsync, you'll actually be running at an effective 30fps, because almost all your averages are below 60fps. You'll notice that FRAPS will report an fps between 30 and 60, but the frametimes indicate what's really going on - it will be locked at 30fps. And when I play Tomb Raider on my HD7870 with vsync, it sometimes drops down to an effective 20fps, the next step below 30fps.

By the way, don't forget to update your sig!

Yeah, I need to update my sig. I've been trying a bunch of things to get things working the way I want and I think I finally nailed WoW last night.

For whatever reason, my CCC profile doesn't stick for IQ settings but the CFX setting sticks. And with Radeon Pro my IQ settings stick but my CFX setting doesn't. I was baffled for the longest time thinking I had a defective card.

So, I got my CFX setup, was happy, turned on WoW and it ran worse than with one card. Forum posts suggested using the AFR-Friendly CFX option, tried that, like magic was getting 180% of one card - was happy. Went back set SSAA and said I'd give Edge Detect a shot (option showed as 24xSSAA.) Loaded game, was having insane FPS assumed all was good, then noticed jaggies. Tinkered more, couldn't get the AA mode to change, was stuck on MSAA.

So tried Radeon Pro with one card first, SSAA+CFAA stuck and I was getting 7-8 FPS. Woof. IQ was AMAZING though. Dropped down to 4xSSAA+CFAA (12xSSAA according to Radeon Pro) 24-25 FPS. Dropped down to 2x better FPS but still I felt something was wrong. I tried alot of the suggestions in another thread, was "happy" with the FPS (40-50) but the stuttering was annoying, and I think it was caused more from FPS dips than Microstuttering though I'm sure the two combined is what I called the time travel effect haha.

So last night I tried CCC again and noticed that setting AFR-Friendly in a CCC Profile about doubled my FPS. So I kept that, then made a Radeon Pro profile with SSAA+CFAA and IT WORKED. I was getting 40-50FPS with the in-game 8x option (so about 24xSSAA) but would tank during any intense action. Set it to 4x (12xSSAA) and it has worked amazingly! 60 FPS every where (except during intense fights), the stutter is similar to one card (ie I notice it, but it doesn't bother me), no tearing (v-sync), I used the internal Frame Limiter seemed to work better than the Radeon Pro one) and, well, I was happy.

Then I crashed HAHA. Something flipped "off" or something, before it crashed I got weird stroble lighting affect and 300 FPS haha. Fun times.

I think I need to be less aggressive on my clocks, I am only running on a 750W I might just upgrade my PSU early just to remove that as the cause of my random crashes. (Both cards are running at 1125/1575 @ 1.175v + 0% Powertune). Temps are acceptable at ~55-60C range.

 

[Will Robinson]

There is something you shoul know abot these forums.
There are some people that cannot deal with flaws in their favourite "toy".

Anyone poiting out these falws are the real problem.
Any flaw must be sidetracked and hidden behind msoke & mirrors.

Take multi-GPU stuttering.
First a lot people rejexted to notion it....they couldn't see it...so it couldn't be real.

Then some people looked into FRAPS and found some clues.
Those findings were rejected...and mocked.

Then both NVIDIA and AMD acknowlegded that multi-GPU had issues.
But the red defense force was (and is still) determnied not to let facts influence their denial.

Suddenly FCAT appears and added the icing on the cake.
But facts don't interest the red defense force.

The mountain of data and documentation dosn't matter.

It's still a "NVIDIA conspiracy".

Don't believe me...just read this thrêad...and the pattern emerges bright as day ^^

Dude.....you need to slow down a bit as it really affects your typing/spelling..
You must be bashing the keyboard so fast it's kinda scary reading it.:biggrin:

 

[SirPauly]

rather than one with poor adherence to good research practices when done for the good of consumers.

Maybe you missed it but TechReport did offer a view and investigation of FCAT as well:


Inside the second with Nvidia's frame capture tools

http://techreport.com/review/24553/inside-the-second-with-nvidia-frame-capture-tool

I don't understand the attacking of PC perspective! If anything, this beyond framerate awareness will improve nVidia and AMD products.

 

[Grooveriding]

Maybe you missed it but TechReport did offer a view and investigation of FCAT as well:


Inside the second with Nvidia's frame capture tools

http://techreport.com/review/24553/inside-the-second-with-nvidia-frame-capture-tool

I don't understand the attacking of PC perspective! If anything, this beyond framerate awareness will improve nVidia and AMD products.

Going by the date of the link you provided, last week, I agree you don't understand.

 

[SirPauly]

OK, but PCPer discovered 'runt frames' in the very first article, before they had the overlay or any other FCAT/Nvidia involvement. They might not have called them 'runt frames' but they certainly spotted their existence.

I guess calling them "runt" frames are not "politically correct" -- some may be overly sensitive to runt!

 

[SirPauly]

Going by the date of the link you provided, last week, I agree you don't understand.

Probably, I do have trouble understanding you!

 

[Homeles]

I don't understand the attacking of PC perspective! If anything, this beyond framerate awareness will improve nVidia and AMD products.

Seriously! The end result of pointing out AMD's flaws is that AMD will be prodded into fixing them. Short term pain for AMD (AMD will have to invest resources into fixing the issue), with long term gains (instilling confidence for AMD in consumers).

But around here, anything that paints AMD in a bad light (even undeniable, objective facts) are wrong or "evil." It's not just here either -- hardware communities in general suffer from this.

 

[bystander36]

It seems all that is talked about is the validity of the new test method, and not the results. It seems this is some sort of denial stage of the process. I guess we'll have to wait a bit before acceptance kicks in. Maybe by then AMD will have fixed the issue, since they have chosen not to deny it, and fix it instead.

 

[railven]

It seems all that is talked about is the validity of the new test method, and not the results. It seems this is some sort of denial stage of the process. I guess we'll have to wait a bit before acceptance kicks in. Maybe by then AMD will have fixed the issue, since they have chosen not to deny it, and fix it instead.

Can you honestly point out someone denying there is an issue?

Actually, can anyone?

 

[SirPauly]

Seriously! The end result of pointing out AMD's flaws is that AMD will be prodded into fixing them. Short term pain for AMD (AMD will have to invest resources into fixing the issue), with long term gains (instilling confidence for AMD in consumers).

But around here, anything that paints AMD in a bad light (even undeniable, objective facts) are wrong or "evil." It's not just here either -- hardware communities in general suffer from this.

To the level of whining about the use of the word runt!

 

[railven]

Seriously! The end result of pointing out AMD's flaws is that AMD will be prodded into fixing them. Short term pain for AMD (AMD will have to invest resources into fixing the issue), with long term gains (instilling confidence for AMD in consumers).

But around here, anything that paints AMD in a bad light (even undeniable, objective facts) are wrong or "evil." It's not just here either -- hardware communities in general suffer from this.

Here-here! The faster them MoFo's fix this, the faster we return to marketshare, driver cheating, shill accussations, and anything but discussing the actual videocards themselves!

All sarcasm aside, I too hope, AMD better fix this else the "their drivers are garbage" monster will return in full force!