FCAT: The Evolution of Frame Interval Benchmarking

Page 6 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

Grooveriding

Diamond Member
Dec 25, 2008
9,147
1,329
126
http://hardforum.com/showpost.php?p=1039741816&postcount=32

Again this comes down to interpretation of the data and what is being compared in any review. If CrossFire was no better than a comparable single GPU card, then real world gaming testing of highest attainable settings and resolutions would expose this easily. Again, it is why we were first in the industry to start this tremendously resource intensive testing years ago.

We have been talking to NVIDIA about frametime testing and collection for a long time now and there is good information back from inside the NVIDIA organization that HardOCP GPU reviews was the catalyst for this coming about. We had the opportunity to help develop the program tools with NVIDIA but chose not to. PCPer has put an incredible about of time and money into this program that we were simply not comfortable with spending. PCPer has done a great deal of needed work on this with NVIDIA, which is commendable, but I am not sure data collection on this front will prove to be the end all be all in GPU reviews. It all still comes down to evaluating the end user gaming experience and how well the hardware allows you to achieve you wants and needs on this front. Frame time data collection will never be something that any users can use at home easily so it will never be more than a review data point. Focus on the user experience will still have the most impact on video card sales making sure the end user gets what he wants and needs.




The NVIDIA involvement might go a bit further back.
__________________


That is a helpful quote. Speaks to my suspicion that nvidia indeed did seek a website out to help get this going, kudos to [h] for not getting involved. I suspect their refusal went beyond merely financial.

I understand not everyone will even be aware of the value of good conduct with regards to research in the consumer interest and that's fine. Obviously some will get personal and feel like the data that validates their personal brand preference is being attacked, rather than see it's the poor adherence to providing impartial reviews and unbiased research practices that is suspect, again that's fine and expected.

Fortunately we have more sites starting to chime in with less vitriol and suspect bias than pcper.
 

railven

Diamond Member
Mar 25, 2010
6,604
561
126
Nice to see mountains out of molehills used less these days!:)

While I love me some drama, as a new Multi-GPU user I figured I could get info from this forum, but reading through all the back-and-forth nonsense, I'm starting to understand why people don't come to AT for technical knowledge. To the few morsels given to me, I am very thankful!

Oh well, it could be worse...
 

blackened23

Diamond Member
Jul 26, 2011
8,548
2
0
Regardless, it is hard to take any benchmark developed specifically by nvidia seriously. Period.

I also find it amusing that PCPer states that crossfire is not better than single. I've used crossfire with the last time being around a year ago - and while it may have issues, it was definitely much better than single card. Conveniently, nvidia "suggests" removing runt frames from the data. Okay, if you say so. NVidia developed benchmark, nvidia suggestion, check, gotcha. Definitely no conflict of interest there. None at all.

Ideally, a 3rd party that isn't being paid by nvidia will develop such tools.
 

SirPauly

Diamond Member
Apr 28, 2009
5,187
1
0
Maybe a third party will. I'm not really too interested in vocal posters attacking the messengers or sites or FCAT but actually interested in what AMD thinks because if it wasn't accurate -- they would or will know and would officially say something to me.
 

blackened23

Diamond Member
Jul 26, 2011
8,548
2
0
Maybe a third party will. I'm not really too interested in vocal posters attacking the messengers or sites or FCAT but actually interested in what AMD thinks because if it wasn't accurate -- they would or will know and would officially say something to me.

Oh. AMD would officially say something to you? That's interesting. Tell us more.

Here's my favorite thing. Some extremely vocal anti AMD GPU posters are suddenly anti-intel in the CPU forum. The guys talking up the Titan here, suddenly in the CPU forum they hate intel CPUs with a passion and can talk about nothing but the Tegra 4 and how it will destroy intel in the future. Interesting stuff! So these guys use their Titans with their Tegra 4s. Lol. Good stuff. I'm not saying there's ulterior motive there though. Nope.

Anyway. Like I said. A lot of people will not accept a benchmark designed by nvidia. There's an obvious conflict of interest there, just as if intel were developing benchmarks for their Atom processor. Would review sites adopt a benchmark designed specifically by intel? Of course not.
 
Last edited:

Homeles

Platinum Member
Dec 9, 2011
2,580
0
0
I also find it amusing that PCPer states that crossfire is not better than single. I've used crossfire with the last time being around a year ago - and while it may have issues, it was definitely much better than single card.
According to the data, the CFX issues are game dependent, which should be a no-brainer.
Maybe a third party will. I'm not really too interested in vocal posters attacking the messengers or sites or FCAT but actually interested in what AMD thinks because if it wasn't accurate -- they would or will know and would officially say something to me.
We already know that AMD will be addressing these issues this summer. They are and have been addressing the frame latency issues found by The Tech Report as well.

It's pretty clear what AMD thinks. People can make vain attempts to discredit the methodology all that they'd like, but the methodology has obviously uncovered something that is a real issue. Therefore, the methodology works.
 
Last edited:

Lonbjerg

Diamond Member
Dec 6, 2009
4,419
0
0
Here-here! The faster them MoFo's fix this, the faster we return to marketshare, driver cheating, shill accussations, and anything but discussing the actual videocards themselves!

All sarcasm aside, I too hope, AMD better fix this else the "their drivers are garbage" monster will return in full force!



Too late...it already have.
And AMD isn't helping by doing 4-5 game fixes...instead of fixing the root cause.

I think what hurt AMD dfenders the most...is that NVIDIA had focus on this for a long time...and whne AMD relaized it was a genuie /faceplam time for AMD.

Now we just neeed to get to the NeXT level...akak stopping trying to smear NVIDIA for AMD's ...lackluster drivers :p
 

VulgarDisplay

Diamond Member
Apr 3, 2009
6,188
2
76
And just like that. Someone pooped in this thread.

The thing is that AMD has their idea of what Multi GPU should provide users and Nvidia has theirs. It's debatable on which is better, but I think most gamers prefer the frame smoothing that Nvidia is doing.

AMD doesn't delay frames or otherwise control how fast the frames are rendering which gives less input lag. Which is better who knows? With my playstyle I would prefer less input lag honestly, but I guess have more input lag and smoother output would be great for single player games.

I know many will say that they can't notice any input lag on SLi with frame metering, but they need to realize that everyone is different.
 

Lonbjerg

Diamond Member
Dec 6, 2009
4,419
0
0
And just like that. Someone pooped in this thread.

The thing is that AMD has their idea of what Multi GPU should provide users and Nvidia has theirs. It's debatable on which is better, but I think most gamers prefer the frame smoothing that Nvidia is doing.

AMD doesn't delay frames or otherwise control how fast the frames are rendering which gives less input lag. Which is better who knows? With my playstyle I would prefer less input lag honestly, but I guess have more input lag and smoother output would be great for single player games.

I know many will say that they can't notice any input lag on SLi with frame metering, but they need to realize that everyone is different.

An what does this hav to do with the

MAINLY AMD MULTI-GPU stutter?

Tha cat is out of the bag...and funny, it confirms one major point of mine:

The useless AT GPU reviews...a single FPS digit...that would NEVER have told the true story...useless!!

But you try and make all the excuses for AMD yoyu can think of...dosn't mattter.

AMD has a major problem....and they need to stop talking and fix it.
And reviers need to include "runt" frames....so AMD dosn't get false high FPS.

But I am sure you have more problems with the people poiting out the problems, the tools showing the problems...than with the prblem itself?

BTW...the bolded part is reported...I am tired of "team red" suing personal attacks all the time, becuase they don't like the facts.
 

bystander36

Diamond Member
Apr 1, 2013
5,154
132
106
So who else thinks Nvidia helping this be known may be a bad thing for them in the long run?

AMD has shown to be clueless about what makes a game feel smooth as shown with the single and multi-GPU stuttering and runt frames. Had Nvidia kept their mouths shut, AMD would likely have never done anything and Nvidia would continue to get reviewers and users alike to recognize, even if subtly, the advantage Nvidia has in this area. Now that AMD knows, it'll be fixed, and Nvidia will no longer have that advantage.

From Nvidia's perspective, was this actually good for them in the long haul?
 
Last edited:

VulgarDisplay

Diamond Member
Apr 3, 2009
6,188
2
76
An what does this hav to do with the

MAINLY AMD MULTI-GPU stutter?

Tha cat is out of the bag...and funny, it confirms one major point of mine:

The useless AT GPU reviews...a single FPS digit...that would NEVER have told the true story...useless!!

But you try and make all the excuses for AMD yoyu can think of...dosn't mattter.

AMD has a major problem....and they need to stop talking and fix it.
And reviers need to include "runt" frames....so AMD dosn't get false high FPS.

But I am sure you have more problems with the people poiting out the problems, the tools showing the problems...than with the prblem itself?

BTW...the bolded part is reported...I am tired of "team red" suing personal attacks all the time, becuase they don't like the facts.

If you took that as a personal attack against yourself that you are obviously aware that you are the problem in every thread you post in.
 

Homeles

Platinum Member
Dec 9, 2011
2,580
0
0
From Nvidia's perspective, was this actually good for them in the long haul?
Possibly. I see it going one of either two ways:

1) AMD gets their act together. We get smooth CFX. People are more confident with CFX, thereby increasing sales for AMD at the expense of Nvidia.

2) Nvidia has just added another ball for AMD to juggle, when they've already been struggling with drivers in the past. I.e., this may add stress to their driver team that's already stretched thin.
If you took that as a personal attack against yourself that you are obviously aware that you are the problem in every thread you post in.
And you consistently blow it up into some big drama-fest.
 

Grooveriding

Diamond Member
Dec 25, 2008
9,147
1,329
126
Oh. AMD would officially say something to you? That's interesting. Tell us more.

Here's my favorite thing. Some extremely vocal anti AMD GPU posters are suddenly anti-intel in the CPU forum. The guys talking up the Titan here, suddenly in the CPU forum they hate intel CPUs with a passion and can talk about nothing but the Tegra 4 and how it will destroy intel in the future. Interesting stuff! So these guys use their Titans with their Tegra 4s. Lol. Good stuff. I'm not saying there's ulterior motive there though. Nope.

Anyway. Like I said. A lot of people will not accept a benchmark designed by nvidia. There's an obvious conflict of interest there, just as if intel were developing benchmarks for their Atom processor. Would review sites adopt a benchmark designed specifically by intel? Of course not.

I think the tool/software package itself are valuable. Now with so many more sites, less partial sites with the tools in hand, if there was anything insidious about it -it would be detected. So that is obviously not the case.

What we will benefit from now are the subjective/objective(in current cases perhaps lack of) opinionated conclusions that are drawn from the results attained from the toolset. This is the main issue with we've seen from pcper, these opinionated and unquantifiable conclusions drawn from results. They've gone fully along with nvidia and agreed with what nvidia has said is worthwhile to signify a frame or not and that is why it's good we have sites like Anandtech now using the tool to give a more objective opinion on what is and is not of value from the data the tool gives.

I'd like to see tests done on legacy hardware as well. Obviously everything we're seeing is all 7XXX and 6XX series hardware. I haven't used CF since having two 5870s, the difference when I went from one to two was obvious. I also noticed a difference in smoothness going from 480s in SLI to 680s, it was probably the first thing I noticed. It would provide more good context to discover if improvements in this metric became drastic in the move from 5XX to 6XX series, which would be a reason to try and promote this improvement all of a sudden.
 

railven

Diamond Member
Mar 25, 2010
6,604
561
126
Is there really a difference? If you cast out the methodology, you cast out the data and issue, or at least leave it to be discovered later.

There is a difference. Again, can you show me where someone is casting out the methodology? I just see stones being thrown because of a few key elements which people should factor in.

I've even seen some of those casting stones saying they'll wait for other review sites (who will be using the same toolset) to report their findings.
 

SirPauly

Diamond Member
Apr 28, 2009
5,187
1
0
The reasons seem pretty clear and obvious: nVidia may have a competitive advantage and desires awareness, and being pro-active about it by helping create tools.

What would you do? Not try to offer this information to the gamer?
 

SirPauly

Diamond Member
Apr 28, 2009
5,187
1
0
From Nvidia's perspective, was this actually good for them in the long haul?

Imho,

Absolutely and goes beyond AMD! With this focus on going beyond frame-rate offers awareness that SLi and multi-GPU is evolving and maturing and potential gamers may find more value based on less limitations than the past.

This builds and improves the SLi, nVidia and GeForce brands.
 

Grooveriding

Diamond Member
Dec 25, 2008
9,147
1,329
126
There is a difference. Again, can you show me where someone is casting out the methodology? I just see stones being thrown because of a few key elements which people should factor in.

I've even seen some of those casting stones saying they'll wait for other review sites (who will be using the same toolset) to report their findings.

Exactly. But as I said, it is inevitable that it could be taken personally or an attack on the raw data if the reviews give you a warm fuzzy. It's the subjective conclusions and accepting nvidia's definition of value to deliver conclusions on material that is not raw data, but open to subjective interpretation, that becomes a grey area with such heavy involvement. Particularly with some of the indications of partiality and unfathomable review practices; see : putting out a review on an unreleased card.. without the card. Having nvidia marketers involved in your nvidia card reviews. etc.

Things will be much more palatable once all the other sites with this tool find themselves in a position of comfort to deliver comprehensive eviews and conclusions from their use.