FCAT: The Evolution of Frame Interval Benchmarking

[BrightCandle]

It was independent. Initially ryan from pcper actually paid a developer to write the overlay software but sounds like that didn't work out. Pcper found the capture card and basically needed the software. In the end nvidia gave them the two pieces they didn't have but it appears they tried to do this without them to begin with. They have also called for an open source overlay program.

Its all in the last couple of podcasts there are hints on the history and how it ended up where it is. I think pcper basically had the idea of how to do this, tried to implement it themselves, used their own money and finally ended up with nvidia who did it all in a couple of weeks!

 

[Imouto]

It was independent. Initially ryan from pcper actually paid a developer to write the overlay software but sounds like that didn't work out. Pcper found the capture card and basically needed the software. In the end nvidia gave them the two pieces they didn't have but it appears they tried to do this without them to begin with. They have also called for an open source overlay program.

Its all in the last couple of podcasts there are hints on the history and how it ended up where it is. I think pcper basically had the idea of how to do this, tried to implement it themselves, used their own money and finally ended up with nvidia who did it all in a couple of weeks!

He's been referring to FCAT as "our" since their first preview. When you're just a beta tester you don't say stuff like that. I've played plenty of game betas and referring that game as "mine" or "ours" never crossed my mind.

Now every reviewer will have the FCAT tool, PCPer isn't even mentioned when reviewing the tool and Nvidia didn't give any credit to PCPer, just a link in a blog. Looks like FCAT wasn't as "ours" as PCPer claimed in their first previews.

 

[Keysplayr]

He's been referring to FCAT as "our" since their first preview. When you're just a beta tester you don't say stuff like that. I've played plenty of game betas and referring that game as "mine" or "ours" never crossed my mind.

Now every reviewer will have the FCAT tool, PCPer isn't even mentioned when reviewing the tool and Nvidia didn't give any credit to PCPer, just a link in a blog. Looks like FCAT wasn't as "ours" as PCPer claimed in their first previews.

And you care if PCPerspective receives, or does not receive, any credit for collaborative effort to get FCAT out there from Nvidia because? I mean, I don't care one way or the other. It won't change the results in any way shape or form.

 

[BrightCandle]

None of that obviously changes the fact that FCAT found some serious problems with AMD crossfire and gives us another good data point on smooth delivery of frames.

I think in all honesty smoothness has become the issue it has because of crossfires bug, more and more people have started to care about it and the presence of this bug is why many people saw an issue at all. But I doubt we'll stop here, we will want to know about latency and latency smoothness as well to complete the picture.

 

[USER8000]

PCPer was beta testing FCAT a few months before its wider release. They already had the equipment, so when NVIDIA was ready to try it outside the company it was a good match for them.

They never mentioned that they were using Nvidia software,instead passing off the solution entirely as their own.

Every other website,mentioned it was using software provided by Nvidia. What if the roles were reversed and it was an AMD or Intel application,then?? It would make no difference. They basically lied,by hiding their links to another company.

Moreover,if they were testing for months,why didn't they wait until the NDA was over,for ALL other sites,then??

Why didn't they mention they were beta testing FCAT??

Anyone who read those articles would have thought,the whole solution they developed(including software) was 100% independently developed by them and had nothing to do with AMD,Intel or Nvidia.

People give flak to politicians if they hid affiliations,and the same should go for journalists.

It just shows how much people compartmentalise these things.

They whinge at the integrity of politicians and don't seem to extend to other things too.

None of that obviously changes the fact that FCAT found some serious problems with AMD crossfire and gives us another good data point on smooth delivery of frames.

Their last article also showed that the HD7970 did quite well against the GTX680 in a single card solution.

However,the problem,is that they kept quiet about the link they had with Nvidia.

So,in the future what undeclared links is the site going to have with?? AMD?? Intel?? Samsung?? Apple??

Places,like TR actually are transparent about their links and it is why people seem to put a great weight to what they say.

Even,in their latest review,they don't mention FCAT let alone ANY link with Nvidia:

http://www.pcper.com/reviews/Graphi...eForce-GTX-690-Radeon-HD-7990-HD-7970-CrossFi

It took Nvidia to highlight the link.

 

[omeds]

Who really cares? It's the data that matters.

 

[USER8000]

They never mentioned that they were using Nvidia software,instead passing off the solution entirely as their own.

Every other website,mentioned it was using software provided by Nvidia. What if the roles were reversed and it was an AMD or Intel application,then?? It would make no difference. They basically lied,by hiding their links to another company.

Moreover,if they were testing for months,why didn't they wait until the NDA was over,for ALL other sites,then??

Why didn't they mention they were beta testing FCAT??

Anyone who read those articles would have thought,the whole solution they developed(including software) was 100% independently developed by them and had nothing to do with AMD,Intel or Nvidia.

People give flak to politicians if they hid affiliations,and the same should go for journalists.

It just shows how much people compartmentalise these things.

They whinge at the integrity of politicians and don't seem to extend to other things too.



Their last article also showed that the HD7970 did quite well against the GTX680 in a single card solution.

However,the problem,is that they kept quiet about the link they had with Nvidia.

So,in the future what undeclared links is the site going to have with?? AMD?? Intel?? Samsung?? Apple??

Places,like TR actually are transparent about their links and it is why people seem to put a great weight to what they say.

Even,in their latest review,they don't mention FCAT let alone ANY link with Nvidia:

http://www.pcper.com/reviews/Graphi...eForce-GTX-690-Radeon-HD-7990-HD-7970-CrossFi

It took Nvidia to highlight the link.

Who really cares? It's the data that matters.

So whats your point? Are you saying the data is wrong? If not, who cares?

So,you support politicians hiding their sources of funding or affiliations with companies?? Of course not. More compartmentalisation as usual.

If Nvidia had not mentioned it this week,we would have never known the link at all.

People read independent review sites,expecting the reviews to be done 100% independently of the companies making the products,with any affiliations stated by the sites clearly.

The same goes with politicians,who in theory are meant to represent the views of the people 100% independently of companies,with any affiliations clearly stated. In fact since this has not worked out in practice(sadly),it has meant many democratic nations have implemented laws,to force politicians to do so.

Even,in peer reviewed scientific research,affiliations and funding sources HAVE to be clearly stated.

I don't see anyone complain about that,so why should be any different for journalists??

Would people trust a government news agency/mouth piece or an independently run one more??

Transparency is very important in journalism and indeed in many other areas. People who don't understand the importance of that and do not think transparency is important,contradict themselves,by reading reviews and not just figures released by companies.

They are extremely naive to not realise the lengths companies will go to.

An example is with FOSS patents,which was referenced by many news sites in the Apple vs Samsung patent wars of fail. The chap behind it was somewhat critical of Samsung and Google as a whole,IIRC.

It turns out,the chap behind the site,had links with Oracle and Microsoft:

http://www.theinquirer.net/inquirer/news/2199752/google-and-oracle-publish-lists-of-paid-bloggers
http://www.groklaw.net/article.php?story=20120817151150419
http://news.cnet.com/8301-1035_3-57...gle-may-signal-ceasefire/?tag=mncol;morePosts

Oracle and Google were themselves involved in a court battle at the time. Both were paying bloggers and most were not indicating that they were paid.

Anyway,since it appears to have gone over your head,I will leave it at that,and leave the thread to the discussion about FCAT,otherwise it will be another circular line of bickering which I CBA with.

TR tests FCAT and FRAPs together:

http://techreport.com/review/24553/inside-the-second-with-nvidia-frame-capture-tools

Despite what AMD and Nvidia say,it still seems to hold up reasonably well.

 

[omeds]

So whats your point? Are you saying the data is wrong? If not, who cares?

 

[Grooveriding]

Who really cares? It's the data that matters.

It's not just the data that matters. Many of the people I work with do nothing but R&D, for them lack of complete transparency on any source of funding, grants, equipment or assistance is professional suicide.

There are different motivations for research, you could have the engineers at Intel working on their next node, or a pharmaceutical company developing their next new drug, both multi-billion dollar projects motivated by profit - meaning no need to disclose a thing. Then we have research motivated out of public or consumer interest determining the efficacy of a given subject for the benefit of consumers, much closer to what hardware review sites are doing.

We have PCPER hiding the fact they were working with nvidia, using nvidia's tools and releasing data that put their products in a good light and their competition's in a poor one. No transparency and no disclosure. Even the vernacular they used - 'runt frames' - was directly from nvidia

http://www.tomshardware.co.uk/graphics-card-benchmarking-frame-rate,review-32658-2.html

'affecting your experience: dropped frames and what Nvidia is introducing to us as runt frames. '

None of which we find out from them until this embargo lifted.

Most recently they've tried to review an unreleased card, the 7990, using different hardware and drivers - one of the most asinine things I have ever seen from a review site. Makes them look eager to dismiss an unreleased card, rather than just show data on the hardware they do have. Makes them look partial.

It's consistent poor adherence to a system of integrity in research that lends its self to not being comfortable with the site's motivations or standards. The reality of hardware review sites is that they are not massively funded organizations, they need site hits and advertising revenue. It's an easily crossed line to go from impartiality to being preferential in order to garner those site hits and income.

Fortunately other sites have this tool and are producing reviews using it, otherwise we'd only have a site with a demonstrated lack of values in transparency and affinity towards one of the two hardware vendors over another to get data from. A source I'd rather not have to take information from. I'm looking forward to seeing what AT produces using this tool and if it paints the same picture PCPER is trying to sling around every opportunity they get, even on unreleased hardware they produce reviews on without even having the hardware... :whiste:

 

[omeds]

But is it of any consequence? Do you really care about their opinions or motivations? I sure as hell don't, I only care about the results. I never trust the results from a single source anyway and will always look for confirmation.

 

[chimaxi83]

But is it of any consequence? Do you really care about their opinions or motivations? I sure as hell don't, I only care about the results. I never trust the results from a single source anyway and will always look for confirmation.

I can only imagine how badly some of the regular Nvidia trolls here would crucify and dismiss any site that did this same exact thing, only with AMD instead of Nvidia. You can literally guarantee the hypoctisy.

Anyway, in the end, the data is the data so it doesn't necessarily matter.

 

[omeds]

Anyway, in the end, the data is the data so it doesn't necessarily matter.

Well, exactly.

 

[Will Robinson]

Unless there was a {if GCN d_tect/false flag initiate>} kinda hanky panky going on..:sneaky:

 

[Lonbjerg]

I have too laugh now.
A new tool appears, illustating even better than before than AMD has isssues.

What does the red defense force do?
Try and invalidtate anything about this.

Too bad for them the rest of us live in the real world....where amd has st-st-st-st-stuttering is-is-is-issues, not ma-ma-ma-matter how ha-ha-hard it's trying to b-b-b-be hi-hi-hidden....

 

[willomz]

Unless there was a {if GCN d_tect/false flag initiate>} kinda hanky panky going on..:sneaky:

Firstly, how could FCAT tell? It just takes in video and analyses it, it's not even on the same machine as the graphics hardware.

Secondly, don't you think Anandtech and others have had a look at these scripts to see how they work?

 

[Grooveriding]

I have too laugh now.
A new tool appears, illustating even better than before than AMD has isssues.

What does the red defense force do?
Try and invalidtate anything about this.

Too bad for them the rest of us live in the real world....where amd has st-st-st-st-stuttering is-is-is-issues, not ma-ma-ma-matter how ha-ha-hard it's trying to b-b-b-be hi-hi-hidden....

nice fallacy

 

[willomz]

Which bit is the fallacy? Are you saying that there haven't been loads of people trying to invalidate the FCAT results?

 

[Firestorm007]

nice fallacy

I'll raise you one red herring!

 

[railven]

But is it of any consequence? Do you really care about their opinions or motivations? I sure as hell don't, I only care about the results. I never trust the results from a single source anyway and will always look for confirmation.

Not trying to beat this dead horse, but my state passed a law that private doctors can't have paid dinners with sales personal from pharmacuticals or vendors.

It has nothing to do with if Drug_A actually works better versus Drug_B or in this case, the data present, but more so the credibility of the person presenting it.

If a site starts receiving incentives (paid dinners, pens, note pads, whatever) one can question their credibility to the point of implying their data may be tainted.

Sure, you say the data is all that matters, and you correlate with various site (that's just being smart) but for the dumb few that use only one site, and they have no idea if that data is tainted - that is an issue [more so for the site than for an individual reader.]

Like someone above said, you don't just listen to Rush Limbaugh (sp?) for your political news...but you'd be surprised just how many people do.

 

[Lonbjerg]

Which bit is the fallacy? Are you saying that there haven't been loads of people trying to invalidate the FCAT results?

There is something you shoul know abot these forums.
There are some people that cannot deal with flaws in their favourite "toy".

Anyone poiting out these falws are the real problem.
Any flaw must be sidetracked and hidden behind msoke & mirrors.

Take multi-GPU stuttering.
First a lot people rejexted to notion it....they couldn't see it...so it couldn't be real.

Then some people looked into FRAPS and found some clues.
Those findings were rejected...and mocked.

Then both NVIDIA and AMD acknowlegded that multi-GPU had issues.
But the red defense force was (and is still) determnied not to let facts influence their denial.

Suddenly FCAT appears and added the icing on the cake.
But facts don't interest the red defense force.

The mountain of data and documentation dosn't matter.

It's still a "NVIDIA conspiracy".

Don't believe me...just read this thrêad...and the pattern emerges bright as day ^^

 

[Lonbjerg]

Not trying to beat this dead horse, but my state passed a law that private doctors can't have paid dinners with sales personal from pharmacuticals or vendors.

It has nothing to do with if Drug_A actually works better versus Drug_B or in this case, the data present, but more so the credibility of the person presenting it.

If a site starts receiving incentives (paid dinners, pens, note pads, whatever) one can question their credibility to the point of implying their data may be tainted.

Sure, you say the data is all that matters, and you correlate with various site (that's just being smart) but for the dumb few that use only one site, and they have no idea if that data is tainted - that is an issue [more so for the site than for an individual reader.]

Like someone above said, you don't just listen to Rush Limbaugh (sp?) for your political news...but you'd be surprised just how many people do.

So you deny the data.
Yes/no?

If no...what does the content of you post mattter, but to remove focus from...and let me be blunt now:

THE MAINLY AMD MULTI-GPU MICROSTUTTER?

 

[railven]

so you deny the data.
Yes/no?

If no...what does the content of you post mattter, but to remove focus from...and let me be blunt now:

the mainly amd multi-gpu microstutter?

Edit: Odd, I thought my post got deleted. Anyways, editing to not be a total ass :awe:

 

[Shmee]

Cmon guys, lets keep this thread at least somewhat civil, and stay away from thread crapping and member callouts.

 

[railven]

Cmon guys, lets keep this thread at least somewhat civil, and stay away from thread crapping and member callouts.

My apologies, just responding in kind. Should have just reported it.

 

[Cadarin]

They never mentioned that they were using Nvidia software,instead passing off the solution entirely as their own.

From Article #1, page 1, last paragraph:

NVIDIA's Involvement
You may notice that there is a lot of “my” and “our” in this story while also seeing similar results from other websites being released today. While we have done more than a year’s worth of the testing and development on our own tools to help expedite a lot of this time consuming testing, some of the code base and applications were developed with NVIDIA and thus were distributed to other editors recently.
NVIDIA was responsible for developing the color overlay that sits between the game and DirectX (in the same location of the pipeline as FRAPS essentially) as well as the software extractor that reads the captured video file to generate raw information about the lengths of those bars in an XLS file. Obviously, NVIDIA has a lot to gain from this particular testing methodology: its SLI technology looks much better than AMD’s CrossFire when viewed in this light, highlighting the advantages that SLI’s hardware frame metering bring to the table.
The next question from our readers should then be: are there questions about the programs used for this purpose? After having access to the source code and applications for more than 12 months I can only say that I have parsed through it all innumerable times and I have found nothing that NVIDIA has done that is disingenuous. Even better, we are going to be sharing all of our code from the Perl-based parsing scripts (that generate the data in the graphs you’ll see later from the source XLS file) as well as a couple of examples of the output XLS files.
Not only do we NEED to have these tools vetted by other editors, but we also depend on the community to keep us on our toes as well. When we originally talked with NVIDIA about this project the mindset from the beginning was merely to get the ball rolling and let the open source community and enthusiast gamers look at every aspect of the performance measurement. That is still the goal – with only one minor exception: NVIDIA doesn’t want the source code of the overlay to leak out simply because of some potential patent/liability concerns. Instead, we are hoping to have ANOTHER application built to act as the overlay; it may be something that Beepa and the FRAPS team can help us with.

:whiste: