Should reviews that don't use FCAT be ignored?

[SirPauly]

I don't care what tools say as long as I am having fun playing my games.

You can ignore the data and simply don't care!

Tools and metrics are offered to help gauge how hardware performs and how smooth the experience is. This is like music to my ears this awareness that goes beyond just frame-rate; beyond just subjective; as gamers receive more data for their potential choices, imho!

 

[Imouto]

@ SKYMTL I believe you can detect screen tearing with FCAT as well? how are the frames displayed in such a case?

Looks like you didn't understand how FCAT works at all.

 

[sushiwarrior]

@ SKYMTL I believe you can detect screen tearing with FCAT as well? how are the frames displayed in such a case?

Every frame that FCAT detects is essentially a screen tear...

 

[SKYMTL]

I can't look through the driver overlay code because as far as I know Nvidia hasn't made the source code public. All I can look at is the perl script that analyses the output. The issue with the coloured bars appears to be A. frame compression of some sort that leaves minor changes in colour, I'm assuming in order to fit large frames across the PCI bridge (which doesn't make sense, considering the bridge is much more massive than the frames) or B. slight differences in colour rendering between each GPU (eg. card 1 renders 0x575757, card 2 renders 0x565656 for some reason).

I am right in that FRAPS has nothing to do with FCAT. That is the unarguable truth. FCAT claiming to know what FRAPS would report is OK, except they inflate the results when errors occur, hence making FRAPS look bad when FCAT is actually at fault.

Dropped frames don't really even exist with either AMD or Nvidia cards. The only time dropped frames occur is with this error. Runt frames are an issue, dropped frames just don't really happen from what I have tested.

FCAT is not unbiased because it doesn't produce identical results on AMD and Nvidia cards. In fact, someone at Nvidia knows about the issue but never fixed it (I can tell you what line in the code there is evidence of this). They also borked the formatting of the run.sps file, making the groupings of 15 dropped frames almost impossible to spot until you fix the formatting....

My point isn't necessarily that FCAT is biased, however, it's that FCAT is a badly coded tool and shouldn't be taken as gospel.

That's more like it.

And you are correct with nearly every single point. However, it should be added that the extractor tool has a built in tolerance for slight color discrepancies so minor differences won't be picked up as issues. This is also why, when placed in sequence, the each of the 16 colors is notably different from the ones directly preceding and succeeding it.

Which version of the tool are you using? From my understanding, the issues with the run.sps file were fixed in the April 18th version.

 

[Jaydip]

Looks like you didn't understand how FCAT works at all.

Every frame that FCAT detects is essentially a screen tear...

I am not asking that at all.Screen tearing is not easy to detect in all games and how to determine who is the worst offender between them.

 

[sushiwarrior]

That's more like it.

And you are correct with nearly every single point. However, it should be added that the extractor tool has a built in tolerance for slight color discrepancies so minor differences won't be picked up as issues. This is also why, when placed in sequence, the each of the 16 colors is notably different from the ones directly preceding and succeeding it.

Which version of the tool are you using? From my understanding, the issues with the run.sps file were fixed in the April 18th version.

Yes, the colour discrepancy shouldn't be an issue (although their "close enough" method is... very odd to say the least ).

The colours aren't necessarily any different, you can have aqua and then teal or navy blue and then blue.

I'm currently running v1.2, I also have 1.6 but it doesn't change anything with the issue at hand.

I am not asking that at all.Screen tearing is not easy to detect in all games and how to determine who is the worst offender between them.

Do you mean screen tearing with Vsync enabled? Or screen tearing in general? Without Vsync, you will get screen tearing. Always. You can't be "better" than the other company at screen tearing, it just happens when Vsync isn't on.

 

[Maximilian]

Ho hum....

http://en.wikipedia.org/wiki/FCAT

Still clueless.

 

[Jaydip]

Yes, the colour discrepancy shouldn't be an issue (although their "close enough" method is... very odd to say the least ).

The colours aren't necessarily any different, you can have aqua and then teal or navy blue and then blue.

I'm currently running v1.2, I also have 1.6 but it doesn't change anything with the issue at hand.



Do you mean screen tearing with Vsync enabled? Or screen tearing in general? Without Vsync, you will get screen tearing. Always. You can't be "better" than the other company at screen tearing, it just happens when Vsync isn't on.

I am not asking which company is better at screen tearing rather which games show it the most.

 

[Imouto]

You're just deflecting the fact that you don't know how FCAT works or you wouldn't ask such a question.

 

[Leadbox]

I am not asking which company is better at screen tearing rather which games show it the most.

Any game were you are panning left and right the most has got to be the answer surely

 

[sushiwarrior]

I am not asking which company is better at screen tearing rather which games show it the most.

It's an issue in any game without Vsync on.

 

[Final8ty]

It's unfortunate my first post here on the AT forums has to be this but here it goes.

Welcome
At least it seems my link on the other forum was useful after all.

Some good contributions SKYMTL :thumbsup:

 

[ocre]

Nobody knows about it because nobody who does FCAT testing cares really. And they kind of shouldn't, with the current drivers it's unarguably broken, but this issue also occurs with the prototype drivers. I can't link to other data because I'm the one who found it.


I can't look through the driver overlay code because as far as I know Nvidia hasn't made the source code public. All I can look at is the perl script that analyses the output. The issue with the coloured bars appears to be A. frame compression of some sort that leaves minor changes in colour, I'm assuming in order to fit large frames across the PCI bridge (which doesn't make sense, considering the bridge is much more massive than the frames) or B. slight differences in colour rendering between each GPU (eg. card 1 renders 0x575757, card 2 renders 0x565656 for some reason).

Dropped frames don't really even exist with either AMD or Nvidia cards. The only time dropped frames occur is with this error. Runt frames are an issue, dropped frames just don't really happen from what I have tested.

FCAT is not unbiased because it doesn't produce identical results on AMD and Nvidia cards. In fact, someone at Nvidia knows about the issue but never fixed it (I can tell you what line in the code there is evidence of this). They also borked the formatting of the run.sps file, making the groupings of 15 dropped frames almost impossible to spot until you fix the formatting....

i would like to see this code

 

[Irishwhitey]

I would take anything that comes out of Tom Peterson's mouth with a grain of salt as he is a bought and payed for nivida shill.

 

[sushiwarrior]

Anyone with the FCAT analysis tool v1.2 can look at lines 569-572 of fcat.pl. I'd rather keep this privy to anyone with access to the tool rather than the whole wide world. SKYMTL should be able to vouch for it to an extent.

Similar lines in v1.6 are 585-587.

 

[BallaTheFeared]

Anyone can download FCAT, you aren't privy to anything special go ahead and snip it.

Nobody claimed FCAT was FRAPS, you claimed FCAT was used for the FRAPS report.

FCAT doesn't run on the cards, it's not even hooking or creating an overlay.


Originally Nvidia provided the overlay tool since since nobody had the tool, but that part, literally the most important part, was handed off to third party members. The guy who does MSI AB picked it up, and if you have any recent beta you have the color overlay.

That overlay is already 3rd party as it was the only part of the program that matters FCAT has no clue who is who, it doesn't even run nor does it have a FRAPS function where it hooks the .DLL.

The overlay is independent of FCAT and is captured on a capture card which is then fed through FCAT. FCAT has no idea which video card was used.

You still have not linked a credible source stating there is a bug in FCAT which renders it ineffectual, nor would you ever with your current approach. The part that matters is the actual frame output less the runt and dropped frames, not some minor spikes in FPS as you state when reporting the front end calls.

 

[sushiwarrior]

Anyone can download FCAT, you aren't privy to anything special go ahead and snip it.

Nobody claimed FCAT was FRAPS, you claimed FCAT was used for the FRAPS report.

FCAT doesn't run on the cards, it's not even hooking or creating an overlay.

FCAT claims a "FRAPS" number that has nothing to do with FRAPS, and reports number FRAPS would never report. FCAT manufactures a guess as to what FRAPS would have reported, and claims it is the actual number. On AMD cards, this is an incorrect guess.

Originally Nvidia provided the overlay tool since since nobody had the tool, but that part, literally the most important part, was handed off to third party members. The guy who does MSI AB picked it up, and if you have any recent beta you have the color overlay.

That overlay is already 3rd party as it was the only part of the program that matters FCAT has no clue who is who, it doesn't even run nor does it have a FRAPS function where it hooks the .DLL.

The overlay is independent of FCAT and is captured on a capture card which is then fed through FCAT. FCAT has no idea which video card was used.

I'm talking about the perl script.

You still have not linked a credible source stating there is a bug in FCAT which renders it ineffectual, nor would you ever with your current approach. The part that matters is the actual frame output less the runt and dropped frames, not some minor spikes in FPS as you state when reporting the front end calls.

Oh I'm sorry, 15 or 30 frame spikes are MINOR? Falsely claiming the card has half the FPS it is reporting is OKAY with you? Are you clueless? The results are WRONG. I don't care what you think, WRONG results don't belong in a review.

Again, go look at lines 569-572 of the code. If you can figure out what the issue is, good for you.

 

[BallaTheFeared]

FCAT claims a "FRAPS" number that has nothing to do with FRAPS, and reports number FRAPS would never report. FCAT manufactures a guess as to what FRAPS would have reported, and claims it is the actual number. On AMD cards, this is an incorrect guess.

Simple enough for you? You really have a problem with wanting to be right.

Sure it's simple enough for me.

It's also completely and utterly meaningless. Nobody cares what FRAPS may have reported when doing FCAT, if that's even true. What people care about are the actual frames less the runt and dropped.

Do you have any proof to show that FCAT is the one creating the FRAPS numbers, and not the program FRAPS itself? Any at all, that would be immensely helpful in your argument if you could actually produce something called proof for a change.

As far as I'm concerned reviewers used FRAPS for FRAPS reports, not FCAT. TechReport even did a direct comparison of the two, though you claim FCAT is creating the FRAPS data, but isn't that a lie and disingenuous to FRAPS to say FRAPS but not actually use FRAPS?

You see seem the have a problem with making things up to support your ignominious arguments.

Yeah and their overlay tool sucked ass Glad there are other ways to use it. Did I ever claim there was an issue with the overlay? I'm talking about the perl script, please pay attention.

How is there a problem with the script if the script is using the overlay to create data and the script has no idea which vendor is which as they're both exactly the same?

You would have to know enough to say the overlay itself is the problem, which you aren't because you don't know enough.

I am paying attention, you're just making things up as you go however. You can't even link a single credible source that backs up anything you say. Something this big would be widely reported in every review of FCAT or any followup or corrections made to every single review once it was public knowledge.

If you can't find a single link to validate your claims than I will continue to assume you're making things up as you go which seems like a pretty good stance currently.

Oh I'm sorry, 15 or 30 frame spikes are MINOR? Falsely claiming the card has half the FPS it is reporting is OKAY with you? Are you clueless? The results are WRONG. I don't care what you think, WRONG results don't belong in a review.

Yes since the point is Actual FPS, not what AMD is losing. You seem to be taking this as a personal attack on AMD and therefore yourself.

I'm ready to admit I am 100% wrong, all you need to do is link a few credible review sites like pcper, anand, techreport saying what you're saying. That shouldn't be too hard for you, presuming such information exists in the first place of course.

Again, go look at lines 569-572 of the code. If you can figure out what the issue is, good for you.

Why don't you just snip it for everyone here so everyone doesn't need to download a free program just to view some lines of code which you could easily snip out and post here.

Are you just that lazy or what?

 

[sushiwarrior]

Sure it's simple enough for me.

It's also completely and utterly meaningless. Nobody cares what FRAPS may have reported when doing FCAT, if that's even true. What people care about are the actual frames less the runt and dropped.

Do you have any proof to show that FCAT is the one creating the FRAPS numbers, and not the program FRAPS itself? Any at all, that would be immensely helpful in your argument if you could actually produce something called proof for a change.

As far as I'm concerned reviewers used FRAPS for FRAPS reports, not FCAT. TechReport even did a direct comparison of the two, though you claim FCAT is creating the FRAPS data, but isn't that a lie and disingenuous to FRAPS to say FRAPS but not actually use FRAPS?

It is very impractical to coordinate FCAT data with FRAPS data. They are on seperate computers.

Like SKY said, FCAT gives you a "FRAPS" number which it infers but doesn't actually use FRAPS. Listen to him, for the love of god.

How is there a problem with the script if the script is using the overlay to create data and the script has no idea which vendor is which as they're both exactly the same?

You would have to know enough to say the overlay itself is the problem, which you aren't because you don't know enough.

I am paying attention, you're just making things up as you go however. You can't even link a single credible source that backs up anything you say. Something this big would be widely reported in every review of FCAT or any followup or corrections made to every single review once it was public knowledge.

If you can't find a single link to validate your claims than I will continue to assume you're making things up as you go which seems like a pretty good stance currently.

Here's where it gets complicated - evidently, something wrong is happening on the AMD cards, as there are odd little blips showing up (2 lines at the top of the screen, too small to notice). Something is wrong, but not "15 frames dropped" wrong.

Yes since the point is Actual FPS, not what AMD is losing. You seem to be taking this as a personal attack on AMD and therefore yourself.

I'm ready to admit I am 100% wrong, all you need to do is link a few credible review sites like pcper, anand, techreport saying what you're saying. That shouldn't be too hard for you, presuming such information exists in the first place of course.


Why don't you just snip it for everyone here so everyone doesn't need to download a free program just to view some lines of code which you could easily snip out and post here.

Are you just that lazy or what?

# If very short number of lines ignore during intial color detection
	if ($lines < 2) {
	    return (17, "Unknown");
	}

They put this code in the wrong part of the program (so it does nothing useful). The point isn't actual FPS, the point is "FRAPS FPS", which FCAT is incorrectly reporting. I don't think you understand, even if this isn't a big deal to you, it's the simple fact that the tool does not report 100% accurate data. In a world where a few percent is the difference between cards winning or losing, when did 15-30FPS error margins become okay? I don't care if its AMD or Nvidia who has the problem, the fact is the tool is wrong and needs to be fixed.

Pcper, anand, techreport don't know about the issue because they don't read into the data. It took lots of digging through output logs and frame colour data to find this error, they obviously haven't made the connection yet.

 

[BallaTheFeared]

Pcper, anand, techreport don't know about the issue because they don't read into the data. It took lots of digging through output logs and frame colour data to find this error, they obviously haven't made the connection yet.

So nobody but you has any idea what you're talking about :thumbsup:

Instead of worrying about FCAT, shouldn't you be solving the worlds biggest problems, like carbon emissions?



Trolled again /\

 

[bystander36]

Why would it be so difficult to run a benchmark with FRAPS running, and the output going to a capture card, then analyze the capture card info with FCAT. You get both sets of data at the same time with the tools they claim to have used.

According to their test setup, they are doing Fraps runs.
http://www.tomshardware.com/reviews/graphics-card-benchmarking-frame-rate,3466-3.html

Since a capture card is designed to behave like a monitor, there is no reason both cannot be done at the same time.

EDIT: As it turns out, they make 2 separate runs. This is from the reviewer of that article in the comments section:

There are a number of factors. Primarily, we can't run FCAT and FRAPS on the exact same run and there's bound to be a bit of variance per run. Having said that, the variance in Batman is more than it should be. Unfortunately, we have no way of being 100% sure what the deal is, we can only show our results.

 

[BallaTheFeared]

Look at what he's saying...

FCAT is producing FRAPS data, which isn't even possible since it analyzes the bar "code" on the side of an AVI file.

FRAPS is taking data from the front end, prior to the entire GPU and Display, whereas FCAT is taking information from a overlay in a AVI file.

Clearly he has no flipping clue what he's talking about, there should be no question at this point.


Also "A bug that effects AMD" the code is "If very short number of lines ignore during initial color detection"

It's just golden at this point.

 

[bystander36]

Look at what he's saying...

FCAT is producing FRAPS data, which isn't even possible since it analyzes the bar "code" on the side of an AVI file.

FRAPS is taking data from the front end, prior to the entire GPU and Display, whereas FCAT is taking information from a overlay in a AVI file.

Clearly he has no flipping clue what he's talking about, there should be no question at this point.


Also "A bug that effects AMD" the code is "If very short number of lines ignore during initial color detection"

It's just golden at this point.

It is clear by what was said on Tom's Hardware review, that they are running two separate runs to get Fraps data and FCAT data.

So ya, sushiwarrior's concern is wrong.

 

[sushiwarrior]

Look at what he's saying...

FCAT is producing FRAPS data, which isn't even possible since it analyzes the bar "code" on the side of an AVI file.

FRAPS is taking data from the front end, prior to the entire GPU and Display, whereas FCAT is taking information from a overlay in a AVI file.

Clearly he has no flipping clue what he's talking about, there should be no question at this point.


Also "A bug that effects AMD" the code is "If very short number of lines ignore during initial color detection"

It's just golden at this point.

Did you listen to any of my posts? What I am saying is FCAT is CLAIMING a "FRAPS" number that doesn't, in fact, have ANYTHING to do with FRAPS. It's like you speak french As I said earlier

As SKY confirms, FCAT is run on a seperate computer and simply "claims" a FRAPS number (which has NOTHING to do with FRAPS, as FCAT can't see DX Present() calls, but instead infers them)

FCAT doesn't involve FRAPS at all. They are 2 seperate, completely different things. Hence FCAT shouldn't make any claims about "this is what FRAPS would show" because those numbers aren't accurate. I can tell you why they aren't, but at the end of the day, what matters is what FCAT claims FRAPS would say and what FRAPS actually does say are very different things on AMD crossfire setups.

It is clear by what was said on Tom's Hardware review, that they are running two separate runs to get Fraps data and FCAT data.

So ya, sushiwarrior's concern is wrong.

And like they said, it didn't work and wasn't accurate. Other sites don't run FRAPS and FCAT, it's inaccurate.

 

[bystander36]

Do you have any examples of sites using the FRAPS number from FCAT or are you assuming this?