[TECH Report] As the second turns: the web digests our game testing methods

[VulgarDisplay]

Flat out "call Shens"

http://forums.anandtech.com/showpost.php?p=34366335&postcount=40



Here you specifically are suggesting it's the games they chose (despite it being a comparison and the issue only occuring on the amd hardware).

http://forums.anandtech.com/showpost.php?p=34366406&postcount=52




Here is someone suggesting they are incompetent

http://forums.anandtech.com/showpost.php?p=34366703&postcount=59



Here you yourself are not asking for more study but are flat out calling the accuracy of the whole thing in to question.

http://forums.anandtech.com/showpost.php?p=34367815&postcount=81





This must be the guy you keep calling no one (i.e. "no one made those claims")

http://forums.anandtech.com/showpost.php?p=34374330&postcount=123





Ok, I got bored after going through 1/3 the other thread. That's just a small sample of what doesn't appear to me to be "claiming more testing is needed" but posts that looked designed to suggest that the whole thing was just wrong.

I don't remember the post from MrK, but the rest of them still don't resemble calling techreport incompetent even when you take them out of context like you just did.

All I see is people saying more testing is needed. Any sane person should be questioning the results. It's all part of the scientific method. I'm sure techreport wouldn't be at all upset to know that people were questioning what they had found because they obviously question it themselves since they have continued to look into the problem.

It more or less appears to be confirmed now that AMD has said they will be fixing it. So they did a great job identifying the problem.

 

[lavaheadache]

You can't play skyrim without vsync. If you use the ini modification to disable it, the physics engine breaks. Although, I thought beth removed the option to disable vsync in the .ini file? Because they acknowledged on their forum a long while ago that disabling vsync screws the game up - they mentioned this not long after release.

Was TR testing with vsync off?

Why did you quote me and then talk about something I didn't mention?

 

[tviceman]

Claiming more testing is needed does not even remotely resemble stating that techreport is incompetent and purposefully misleading. What I said stands, no one made those claims. Stop being dramatic.

http://forums.anandtech.com/showpost.php?p=34374330&postcount=123 <- LOL
http://forums.anandtech.com/showpost.php?p=34329111&postcount=53
http://forums.anandtech.com/showpost.php?p=34326511&postcount=19
http://forums.anandtech.com/showpost.php?p=34326610&postcount=25
http://forums.anandtech.com/showpost.php?p=34366287&postcount=37
http://forums.anandtech.com/showpost.php?p=34367516&postcount=76
http://forums.anandtech.com/showpost.php?p=34378025&postcount=140
http://forums.anandtech.com/showpost.php?p=34378632&postcount=146
http://forums.anandtech.com/showpost.php?p=34380011&postcount=182

That's with 5 minutes of skimming two threads on the same tests / subject. People were trying hard to invalidate the tests in their own minds, and SEVERAL said techreport was being misleading (read the first and last links). Again, it happens when something is reported for / or against either GPU vendor. It doesn't matter. People with their heads so far up their video card's exhaust can't read anything without getting offended at the data or whichever website reported the data.

 

[Ferzerp]

http://forums.anandtech.com/showpost.php?p=34374330&postcount=123 <- LOL
http://forums.anandtech.com/showpost.php?p=34329111&postcount=53
http://forums.anandtech.com/showpost.php?p=34326511&postcount=19
http://forums.anandtech.com/showpost.php?p=34326610&postcount=25
http://forums.anandtech.com/showpost.php?p=34366287&postcount=37
http://forums.anandtech.com/showpost.php?p=34367516&postcount=76
http://forums.anandtech.com/showpost.php?p=34378025&postcount=140
http://forums.anandtech.com/showpost.php?p=34378632&postcount=146
http://forums.anandtech.com/showpost.php?p=34380011&postcount=182

That's with 5 minutes of skimming two threads on the same tests / subject. People were trying hard to invalidate the tests in their own minds, and SEVERAL said techreport was being misleading (read the first and last links).

Nope, those are just people asking for "more testing"

 

[VulgarDisplay]

http://forums.anandtech.com/showpost.php?p=34374330&postcount=123 <- LOL
http://forums.anandtech.com/showpost.php?p=34329111&postcount=53
http://forums.anandtech.com/showpost.php?p=34326511&postcount=19
http://forums.anandtech.com/showpost.php?p=34326610&postcount=25
http://forums.anandtech.com/showpost.php?p=34366287&postcount=37
http://forums.anandtech.com/showpost.php?p=34367516&postcount=76
http://forums.anandtech.com/showpost.php?p=34378025&postcount=140
http://forums.anandtech.com/showpost.php?p=34378632&postcount=146
http://forums.anandtech.com/showpost.php?p=34380011&postcount=182

That's with 5 minutes of skimming two threads on the same tests / subject. People were trying hard to invalidate the tests in their own minds, and SEVERAL said techreport was being misleading (read the first and last links).

Like I said, the one post I did not remember reading. The others may be misquoted because some of them have less than nothing to do with what we are talking about. A few of them question the testing methodology which is what is done to reach a conclusion about any scientific question.

 

[tviceman]

Like I said, the one post I did not remember reading. The others may be misquoted because some of them have less than nothing to do with what we are talking about. A few of them question the testing methodology which is what is done to reach a conclusion about any scientific question.

Apparently you aren't reading the posts nor did you read that I only skimmed for a few short minutes. When people are saying

"fail test" and
"because it would invalidate their nv sponsored subjective feelings" and
"TR's flawed methodology" "So you must break the game by not supported tweaks to see the stuttering" and
"Lol they "feel" it's better using the ultra advanced tools knows as eyeballs, which of course we all know is objective and scientific."

among other things it's pretty obvious they don't want to accept the truth or the possibility of the truth. The attempts to call the tests invalid, flawed, wrong, incompetent, biased, etc. were ALL OVER. Take the time to read threads and not just replies to you or posts you concentrate on.

 

[Ferzerp]

Apparently you aren't reading the posts nor did you read that I only skimmed for a few short minutes. When people are saying

"fail test" and
"because it would invalidate their nv sponsored subjective feelings" and
"TR's flawed methodology" "So you must break the game by not supported tweaks to see the stuttering" and
"Lol they "feel" it's better using the ultra advanced tools knows as eyeballs, which of course we all know is objective and scientific."

among other things it's pretty obvious they don't want to accept the truth or the possibility of the truth. The attempts to call the tests invalid, flawed, wrong, incompetent, biased, etc. were ALL OVER. Take the time to read threads and not just replies to you or posts you concentrate on.

It's obvious is doesn't matter what proof you present. His mind is made up. He really should stop appealing to "science". His stance here is one more of faith than science. The truth isn't important, adhering to his pre-formed notion is.

 

[VulgarDisplay]

Apparently you aren't reading the posts nor did you read that I only skimmed for a few short minutes. When people are saying

"fail test" and
"because it would invalidate their nv sponsored subjective feelings" and
"TR's flawed methodology" "So you must break the game by not supported tweaks to see the stuttering" and
"Lol they "feel" it's better using the ultra advanced tools knows as eyeballs, which of course we all know is objective and scientific."

among other things it's pretty obvious they don't want to accept the truth or the possibility of the truth. The attempts to call the tests invalid, flawed, wrong, incompetent, biased, etc. were ALL OVER. Take the time to read threads and not just replies to you or posts you concentrate on.

Other than the one post from MrK6 I still fail to see how questioning their testing methods is the same as calling them incompetent or intentionally misleading. Also, much of the posts you have pulled out of context are in response to posts from members of this forum who were actually being misleading and had little to do with techreport.

I still think using skyrim for this type of testing is less than stellar because you do have to in fact break the game to test FPS.

Just because I don't agree with their choices for testing their hypothesis doesn't mean I think they are incompetent or misleading. I do now accept that they are right about the problem because AMD themselves have confirmed it through their own testing.

 

[Final8ty]

Some of the quotes were about motives of people and not the test.

It also would of helped if the test was not on a game that is known to have issues with Vsync off, that alone would of saved lots of arguments..

 

[blackened23]

It's obvious is doesn't matter what proof you present. His mind is made up. He really should stop appealing to "science". His stance here is one more of faith than science. The truth isn't important, adhering to his pre-formed notion is.

Whether TR disabled vsync or not is a valid point, because it does break the game to disable it. Whatever though. I sense that this thread is about to go full-on stupid, let the circular amd vs nvidia arguments commence I guess.

 

[Ferzerp]

Some of the quotes were about motives of people and not the test.

It also would of helped if the test was not on a game that is known to have issues with Vsync off, that alone would of saved lots of arguments..

Since the word "science" keeps coming up, I'll pull in "logic". It only takes a single counter example to disprove a claim that says "no one made those claims"

Obviously this has been shown to be false, as examples of people "making those claims" have been provided.

 

[Hitman928]

Other than the one post from MrK6 I still fail to see how questioning their testing methods is the same as calling them incompetent or intentionally misleading. Also, much of the posts you have pulled out of context are in response to posts from members of this forum who were actually being misleading and had little to do with techreport.

Just because I don't agree with their choices for testing their hypothesis doesn't mean I think they are incompetent or misleading. I do now accept that they are right about the problem because AMD themselves have confirmed it through their own testing.

Yeah, you really have to look at the thread to see the posts in context. For instance, many were questioning how their pure FPS performance in Skyrim went down after the 12.11 when everywhere else saw the same or better FPS. Hence, questioning the methodology. It doesn't mean you throw out all the data and good info though. Sure, there may have been a few who were more or less trolling, but on this forum at least people were just being very conservative about things and simply saying, wait for validation.

Heck, I still think the way they use fraps to measure things is insufficient, partly because:

Does it, or does it just fool fraps? When I set a frame limiter to say around 60fps on my 120hz monitor my frame times in fraps may look great, but the output is horrendously rough looking. Perhaps I'm just able to detect frames better than most people.

Maybe the upcoming drivers that address the problem will give me smooth output at 60fps on my 120hz monitor.

Now, it depends on the game for me, but when I tested Max Payne 3, the worst stutter I experienced was with a frame cap, yet the fraps frame times were a solid horizontal line. Other testing (of my own) has shown me that you can't simply rely on fraps to do what they are trying to do (as discussed in one of their own articles). As always, I welcome what they do and I'm glad that it helped AMD know that they have some more tweaking to do, but to me, it is still not reliable enough to base reviews off of. Also why anandtech and [H] and others haven't started using it, it's not reliably consistent.

 

[VulgarDisplay]

Since the word "science" keeps coming up, I'll pull in "logic". It only takes a single counter example to disprove a claim that says "no one made those claims"

Obviously this has been shown to be false, as examples of people "making those claims" have been provided.

I already said that I had never seen the post from MrK6 which confirmed that you were right and I was wrong. You can continue to try and troll me all night if you want.

I was wrong out of ignorance not out of what you obviously seem to think is some type of anti-Techreport/pro AMD bias.

 

[Final8ty]

Since the word "science" keeps coming up, I'll pull in "logic". It only takes a single counter example to disprove a claim that says "no one made those claims"

Obviously this has been shown to be false, as examples of people "making those claims" have been provided.

Regardless of a few individuals the fact still remains that another game should of been used initially where turning off Vsync would not be detrimental to the mechanics of the game and that the game is not dependent on it.

Why demonstrate a situation between cards on a game that most people would not run that game like that to show something else that is only apparent when run like that, people are going to question it.

Its like saying when all 4 wheels are on the vehicles the differences is barely noticeable but when we remove one of the wheels one is noticeably smoother even though you should not be driving with 3 wheels.

There are better ways to go about it, just being right is not the only thing that matters, how you go about it can be just as important.

 

[Will Robinson]

MezzanineSo this will be fixed with a driver revision and the 7950 will continue to roflstomp the 660Ti...business as usual.

lol...awesome :thumbsup:

 

[jackstar7]

Hey ladies, can we focus on the news and not re-live the drama of soap operas past?

I am glad to see that AMD sees the issue and will be working on it. I'm hopeful for maybe 13.2 Betas or something along those lines.

 

[VulgarDisplay]

Hey ladies, can we focus on the news and not re-live the drama of soap operas past?

I am glad to see that AMD sees the issue and will be working on it. I'm hopeful for maybe 13.2 Betas or something along those lines.

Would be nice, and I am curious to see if their improvements to how the GCN cards manage their memory can yield additional performance gains as well as increased smoothness.

 

[BrightCandle]

I wonder if I now go to AMD and claim the difference in sale prices due to their fault if they will give it to me. They have just admittedly liability to a fault, a fault that made their product useless to me.

I feel mighty vindicated at this point, its been just a few days short of a year since I first reported the problem to AMD and this forum. I have put up with hundreds of angry responses and outright ridiculous behaviour in response to the assertion that AMDs 7000 series cards stutter. Here we are finally the truth is out and now we know who did it. AMD drivers......again.

What did it take to get the AMD owners to finally admit their was a fault, nothing short of an AMD statement admitting they did it. But its not just this forum, Ryan from Anandtech called the method into question and would not investigate it. Almost every review out there missed this completely. Clearly most people who use AMD cards have no idea what the frame delivery should look like, it's a proven now. Worse still most of the bench markers don't either.

So those that naysayed, said it wasn't a problem, not only were you wrong but you don't have the competence to see the problem, you just added noise and attitude. Shame on the whole lot of you.

 

[nextJin]

Does it, or does it just fool fraps? When I set a frame limiter to say around 60fps on my 120hz monitor my frame times in fraps may look great, but the output is horrendously rough looking. Perhaps I'm just able to detect frames better than most people.

Maybe the upcoming drivers that address the problem will give me smooth output at 60fps on my 120hz monitor.

It most certainly does not fool fraps, limiting the frame rate to 58-59 and using vsync on a 60hz panel pretty much solves any microstuttering for most people. Using my CFX 7970's it is so bad in Far Cry 3, Skyrim, BF3 that I pretty much have Dxtory as a startup program where EVERYTHING I play is frame limited.

So this news is very much of interest to me lol, I hope they fix it promplty.

 

[VulgarDisplay]

It most certainly does not fool fraps, limiting the frame rate to 58-59 and using vsync on a 60hz panel pretty much solves any microstuttering for most people. Using my CFX 7970's it is so bad in Far Cry 3, Skyrim, BF3 that I pretty much have Dxtory as a startup program where EVERYTHING I play is frame limited.

So this news is very much of interest to me lol, I hope they fix it promplty.

I think you misunderstood what I said. I can have a completely smooth looking frametime graph that is a stuttering mess on my monitor when using a frame limiter.

That is why I think that FRAPS can be fooled.

 

[lavaheadache]

What does fraps have to do with what he is saying regarding a frame limiter?

 

[Final8ty]

I wonder if I now go to AMD and claim the difference in sale prices due to their fault if they will give it to me. They have just admittedly liability to a fault, a fault that made their product useless to me.

I feel mighty vindicated at this point, its been just a few days short of a year since I first reported the problem to AMD and this forum. I have put up with hundreds of angry responses and outright ridiculous behaviour in response to the assertion that AMDs 7000 series cards stutter. Here we are finally the truth is out and now we know who did it. AMD drivers......again.

What did it take to get the AMD owners to finally admit their was a fault, nothing short of an AMD statement admitting they did it. But its not just this forum, Ryan from Anandtech called the method into question and would not investigate it. Almost every review out there missed this completely. Clearly most people who use AMD cards have no idea what the frame delivery should look like, it's a proven now. Worse still most of the bench markers don't either.

So those that naysayed, said it wasn't a problem, not only were you wrong but you don't have the competence to see the problem, you just added noise and attitude. Shame on the whole lot of you.

I'm not point to go into details but you have a habit of over blanketing.

First there are people who don't notice.
Then there people who Vsync all the time.
There are are combination's of set-ups that just play better than others and the use of 3rdparty apps which both brands use.
Then there are people when went from NV to AMD and had a better experience

Not everyone suffered from the 6xx Vsync stutter bug but it was very real, NV issued a fix for many it but there was plenty of initial denial on the NV forum, some of the NV 5XX range also suffered from big latency, no big out cry from NV or AMD users about it.

AMD 7xxx have had some mobo compatibility issues in some configuration, NV has some PCI 3.0 compatibility issues some mobo,s.

NV has had Drivers that kill cards and the 590 that was very sensitive to going pop.

ect... no side is absolute and you can not guarantee a good or bad experience on any side.
.

 

[AnandThenMan]

I wonder if I now go to AMD and claim the difference in sale prices due to their fault if they will give it to me. They have just admittedly liability to a fault, a fault that made their product useless to me.

I feel mighty vindicated at this point, its been just a few days short of a year since I first reported the problem to AMD and this forum. I have put up with hundreds of angry responses and outright ridiculous behaviour in response to the assertion that AMDs 7000 series cards stutter. Here we are finally the truth is out and now we know who did it. AMD drivers......again.

What did it take to get the AMD owners to finally admit their was a fault, nothing short of an AMD statement admitting they did it. But its not just this forum, Ryan from Anandtech called the method into question and would not investigate it. Almost every review out there missed this completely. Clearly most people who use AMD cards have no idea what the frame delivery should look like, it's a proven now. Worse still most of the bench markers don't either.

So those that naysayed, said it wasn't a problem, not only were you wrong but you don't have the competence to see the problem, you just added noise and attitude. Shame on the whole lot of you.

What the hell...

 

[VulgarDisplay]

I wonder if I now go to AMD and claim the difference in sale prices due to their fault if they will give it to me. They have just admittedly liability to a fault, a fault that made their product useless to me.

I feel mighty vindicated at this point, its been just a few days short of a year since I first reported the problem to AMD and this forum. I have put up with hundreds of angry responses and outright ridiculous behaviour in response to the assertion that AMDs 7000 series cards stutter. Here we are finally the truth is out and now we know who did it. AMD drivers......again.

What did it take to get the AMD owners to finally admit their was a fault, nothing short of an AMD statement admitting they did it. But its not just this forum, Ryan from Anandtech called the method into question and would not investigate it. Almost every review out there missed this completely. Clearly most people who use AMD cards have no idea what the frame delivery should look like, it's a proven now. Worse still most of the bench markers don't either.

So those that naysayed, said it wasn't a problem, not only were you wrong but you don't have the competence to see the problem, you just added noise and attitude. Shame on the whole lot of you.

For some reason I can't help but imagine you bashing your head into your keyboard to compose this message. From what I gathered AMD intentionally did this just to piss you off.

 

[Homeles]

For some reason I can't help but imagine you bashing your head into your keyboard to compose this message. From what I gathered AMD intentionally did this just to piss you off.

Just as bad as the "Nvidia bought the 7950 vs. 660 Ti review" conspiracy theory that was thrown around at OCN.