[TECH Report] As the second turns: the web digests our game testing methods

[Ferzerp]

Those are frame times, unless you mean the raw data . . .

Yes, it's radeon pro.

I've never heard/seen anyone claim that using a frame limiter caused frame skipping. . .

If you have to limit your FPS to 40 to get rid of stuttering, I don't really think it unreasonable to never report any benchmark framerate score above 40 fps, since that's all you can do without stutter. Pretty much, if that's needed, then you'd better be benchmarking with it on as well or you're being misleading.

 

[Hitman928]

If you have to limit your FPS to 40 to get rid of stuttering, I don't really think it unreasonable to never report any benchmark framerate score above 40 fps, since that's all you can do without stutter. Pretty much, if that's needed, then you'd better be benchmarking with it on as well or you're being misleading.

You're commenting on something without understanding the context. I don't need a 40fps cap for a smooth frame time graph. I could turn on vsync and get very smooth frame times @60 fps, no problem, or cap it at 60 fps (maybe 55 at worst) and get a horizontal line. I did it at 40 fps to have a clear distinction from vsync.

 

[MTDEW]

If you have to limit your FPS to 40 to get rid of stuttering, I don't really think it unreasonable to never report any benchmark framerate score above 40 fps, since that's all you can do without stutter. Pretty much, if that's needed, then you'd better be benchmarking with it on as well or you're being misleading.

Finally, someone who "gets it". :thumbsup:

Everyone is like, here buy this 7xxx series card, look how fast it is in the benchmarks i just linked.

Then person buys it, posts a few days later about stutter.
And everyone says "just use a frame-limiter".

And that person is supposed to be too stupid to realize that now he's not getting near the performance the benchmarks claimed or others said he would with the card because he's forced to limit the cards performance with a frame-limiter to achieve smooth gameplay.

 

[Hitman928]

Finally, someone who "gets it". :thumbsup:

Everyone is like, here buy this 7xxx series card, look how fast it is in the benchmarks i just linked.

Then person buys it, posts a few days later about stutter.
And everyone says "just use a frame-limiter".

And that person is supposed to be too stupid to realize that now he's not getting near the performance the benchmarks claimed or others said he would with the card because he's forced to limit the cards performance with a frame-limiter to achieve smooth gameplay.

Again, pay attention to context as this has nothing to do with the thread right now.

 

[3DVagabond]

It's amazing to me that after AMD has stated that there is an issue, there is post after post claiming that there isn't (and that's putting what many of you have posted in the most generous of lights).

What's amazing is how far people have run with what Baumann actually said. Tells me that people really don't know what he said, or are purposely being misleading in interpreting it.

I think it's important to reiterate what Dave Baumann (AMD) actually said about this:

Thanks for posting this.

Quote:
Originally Posted by DaveBaumann
Curious, where did you read any of that?

First off, I mentioned that we have a memory management rewrite underway for orthogonal reasons and it was at a point where we could just try it and see if it makes any improvements; it transpires that in some cases it has helped improve things. This has raised the priority to trying to get this through earlier; that doesn't mean that we "weren't going to bother", otherwise what would have been the point of investing the development time in the first place? Again, the memory manager rewrite was being done for other reasons, and in fact there was no indications from anywhere (including TR's prior reviews) was this an area that needed to be addressed.

Secondly, the memory manager change is not "magic bullet". The quote Scott originally highlighted from me was that these types of things can occur from multiple different areas of the driver - for instance the BL2 improvement primarily comes from resizing a buffer, not related to generic the memory management at all.
Quote:
This is not due to solely chasing average FPS. One of the software engineers looking into this contacted me to point out that for at least two of the titles where we've made improvements the average FPS should be improved, maybe by a reasonably large degree in some cases.
Quote:
I can't say that in the games I've played they have been noticeable, likewise I do not see end user feedback for this type of issue (except, maybe outside of a few known problem titles).
Quote:
As far as I had seen, TR's prior reviews using this method didn't highlight anything particularly untoward.

Likewise, we all use our GPU's and, for instance, I've put may many hours in to Borderlands 2 (207 in fact :gasp:!) on other an HD 7970 Ghz Ed or a HD 7870 and I genuinely can't notice any affects on the gameplay related to this to have even flagged it up as something I can perceive. However, we take it seriously and this has occupied quite a lot of architectures and software engineers time over the past few weeks to see where this is coming from and whether it can be improved.
Quote:
Yes, I think this is the area that needs further investigation. It is interesting that TR has does this type of analysis, and flagged an "issue" but to all intents and purposes I have no frame of reference that indicates that this is an "issue" that affected people in most of the cases; as mentioned nobody has really reported this an issue that affects their gameplay.

The bolded part is what he actually said. The rest is him explaining away what others are claiming he said. Nobody is saying the frame latency doesn't exist. The issue is not as big as some are trying to make it out to be, though. Thus the "mountains out of molehills" statement from Ryan.

Unfortunately we got a big "headline" but not much scientific procedure to go along with it. It's obviously particular games, and in some cases particular spots in games. Is it with particular settings? For example are they forcing something through drivers? Is it with a certain type of AO? We don't know.

 

[AnandThenMan]

Which member? And show us the quote.

Oh sorry you only admitted a strong Nvidia bias, so I'll only consider the vast majority of your posts as having an agenda.

What do you mean? I have a strong bias towards Nvidia.

It's like they discover this anew every conversation we have.

 

[Keysplayr]

Oh sorry you only admitted a strong Nvidia bias, so I'll only consider the vast majority of your posts as having an agenda.

That's better. You always did have the gift of exaggeration. So, i'll only consider the vast majority of your posts as inaccurate exaggerations meant to mislead. So we have an understanding then. :thumbsup:

 

[AnandThenMan]

That's better. You always did have the gift of exaggeration. So, i'll only consider the vast majority of your posts as inaccurate exaggerations meant to mislead. So we have an understanding then. :thumbsup:

Now who's exaggerating. I never said I have a strong bias to any brand. Not to mention I don't get paid to pump Nvidia's tires.

 

[Will Robinson]

The sweet irony is that he has turned far more people off buying an NVDA graphics card than just about any other single source on the Net.
AMD should probably be sending him free hardware as a thank you gesture.:thumbsup:

 

[Keysplayr]

Now who's exaggerating. I never said I have a strong bias to any brand. Not to mention I don't get paid to pump Nvidia's tires.

This is too funny. Please fight this infatuation with me. Get back to the topic and lets not make this all about me, your arch enemy. K? Thanks.

And Will. I gotta luv ya. Don't like what the man is saying, so go after the man himself. Not his arguments.

So, why did you post? A one liner zing toward me? Do you feel better? I bet you do big guy. Doesn't take much to sate you does it. Just a one liner, maybe two here and there. Very hollow boy.

 

[VulgarDisplay]

AMD has had security issues too, such as their lack of ASLR support in EMET earlier this year.

http://www.anandtech.com/show/5966/uscert-takes-amd-to-task-on-driver-security

Security vulnerability issues happen from time to time for every single software and hardware company in existence. That is just how computers work — I'm glad that you've given me the opportunity to enlighten you.

Citing one issue by its lonesome and implying that the company with that is less competent than another is completely inane. Like the people pointing out Nvidia shimmering issues that happened a good 6 years ago. What else has happened since then? If you are incapable of producing a series of failures over a period of time, your argument is not worth the pathetic amount of storage space it takes up on these servers.

I said the same thing about 2 posts later. I'm glad you enlightened me though. My whole point still remains that both sides have driver issues. The fact that this one is such a MASSIVE issue at a time when nvidia doesn't maintain the outright single GPU performance crown is extremely suspect. All I was doing was responding in kind to the people trolling his thread to cause drama, but I don't think you "got it."

Truth be told I've had more driver issues with Nvidia hardware. The only reason why is because I've used more nvidia GPU's over a longer period of time than I have AMD cards. Both companies have been great while I've used them, so I guess I'm just lucky.

 

[HurleyBird]

The sweet irony is that he has turned far more people off buying an NVDA graphics card than just about any other single source on the Net.
AMD should probably be sending him free hardware as a thank you gesture.:thumbsup:

I don't think he's turned people off... perhaps individually, but not on the whole. The fact that he wears his bias on his sleeve (focus group) probably helps in that regard.

On the other hand, I have no doubt a lot of Nvidia fanboys (such as one guy with GTX 470's in SLI) have turned people off immensely.

And although I haven't had a problem with them historically, the pro-AMD crowd is starting to turn me off in this thread and it's predecessor.

 

[notty22]

The fact that this one is such a MASSIVE issue at a time when nvidia doesn't maintain the outright single GPU performance crown is extremely suspect

What does this even mean? What does one have to do with the other?
It was a rather obscure , potential opening, on a update web service. Not sure what that has to do with GPU performance ?

 

[chimaxi83]

What does this even mean? What does one have to do with the other?
It was a rather obscure , potential opening, on a update web service. Not sure what that has to do with GPU performance ?

What he meant is that it's funny how the Nvidia zombies are foaming at the mouth over this frame time issue now, when Nvidia is not in the performance lead, because in their minds, it somehow makes Nvidia look better. Mountains out of molehills, indeed. For all of you AMD users affected by this, hope your fix comes soon :thumbup:

 

[f1sherman]

I said the same thing about 2 posts later. I'm glad you enlightened me though. My whole point still remains that both sides have driver issues.

Truth be told I've had more driver issues with Nvidia hardware. The only reason why is because I've used more nvidia GPU's over a longer period of time than I have AMD cards. Both companies have been great while I've used them, so I guess I'm just lucky.

Why do you want to talk about everything but the topic itself. Why.... I don't understaind.

This thread is no more about your hardware luck, or about Nvidia driver issues than it's about feeding the poor of sub Saharan Africa.

 

[Keysplayr]

What he meant is that it's funny how the Nvidia zombies are foaming at the mouth over this frame time issue now, when Nvidia is not in the performance lead, because in their minds, it somehow makes Nvidia look better. Mountains out of molehills, indeed. For all of you AMD users affected by this, hope your fix comes soon :thumbup:

He's wrong. It just so happens that AMD has the fps lead currently. Whether or not Nvidia has that lead or not has little to do with smooth gameplay. Right?
So what is he talking about then because this has NOTHING to do with framerates. It has everything to do with frametimes. And AFAICT, Nvidia doesn't need more fps to offer a smoother performance over it's higher fps competitor. Interesting, right? Yeah, I thought it was interesting also, hence all of this discussion and more review sites jumping on board found it all very interesting to.

Oh, and it's zAMDies btw.

 

[Keysplayr]

What does this even mean? What does one have to do with the other?
It was a rather obscure , potential opening, on a update web service. Not sure what that has to do with GPU performance ?

AHHHHHH I get it. Because they have fps on the brain and AMD has the fps lead currently, they must truly believe that everybody else in the world supporting this frame time movement, is pissed about that. Pretty shallow huh?
But Vulgar Display has actually said that he likes to get a rise out of people, so I wouldn't buy into his posts as so much believing what he is saying, and more so that he just might be trying to get that rise out of ya. Don't give him the satisfaction bro.

 

[notty22]

What he meant is that it's funny how the Nvidia zombies are foaming at the mouth over this frame time issue now, when Nvidia is not in the performance lead, because in their minds, it somehow makes Nvidia look better. Mountains out of molehills, indeed. For all of you AMD users affected by this, hope your fix comes soon :thumbup:

I see. I remember the same poster using (along the lines)MASSIVE exploit, when describing the security issue, with the last Nvidia driver. The references were unclear.
Thanks.

 

[VulgarDisplay]

I see. I remember the same poster using (along the lines)MASSIVE exploit, when describing the security issue, with the last Nvidia driver. The references were unclear.
Thanks.

Even when I was talking about the exploit I was simply just talking about how drivers have problems, and not calling out nvidia for being worse than AMD.

It's funny how not drinking the kool-aid automatically labels you biased towards their opposition...

 

[Rikard]

And thx for being honest

*sigh* Why does every thread have to be turned into a fight between AMD and Nvidia? My previous six cards were Nvidia, now I have an AMD card, but apparently that makes me and AMD fanboy, is that it? Of course we are being honest, what would be gained by fabricating results? I wish you guys would stay on topic (other posters than you really f1sherman) and discuss TR's method since all this flame baiting is getting old fast.

If you have to limit your FPS to 40 to get rid of stuttering, I don't really think it unreasonable to never report any benchmark framerate score above 40 fps, since that's all you can do without stutter.

No that is not what removes stuttering, that introduces it, at least in Hitman's case. It is other options in Radeon Pro that makes 1 frame in 100 go from instantaneous 40-something FPS to instantaneous 50-something FPS also outside. Obviously, this is not a game breaker for me. My issue is that no matter what I do, I never obtain as poor results as TR reported. If there is anybody else with the exact same card model as they used, please post your results so we can help TR find their problem!

Since I find FRAPS to be a blunt instrument, and video recording itself introduces stuttering, and I do not have a video capture card like PCPER, I used other people to look at my screen in an effort to do a "blind test". They also see no stuttering.

I guess what I am saying is that yes, there is some microstuttering that can be measured, but at least on my card it is not at all noticeable, not for me nor anybody else who looked for it. This is not a defense of AMD, but I feel one needs to put it into perspective and see the bigger picture.

 

[Will Robinson]

A little off topic perhaps but every time I run FRAPS the screen gets so jerky it actually effects my flying in BF.
It's been like that with every system I've owned so far.
When I play back the captured video it is perfectly smooth however.
Is that usual with FRAPS?

 

[ArchAngel777]

A little off topic perhaps but every time I run FRAPS the screen gets so jerky it actually effects my flying in BF.
It's been like that with every system I've owned so far.
When I play back the captured video it is perfectly smooth however.
Is that usual with FRAPS?

When I record in-game video from FRAPS or MSI Afterburner on a regular HDD, I get what you are describing. But when I write it to my SSD array, it is butter smooth. So most of my videos are recorded on my SSD array and transfered later.

 

[KCfromNC]

You do? Sounds like you've predetermined your answer before taking the blind test yourself. It's like you already know that you wont be able to tell the difference.

Yes. Lets make it more about Keysplayr and less about the subject. That'll work.

Yeah, complain about people making the thread about you and then cherry pick a single line out of context to manufacture a post making it about my implied biases. That's consistent.

Unfortunately, not all of us get free hardware from these companies to test with. I'm just going by what various hardware review sites have already said after looking into the matter, and combining that with the lack of observed problems in other chips which measure similar levels of stutter to the supposed problem AMD cards (e.g. GTX5xx). So far, I have seen lots of cases of blind testing showing little or no problem, and no cases where it has turned up a consistent result for one or the other companies being better.

I don't see using data like this to inform my views as a bad thing. Keep trying.

 

[Keysplayr]

Yeah, complain about people making the thread about you and then cherry pick a single line out of context to manufacture a post making it about my implied biases. That's consistent.

Unfortunately, not all of us get free hardware from these companies to test with. I'm just going by what various hardware review sites have already said after looking into the matter, and combining that with the lack of observed problems in other chips which measure similar levels of stutter to the supposed problem AMD cards (e.g. GTX5xx). So far, I have seen lots of cases of blind testing showing little or no problem, and no cases where it has turned up a consistent result for one or the other companies being better.

I don't see using data like this to inform my views as a bad thing. Keep trying.

That line was taken at 100% in context. Anyone can see that. I'm calling them like I see them bud.

 

[Homeles]

What he meant is that it's funny how the Nvidia zombies are foaming at the mouth over this frame time issue now, when Nvidia is not in the performance lead, because in their minds, it somehow makes Nvidia look better. Mountains out of molehills, indeed. For all of you AMD users affected by this, hope your fix comes soon :thumbup:

Let's not forget all the ridiculous laughing in the face of logic by the AMD crowd in response to Scott Wasson's findings.

And the assertion that Nvidia is behind AMD simply because AMD put out the faster chip is a horrifying display of ignorance. Just a little over one year ago, AMD had far and away the better performance per watt over Fermi. They had significantly better perf/mm^2. And you'd always hear about better performance per dollar. Where was the opposite side clamoring about their better performance? No -- the arguments made from the green team are almost always based around having superior driver support.

But now, all of a sudden, it's all about who's got the biggest performance at the top end -- power, and all of those other areas where AMD excelled, be damned. The hypocrisy is rather laughable.

The fact of the matter is that Nvidia has had a far more successful 28nm line of products than AMD has. It's indisputable -- the OEM wins and financial statments are incontrovertible proof of this.

But hardware communities tend to have a massive soft-spot for AMD, and to support their undying love, they defend their business with religious fervor.