Techreport 7950 vs. GTX 660 Ti "Smoothness" videos

[ICDP]

See, the important thing is that you've recognized this. Things over here at AnandTech tend to be a bit more mature than over at OCN, where I normally lurk. We had people (as in >1) over there claiming that Nvidia forced Scott Wasson's hand in this.

I always recognised it, I just felt they were completely wrong until I actually tested Skyrim and found the same stutter. I also tested other games including Hitman AB and found my results totally different to TRs. This is why I won't take everything they post at face value and neither should anyone else. More testing from various sites and sources is required.

My theory is that Scott Wasson is having an affair with Jen-Hsun Huang.

I'm a firm believer in Hanlon's Razor. "why attribute to malice what can adequately be explained by ignorance".

Exactly. And I would stick with phrasing that as "not reporting" the problem, because most sites don't comment on smoothness. I know SweClockers is looking into it and are going to publish a review, and they've already mentioned that they have found similar issues, although far less pronounced. Then there's the apparent investigation going on over at AMD that would be interesting to here the results of.

Bear in mind that "not reporting" could simply mean nothing to report.

http://www.youtube.com/watch?v=_w5JqQLqqTc

 

[blastingcap]

It's clear to me that nothing will be resolved until we have a LOT more high-speed camera tests done on way more games at various realistic settings.

 

[antihelten]

So I looked in to ICDP's frame numbers, and found some (IMHO) quite interesting things.

For the run you made at 19:20:49 (with flip queue size at default I assume, this run was the one with the most stutter at least), I could find the same pattern as TR showed, with one slow frame, followed by one average frame then followed by one fast frame repeated for upwards of 100 frames at a time.

For the other run you made (19:16:43, flip queue size at 1), this pattern was no where to be found.

The reason why I think this pattern is interesting is that the slow frames are rendered as slowly as 20-25 fps, whereas the fast frames are rendered at upwards of 200 fps, and it seems extremely unlikely to me that the load the game puts on the GPU can change that quickly (within a 3 frame period), and do so repeatedly. So my guess would be that this pattern (and the stutter) is actually a buffer problem. This also makes sense seeing as changing the flip queue size to 1 apparently solved the problem.

Funny thing is that to my knowledge the default flip queue size should be 3, which matches up with a repeated pattern of 3 frames.

The bright side is, if this is all just a problem with the buffer, then I would imagine that it is quite fixable for AMD

 

[lavaheadache]

I just want to comment on this one point. It would be completely irresponsible for AMD to ignore this. Them looking into it shouldn't be misconstrued as endorsing anything.

To be clear, I'm not accusing you of this.

:thumbsup:

 

[VulgarDisplay]

Are you on Crack or something? Truths of this world? Sensational comments that have no basis in fact? You my friend have just changed the dictionary definition of delusional. You believe TR's findings are all one giant untruth. A lie. A scam. Are you serious? You know what, don't answer that. I know it already. We will have a shot in your name tonight at the local pub.

If you took off the green tinted glasses and actually read what he said you would know that he's not calling TR's findings a scam. He's calling for more independent reviewers to research the issue to come up with a definitive answer to the question of "smoothness."

Taking one review and then using that review to make a fact out of something that has not been corroborated by others and running with it is the sensational comments he is referring too. In the end it may be found that Tech Report was 100% correct and AMD does need to do something to fix the issue. As for now, all rational people regardless of what brand of hardware is in their computer want more investigation into the issue before jumping to any conclusions.

Just because someone wants to see more research about this doesn't mean they want to sweep it under the rug, or distract from the problem. You are the one that is doing all of the distracting by throwing a tantrum whenever someone disagree's with you. I am brand agnostic when it comes to video cards. I choose based on value when I am in the market.

It's hilarious that myself and others have been questioning TR's findings, just questioning, not dismissing or calling them lies and we are branded as AMD viral marketers. We need more facts before any conclusions can be reached and that is the reality of the situation.

 

[Keysplayr]

No kidding. We should wait for more results... What do you say Keys? Maybe we shouldn't just take bad news against the competition and run with it without question... Ahh, but I see who I am talking too, or rather, what, so I guess my expectations are too high to expect any reasonable discussion.

With that said, the initial results should alarm anyone with an ATI card. I read this review and could easily see that ATI was lagging in the videos. I have a 7970, so this matters to me. But since I am not particularly vested in AMD or nVidia I have no problem with more investigation on the issue. If true, it is dissapointing. But my guess is that it may be a driver related issue, or possibly something with a specific configuration. I hope I am right.

The good news is that so far I have not noticed anything like those videos on my own rig. Buttery smooth.

All right. Have at it boys. :thumbsup:

 

[Firestorm007]

If you took off the green tinted glasses and actually read what he said you would know that he's not calling TR's findings a scam. He's calling for more independent reviewers to research the issue to come up with a definitive answer to the question of "smoothness."

Taking one review and then using that review to make a fact out of something that has not been corroborated by others and running with it is the sensational comments he is referring too. In the end it may be found that Tech Report was 100% correct and AMD does need to do something to fix the issue. As for now, all rational people regardless of what brand of hardware is in their computer want more investigation into the issue before jumping to any conclusions.

Just because someone wants to see more research about this doesn't mean they want to sweep it under the rug, or distract from the problem. You are the one that is doing all of the distracting by throwing a tantrum whenever someone disagree's with you. I am brand agnostic when it comes to video cards. I choose based on value when I am in the market.

It's hilarious that myself and others have been questioning TR's findings, just questioning, not dismissing or calling them lies and we are branded as AMD viral marketers. We need more facts before any conclusions can be reached and that is the reality of the situation.

:thumbsup: Nice Post. Maybe we should direct him over to GeForce forums and all the users reporting stuttering with the last three sets of 3 series drivers. I call for an investigation into in this issue!

 

[Will Robinson]

Uh oh.....not stuttering with NV cards?

Hmm:sneaky:

 

[Aristotelian]

Are you on Crack or something? Truths of this world? Sensational comments that have no basis in fact? You my friend have just changed the dictionary definition of delusional. You believe TR's findings are all one giant untruth. A lie. A scam. Are you serious? You know what, don't answer that. I know it already. We will have a shot in your name tonight at the local pub.

Are you even moderately literate? The sensational comments are falsehoods like 'Nvidia wins hands down', and you claimed that statements like that ought to be respected because they are opinions. Now look at the way you respond to a post that essentially argued that objectivity is necessary (and much more rigorous testing, too) to get to the bottom of this. Quote a part of any of my post that says that TR's findings are an untruth, and I'll buy you that shot, but until then - you're the one who is on crack.

 

[Aristotelian]

If you took off the green tinted glasses and actually read what he said you would know that he's not calling TR's findings a scam. He's calling for more independent reviewers to research the issue to come up with a definitive answer to the question of "smoothness."

Taking one review and then using that review to make a fact out of something that has not been corroborated by others and running with it is the sensational comments he is referring too. In the end it may be found that Tech Report was 100% correct and AMD does need to do something to fix the issue. As for now, all rational people regardless of what brand of hardware is in their computer want more investigation into the issue before jumping to any conclusions.

Just because someone wants to see more research about this doesn't mean they want to sweep it under the rug, or distract from the problem. You are the one that is doing all of the distracting by throwing a tantrum whenever someone disagree's with you. I am brand agnostic when it comes to video cards. I choose based on value when I am in the market.

It's hilarious that myself and others have been questioning TR's findings, just questioning, not dismissing or calling them lies and we are branded as AMD viral marketers. We need more facts before any conclusions can be reached and that is the reality of the situation.

Exactly. it is nice to see posts like this and BlastingCap's too.

 

[Rikard]

Are you even moderately literate? The sensational comments are falsehoods like 'Nvidia wins hands down', and you claimed that statements like that ought to be respected because they are opinions. Now look at the way you respond to a post that essentially argued that objectivity is necessary (and much more rigorous testing, too) to get to the bottom of this. Quote a part of any of my post that says that TR's findings are an untruth, and I'll buy you that shot, but until then - you're the one who is on crack.

Do not feed it!

 

[f1sherman]

Can't say I'm surprised. Usually you get what you pay for.
AMD getting you higher fps/$ with Nvidia providing smoother gameplay is what much of the "sheeple" thought to be the true even without Techreport/Hardocp tests.

Turns out sheeple was spot on, and self proclaimed experts with their eyeballs fixated on FPS charts were bamboozled. Imagine that :whiste:

Some more from Techreport for you guys to chew on.
Does the Radeon HD 7950 stumble in Windows 8?

 

[Will Robinson]

Aren't those FPS charts the ones we use to decide the winner?
You know..like when NVDA cards had faster frame rates...
Let it go...a new generation card will come out next year and may regain the performance crown from AMD.Get excited about that....this round is lost.
SLI seems to be a bit better than Crossfire according to most reviewers but single card they say GCN rules this generation both for performance and price.
I can live with that.

 

[f1sherman]

Thread is about 7950 vs. GTX 660 Ti "Smoothness" ...
What do you mean let it go :hmm:

Techreport for one has been consistent about Kepler being a better buy than GCN when it comes to what they consider the most important performance metrics, ie. smoothness.

For all the screaming in past few months about AMD offering better FPS/$, we might now have to endure with the notion that even on lower fps Nvidia offers smoother gameplay :thumbsup:

 

[NIGELG]

Thread is about 7950 vs. GTX 660 Ti "Smoothness" ...
What do you mean let it go :hmm:

Techreport for one has been consistent about Kepler being a better buy than GCN when it comes to what they consider the most important performance metrics, ie. smoothness.

For all the screaming in past few months about AMD offering better FPS/$, we might now have to endure with the notion that even on lower fps Nvidia offers smoother gameplay :thumbsup:

Techreport is one site out of so many.If you want to make a decision based on that then that's your right.

''Smoothness'' is subjective.All happy 7950 users must like ''rough'' gameplay then.

 

[f1sherman]

Techreport is one site out of so many.If you want to make a decision based on that then that's your right.

''Smoothness'' is subjective.All happy 7950 users must like ''rough'' gameplay then.

I'm not dishing out anyone. Who am I to complain about someone else's gameplay experience, particularly if they're happy.
Calling people sheeple because you disagree with their hw choice... I leave that to others.

And true... Techreport is one of many, but same as [H] is the only one with subjective gameplay tests, Techreport is the only reviewer with advanced frame statistics.
It's also the original CRYSIS 2 INVISIBLE WATER, CONCRETE SLAB TESSELATION article website. Not that I give any credibility to conspiracy theoretics, but all this kinda confirms their impartiality.

 

[BrightCandle]

There are two review sites that focus on smoothness of motion delivered by video cards.

HardOCP use their eyes to find the best looking settings they are comfortable playing at and report those. They do apples to apples but it isn't the focus of their reviews. They don't talk about the methods for choosing scenes.

Tech report play the games, find scenes that represent typical game experience and then play on settings that should push the card but still work. In their recent review they took high speed video of the play and also captured frame times. Frame times here are an objective measure of smooth play, the videos a subjective view but still comparable chunk of data and then combine it with an objective view of how they perceived the motion.

Anandtech, toms hardware .... etc take minimum and average frames per second and compare those for some scene.

Who else are we going to compare tech reports data to exactly? I don't know of a single other site that has gone within the second like they have. That is why Anandtech and others are being asked to investigate it.

If you listen to their podcast you can hear all about what goes into validating their results, they really are quite thorough. They are ahead of everyone else in reporting game performance. They are possibly the only site that genuinely reviews GPUs for what matters. They could be wrong, but I can reproduce their data on my 7970s and 680s as it seems can others. I think yet again tech report has genuinely found a problem and everyone else failed to notice.

 

[SirPauly]

but I can reproduce their data on my 7970s and 680s as it seems can others.

Can you share some of this data in this thread? More investigations are welcomed.

 

[Hitman928]

Who else are we going to compare tech reports data to exactly? I don't know of a single other site that has gone within the second like they have. That is why Anandtech and others are being asked to investigate it.

If you listen to their podcast you can hear all about what goes into validating their results, they really are quite thorough. They are ahead of everyone else in reporting game performance. They are possibly the only site that genuinely reviews GPUs for what matters. They could be wrong, but I can reproduce their data on my 7970s and 680s as it seems can others. I think yet again tech report has genuinely found a problem and everyone else failed to notice.

Ryan (Anandtech), Kyle [H], and a couple others have commented and said that they've known about frame times for quite a while but do not feel that it is nearly accurate enough a way to compare "smoothness" or one's gaming experience and so they don't use it. Ryan said that they are working on getting more accurate tools and hope to have them sooner than later and does not feel like they can backup any data they might post without them.

I give TR props for embracing a new methodology and trying to provide a different take on it, but when you do this, you better be prepared to defend your stance and take critiques of your method, especially when there seems to be inconsistancies. This is the same for any, even remotely, scientific review. I for one like the frame times but only trust them to a very short degree. For instance, I've seem frame times using tripple buffering that someone would say should be a stuttering mess according to TR's standards, but was a solid 60fps and smooth as silk, as you would imagine tripple buffered output to be.

I guess we'll have to wait and see how all this comes out in the wash. I hope we can get a "verified" way of testing for the microstutter type of effect because that is something I would take into consideration when making a purchase. This may end up being TR's way, they may need to tweak their process, or it could mean something different entirely, we'll see.

 

[KCfromNC]

So I looked in to ICDP's frame numbers, and found some (IMHO) quite interesting things.

For the run you made at 19:20:49 (with flip queue size at default I assume, this run was the one with the most stutter at least), I could find the same pattern as TR showed, with one slow frame, followed by one average frame then followed by one fast frame repeated for upwards of 100 frames at a time.

For the other run you made (19:16:43, flip queue size at 1), this pattern was no where to be found.

The reason why I think this pattern is interesting is that the slow frames are rendered as slowly as 20-25 fps, whereas the fast frames are rendered at upwards of 200 fps, and it seems extremely unlikely to me that the load the game puts on the GPU can change that quickly (within a 3 frame period), and do so repeatedly. So my guess would be that this pattern (and the stutter) is actually a buffer problem. This also makes sense seeing as changing the flip queue size to 1 apparently solved the problem.

Funny thing is that to my knowledge the default flip queue size should be 3, which matches up with a repeated pattern of 3 frames.

The bright side is, if this is all just a problem with the buffer, then I would imagine that it is quite fixable for AMD

Again, assuming it is even a problem with the driver and not in the way FRAPS happens to measure certain types of buffering schemes. For example, see how changing from single to double to triple buffering turns another card into a stutter-fest : http://www.overclock.net/t/1337206/...d-7950-stumble-in-windows-8/270#post_18836123. Does enabling triple-buffering really make the output stutter like that, or is there something weird about how FRAPs measures frame times with it enabled?

Edit - semi-ninja'd by the previous poster, but at least I linked to pretty pictures

 

[KCfromNC]

Uh oh.....not stuttering with NV cards?

Hmm:sneaky:

Who knows, maybe nVidia will deliver a new beta tomorrow and fix all this mess in one shot.

 

[ArchAngel777]

Who else are we going to compare tech reports data to exactly? I don't know of a single other site that has gone within the second like they have. That is why Anandtech and others are being asked to investigate it.

If you listen to their podcast you can hear all about what goes into validating their results, they really are quite thorough. They are ahead of everyone else in reporting game performance. They are possibly the only site that genuinely reviews GPUs for what matters. They could be wrong, but I can reproduce their data on my 7970s and 680s as it seems can others. I think yet again tech report has genuinely found a problem and everyone else failed to notice.

I agree. I believe they are testing to the best of their abilities. They are gambling on the farm though. If it turns out they are right after more validation (with time) they will be known as the pioneers and will be well respected. If it turns out they are wrong... Well, we can just call them tweaktown or something that no one ever visits anymore.

 

[MrK6]

I agree. I believe they are testing to the best of their abilities. They are gambling on the farm though. If it turns out they are right after more validation (with time) they will be known as the pioneers and will be well respected. If it turns out they are wrong... Well, we can just call them tweaktown or something that no one ever visits anymore.

Being "right," even by chance, doesn't give them much credibility. There's way too many liberties taken with their testing to take it with anything but a grain of salt. IIRC, there were reports that frametime reading on nvidia's cards is borked anyway due to the way their drivers work, so how accurate is FRAPS? I like using high speed cameras to film the action and slow it down, but so far the only game they've tested they ran modded, which completely confounds the test. If the results are more consistent across different games that are run at default settings, then we have something. Currently it's impossible to tell if the discrepancies come from the hardware, the drivers, the game/application, or the software benchmarking everything.

While I'd like to see this investigated, it must be done so properly. I honestly think most of the reviewers today must have never passed high school as it seems the scientific method is an alien concept to them. Also, the rabid fanboyism jumping at any crumb to try to prove their side better is getting ridiculous. I honestly don't care if the mods want this forum to go to waste, but for those who are truly interested in seeing results, I should think they would want to clean up their acts.

 

[BrightCandle]

My realisation in all this is that all the reviews are near worthless. Fps is clearly rubbish as a measure, frame times have potential issues as well and all that remains is whether someone on the internet thought it was smooth. No one has a test that is objectively measuring perception of motion in all cases, the closest is hardOCP and I don't really trust those reviews when I don't find their settings smooth myself.

Everything this industry is doing to measure performance is wrong.

 

[MrK6]

My realisation in all this is that all the reviews are near worthless. Fps is clearly rubbish as a measure, frame times have potential issues as well and all that remains is whether someone on the internet thought it was smooth. No one has a test that is objectively measuring perception of motion in all cases, the closest is hardOCP and I don't really trust those reviews when I don't find their settings smooth myself.

Everything this industry is doing to measure performance is wrong.

I think that's a little much Chicken Little to be honest. FPS measured performance for the last decade and everyone was fine with it. No one bothered getting a frametime graph because we were all just happy if the newest graphics cards could actually max games at playable FPS.

Since then, game development has graphically stagnated, as has LCD tech (we're stuck at 1080p and 1600p), and therefore gaming cards can max today's games with room to spare. In addition, many performance differences usually come down to how well optimized a game and drivers are, not the hardware itself. This rut allows people to explore their experiences further and notice other things, such that AFR sucks.