Techreport 7950 vs. GTX 660 Ti "Smoothness" videos

[WaitingForNehalem]

More testing with both Windows 7 and Windows 8: http://techreport.com/review/24022/does-the-radeon-hd-7950-stumble-in-windows-8

 

[SPBHM]

this is unacceptable for this level of VGA, 50ms? that's some horrible stutters, no need for a video...
they should have tried lowering details, vsync, manual framerate limit and maybe a different platform/CPU.

edit: also if there is any "lag", or "input/output delay"

 

[Ferzerp]

More testing with both Windows 7 and Windows 8: http://techreport.com/review/24022/does-the-radeon-hd-7950-stumble-in-windows-8

I'd like to see that done with the high end cards, and multi-gpu as well. This is the kind of data that we need to be seeing instead of just "fps"

 

[blastingcap]

Thread title needs to change. 7950 vs 660Ti, not 7970.

Apparently NV has some sort of smoothness advantage for single-GPU, not just multi-GPU, at least for Skyrim but likely for other games as well. This isn't terribly surprising to me. I would like to see Cat 12.8 used to see if that helps with the smoothness, as I suspect Cat 12.11 was rushed out, or maybe AMD even knowingly traded off smoothness in frametime for a few more frames per second, knowing that most people just look at fps.

Shame on all the other websites for being beaten by TechReport not only on frametimes, but on capturing high-speed video like I and others had suggested time and again. Shame on Anandtech. Shame on Ryan Smith in particular for ignoring our repeated requests for doing a high-speed video comparison. Only partial shame on HardOCP because they talked about smoothness for months, even years, yet never got around to actually trying to capture it objectively. TechReport used to be such a lightweight when it came to video card reviews; yet it is forging ahead while others rest on their laurels. Time to make an account at TechReport and ditch VC&G graphics forum here. (I'm only half-joking.)

 

[cmdrdredd]

i would definitely have tried a fps limiter and see what happens. It may smooth things out. As for the different platform and CPU I can only say that an i7 3820 should be plenty CPU for either of these cards but perhaps an overclocked 3570k on a z77 would show a different result with higher clock speed and PCIe 3.0(minor difference though it may be)?

Thread title needs to change. 7950 vs 660Ti, not 7970.

Apparently NV has some sort of smoothness advantage for single-GPU, not just multi-GPU, at least for Skyrim but likely for other games as well. This isn't terribly surprising to me. I would like to see Cat 12.8 used to see if that helps with the smoothness, as I suspect Cat 12.11 was rushed out, or maybe AMD even knowingly traded off smoothness in frametime for a few more frames per second, knowing that most people just look at fps.

Thread title needs to change. 7950 vs 660Ti, not 7970.

I wouldn't rule that out, similar things have happened before in the past. I remember a few times where image quality suffered in order to pull ahead in a few benchmarks. Around the time when Quake 3 was popular.

 

[Rvenger]

More testing with both Windows 7 and Windows 8: http://techreport.com/review/24022/does-the-radeon-hd-7950-stumble-in-windows-8

Interesting article

 

[SirPauly]

Thanks, more data and further investigations are welcomed.

 

[Jaydip]

@ blastingcap

Did you advocate these kind of tests? if not what's your idea to improve it?sorry didn't read the whole thread so not sure if you already posted that

 

[Hitman928]

this is unacceptable for this level of VGA, 50ms? that's some horrible stutters, no need for a video...
they should have tried lowering details, vsync, manual framerate limit and maybe a different platform/CPU.

I'll post in this thread what I posted in the other, just because I think this seriously needs to be looked into before these types of comparisons are made, IMO:

Here's what I don't understand and why I wanted them to revisit the same scene with the new system:

Time spent beyond 16.7ms ("whiterun" scene)

First Review:
660Ti 132
7950 152

Follow-up:
660Ti 30
7950 259

And time spent beyond 50ms from first review ("whiterun" scene)
660Ti 0
7950 0

Their conclusion of the data: "Interesting. There isn't much change from our older review"

...?

To be fair, the numbers they point at are 99th percentile, but do they really not see the massive change in 7950 latency? You have a 70% increase for time spent over 16.7ms. Also, they're percentile graph has the 7950 and 660Ti basically swapping places. They also are the only review I have found thus far (maybe there's another?) that shows a decrease in fps from 12.7 to 12.11 catalyst despite using a card with a higher boost clock?

I'm not saying we should ignore these results as obviously AMD is even looking into them, but can anyone show me any other review of a single card configuration that in any way corroborates what techreport is getting? Until then I will treat this as I always treat a single review site showing something different than everywhere else I look, an anomaly. If there is any other place showing the same results, I'd love to see it. . .

 

[blastingcap]

@ blastingcap

Did you advocate these kind of tests? if not what's your idea to improve it?sorry didn't read the whole thread so not sure if you already posted that

What I advocated was taking a high speed cam and literally counting the frame times manually frame by frame, since I didn't know of any other way to do it. Yes it would take a few days but you wouldn't need to run more than a minute or so of footage per card. Very doable and leaves nothing to software bugs or chance. Ryan mumbled something about doing software capture of things in the graphics pipeline, and then didn't respond to my suggestions of using a cam. HardOCP identified the issue but didn't bother camming it up either. Only TR did something, just like they did the frametimes review as well. This is the equivalent of a small independent oil exploring firm outhustling the likes of Exxon and Chevron to a known oil field. It was known. Exxon and Chevron have little excuse for why they weren't there first, other than sheer inertia because exploring it would make more work for them.

TR's side-by-side comparison is obviously imperfect; they should test more games and more driver versions and such. But it's a step in the right direction, to slice things up more finely than a whole second. 240fps is a little low, though; I would have hoped for more like 400fps, but even 240fps is enough to get the point across and so perhaps 400fps is overkill.

 

[Jaydip]

But I have a point though, suppose your card is generating frames @ x fps and you capture that using a camera which can shoot @ 2x fps.Now I would be looking at the same frame twice, how would I distinguish between them?

 

[Will Robinson]

i would definitely have tried a fps limiter and see what happens. It may smooth things out. As for the different platform and CPU I can only say that an i7 3820 should be plenty CPU for either of these cards but perhaps an overclocked 3570k on a z77 would show a different result with higher clock speed and PCIe 3.0(minor difference though it may be)?



I wouldn't rule that out, similar things have happened before in the past. I remember a few times where image quality suffered in order to pull ahead in a few benchmarks. Around the time when Quake 3 was popular.

Oh you mean when NVDA was found cheating on 3D Mark?

 

[RussianSensation]

I want to see more examples... Skyrim is just one example and you can't make a conclusion on it.

Since Tech Report noted that they felt stutter with HD7950 on 12.8 or 12.11 drivers, I went to look at their GPU reviews to even later reviews that didn't used older drivers for high-end GPUs. I found their HD7970Ghz launch review in June with Catalyst 12.7 beta.

http://techreport.com/review/23150/amd-radeon-hd-7970-ghz-edition/3

Quote from their review:

"We're relying on our 99th percentile frame time metric for our performance summation, but we've converted the result to FPS to keep our scatter plot readable."

^ What you see is HD7970 Ghz clearly delivering lower frame times than GTX680 or even an after-market pre-overclocked 680. In that review, I even looked closer to the individual tests, games that performed well on NV like Max Payne 3, BF3 or Crysis 2. No unusual stuttering on their HD7970 Ghz card at all. If AMD's drivers had these permanent issues in place, these games should have been stuttering on 7970Ghz compared to 680, but yet the results in that review are nothing worth talking about.

All these games were tested at 2560x1600 with AA and you can see the frame times are imperceptible in smoothness between 680 and HD7970Ghz cards, except Batman AC and Dirt Showdown where each camp nets a win.

I am not debating Tech Report's testing methodology but this idea that AMD cards stuttered all this time is being spread like the plague and it contradicts even TR's own testing not long ago. If you ask AMD/ATI users who switched back and forth between those brands and NV, I would bet you if they noticed some stutter fest on ATI/AMD cards, they would never go back after trying NV.

If we look at HD7950 vs. GTX660Ti review, the only game that's the same in that review and HD7970 Ghz vs. GTX680 review is Skyrim.

This is TR's HD7970Ghz vs. GTX680 testing of Skyrim with Catalyst 12.7 betas. Not only do HD7900 cards have no problems, but HD6970 is easily outperforming GTX570 under the same methodology. In fact, GTX570 significantly lost to HD6970 in smoothness in BF3, a game that was a staple for NV's good performance. So this idea that AMD cards stutter more as a generalization is unsubstantiated by real world evidence even from the same website!

And now Skyrim retested with Catalyst 12.11 for 7950 vs. 660Ti:

^ The Skyrim testing area is not the same, and I already noted how the frames per second seem way off to begin with as HD7950 boost demolished GTX660Ti in Skyrim in most other professional reviews that tested 2560x1440/1600 with AA, but still, if AMD cards had these stuttering driver issues for single GPUs all these years, it sure was nowhere to be found in TR's testing this summer. They even concluded that HD7970Ghz was faster and their frame times testing showed it was smoother too (See 1st graph).

This conspiracy theory that AMD was trading smoothness for frames per second all this time and that TR was the first website to expose them is not a valid one since even as of June 2012, these issues are nowhere to be found for single AMD GPUs. My guess is AMD's drivers for recent games have not been polished enough, which is not a surprising given AMD's financial struggles and layoffs, including you guessed it their graphics card department, and the fact that they are still betas. I personally have never felt any stutter difference for single GPUs between AMD and NV. I am currently running 12.8s because 12.11s cause instability for me in bitcoin mining on desktop. I wouldn't have continued buying AMD cards after my Fermi experience if my HD6950 @ 6970 was a stutter mess, that's for sure.

And as I said, frame times is one thing but the Skyrim frames per second are so off, it's almost mind-boggling how they got GTX660Ti to beat HD7950 in this game at high resolution with MSAA. Even a GTX670 max OC cannot even manage to come close to an HD7950 OC and GTX660Ti is way behind by nearly 40%!

This was as of August 2012.

Source

Once again, where is this frames per second differential in Tech Report's review? If the card is producing low frame rates, the frame times will be higher. There is still a relationship between them. How in the world did GTX660Ti not only make up nearly 40% deficit but is now beating an HD7950 in Skyrim at 2560x1400 + 4xMSAA? I can't explain that, I really can't.

TR's review is raising more questions:

1) HD7970/Ghz cards exhibited no such issues in June
2) HD7950 mysteriously gets creamed in Skyrim by GTX660Ti at high resolution with AA, contrary to 95% of reviews out there.
3) Older AMD cards like HD6970 often beat GTX570 in smoothness. Therefore, we know these developments are recent for single GPUs and in fact, most likely related to the specific games tested in that HD7950 vs. 660Ti review.

In other words, starting to generalize much from this review of HD7950 vs. GTX660Ti outside of those 2 cards and those specific games tested is not exactly confidence inspiring without other professional reviewers also vetting this data.

 

[boxleitnerb]

4xAA vs 8xAA maybe. 8xAA is very hard on bandwidth.
Also indoor vs outdoor - they can have quite different performance characteristics.

 

[3DVagabond]

*snip*
Once again, where is this frames per second differential in Tech Report's review? If the card is producing low frame rates, the frame times will be higher. There is still a relationship between them. How in the world did GTX660Ti not only make up nearly 40% deficit but is now beating an HD7950 in Skyrim at 2560x1400 + 4xMSAA? I can't explain that, I really can't.

TR's review is raising more questions:

1) HD7970/Ghz cards exhibited no such issues in June
2) HD7950 mysteriously gets creamed in Skyrim by GTX660Ti at high resolution with AA, contrary to 95% of reviews out there.
3) Older AMD cards like HD6970 often beat GTX570 in smoothness.

In other words, starting to generalize much from this review of HD7950 vs. GTX660Ti outside of those 2 cards and those specific games tested is not exactly confidence inspiring without other professional reviewers also vetting this data.

So, are we ready to call Shens. on this, yet? There's just too much evidence, even from their own reviews, that contradicts their findings. Even looking at last gens results, where latencies were well into the range we're being lead to believe is noticeable, nobody's cards were stuttering then. And as you point out, nobody was saying AMD was "slower but smoother".

 

[RussianSensation]

nvmd.

 

[blastingcap]

But I have a point though, suppose your card is generating frames @ x fps and you capture that using a camera which can shoot @ 2x fps.Now I would be looking at the same frame twice, how would I distinguish between them?

For TR both cards are affected the same way.

For my proposal, you would wait till the frame changed over to record a new frame.

 

[boxleitnerb]

RS, where did you get that graph? What I see is this:

 

[RussianSensation]

4xAA vs 8xAA maybe. 8xAA is very hard on bandwidth.
Also indoor vs outdoor - they can have quite different performance characteristics.

That's not enough of an explanation. I checked reviews with 4xAA and GTX660Ti still loses by ridiculous amounts.

Skyrim at 2560x1440/1600 + 4xMSAA or 8xMSAA performs way faster on AMD HD7900 cards. This has been known for a long time which is why on our forum we have been recommending GTX680 for BF3 players and AMD cards for Skyrim players (at least until Cats 12.11s fixed BF3 issues).

Even going back to older drivers, HD7950 Boost was still faster than a reference GTX670 in this game at high rez.

RS, where did you get that graph? What I see is this:

Linked the wrong game by accident. I corrected the mistake above. The entire message still stands. The testing for Hitman was inconsistent / or 0 frame times point I can't explain. Maybe someone else can

 

[boxleitnerb]

The graphs don't say anything about duration. In Hitman the 660 Ti renders fewer fps during the benchmark, hence the graph is shorter. The x-axis is not time, it is number of frames and thus dependent on performance. Lower performance -> shorter graph.

And regarding performance:
Different benchmark scenes can have different results. I remember that especially in some outdoor scenarios, Geforces were very strong due to the seemingly better frontend.

 

[notty22]

You can't compare different driver (especially beta) versions, for stuttering or other issues, that's the whole point. Is something sacrificed for a FPS counter result. In the past that might have been flickering or disappearing textures, now it may be some other optimization. Which could come from either competing company, in theory.

 

[RussianSensation]

Same Zotac GTX660Ti was tested against MSI HD7950 880mhz and reference HD7950 Boost on September 12, 2012 at Tech Report, using Catalyst 12.7 betas as well.

GTX660Ti has no advantage in smoothness.

The games tested were BF3, Batman AC, Dirt Showdown (excluded from that graph), Skyrim, Max Payne 3 and Skyrim.

You can't compare different driver (especially beta) versions, for stuttering or other issues, that's the whole point. Is something sacrificed for a FPS counter result. In the past that might have been flickering or disappearing textures, now it may be some other optimization. Which could come from either competing company, in theory.

I don't think you understood why I did that. To diagnose the problem, we need to start isolating variables. The first variable is drivers because it's easy to track this data from the same website. I went back to check for myself and saw that TR doesn't report any unusual stuttering with older drivers. That's the reason I am posting this to show this information. The stuttering issues did not exist in any of their HD7950 Boost vs. Zotac GTX660Ti then, or HD7970Ghz vs. GTX680 reviews. So that means:

1) There are possible driver issues that exist in Catalyst 12.8 and beyond; perhaps the recent drivers broke frame metering somehow.
2) AMD simply didn't optimize the drivers well enough for the newly tested games on TR (Assassin's Creed 3, etc.) and need a lot of work on this front.
3) Other

This changes nothing about the results for MOH:W and Skyrim especially. The frame rates for those games make no sense.

The stuttering must be recent since it's just not there at all in any of their previous reviews.

 

[SirPauly]

The Never Settle drivers may need more maturity on smoothness!

 

[RussianSensation]

Remember this is only 1 review and smoothness aside, it's still showing FPS anomalies in 2 games: MOH:W and Skyrim. Those 2 benchmarks do not make sense whatsoever in FPS measurements using fairly recent drivers compared to other online reviews. I am not about to ignore every other review and consider TR's FPS numbers correct without at least seeing other websites corroborating on their findings.

Here is TechSpot's with Nvidia Forceware 310.61 Beta and AMD Catalyst 12.11 (Beta 8).

Source - Review dates November 26, 2012

TR's testing gave GTX660Ti 35-40% performance in both of those games...out of nowhere. Did you see any other recent review that aligns with TR's FPS data in those 2 games?

TR can do further testing but I doubt they will. 2 key ways to find out: (1) wait months to see if these issues are fixed with newer non-beta drivers and revisit the same games used in the TR review in the OP, (2) TR can go back and retest all those same games they used in GTX680 vs. HD7970Ghz and GTX660Ti vs. HD7950 Boost reviews I linked earlier. If games such as BF3, Max Payne 3, Crysis 2 start stuttering way beyond their previous findings, we know Cats 12.7 Betas were the last "good" driver that works. If those games run well, then chances are the new titles need more driver optimizations to achieve smoothness. I can say I see no such stuttering on my 7970 with Catalyst 12.8 though. I would have downgraded to 12.7s.

And like I said before, even HardOCP's recent testing of HD7970 CF vs. GTX680 SLI showed that HD7970Ghz provided a better gaming experience than GTX680, while HardOCP's Fall GPU Roundup done in November clearly showed HD7950 beating GTX660Ti in smoothness and FPS. I don't think we should start discounting all these other reviews online done recently and put all our eggs into TR's review especially since their FPS numbers for MOH:W and Skyrim are so strange.

 

[boxleitnerb]

While I admit that these results are a bit strange, you should keep in mind that different scenarios can yield different results. Unless you use the exact same scene, benchmarks will not necessarily be comparable. Maybe techreport picked a worst case scenario for AMD by chance, who knows?

Do we even know what scenes techspot and hardocp are benchmarking? Indoor, outdoor?