[Tom's Hardware] AMD FreeSync Versus Nvidia G-Sync

[bystander36]

Reading the article, I am amazed the OP didn't mention some key details:

Why do I need to point out every detail? I gave the summary, the article had the details.

"Four of those who mentioned Crysis preferred their experience on AMD’s hardware, eight chose Nvidia’s and one said the two technologies were of equal quality, though his Nvidia-based platform did stutter during an intense sequence."

> This calls into question the validity of the testing where some participants would consider experience similar but the NV-based platform stuttered more?!

What makes that call into question the validity of the comparison? If only Nvidia had any stutters, yet gamers still preferred G-sync, wouldn't that mean G-sync had to be a lot better to overcome some stutter at one point?

"Borderlands turned out to be a gimmie for Nvidia since the AMD setups were destined to either tear (if we left v-sync off) or stutter/lag (if we switched v-sync on)."

> Ok so this isn't really a test of GSync vs. FreeSync but how well optimized a GameWorks title is to run on AMD vs. NV. Figures! But yet, it's a data point that contributes to showing that GSync is better than FreeSync? That makes no sense sense.

Maybe.

This next one is downright shocking:

"Right next to them, we had another AMD machine at Ultra settings and an Nvidia box dialed down to the High preset. Again, three respondents picked AMD’s hardware. Seven went with Nvidia, while two said they were of equal quality"

> This right now calls into question the experience/quality of gamers they have chosen where 7 people picked NV and 2 said equal despite BF4 running on Ultra on AMD and High on NV.

They did choose a more expensive, faster GPU for AMD. They likely turned down settings to get similar FPS. And even though AMD had better IQ settings, they still picked G-sync, most the time.

What does that tell you? I'm not really sure to be honest.

As to why they didn't do all those great, proper research things:

We especially like Tom's suggestion to use a control group in a fixed refresh mode for comparison. Given a longer day and perhaps more activities to keep other folks busy, we would like to see gamers on three systems, one of them being a control of some sort.

A larger sample size was on our wish list all along, but there's only so much you can do with eight machines and one Saturday afternoon. This event was already several times as large as our last one, and we'll definitely shoot for something even larger next time.

The idea to keep the purpose of the experiment under wraps is also intriguing, though I'm not sure we'd have as much luck getting volunteers to sign up without some sort of teaser ahead of time. This and volunteer-run testing might be ideal, but they present us with some practical challenges we'll have to think about.

It would be great to see, but they aren't really a research team with a grant to do such things. They are a review team. If AMD or Nvidia, probably the only two companies who would like to see this research done offered funding, I'm sure no one would trust it.

 

[VulgarDisplay]

Did they include a regular monitor in there to weed out the people who legitimately cannot tell a difference and are choosing based on colors and brightness?

 

[Hitman928]

There was no control group or control test (e.g. a standard, non variable refresh monitor).

 

[3DVagabond]

I think it would have been a better tech comparison to have compared the Swift to the 2730z and 390 vs. 970. So, what we have is 90Hz vs 144Hz (Hmm, don't see any advantage there ) and O/C'ing the lower priced card to match the performance of the more expensive one (I'm sure this doesn't in any way add value to the Gsync components ). Why go through the effort of perfectly matching the systems on the components that won't make a noticeable difference and then mismatching the components that they are actually evaluating?

Wow! only 60% of the participants favor 144Hz to 90Hz. (Disclaimer the "Blind test @ 2560x1440, 27", IPS 144Hz is a lie.) This is probably the most revealing part of this comparison. Surely I would have thought that a larger percentage of participants would have been able to tell the difference.

Lastly, without the testing being administered double blind, enough redundant testing to be certain the subjects could reliably pick out any difference, the test is flawed beyond any hope to reliably determine any objective difference.

I'm stunned that anyone from AMD would have signed off on this. They should either be fired or reassigned to a job they have a clue about.

 

[Keysplayr]

Well, according to the blind test graph here, G-Sync is the overwhelmingly preferred experience. The interpretations are very entertaining though.

 

[Spanners]

Well, according to the blind test graph here, G-Sync is the overwhelmingly preferred experience. The interpretations are very entertaining though.

You have to admit the methodology is far from ideal? Tom Petersen from Nvidia makes some pretty good points that are acknowledged by the author.

The input lag of the two panels is disparate according to TFT Central the Asus has a measurement of 17ms at 60Hz vs 3ms for the Acer. Never mind all the other variables introduced by using different screens.

Are you honestly going to say if the results went the other way you wouldn't have a single comment regarding the testing conditions? No need to answer that. Glad you're finding entertainment in odd places though

G-sync may be the superior technology and I can't stress enough that I'm not bothered by that in any way at all but the flaws in the testing methods still stand out to me. That said some semi-flawed information is probably better than none.

 

[Azix]

I think it would have been a better tech comparison to have compared the Swift to the 2730z and 390 vs. 970. So, what we have is 90Hz vs 144Hz (Hmm, don't see any advantage there ) and O/C'ing the lower priced card to match the performance of the more expensive one (I'm sure this doesn't in any way add value to the Gsync components ). Why go through the effort of perfectly matching the systems on the components that won't make a noticeable difference and then mismatching the components that they are actually evaluating?

http://media.bestofmicro.com/Y/5/514445/original/freesync-or-gsync.png[/img
Wow! only 60% of the participants favor 144Hz to 90Hz. (Disclaimer the "Blind test @ 2560x1440, 27", IPS 144Hz is a lie.) This is probably the most revealing part of this comparison. Surely I would have thought that a larger percentage of participants would have been able to tell the difference.

Lastly, without the testing being administered double blind, enough redundant testing to be certain the subjects could reliably pick out any difference, the test is flawed beyond any hope to reliably determine any objective difference.

[B]I'm stunned that anyone from AMD would have signed off on this. They should either be fired or reassigned to a job they have a clue about.[/B][/QUOTE]

They probably didn't take it seriously enough. Another nail in their PR coffin. I don't think they are doing enough with freesync at all. Still only featured on expensive monitors and high resolution monitors, keeping out the vast majority of gamers. They are currently the only ones supporting this Somewhat revolutionary feature that is supposedly affordable, push the hell out of it and push manufacturers to do better.

The first company to start putting out affordable freesync monitors should make a killing. 1080p 25/30 - 75hz and up

 

[Paul98]

All I see here is a showdown between amd marketing, and nvidia marketing

 

[Keysplayr]

You have to admit the methodology is far from ideal? Tom Petersen from Nvidia makes some pretty good points that are acknowledged by the author.

The input lag of the two panels is disparate according to TFT Central the Asus has a measurement of 17ms at 60Hz vs 3ms for the Acer. Never mind all the other variables introduced by using different screens.

Are you honestly going to say if the results went the other way you wouldn't have a single comment regarding the testing conditions? No need to answer that. Glad you're finding entertainment in odd places though

G-sync may be the superior technology and I can't stress enough that I'm not bothered by that in any way at all but the flaws in the testing methods still stand out to me. That said some semi-flawed information is probably better than none.

I don't know that a whole lot can be said about blind testing. That IMHO is the purest form of testing. The testers have NO clue which system they are using and have to give honest answers, even if they are biased. So, the only thing that has to be admitted is that this form of testing is obviously the best. Blind means blind. No prior knowledge. You see what you see. Done.

 

[Keysplayr]

All I see here is a showdown between amd marketing, and nvidia marketing

I see a blind test result. Are you sure marketing is all you see?

 

[Hitman928]

I don't know that a whole lot can be said about blind testing. That IMHO is the purest form of testing. The testers have NO clue which system they are using and have to give honest answers, even if they are biased. So, the only thing that has to be admitted is that this form of testing is obviously the best. Blind means blind. No prior knowledge. You see what you see. Done.

Umm, no. They created double blind (and even triple blind) experiments for a reason and they left out a control test which makes it almost impossible to actually draw a conclusion. Add to that the significant difference in monitors and this is not really even a blind test of the two technologies. There was a real lack of variable control, statistically significant number of participants, subject screening, control tests, and even how "blind" the participants were (e.g. one person responded clearly he knew which systems were AMD because of the heat exhaust). It is interesting data and like I said, to me the most interesting thing was how much they were willing to spend for what they felt was a better experience, but it is far from reliable.

 

[dark zero]

F-Sync and the Adaptive Sync (the supposed free tech) are destined to fail since G-Sync showns up their fangs. Well... Seems that NVIDIA won big time.
And as always, propietary stuff are far better than free or open ones.

 

[DustinBrowder]

I completely disagree here. It's not about costs, just simply what tech is preferred, but if it makes you feel better, they did ask what people were willing to pay for the advantage. They also spent an extra $80 on the AMD GPU.

I disagree. Only upper class people in the USA who are also huge gamers would spend $150 more for a bit better viewing experience on top of an already big spending.

$150 for a G-sync on the monitor is a HUGE premium, most people would have a budget of say $1200 for a computer, including a monitor, peripherals and OS. If the choice is to get lets say a GTX 970 for $350 and a GTX 960 for $200 in order to get a monitor with G-sync, most will choose to get a better graphic card over G-sync monitor.

Now if you are upper class and huge gamer and you had an open ended budget or at least a big budget of say $2000 to $2500, you would pay the $150 premium when you already got the money and you have the top of the line GPU as well, you are not sacrificing on the actual computer hardware.

Most people, especially in other countries that have smaller annual household income would not pay ANY premium for a bit better viewing experience.

 

[dark zero]

I disagree. Only upper class people in the USA who are also huge gamers would spend $150 more for a bit better viewing experience on top of an already big spending.

$150 for a G-sync on the monitor is a HUGE premium, most people would have a budget of say $1200 for a computer, including a monitor, peripherals and OS. If the choice is to get lets say a GTX 970 for $350 and a GTX 960 for $200 in order to get a monitor with G-sync, most will choose to get a better graphic card over G-sync monitor.

Now if you are upper class and huge gamer and you had an open ended budget or at least a big budget of say $2000 to $2500, you would pay the $150 premium when you already got the money and you have the top of the line GPU as well, you are not sacrificing on the actual computer hardware.

Most people, especially in other countries that have smaller annual household income would not pay ANY premium for a bit better viewing experience.

Sadly if this is the destiny of the gaming industry, so be it. Fewer and fewer people will still be gamers and they will buy the tech at not matter what are the cost. Just like Apple users. If the gaming industry returns to the origins (few gamers), is what the gamers want.

 

[Paul98]

I don't know that a whole lot can be said about blind testing. That IMHO is the purest form of testing. The testers have NO clue which system they are using and have to give honest answers, even if they are biased. So, the only thing that has to be admitted is that this form of testing is obviously the best. Blind means blind. No prior knowledge. You see what you see. Done.

You should read the article before commenting on it.

 

[Hitman928]

BTW, does anyone remember this?
http://www.legitreviews.com/amd-reality-check-at-fx-gamexperience_1838

A blind test of two systems, one with an 8150 and one with a 2700k. Participants chose the AMD system as the better gaming experience at an almost 2:1 rate with over one hundred participants. These types of tests aren't meant to be conclusive at all

 

[UglyDuckling]

AMD won it by a big enough margin, sadly the fan base is a lot more savvy of NV tech.

 

[ShintaiDK]

BTW, does anyone remember this?
http://www.legitreviews.com/amd-reality-check-at-fx-gamexperience_1838

A blind test of two systems, one with an 8150 and one with a 2700k. Participants chose the AMD system as the better gaming experience at an almost 2:1 rate with over one hundred participants. These types of tests aren't meant to be conclusive at all

Not sure I trust a test setup by a vendor wanting to promote its own. Contra a 3rd neutral party.

 

[Hitman928]

Not sure I trust a test setup by a vendor wanting to promote its own. Contra a 3rd neutral party.

I don't trust either of them, that's the point. There's a lot more to getting reliable data than a few sample points in a "blind" test.

 

[Silverforce11]

Wow! only 60% of the participants favor 144Hz to 90Hz. (Disclaimer the "Blind test @ 2560x1440, 27", IPS 144Hz is a lie.) This is probably the most revealing part of this comparison. Surely I would have thought that a larger percentage of participants would have been able to tell the difference.

Considering most people still game on 60Hz, I doubt they can tell the difference at 90+..

Also, not sure why Tom's did such a rigged "blind test", wonder how much NV paid them.

 

[Keysplayr]

Considering most people still game on 60Hz, I doubt they can tell the difference at 90+..

Also, not sure why Tom's did such a rigged "blind test", wonder how much NV paid them.

Wow.

 

[Silverforce11]

Wow.

Wow indeed. Why would such a big site do such a rigged test?

This is a 144hz vs 90hz test, on different panels.

Then they have the balls to conclude GSYNC wins. WTF.

Edit: It's not even a blind test when most of those involved knew which was which AND a large segment were self-labeled as "NV fans" with one who admitted GSYNC stutters but they were "equal" LOL. What a total marketing stunt. So back to the how much NV paid for this marketing stunt..

 

[Keysplayr]

Wow indeed. Why would such a big site do such a rigged test?

This is a 144hz vs 90hz test, on different panels.

Then they have the balls to conclude GSYNC wins. WTF.

Edit: It's not even a blind test when most of those involved knew which was which AND a large segment were self-labeled as "NV fans" with one who admitted GSYNC stutters but they were "equal" LOL. What a total marketing stunt. So back to the how much NV paid for this marketing stunt..

Again. Wow. There aren't any other words.

 

[stahlhart]

Except these three.
-- stahlhart