Even at a sample size of 30+M, is the Steam GPU survey accurate?

[-Slacker-]

But, according to the last couple of posts, the entire HD 6000 series seems like a complete faliure, because cayman sold like crap too, judging from the steam survey. How did AMD manage to make a profit, then?

 

[Grooveriding]

But, according to the last couple of posts, the entire HD 6000 series seems like a complete faliure, because cayman sold like crap too, judging from the steam survey. How did AMD manage to make a profit, then?

Simple, the 5XXX series dominated DX11.

-They had DX11 all to themselves for seven months.

-The 5770 is the best selling DX11 card by a wide margin.

-The 5850 is still today the best value going in a DX11 card.

-They likely unloaded piles of 5870s when there was no competition for those cards. A high margin card.

-They've been scooping up mobile gpu market share.

-They've had the fastest card on the market since Sept/09 until today and that helps to command sales.

 

[happy medium]

Simple, the 5XXX series dominated DX11.

-They had DX11 all to themselves for seven months.

-The 5770 is the best selling DX11 card by a wide margin.

-The 5850 is still today the best value going in a DX11 card.

-They likely unloaded piles of 5870s when there was no competition for those cards. A high margin card.

-They've been scooping up mobile gpu market share.

-They've had the fastest card on the market since Sept/09 until today and that helps to command sales.

yet the company STILL makes no profit. amazing.

 

[Arzachel]

Amd cards are underrepresented in the Steam survey due to a bug. So no, its not representative.

 

[Lonyo]

Steam survey is accurate.

Prove it.

1) Me, from May

You really don't want to look at the Steam numbers for the simple fact that they don't really bear any relation to the actual market.
I say this based on the fact that the 2 products that hard numbers have ever been released for, the HD5700 and HD5800 cards, shortly after release showed a 62.5 to 37.5% sales split (500k vs 300k said AMD) while the Steam figures for these cards at the end of the month of that release following showed them to have a 44/56 split (end of December).

Now, when the company making the products says the split is 62.5/37.5 and Steam says it's 44/56, that shows that there is a CRAZY margin of error when it comes to the Steam figures. Which means that when talking about the overall market or general trends trying to use Steam figures is a terrible idea which in no way presents an accurate picture of anything.
And that's based on Steam numbers which are fairly accurate because basically they were the only DX11 cards available and had only been available a short time and were in a category of their own.

But if you would like to use the Steam numbers for saying anything beyond "this is a typical computer for a Steam user", feel free, but be aware that anything you try and say using said numbers is likely to be utterly out of whack with the real world.

Oh, and one more thing. If Steam is accurate and therefore the GTX470 and GTX480 account for the 1.09% of sales as given, and ATI have indeed (as they claim) sold 6 million DX11 GPUs, that means that NV have sold only 66k GTX470s and GTX480s combined, which would in fact agree somewhat with the low estimates given by most analysts and crazy people (e.g. Charlie). Of course, that wouldn't seem that close to the hundreds of thousands of Fermi chips NV claim to have shipped, but it could be that they shipped more than 2/3rds (figuring minimum of 200k for hundreds of thousands) to non-consumer products.

2) Here's a picture showing another problem with month to month figures:

3) Continuing from 1, which has hard numbers, here are some segments:
http://www.xbitlabs.com/news/video/...h_End_Mainstream_Graphics_Cards_Research.html
Steam is clearly top heavy, towards "gaming cards", which means it is in no way representative of overall sales, as some people seem to use it for, be it in the overall market, or within a specific area of the market (e.g. DX11).



So, with #1 I have established that there is no way in which Steam is accurate for arguably "gamer" cards (assuming you consider the 5700 level performance to be gamer level, I would certainly hope you do).
With #2 I have established that Steam is full of inconsistencies and the data seems to have holes in it.
With #3 I have shown that Steam in no way represents the overall market.

So with the Steam hardware survey we have something which:
Doesn't accurately reflect sales within the 'gamer' product category.
Doesn't have any degree of reliability.
Doesn't represent overall sales of graphics cards.

Please stop trying to use it to show anything other than what the Steam market is made up of.

Steam has ~25 million accounts, and Intel estimates the gaming market at ~200 million PCs as of 2010 (some graph somewhere I found via google - here it is http://software.intel.com/en-us/blogs/wordpress/wp-content/uploads/2010/03/Picture12.png). At best Steam shows 12.5% of the market, but it is not a representative sample, despite the size of the sameple.
If I surveyed the most populous US state (California) which has 12% of the US population (almost the same % as Steam is of gaming PCs based on Intel's estimate), I would not get an accurate view of the United States. I would get an accurate survey of California.

 

[happy medium]

Amd cards are underrepresented in the Steam survey due to a bug. So no, its not representative.

I said this in post #5 ,but I don't think anyone believes me.:\

 

[Tempered81]

Prove it.

Well you can be particular and paranoid all you want, but it's real simple man.

Steam pulls the data off 30 million PC gamers running their software. Then they slap the data into tables and pie charts. I'm sure they aren't lying to us.

I would guess that this much is accurate.


I hear what you're saying about 44/56 for steam numbers on 5800/5700 versus what AMD said they actually shipped for those cards. Who knows. There's obviously an issue here, but how does that disprove that steam gathers data incorrectly?

I would imagine the people who have steam on a PC know what a video card is and do not purchase mobile gpus, IGPs and htpc cards to play their games. Even though these type of cards sell plenty and are in Nvidia & AMD's sales figures, they likely DO NOT give as large a representation in the steam data as discreet cards.

The steam data IMO reflects more discreet gpu marketshare. I'll bet 90% or more of the 30mil on that sample have a discreet card, whereas actual sales are 50/50 mobile products

Lonyo, in your opinion, if Steam GPU survey isn't accurate, I challenge you to provide a more accurate source of data. You could pull every press statement from both camps (like you did with 57xx and 58xx) but do you expect that to be more accurate than steam or gpuz?

GPU-Z's data says
Nvidia 58%, AMD 36%, Intel 6%
http://www.techpowerup.com/gpuz/

Steam's data says
Nvidia 59%, AMD 33%, Intel 6%
http://store.steampowered.com/hwsurvey

 

[Voo]

I said this in post #5 ,but I don't think anyone believes me.:\

Yeah which probably has to do with the fact that most people don't seem to have heart of that bug and nobody so far has shown any example of what this bug looks like.. not especially helpful

@Lonyo: Huh, so steam overrepresents cards that are most likely to be used in gaming PCs contrary to those often enough included in cheap built PCs, HTPCs and whatnot? Not especially surprising and I've yet to see people using steam to judge the relation of cheap to expensive cards - because that's obviously flawed. But for other arguments, eg comparing cards in the same market segment that argument doesn't hold.

 

[Martimus]

The Steam Survey doesn't cover all 30+ M users. It only covers those that take the survey. I have only been asked 3 times to take the survey over the many years I have used Steam. Twice within a few months of each other, then again about 2 years later. It seems rather random.

 

[titan131]

As often as I disagree with ocguy, he's absolutely right on this. 6850 is a POS card. People who do their homework would buy a 580. 6850 is crap.

lol... they are in completely different price brackets. If money is no issue then u are right but it is for most buyers...

 

[styrafoam]

I think the whole issue of people responding to the steam survey being taken as the last word on an install base is kind of sketchy. It is a good snap shot of steam users who respond to the survey, but trying to project that on to the rest of the planet is a little off. I have never done the survey even though i have a steam account, i have at least 5 friends at work who are mainly MMO players and they don't even have steam accounts. Every holiday sale i try and talk them in to signing up to score some cheap games and they just shrug thier shoulders.

 

[Lonyo]

Yeah which probably has to do with the fact that most people don't seem to have heart of that bug and nobody so far has shown any example of what this bug looks like.. not especially helpful

@Lonyo: Huh, so steam overrepresents cards that are most likely to be used in gaming PCs contrary to those often enough included in cheap built PCs, HTPCs and whatnot? Not especially surprising and I've yet to see people using steam to judge the relation of cheap to expensive cards - because that's obviously flawed. But for other arguments, eg comparing cards in the same market segment that argument doesn't hold.

See the example of something disappearing from the survey for a month GTX260M), or not appearing while another card does (HD6850 vs 6870).

So, did AMD sell absolutely no HD6850s in the first month while selling some HD6870s?
What happened to all the GTX260Ms?

Also one bug is the fact that Crossfire simply doesn't register for most people, so NV has >95% multi GPU marketshare vs <5% for AMD. Not representative at all.

Now your argument is saying that it's fine comparing cards within the same "bracket".
So if we compare the GTX460~480 (IIRC) which almost all shipped with Mafia 2 (a game which REQUIRES Steam) or Just Cause 2, which is the same, to the relevant AMD competition which didn't ship with a Steam requiring game, how do you judge sales numbers accurately?
One card is more likely to appear on Steam simply based on the fact that every user gets a free piece of software which is tied to Steam.

 

[Genx87]

Those cards represent such a small fraction of the market. It is possible the amount of users who took the survey didnt register to a meaningful number on the survey for the month. We are talking about cards according to the numbers represent 1:200 cards on the survey. It isnt hard to imagine how from a statistics standpoint they can be left out. But so what? If a card that represented 5-10% disappeared for a month. I would be more worried.

 

[Lonyo]

Those cards represent such a small fraction of the market. It is possible the amount of users who took the survey didnt register to a meaningful number on the survey for the month. We are talking about cards according to the numbers represent 1:200 cards on the survey. It isnt hard to imagine how from a statistics standpoint they can be left out. But so what? If a card that represented 5-10% disappeared for a month. I would be more worried.

The comments in the OP were comparing something with 2.49% of the DX11 market to something with 1.85% of the DX11 market.
I showed a card with 0.56% of the entire market disappearing.

So yes, it does matter, because if you are trying to compare two things with small percentages, and significant percentages can simply disappear (0.56% of the whole is greater than the 0.66% difference between two DX11 cards), you can't reliably compare them, can you?

 

[SirPauly]

Imho,

It depends on extreme posters cheer leading their favorite color.

What matters more-so is the bottom line: Revenue, profit and margin and where the GPU over-all share and what the discrete share is.

Many posters here may be passionate about technology and what this technology may do for the games they play and GPU discrete desktop may matter most. Even though the numbers don't look great in Steam, well, AMD did gain some share in the desktop discrete.

However, what is interesting about steam is the break downs on the different sku's.

 

[-Slacker-]

Wait, so the survey isn't automated - as in, everybody who makes a steam account has his rig's specs automatically registered in the steam data banks? It requires you to actively partake in the survey and "fill in the blanks", just like regular internet surveys?

Well, then, the sample size is not 30+ million ... it could be just a small fraction and, combined with the fact that the survey is not unbiased, then I don't see how it could hold much credibility ...

 

[Seero]

Once upon time, I believe that the percentage on the piece of paper people called "report card" is inaccurate, it can't be used to represent my ability in whatever I was in. Everyone who got below 100% agreed with me, while those who doesn't belong to the class/school disagree, especially teachers and parents. Interestingly, those who got higher marks believe that are superior than those who got lower marks, even if the difference is only 1%.

A: "I got 79. So unfair", a student who study hard.
B: "It is fair, I got 80.", a student who copies off A.

 

[-Slacker-]

You make an interesting point

 

[notty22]

Once upon time, I believe that the percentage on the piece of paper people called "report card" is inaccurate, it can't be used to represent my ability in whatever I was in. Everyone who got below 100% agreed with me, while those who doesn't belong to the class/school disagree, especially teachers and parents. Interestingly, those who got higher marks believe that are superior than those who got lower marks, even if the difference is only 1%.

A: "I got 79. So unfair", a student who study hard.
B: "It is fair, I got 80.", a student who copies off A.

This is my same experience while visiting television show forums and The Nielsen ratings ! Oh boy, are fans ever in denial about those results. When someones favorite show is about to be canceled nothing is what it seems !
The reality is , Nielsen is probably very accurate, everyone's mother, aunt and grandmother do watch DWTS !

 

[Vesku]

Considering the Steam survey claims to have no idea what sort of card my EVGA GTS450 is, I'll have to side with not using their numbers for anything requiring you actually hit the barn on it's broadside.

 

[PingviN]

yet the company STILL makes no profit. amazing.

AMD is much more than their graphics department. You either know this and act stupid or you need to read up. AMDs main business is CPUs and that business is what decides profit or loss.

 

[Genx87]

The comments in the OP were comparing something with 2.49% of the DX11 market to something with 1.85% of the DX11 market.
I showed a card with 0.56% of the entire market disappearing.

So yes, it does matter, because if you are trying to compare two things with small percentages, and significant percentages can simply disappear (0.56% of the whole is greater than the 0.66% difference between two DX11 cards), you can't reliably compare them, can you?

I'd chalk it upto the error range. We are talking about fractions of a % point. So yes, you can compare them. Looking at your data. I would say the range can vary about a half % point in either direction. I would look at trends on those cards. Over time if the rate stays the same. It is a fair assumption the data is reliable as far as steam is concerned.

 

[Arzachel]

I'd chalk it upto the error range. We are talking about fractions of a % point. So yes, you can compare them. Looking at your data. I would say the range can vary about a half % point in either direction. I would look at trends on those cards. Over time if the rate stays the same. It is a fair assumption the data is reliable as far as steam is concerned.

The Steam survey isn't unrepresentative because the HD 6850 didn't show up for a month, its because the system itself is flawed. It just plain doesn't recognise some kinds of GPU's. Chalk me up as another person, that has an unrecognisable GPU.

Would you cite a benchmark, that randomly skews its results, as fact because you don't have anything better to go off of?

 

[Genx87]

The Steam survey isn't unrepresentative because the HD 6850 didn't show up for a month, its because the system itself is flawed. It just plain doesn't recognise some kinds of GPU's. Chalk me up as another person, that has an unrecognisable GPU.

Would you cite a benchmark, that randomly skews its results, as fact because you don't have anything better to go off of?

A benchmark and a poll are apples and oranges. A benchmark is measuring a reproducible result based on the performance of the part. A poll is taking a sample of what is out there and is subject to an error variance.

Take for instance presidential polls. Those have error ranges upto 5-7%. We are talking a possible .5-1.0% error rage here. That is miniscule. People's GPU's that dont show up in the survey or represents too small of a sample to show as an independent data point fall under the title of "other". "Other" represents .74% of DX11 GPU's as of April.

 

[bryanW1995]

GTX 580 owners tend to be smarter, better looking, and drive nice cars.

Haha, they would be "better in a dodge", however.