Why the Steam hardware survey is a joke for what people try to use it for.

[Lonyo]

1) Me, from May

You really don't want to look at the Steam numbers for the simple fact that they don't really bear any relation to the actual market.
I say this based on the fact that the 2 products that hard numbers have ever been released for, the HD5700 and HD5800 cards, shortly after release showed a 62.5 to 37.5% sales split (500k vs 300k said AMD) while the Steam figures for these cards at the end of the month of that release following showed them to have a 44/56 split (end of December).

Now, when the company making the products says the split is 62.5/37.5 and Steam says it's 44/56, that shows that there is a CRAZY margin of error when it comes to the Steam figures. Which means that when talking about the overall market or general trends trying to use Steam figures is a terrible idea which in no way presents an accurate picture of anything.
And that's based on Steam numbers which are fairly accurate because basically they were the only DX11 cards available and had only been available a short time and were in a category of their own.

But if you would like to use the Steam numbers for saying anything beyond "this is a typical computer for a Steam user", feel free, but be aware that anything you try and say using said numbers is likely to be utterly out of whack with the real world.

http://forums.anandtech.com/showpost.php?p=29892571&postcount=33


2) Here's a picture showing another problem with month to month figures:

3) Continuing from 1, which has hard numbers, here are some segments:
http://www.xbitlabs.com/news/video/...h_End_Mainstream_Graphics_Cards_Research.html
Steam is clearly top heavy, towards "gaming cards", which means it is in no way representative of overall sales, as some people seem to use it for, be it in the overall market, or within a specific area of the market (e.g. DX11).



So, with #1 I have established that there is no way in which Steam is accurate for arguably "gamer" cards (assuming you consider the 5700 level performance to be gamer level, I would certainly hope you do).
With #2 I have established that Steam is full of inconsistencies and the data seems to have holes in it.
With #3 I have shown that Steam in no way represents the overall market.

So with the Steam hardware survey we have something which:
Doesn't accurately reflect sales within the 'gamer' product category.
Doesn't have any degree of reliability.
Doesn't represent overall sales of graphics cards.

Please stop trying to use it to show anything other than what the Steam market is made up of.

Steam has ~25 million accounts, and Intel estimates the gaming market at ~200 million PCs as of 2010 (some graph somewhere I found via google - here it is http://software.intel.com/en-us/blogs/wordpress/wp-content/uploads/2010/03/Picture12.png). At best Steam shows 12.5% of the market, but it is not a representative sample, despite the size of the sameple.
If I surveyed the most populous US state (California) which has 12% of the US population (almost the same % as Steam is of gaming PCs based on Intel's estimate), I would not get an accurate view of the United States. I would get an accurate survey of California.

 

[Wreckage]

I'm starting to think Steam scans a small percentage of it's customers every month and that's what they post. This may explain all the whacky numbers. I can't remember the last time they asked me to participate in the survey.

By the end of the month we should have market share numbers from JPR and financial reports from both companies. So for those who are concerned about numbers, this will be hard data and not poll results.

I like the steam survey, however the market share numbers will be a truly accurate depiction of what's going on.

 

[Grooveriding]

1. The numbers are delayed and represent last month.

2. Each column represents the change on a month to month basis. Some months there will be no variance. There does not have to be a change every month. This is even more true when you are discussing cards that take a small segment of the market, 260M, 580.

3.) Most every DX11 card is a gaming card and common sense is that the majority of DX11 cards are going in the machines of people who game, and likely, use Steam.

4.) Statistics are never perfect, particularly when you are drawing on a huge segment. Steam takes a large enough picture of that segment to have some credence.

5.) http://forums.anandtech.com/showthre...md+marketshare Alternate source in that thread that gave numbers almost exactly the same as Steam's for that time.

6.) Provide a better source if you have one. Steam is currently the best. No one claims it is infallible, but your claims that it is useless are as incredulous as a claim that the survey is perfect would be.

 

[Patrick Wolf]

Eh, who cares. Why are people concerned with what hardware people are using anyway?

 

[Texashiker]

Eh, who cares. Why are people concerned with what hardware people are using anyway?

When making hardware purchases, I like to know what the current trends are.

4.) Statistics are never perfect,

6.) Provide a better source if you have one. .

+1 on both of those.

If you take steam away, or discredit it, what do you have to replace it with? What other service directly measures its user base like what steam does? On top of that, what service poles such a targeted niche?

From what I remember, users might get asked every 3 months what their hardware configuration is. I dont knwo why, but the time line of 3 months comes to mind.

Its not important to look at the last couple of months. When looking at the steam stats, look at the overall trends. At what rates are people adopting DX11, at what rates are people going to quad and hexacore CPUs,,,,, and so on.

Steam hardware survey is a great service, just dont try to put one little section under a microscope, or else your going to miss the big picture.

 

[Absolution75]

Eh, of course steam surveys are inaccurate. . .

I believe what they do is survey a single account every year or at the start of your account.

Sometimes they completely reset the stats though.


steam hw survey is more of a long term thing than something you can show on a month to month basis. The survey group is large enough though for most cases - niche products won't really show up because of the way they survey though.

 

[blanketyblank]

All evidence is relevant as long as it tends to show something is more likely. It doesn't have to prove something beyond any doubt. When you have a lot of evidence that points to something chances are that event actually happened, or more importantly people believe it happened and treat it accordingly.

If you're going to ban everything that's imperfect or disagreeable next step is to start book burnings.