First off I would like to say that I am a frequenter of HardOCP. My post is likely biased in favor of HardOCP, but I am going to do my best to remain unbiased and looks solely at the "facts".
Some of the posts here are absurd, especially in light of the fact on how everyone is claiming superiority for the scientific method and people able to back up claims and have proof for your statements.
For all the people claiming that crysis is one game, and that because HardOCP did not show the same results for other games that they are wrong. In scientific theories it only takes one case to prove an entire theory wrong, even if there are million cases that support the theory all you need is one case where the theory is wrong to invalidate the entire theory. Scientific method/theory doesn't work on a only when its convenient for us basis.
Then there are the people who are claiming they need 100% repeatable proof before they are convinced of anything, but then they turn around and make absurd claims that Kyle probably picks the best run for card a and the worst run for card b, or picks a scenario to favor card a over card b without any sort of proof. If you guys are going to stick to your proof guns at the very least don't make up stuff about Kyle cause you don't like him calling AT out. I am not saying Kyle is right for calling AT out, but for you guys to say Kyle calling out AT was wrong and unprofessional and then to turn around and say that Kyle picks and chooses data to fit the results they want, without any sort of proof, is outright hypocritical.
To the guys discussing population and samples, the population would be all the frames one could create in the entire game, or at least that level, and Kyles run-through would be a sample of that. Now granted both samples are likely different and it is impossible to recreate the exact same sample by hand. But two random samples from the same population should give the same averages. Now I understand they aren't taking random samples, but we can conclude that two similar samples from the same population, will give similar results.
To the people saying that because HardOCP compare apple to oranges that it is impossible to distinguish performance differences because you could run card a at 600 x 800 and card b at 1600 x 1200 and get the same frame rate and determine that the cards are equally fast is again absurd. First HardOCP determines the "fastest" card as the one that can play at the highest settings IE: 1600 x 1200 would be pronounced a faster card then one run at 600 x 800. HardOCP doesn't rate the fastest card by the best average frame rates in their apple to oranges comparison like everyone seems to going off about. Since HardOCP gives its readers all the setting information readers can come to their own conclusions IE: card a has an average fps of 20 and card b has an average fps of 20 at the same resolution, but card b is running with 4xAA and 16x AF while card a is running no aa or af, anyone can easily see that card b is the faster card. There is usually enough information present that you can tell which card is faster even though they are using apple to orange comparisons.
http://www.hardocp.com/article...wzLCxoZW50aHVzaWFzdA==
We can see here that the geforce has an avg fps of 28 and the radeon has an avg fps of 26.2. We can also see that the shader quality on the geforce is set at high, while the 1.8 avg fps difference is negligible especially in RW testing, we can conclude that the geforce is a faster card cause it is running at a higher setting.
Also note on this page that they describe the exact map they choose to play, and some of the various effects, affecting the graphics card, and the length in which they played. This is for all the people claiming they chose specific effects for one video card over another, and for those saying they probably just played for 10 seconds. I understand that this isn't a save point or a video but it does give a lot of data about their run through the game. Someone could easily go to the selected map, and make a 10 or so minute run through containing the listed effects and if the data gotten was drastically different in that scenario then HardOCP's we could conclude that something was wrong with someones data.
And while what the highest playable setting is subjective to each reader, for the most part the criteria for a game being unplayable should be the same for either card for the reviewer. Also Kyle many times addresses why the card wasn't capable of running at higher settings.
Also making demands that someone do something or all there data is false, does not make it so. And if all data was held to the meet my demands or your data is no good standard no data would be good, as everyone would have ever increasing demands. While I would love to see more openness from HardOCP reviews, including save points and videos of the run through, the fact that they don't doesn't make there data somehow false. Making demands on how things be done on someone else's forum, and making threats (if you don't do it, your a bunch of pussy's/liars/corporate whores/etc...) has never been a good way to get things done the way you want them. Although I don't frequent the HardForum from what I have read Kyle seems pretty open about there testing procedure and want to do what their audience would like to see from them. Perhaps if you had written Kyle a nice message saying you would like to see the save points and videos added to the gfx card reviews on the HardForum you might have gotten a better response. But making demands from the AT forums is surely not going to get you what you want, and you'll continue to claim the HardOCP sucks because they didn't bow down to your every demand.
Again going back to the data HardOCP gives its users, if the fps are within 5 frames of each other or 10% at 50fps, that it could be chalked up to human error, but usually the data presented in that card a is clearly faster then card b or that card a and b perform similarly in this game. Again there data is their for the user to come to their own conclusions. While I agree that without apples to apples it may be hard to directly compare how much faster card a is then card b, but you can always tell which card is faster, and what kind of performance in you can expect with card a in game c.
Some people said it already, and I'll repeat it if card a performs better in game, but card b benches better, and AT says card b is the superior card, and when you go to play game c and card b isn't faster then the card a you upgraded from your going to feel cheated and misled.
I realize that this is biased in favor of HardOCP. Also I know I gave Kyle the benefit of the doubt in this thread, but it only seems fair to me to give someone the benefit of the doubt unless there is proof stating otherwise.
Facebook
Twitter