If they play through with FRAPS, how do they guarantee they are getting the same thing each time for the different cards?
Also, did they ever post what settings they used for the Nvidia AF and Trilinear optimizations? I can never seem to find out how the different sites are settting those, but maybe I'm just missing it when I read the article.
My only issue with the H review is that they make a judgement call on what they consider "equally playable" but often don't let the viewer see the other results. I'm glad they added the "apples to apples" section, since it helps clear that up a bit, but when they do the "apples to apples" why can't they use the same setting they use in the initial test. If you look at the Far Cry "applies to apples" they use 1600x1200 4/16, whereas they use 1600x1200 2/8 in the initial test. The ATI card performs essentially the same at those two settings, (min FPS is 1 different, average is 3, max is quite a bit more, but not really important). So if they are really evaluating "best image quality @ a certain speed" then they should have used 4/16 AA/AF in the original test. I don't fault their layout, and I really like the fact that they show the graph for the entire demo instead of just min, max, and average. But they use inconsistent judgement and don't give us enough information to read between the lines. If they could improve their scientific method I feel that their reviews would be much more informative since we could use them in conjunction with the others out there to try and determine how the cards would work in our situation. Currently though, I feel their reviews are incomplete.
Also, I wonder what causes the low points in the charts. It seems like sometimes you see all the cards take a dip at certain points, which makes sense, but then other times it seems like only one of the cards will take a dip, while the other doesn't. It makes me wonder if there are other things happening that cause that. For example, if you look @ the frist Far Cry chart, you will see that there are two occasions (51 and 61 time) where the 6800U/OC drops down to it's lowest point, but the normal 6800 is about 5 FPS faster in those specific locations. Anyhow, little things like that just make me wonder how repeatable the results are.
I don't know if they are ATI biased or Nvidia biased (and don't really care), and instead use their site just like I use all the others: as another data point to use to find a trend that indicates whether one card is significantly better than another. Since they leave many unanswered questions in their reviews, I usually don't give them as much weight as other sites, but data is still data, and the more the merrier IMO. So far, it looks like the X800XT is the fastest card in pure FPS, but the 6800's are close behind and have some additional features so all these cards are good choices IMO. Nice not to have to worry about getting a looser card this time around.
-D'oh!
