Are you happy with current Anand video reviews?

[Ackmed]

First off, this is not a slam on the reviews done here. Reviewing is a pretty thankless job, even though they get to play with new hardware first. Some are even full time employees and get paid. Recently (few weeks ago) HardOCP's reviewer Brent, made a thread asking what we they readers would like to see more of, less of, in reviews. So I thought I would make a poll, to see if readers here would like to see more, or are happy like they are.

My opinion is that reviews done like they have been for years, is not good enough anymore. The "mold" of choosing a few resolutions, and running timedemos without AA/AF, and then with 4xAA8/AF is not good enough. In fact, timedemos themselves are a flawed way of testing games in my opinion. It doesn not take into accoount A.I., physics, or other things. One glaring error with Quake4 and the 81.85 drivers, in a timedemo it doesnt even render shadows on the GTX. Making the numbers even further off from what you would get if you actually played the game. Someone said it did the same for ATi's card, but I couldnt verify that.

To me when someone is going to pay a lot of money, such as $750 for the 512MB GTX, they need as much info as possible. Testing it with TAA, HDR, 16xAF, and other high end graphical options are a must to me. I need to know how much these features take from my frames, and what they look like, and if the 512MB card gains the same percentage of performance as without. In the latest 512MB GTX review, Anand's didnt even use AF. This to me, is selling the reader way short in information that they need.

Perhaps this poll with help the reviewers to know what their readers want, and can advance their reviews in doing that. Or perhaps it will show I am the only one who thinks like this...

edit, ok I made the poll, but for some reason its not showing. How do I edit the poll in this post? When I hit "edit poll" all the info I put is still there, but its not showing up in the post?

 

[xtknight]

I'm happy with the benchmarks myself for relative comparison. If they added 3 or so highest possible setting combos for the 3 highest performing cards I'd be happy. That means 9 more tests.

I see space for the poll you just haven't added anything to it yet. Click the answers column for the specified poll.

 

[AnonymouseUser]

Originally posted by: Ackmed
One glaring error with Quake4 and the 81.85 drivers, in a timedemo it doesnt even render shadows on the GTX. Making the numbers even further off from what you would get if you actually played the game. Someone said it did the same for ATi's card, but I couldnt verify that.

Interesting. How could Nvidia's (81.85) drivers affect an ATI card?

To answer your question, I would like to see some variations in Anandtech's reviews similar to the way HardOCP does theirs.

 

[aatf510]

Actually, I am finding a lot of the video card review incomplete or they inaccurately represent what the users would see in term of real life performance. For example, you can see in Anand's review that a 7800GT would get over average 60fps in BF2 with 1600x1200 4xAA, which I believe it's impossible to achieve in real life situation. I honestly don't even find 1920x1200 0xAA quite playing in with a GTX KO because the average fps no where near 60fps.

 

[CP5670]

I thought the 512 GTX one was very good, better than the preceding ones, but a few things are still missing. They really need to show minimum framerate scores in particular; that's by far the biggest omission in most video card articles I see around the internet. If AT added that in (without doing the whole "playable settings" thing of HardOCP), they would easily have the best VC reviews as far as I am concerned.

I like the fact that AT tests things at 2048x1536, although it seems that a few more sites are starting to do that now than before. As you say some more combinations of AA/AF/HDR/etc. scores would also be nice. I recently read some of Rage3D's articles on nvidia cards and was surprised at how good they were, especially since they're an ATI fansite, and they have a lot of these types of benchmarks.

 

[AnonymouseUser]

Originally posted by: toattett
Actually, I am finding a lot of the video card review incomplete or they inaccurately represent what the users would see in term of real life performance. For example, you can see in Anand's review that a 7800GT would get over average 60fps in BF2 with 1600x1200 4xAA, which I believe it's impossible to achieve in real life situation. I honestly don't even find 1920x1200 0xAA quite playing in with a GTX KO because the average fps no where near 60fps.

Where did you get those numbers? According to the 7800GTX 512MB review, and using an "AMD Athlon 64 FX-57 (2.8GHz)" (faster than your OC X2 4400+), the 7800GT only gets 37.2 fps at 16x12 4xAA.

 

[Ackmed]

Originally posted by: xtknight


I see space for the poll you just haven't added anything to it yet. Click the answers column for the specified poll.

There is no button to "submit" the poll that I see.

 

[xtknight]

Originally posted by: CP5670
I thought the 512 GTX one was very good, better than the preceding ones, but a few things are still missing. They really need to show minimum framerate scores in particular; that's by far the biggest omission in most video card articles I see around the internet. If AT added that in (without doing the whole "playable settings" thing of HardOCP), they would easily have the best VC reviews as far as I am concerned.

I like the fact that AT tests things at 2048x1536, although it seems that a few more sites are starting to do that now than before. As you say some more combinations of AA/AF/HDR/etc. scores would also be nice. I recently read some of Rage3D's articles on nvidia cards and was surprised at how good they were, especially since they're an ATI fansite, and they have a lot of these types of benchmarks.

I agree. Minimum/lower quartile/upper quartile/maximum would be nice.

As long as the whole data set's there, it can't be that hard to do. I'm sure theres software to calculate the quartiles, and if not I'll gladly jump off a cliff now.

 

[lifeguard1999]

The problem I find is that games are CPU bound at low resolutions, so AT cranks up the resolution to 1920x1200 or 2048x1600 with 4xAA and 8xAF. That's great for showing that there is a difference between GPUs at ultra-resolutions, but not too useful for most people.

Personally I like the idea of using FRAPS to show how a game plays over time, or F.E.A.R.S. benchmark where it records percent of frames below 20 fps, percent between 20 and 40 fps, and percent over 40 fps. That tells me more than just a Avg fps on a benchmark.

 

[Nextman916]

What they need to do is get around to reviewing the R580.

 

[crazydingo]

AT's reviews arent extensive anymore.

Xbitlabs, techreport, firingsquad etc. all have 20+ pages of very extensive reviews of latest videocards.

I think AT is spreading itself too thin over other areas that they are probably finding it difficult to keep pace.

 

[otispunkmeyer]

good thread

i think AT's reviews are concise and provide all the relavent info needed, but yes at the end they still feel abit lacking. i usually complement AT's reviews by reading all the other major sites reviews.

i find HardOCP's abit too much to take in....with their fancy graphed outputs which to me look messy and too small to be useful, then you gotta root through a table to find and then compare numbers, takes too much time.

I like the way Guru3d do their's and their graphs are pretty swish, but again they are much like AT in terms of format.

what i do really like though, is Bit-tech.nets format, initially they dont show graphs of 'x' resulotions with and without AA/AF, they just have one table for each game containing what they consider "the best playable settings". they later follow up with apples to apples tests so the fanboys can say ZOMG!!1!!111! X1800XT PWNS THE 7800 BY 1FPS!! ATI RULEZ ATI PWNS j00 IN THE FACE!

i know its just their opinion of whats playable, but for me at least i usually find their suggestions spot on or even a little conservative sometimes.

 

[johnnqq]

just run around and play a specific level for 10 minutes(or whatever is most accurate) with each card.

 

[dorkbert]

It would be nice if review can go over VIVO feature, seeing that it's rapidly becoming a rather "standard" thing to be included on video cards. After all there are only so many different ways a review site can run the bench marks, and pretty much that's the only thing every review site I've seen ever does. It would be nice to see comparison of VIVO quality on different cards in reviews.

 

[Avalon]

I find AT graphics articles too short nowadays, and I know that in some (not sure if all) of the reviews, they disable sound and bench that way. While I know this lets them get a more direct GPU to GPU comparison, no one plays with sound disabled, so the numbers they get are not realistic.

 

[BFG10K]

The best results I've seen are in the Rage3D reviews - clear graphs at various resolutions (including 1920 and 2048) with the ability to mix-and-match AA and AF as you please. I wish every reviewer adopted this approach.

Timedemos however are still the best way to test games since they benchmark actual gameplay. If you take them away you have nothing with which to compare.

One glaring error with Quake4 and the 81.85 drivers, in a timedemo it doesnt even render shadows on the GTX.

Quake 4 has issues with demo playback, not the two vendors. It's a problem with Quake 4 and it actually drops a lot more than just shadows.

 

[Rage187]

I like Driverheavens testing methods and display of results.

 

[aatf510]

Originally posted by: AnonymouseUser
Where did you get those numbers? According to the 7800GTX 512MB review, and using an "AMD Athlon 64 FX-57 (2.8GHz)" (faster than your OC X2 4400+), the 7800GT only gets 37.2 fps at 16x12 4xAA.

http://anandtech.com/video/showdoc.aspx?i=2519&p=4

 

[mwmorph]

AT needs a few things. xbit's sometimes used max, min average graphs, and hardocp's sometimes used line graphs showing fps during a section of gameplay that plots second to second variations. Also, some cpu scaling grapgs would be nice to end the ageless question, will my 3200+ bottleneck my 7800gt?

most importantly, more resolutions. the mobile x1600 review was pitiful. 3 pages? no graphs?? wtf? ghet more resolutions and games. maybe throw in a age of empires 3?sims 2?

 

[Hacp]

IMO, Xbitlabs does the most extensive cpu/video reviews out there. Most of their articles are top quality. Some are written by crap writers however, but I would say 70% of their articles are good enough to be feature articles in a magazine.

 

[Sunrise089]

3 changes, in order of desire:

1) Include many cards in each comparison - stop not including popular cards like x800xl and x850xt in reviews of new games.
2) Test on a variety of hardware - even though many overclock, we don't all have 2.8ghz FX-57s, so maybe test with either a 512k cache Athlon 64, or at least test with something clocked at like 2.4ghz.
3) Test as many resolutions as possible with and without AA/AF - especially low resolutions for those with CRTs that may lower resolutions in order to enhance playability (sp?) and widescreen resolutions

 

[AnonymouseUser]

Originally posted by: toattett

Originally posted by: AnonymouseUser
Where did you get those numbers? According to the 7800GTX 512MB review, and using an "AMD Athlon 64 FX-57 (2.8GHz)" (faster than your OC X2 4400+), the 7800GT only gets 37.2 fps at 16x12 4xAA.

http://anandtech.com/video/showdoc.aspx?i=2519&p=4

Why there is so much of a disparity between those two tests?

 

[lifeguard1999]

Originally posted by: Sunrise089
3 changes, in order of desire:

1) Include many cards in each comparison - stop not including popular cards like x800xl and x850xt in reviews of new games.
2) Test on a variety of hardware - even though many overclock, we don't all have 2.8ghz FX-57s, so maybe test with either a 512k cache Athlon 64, or at least test with something clocked at like 2.4ghz.
3) Test as many resolutions as possible with and without AA/AF - especially low resolutions for those with CRTs that may lower resolutions in order to enhance playability (sp?) and widescreen resolutions

The problem is that with 2 & 3. the CPU becomes the limiting factor. Would you really want to buy the 7800 GTX if a 6600GT is just as fast since you are CPU limited??? (In theory I agree with you though.)

 

[Munky]

I would like to see some tests with TRAA/AAA running, since both companies tout this feature, but almost no review benches with those features enabled. Also, some IQ tests would be nice. I know it's alot easier to just pretend both cards are rendering what the game developers intended, but now that both sides make app-specific optimizations, it's pretty naive to assume none of those optimizations degrade IQ.

Another nice thing to see is to max out the IQ settings on each new card and see how playable it is. That means if Nv claims 8xAA and Ati claims 16xAF, I'd like to know if those features result in a slideshow or not, and how big of a hit they induce.

 

[Sunrise089]

Originally posted by: lifeguard1999

Originally posted by: Sunrise089
3 changes, in order of desire:

1) Include many cards in each comparison - stop not including popular cards like x800xl and x850xt in reviews of new games.
2) Test on a variety of hardware - even though many overclock, we don't all have 2.8ghz FX-57s, so maybe test with either a 512k cache Athlon 64, or at least test with something clocked at like 2.4ghz.
3) Test as many resolutions as possible with and without AA/AF - especially low resolutions for those with CRTs that may lower resolutions in order to enhance playability (sp?) and widescreen resolutions

The problem is that with 2 & 3. the CPU becomes the limiting factor. Would you really want to buy the 7800 GTX if a 6600GT is just as fast since you are CPU limited??? (In theory I agree with you though.)

Then at least show it to me. I'm thinking in reviews such as FEAR though, even if it was CPU limited I would see how well a possible system could run the game at all. I know that isn't important for a site that only does GPU articles, but since AT deals with CPU performance anyways, seeing how a game performs, even if CPU limited, can still be very usefull.

Cliff note: Knowing whether or not I need to upgrade my CPU for a game like FEAR if I know I am already buying a 7800GT is usefull.