Suggestions for Individual Video Card Reviews

[First]

By next week I expect to release our first individual video card review in quite some time. It'll most likely be a shootout between a retail 9800 Pro and a retail 5900 (or Ultra). I need to know what you guys would like to see in this review. What type of AA and AF combinations and at what resolutions do you want to see these benchmarks being run at? What type of information do you want explicitly underlined in this review, be it related to benchmarks, accessories, drivers, etc. This doesn't just apply to next week's review, you can list general suggestions for all future individual video card reviews and roundups.

Let us know as soon as you read this thread so our L.A. team can get these things included by the end of this weekend.

Take care,

Evan

P.S. We also plan on having a retail 9200 versus 5200 and 5600 versus 9600 shootout posted before the end of July, with a full blown "VGA Charts" thrown in, sort of like what Tomshardware does.

 

[stardust]

thnks, im looking forward for this.
may i suggest theoretical tests, maybe a word on the VIVO technology and what to expect for each card's future proof"ness".
I would also like a rarer 2XAA/4XAF benchmark for those who want a blend of image quality and performance.

 

[beserkfury15]

wow, that's a great idea, should help me on my next vid card upgrade. by chance, can you put in some vice city benchmarks, and maybe mafia too, running at 4xaa/8xaf. i don't play first person shooters too often, and these are common played cames.

 

[Tab]

Agreed. I would like to have some information on the TV-Out system. Maybe a explanation for the noobs about the situation with 3dmark03? Along with a comparison between the 9100 Pro, 9200, 9200 Pro and 9700, 9700 Pro and 9800 and 9800 Pro and the same with nvidia's card lineup.

 

[oldfart]

First, custom benchmarks that do not have the possibility of cheats. Many other sites have done this with some eye opening results.

Second, benchmark with equal IQ. The actual IQ of the two cards must be set as close as possible. This does not mean driver settings. If card "A" 2X AA has the same IQ as card "B" 4X AA, then that is how the test should be run. Same for AF. Equal actual IQ, not IQ driver settings.

Since many people have moved to flat panel LCDs with 1280 x 1024 native resolution, that would be a good res to use as one of the tests.

For AA/AF settings, a min mid and max would be nice. Again, same actual IQ, not driver settings when comparing cards. I know this is difficult and somewhat subjective, but, it is really the way it should be done.

With the high GPU clocks we have these days, cooling and therefore, noise of the card is now also a major consideration.

 

[sash1]

I would like to suggest PowerVR's village mark!

Thanks,

~Aunix

 

[DaveSimmons]

^^ I second oldfart's suggestions on 1280x1024 resolution and on custom demos / timedemos that haven't yet been "optimized" for. You might as well skip Quake3 since 290 fps vs. 293 fps isn't too relevant but UT2003 is worth seeing.

(edit) if you can work around this:

Rand Now it seems nVidia forcefully disables trilinear filtering in UT2003 regardless of in-game/driver settings.

Too bad you probably can't get another, un-sponsored look at Doom3 to see if id has improved the ATI code path.

 

[rbV5]

If you had the chance to toss in an AIW 9800 pro and give an indepth- multimedia revew on MMC 8.5 including the EZLOOK interface and the Divx enhanced player with the Cat 3.5 drivers. I'm hearing its equipped with faster rated ram than the standard 9800 pro, so maybe overclock it as well, could be the fastest 9800 pro card available.

Also, it would be nice to see some tests run on older popular games like CS ect. for compatability and driver quality using both AMD and Intel platforms.

 

[Rand]

Synthhtic test of Pixel/Texel trhoughput using 3DM2001.

Synthetic VS/PS 1.1/1.4/2.0 tests, most noteably using 3DMark and ShaderMark.. make note of the known cheats in both.

Compare anisotropic filtering using "samX"'s Anisotropic Filtering Test application. D/L

Compare FSAA using Colorless's 'FSAA Viewer'. D/L

Compare Hardware vs. Software geometry performance.

Compare overdraw reduction capabilities depending upon rendering mode using Humus GL_EXT_reme benchmark. D/L

Make note of known cheats in UT2003/3DM2001/3DM2003/Quake3/ShaderMark if you use any of those benchmarks.

Testing Stencil Buffering using PowerVR's FabelMark, and perpass texturing performance difference using Serious Sam would be nice.

Test with and without UniWinder's Anti-Detector scripts, to show performance differces with/without driver application detection. D/L

Compare in-game FSAA/Anisotropic filtering image quality across the various options in application/performance/quality modes on all boards.

Test that Trilinear filtering is indeed being done in those games that allow it, as some drivers seem to be forcefully disabling depending on the game in question.

Test games from MULTIPLE GENRES!!!!!, use FRAPS if necessary (There are more then just FPS in this world) , along with SpecViewPerf and other Pro3D tests if viable.

Test using CUSTOM DEMOS, as well as readily available publically distributes demos.

Compare TV-Out, VIVO, Multi-Monitor capabilities.
Test TV-Out, DVD image quality.


Obviously including all of the above would be asking for way too much, and would require a ridiculous amount of testing time... but please do try to cover at least a few of the above requests.

 

[JSSheridan]

Splinter Cell if at all possible. Keep up the good work. Peace.

 

[Pete]

I kind of like GamePC's simpler yet informative take. I suggest two or three resolutions (10x7, 12x9, and 16x12 optional, depending on the cards tested) and four settings (plain, max performance AF, 2xAA 4xAF, 4xAA 8xAF). I'm forced to include AF-only as a separate category (rather than stick with plain and AA+AF) as it's often so cheap to select and can greatly improve IQ. It's another graphs to bench and make and view, but it's worthwhile. I'd like a comparative screenshot of each card at each settings for each test (in anticipation of when I can look at screenshots, determine which looks equal, and then view only those benchmarks).

I'd like a mix of games, not just the standard FPS crew with their built-in timedemos. Experiment with FRAPS. Choose popular titles at the time of the review, perhaps one unique one per review, to keep things honest. Try including a racing and flight sim title into your standard benchmark suite. Obviously, use custom demos (one public and one private, so the masses can compare on their own systems, and so that you have a reference to make sure no illegal optimizations are being added). I found the Hardware.fr racing sim benchmarks very interesting, as the 5600 performed unexpectedly poorly in non-standard benchmark titles compared to its standard-suite performance. Investigate this sort of thing! Tell us how various AA/AF implementations look in games other than FPSes: racings sims, flight sims, RTSes. Is AA effective? Does AF introduce texture shimmering?

Then, give various reviewers a card for a week and tell them to use it as they normally would (office work, gaming, video viewing), and include their comments on the "whole card" experience at the end of the review: driver setup, fan noise, signal quality, DVD and video playback, AA and AF implementations in-game. This will obviously take some time, so perhaps it's better to include in a separate article, after the initial benchmark and screenshot-fest--a second look.

Thanks. It's obvious reviewing now requires more work, but I'd prefer to see fewer, in-depth reviews rather than many benchmark-fests. I understand it may be important to have a review out on release day, so I suggest a benchmark-heavy article then, followed a week or two later by an in-depth, "after-the-honeymoon" review of the full card.

 

[FearoftheNight]

Originally posted by: oldfart
First, custom benchmarks that do not have the possibility of cheats. Many other sites have done this with some eye opening results.

Second, benchmark with equal IQ. The actual IQ of the two cards must be set as close as possible. This does not mean driver settings. If card "A" 2X AA has the same IQ as card "B" 4X AA, then that is how the test should be run. Same for AF. Equal actual IQ, not IQ driver settings.

Since many people have moved to flat panel LCDs with 1280 x 1024 native resolution, that would be a good res to use as one of the tests.

For AA/AF settings, a min mid and max would be nice. Again, same actual IQ, not driver settings when comparing cards. I know this is difficult and somewhat subjective, but, it is really the way it should be done.

With the high GPU clocks we have these days, cooling and therefore, noise of the card is now also a major consideration.

 

[Schadenfroh]

n/m

 

[Pete]

?

 

[BenSkywalker]

Talking about the software bundle would be nice. If one board comes with four or five games that someone is interested in, that $50 premium over the other brand could become a raging bargain(a simple yet extremely frequently overlooked item).

Has anyone at AT managed to get a workaround for the post filter/screenshot hosing going on with nV boards running AA? It would be nice to see a review where the actual end visuals end up posted as screenshots

A theoretical test of OD reduction, Pixel Shader performance and also Stencil fill would be nice. AF tests on varrying angles, AA tests on varrying angles would be welcome additions.

If you happen to have both an 59U and a 59 around, could you please run side by side benches(ideally at identical clock speeds) of UT2K3 @1600x1200 w/4x AA and show the performance gap? A lot of people are under the impression that the extra 128MB of RAM won't do them anything because it doesn't help the R9800 out that much not realizing that nV's AA functions in a very different fashion and can already be useful today(which actually is a good selling point on the more expensive 5900U, but also tilts the field towards ATi a bit as the $100 premium isn't as important).

A nice look at what exactly the xS modes of AA bring to the table would also be great. Counterstrike's Italy map showcases it nicely. Great tool for legacy titles with heavy alpha texture useage, and makes things look overall a lot smoother. This would tend to be in nV's favor although if a site the caliber of AT started talking about SSAA still having a place, maybe ATi would finally enable it in their drivers

If you do use SplinterCell as a bench, please make note that AA does not work in the game without serious image corruption and note that nV has disabled it due to this and ATi shows artifacts because they don't disable it while the game itself is the one with issues.

Would it be possible to bring back somewhat in depth 2D quality analysis? On the ATi side this isn't all that big of a deal, but it is very big on the nV side of things. Obviously when looking at a reference board it isn't going to make much of a difference, but I am extremely curious if any vendor of nV boards has caught up to Gainward on the 2D front yet.

VIVO capabilities would be nice to talk about also for boards that have it. Often times we see mention that a board has VIVO, but nothing on how well it works. Total experience as Pete mentioned would be an extremely nice addition. Grabbing a random group of twenty to thirty games and testing them all out in terms of playability would be nice too(from say, CounterStrike era on), but don't release what games you are testing with. That way, we don't have a list of games that any IHV fixed cause AT nailed them

 

[FluxCap]

ahh the fanATIcs are out in force tonight.

Huh? Mad because the latest benchies show 9800 beating the 5900? I hope that is not the case since you can't go wrong with either card and I would be happy to have either. I just chose to have the one that was a little less expensive.

Also, a second for a similar no BS review like this ONE , except with more detail of course :beer:

 

[Blastman]

Oldfart summed things up pretty well for me.

I think the biggest issue is what IQ settings are going to be used for the comparison. There is a huge difference in performance between the Quality and Performance settings. In this ukgamer article they ran several settings on the Nvidia and ATI cards but that may be too much work.

UT2003 : dm-suntemple performance

5900U ?

150.3 ? High-performance
126.8 ? Performance
107.3 ? Quality

9800 pro

136.2 ? Performance
116.7 ? Quality

My impression from this 3dvelocity review is that the High-performance settings on the Nvidia cards (old aggressive setting) produces terrible IQ.

? ?(Page 5 on the 5800U) ? ........the problem is that unless you really dial up the image quality options what you get is a final rendered screen that just plain sucks I'm afraid to say.

As I mentioned earlier though, the "Aggressive" setting really is too severe to be considered a fair option.

So having seen just how poor image quality is when the GeForce FX is set to "aggressive" I think we're perfectly justified in dropping this setting from all further benchmarks?.?

I think the Quality settings should probably be used if only one set of benchmarks is produced.

 

[oldfart]

If you do use SplinterCell as a bench, please make note that AA does not work in the game without serious image corruption and note that nV has disabled it due to this and ATi shows artifacts because they don't disable it while the game itself is the one with issues.

SC seems to be problematic in general with nV cards as well.
UKGamer on SC

Before we make any comments on the results, the first thing to note is that the NV35 cards doesn't render the ocean completely. There are vast gaps of "plain" sea. By plain we mean no ripples. The excuse we've been given is that it's a beta benchmark, now of course this is true, but all the ATI cards managed to render the ocean perfectly fine and considering Splinter Cell was made as an Xbox game in the first place (which has a NV2x derived part) it's quite frankly worrying.

However that aside, the R3xx cards dominate here, with the Gainward Ultra/1200 being the pick of the 5900s. So even though the NVIDIA cards aren't rendering vast areas of the ocean properly, they seem to taking the bigger performance hit.

 

[oldfart]

Originally posted by: Blastman
Oldfart summed things up pretty well for me.

I think the biggest issue is what IQ settings are going to be used for the comparison. There is a huge difference in performance between the Quality and Performance settings. In this ukgamer article they ran several settings on the Nvidia and ATI cards but that may be too much work.

UT2003 : dm-suntemple performance

5900U ?

150.3 ? High-performance
126.8 ? Performance
107.3 ? Quality

9800 pro

136.2 ? Performance
116.7 ? Quality

My impression from this 3dvelocity review is that the High-performance settings on the Nvidia cards (old aggressive setting) produces terrible IQ.

? ?(Page 5 on the 5800U) ? ........the problem is that unless you really dial up the image quality options what you get is a final rendered screen that just plain sucks I'm afraid to say.

As I mentioned earlier though, the "Aggressive" setting really is too severe to be considered a fair option.

So having seen just how poor image quality is when the GeForce FX is set to "aggressive" I think we're perfectly justified in dropping this setting from all further benchmarks?.?

I think the Quality settings should probably be used if only one set of benchmarks is produced.

There needs to be some sort of cross reference of the quality modes between the cards. A low/mid/high IQ with equal or as close to equal as possible actual IQ settings is needed. This would take a bit of work and time to sort out.

 

[BenSkywalker]

SC seems to be problematic in general with nV cards as well.

They could have covered the fact that SC can't run using the highest quality settings on ATi cards at all too, but that seems to be ignored most of the time Actually, they failed to mention if they were comparing them at equal settings or ATi at medium quality(the best it can do) and nV at high(the best it can do).

There needs to be some sort of cross reference of the quality modes between the cards. A low/mid/high IQ with equal or as close to equal as possible actual IQ settings is needed. This would take a bit of work and time to sort out.

With ATi being incapable of matching nV"s IQ in AF and nV being incapable of matching ATi's IQ in AA how should they go about that..?

 

[chsh1ca]

- Non-standard benches and/or demos.
- 1280x1024.
- I would love to see both OpenGL and DirectX benchies, in order to see which is better for which card in terms of raw speed (OGL always seems to win in the IQ dept).
- Perhaps a second testbed system with more standard configuration (AXP 2400+ or P4C 2.4GHz), just to give a rough idea how well the processors scale.
- Address the UT2K3 Trilinear filtering issue.
- Discuss driver support that each card has for alternate OSes (MacOS, Linux, etc.).
- Detailed breakdowns of everything.

In short, more than the last review I bothered reading, which consisted only of some bar graph pics and the next/prev buttons. (I was really disappointed, what can I say?)

 

[BoomAM]

When doing the 9800 vs 5900, in the benchmarks, include results from other cards, such as the 9500/9600/9700. Especially the 9700s, cos ive seen in alot of reviews lately, the 9700s being missing from the benchmarks.
Test at 1280x1024.
Zoomed in FSAA and AF comparisons.
General/2D image quality.
Testing on a veriety of pcs, with various video cards, from the 9500-9800 and the 5200-5900. Test on PCs at various speeds, like XP1800, XP2200, XP2600, XP3000, XP3200, and P4 1.8, 2.2, 2.6, 2.8, 3.0, 3.2.

 

[themanwithapc]

Well just a quick suggestion. How about giving a slightly more detailed look at the non-ultra and non-pro cards. For the mainstream market, many people will consider buying these, and in my opinion, they just aren't given enough coverage.

 

[Vonkhan]

How about something like a balanced scorecard ... its used in accounting to produce a holistic view: quantitative + subjective aspects taken into consideration

 

[oldfart]

Originally posted by: BenSkywalker

SC seems to be problematic in general with nV cards as well.

They could have covered the fact that SC can't run using the highest quality settings on ATi cards at all too, but that seems to be ignored most of the time Actually, they failed to mention if they were comparing them at equal settings or ATi at medium quality(the best it can do) and nV at high(the best it can do).

There needs to be some sort of cross reference of the quality modes between the cards. A low/mid/high IQ with equal or as close to equal as possible actual IQ settings is needed. This would take a bit of work and time to sort out.

With ATi being incapable of matching nV"s IQ in AF and nV being incapable of matching ATi's IQ in AA how should they go about that..?

I said it would be difficult. Other sites have done a reasonable job of getting as close as possible. Its not perfect and cant be, but you cant just set them both to 2X/8X and bench them. You do the best you can.