What would you like to see in a graphics card review.

Keysplayr

Elite Member
Jan 16, 2003
21,211
50
91
Hey guys,

I'm just looking for some concerns that you all have regarding graphics card reviews. What do you not see that you wish were included in a review?

In other words, some input for "follow up" reviews. Most sites have their initial reviews of new cards, and sometimes afterward, may publish follow ups if newer drivers come out, or an image quality comparison.

I want to know what you want to know. Keep in mind that initial reviews take up HUGE amounts of time, and not everything you wanted to see may have been included. So we have follow up reviews for certain aspects of the initial review not covered to satisfaction.

Give me some ideas.

Thanks,

Keys

Comments:

Where bottlenecks are. Sweet spot for games with lower speed CPU's.
Keep in mind that something like this can turn into an IMMENSE review. There is almost an
unlimited combination of CPU/GPU out there, so I would just stick to identical CPU's and the only variable be the graphics cards.


Power Consumption
This is done on most initial reviews.

Links to benchmark or benchmark methods used for members to try out themselves to compare.
Good Idea. Offers the members a chance to see how their rigs compare with repeatable benchmarks.

A round up of different manufacturer's cards like which gt is best at what from evga, xfx, bfg, ect...
Basically, with only a few exceptions, all the manufacturers card are designed after a reference model and stick to it. Both Nvidia and AMD. So using up huge amounts of time to compare say an 8800GT from eVGA, XFX, BFG, MSI, would yield next to nil differences, excluding overclocked models.

Include various ranges of AA and AF at all resolutions.
Yes, good.
 
Nov 26, 2005
15,189
401
126
Well, I'm glad to see testings at 1280x1024 on these new card but I really have no relevance in how they compare to my old X1900XTX paired with my FX60 - i guess including more cards on the comparison scale would help. Seeing the power consumption is important, games like Red Orchestra would be a nice to see included in a benchmark as well as UT3.
 

MTDEW

Diamond Member
Oct 31, 1999
4,284
37
91
A link to download the exact benchmarks / time demos used in the review so we can run the same tests on our systems.
 

andrei3333

Senior member
Jan 31, 2008
449
0
0
i would like to see group round ups, for example once a person chooses their purchase price point and selects a card (say a 8800gt since it was so popular) then they would need to know which gt is best for them, vanila, overclocked, superclocked, modified cooler (noise/heat), extra memory (1gb), and ultimately price

so a round up of different manufacturer's cards like which gt is best at what from evga, xfx, bfg, ect...
 

taltamir

Lifer
Mar 21, 2004
13,576
6
76
its a well known fact andrei that those things matter very, VERY little... and you can OC any card to its "superclocked" version... And it will be as stable. Generally speaking, the cards will OC to very VERY close max speeds. Unless they have aftermarket coolers, or a custom PCB, then its a whole different animal. Those are worth reviewing. But even then, the speed difference is minimal. Go with what gives you the best price+warranty

But on the core I agree with andrei premise, the only reason (other then idle curiosity) to check a specific card review is if you are in the market to BUY a new card. That means it needs to be comprehensive review compared MANY games, at MANY resolutions, and MANY cards.

Oh, and definitely test each resolution without AA/AF, and with AA/AF maxed out. it gives a much better picture of what to expect.
 

Lithan

Platinum Member
Aug 2, 2004
2,919
0
0
I wish they'd break results up by RESOLUTION, not games.

With everyone on LCD's with a default reso, it would make reading the results MUUUUCH easier. For instance. I'm on 2005fpw The ONLY resolution I care about right now is 1650x1080. So basically I'm loading 20-30 graphs to get the info that if arranged as I need it would take only 3-4 graphs. Better yet, make your site use datastudio/excel like graphs that allow the user to take a blank graph and apply his selected data to it. IE, you have a 50 card shootout at 640x480 through 3800x2400. But I am only interested in 1280x1024 and 1650x1080 and only want to see 8800gt vs 8800gts vs 9600gt vs 3870x2. So I click and drag the resos into the header of your initially blank graph, and then click and drag each card onto the Y axis.
 

Aberforth

Golden Member
Oct 12, 2006
1,707
1
0
A good product doesn't require a review...but a review should never be extensively based upon "Frame Rates" and "value for money". These are two areas where people make mistakes, the reviews should always be enlightning by explaining in and outs of the technology in a more easy to understand fashion. So if a reader is tech savvy he would understand well enough - an make his own independent decisions or if he is just a casual geek then he would find it easy to understand the gist of it.

Nowadays it's all about the fastest card (highest numbers to be precise)- not about how good the technology is. It's like how best one can sneeze, there is always a limiting factor.
 

Keysplayr

Elite Member
Jan 16, 2003
21,211
50
91
Aberforth: Technical details only matter to some. Basically, most people are interested in just that, the best for the money. Image quality is also a factor. But at the end of the day (review) most just want to see which one is the fastest. Nothing wrong with that.

Then there are those who wish for detail down to the actual way the transistors were fabricated. Is it required to know this? No way, but some people are interested in it. Nothing wrong with that either.
 

Keysplayr

Elite Member
Jan 16, 2003
21,211
50
91
Originally posted by: Lithan
I wish they'd break results up by RESOLUTION, not games.

With everyone on LCD's with a default reso, it would make reading the results MUUUUCH easier. For instance. I'm on 2005fpw The ONLY resolution I care about right now is 1650x1080. So basically I'm loading 20-30 graphs to get the info that if arranged as I need it would take only 3-4 graphs. Better yet, make your site use datastudio/excel like graphs that allow the user to take a blank graph and apply his selected data to it. IE, you have a 50 card shootout at 640x480 through 3800x2400. But I am only interested in 1280x1024 and 1650x1080 and only want to see 8800gt vs 8800gts vs 9600gt vs 3870x2. So I click and drag the resos into the header of your initially blank graph, and then click and drag each card onto the Y axis.

Jesus Jim, I'm a doctor!! Not a pool man!!

Seriously, I wouldn't know where to begin doing something that specific. Good idea, but may be a monster.
 

CP5670

Diamond Member
Jun 24, 2004
5,660
762
126
Minimum framerates. Only a few review sites show these, and those that do often only have them for some games. The best thing would be if the review provided the entire framerate curve at each game/resolution that was benchmarked (like the graphs HardOCP shows, but at a lot more settings).

The average framerates can be misleading by themselves, either for getting an idea of what actual gameplay will be like or for comparing different cards. If you look at Xbit's multi GPU review for example, there are many games where the 9800GX2 seems to perform much better than a single 8800GTS 512MB based on the averages, but the minimums are very similar or even lower in some cases.
 

SickBeast

Lifer
Jul 21, 2000
14,377
19
81
What I think would be interesting would be:

Show 3 vertical columns for each game. Each column would represent "low", "medium", and "high" quality settings. Maybe show a thumbnail of a screenshot at each of those settings at the top of each column that users can click on to enlarge it. Or, just link to a side-by-side comparison of the screenshots.

Then, list the bar graphs for each resolution, from highest to lowest resolution, and overlay AA performance on the bar graphs by using a different color. This would help to show the hit that each card takes enabling AA.

It would also be interesting to see how much texture memory each test uses.

I would prefer to see all tests run under Vista 64-bit with at least 4gb of system memory installed, as I feel that is more indicative of performance going forward.

Also, list overclocking results right with the main results. Perhaps overlay it similar to the AA results. I don't like how reviews always leave the OC results to the end, and then maybe run a single test using those settings. I think a large portion of people now overclock their cards by default. It's more reflective of the performance most people will actually experience.
 

Piuc2020

Golden Member
Nov 4, 2005
1,716
0
0
Originally posted by: CP5670
Minimum framerates. Only a few review sites show these, and those that do often only have them for some games. The best thing would be if the review provided the entire framerate curve at each game/resolution that was benchmarked (like the graphs HardOCP shows, but at a lot more settings).

The average framerates can be misleading by themselves, either for getting an idea of what actual gameplay will be like or for comparing different cards. If you look at Xbit's multi GPU review for example, there are many games where the 9800GX2 seems to perform much better than a single 8800GTS 512MB based on the averages, but the minimums are very similar or even lower in some cases.

I agree, they should show something like the FEAR built-in benchmark that gives you lowest, average and max but also tells you what percentage of the benchmark you were over 20FPS, over 40, etc. I don't care if the minimum FPS is 10 if I only experience it for a few seconds over a playing period of 1 hour but I would be pretty pissed if those dips to 10 are quite constant even if the average is something "playable" like 30 or 40.

I guess that would say much more than average framerates about the playability of cards.
 

Aberforth

Golden Member
Oct 12, 2006
1,707
1
0
Originally posted by: keysplayr2003
Aberforth: Technical details only matter to some. Basically, most people are interested in just that, the best for the money. Image quality is also a factor. But at the end of the day (review) most just want to see which one is the fastest. Nothing wrong with that.

Then there are those who wish for detail down to the actual way the transistors were fabricated. Is it required to know this? No way, but some people are interested in it. Nothing wrong with that either.

:D

If you have lots of money keys, will you buy the fastest car? If you do - would you be able to take full advantage of it? naturally not because you are sure to hit a ticket or a unexpected dead end. So it would be wise to buy a car with best features, safety and mileage.

Video cards are no different.
 

Keysplayr

Elite Member
Jan 16, 2003
21,211
50
91
Originally posted by: Aberforth
Originally posted by: keysplayr2003
Aberforth: Technical details only matter to some. Basically, most people are interested in just that, the best for the money. Image quality is also a factor. But at the end of the day (review) most just want to see which one is the fastest. Nothing wrong with that.

Then there are those who wish for detail down to the actual way the transistors were fabricated. Is it required to know this? No way, but some people are interested in it. Nothing wrong with that either.

:D

If you have lots of money keys, will you buy the fastest car? If you do - would you be able to take full advantage of it? naturally not because you are sure to hit a ticket or a unexpected dead end. So it would be wise to buy a car with best features, safety and mileage.

Video cards are no different.

Bold above ^ :D

 

nevbie

Member
Jan 10, 2004
150
5
76
CPU/GPU scaling/bottleneck results would be very useful. Instead of adding this kind of information to GPU reviews, scaling/bottleneck results would qualify for reviews of their own.. not review of a certain product (CPU or GPU) but a review of a certain situation in time. Of course it would be quite massive.. for example, 3 GPUs, 3 CPUs, 3 kind of gameplay setting configurations and 10 games would result in 3*3*3*10=270 benchmark runs *laughs*. "Simulating" lower end parts with underclocked higher end parts would help slightly.

..hmm now that made me wonder how a graph would look like that shows required CPU speed at Y-axis and required GPU speed at X-axis when the objective is to hit a certain (60fps?) game performance value. complex.
 

ArchAngel777

Diamond Member
Dec 24, 2000
5,223
61
91
I would like to see the reviews that AT did two years ago. The video card reviews for the past year have been really horrible to extract the information I want. In some cases, it would require me to enter the information into a spread sheet from a few different reviews that AT had in order to get an understanding of the differences between the cards. In fact, the worst review they did, IMO, was the G92 reviews. I want all the results from all the cards they tested in the SAME graph. Not some 8800GT Versus 3870 only, and then magically have the 8800GTS versus the ULTRA in the same review, but seperated out. I'd like to see image quality address and more AA/AF modes.

Basically, the reviews that AT did two years were great... Not fond of the ones that come out now.

To rehash

1) All graphics cards tested in the same graph
2) CPU Scaling
3) AA and AF modes, more of them.
4) Image Quality
5) Minimum Frame Rates
6) Roundup of most graphics cards released in the last 2-3 years. everything from the 7600GT to the 9600GT

Tom's hardware does it right with their performance databases.
 

taltamir

Lifer
Mar 21, 2004
13,576
6
76
some very good ideas i have seen are:
1. Group by resolution. With LCDs you want to know how well things work at YOUR resolution. No reason to check other resolutions unless you want to upgrade the monitor or resell the card.
2. min frame rate, and how often they occur. We only STIPULATE that if the game is running at 30 to 40 fps it will likely be smooth. But that is not necessarily true. Min frame rates matter.
I honestly couldn't care less about max frame rates, I don't know why anyone would other then bragging rights. Unlike average and min, max frame rate has no effect on smoothness.

I don't know if it was mentioned but... test with REALISTIC CPUs.
I am tired of seeing every single benchmarking site testing all video cards on 1000+$ CPUs... that is useful information, for the 5 people who have such a CPU. most of us are running a more reasonable 200$ cpu and I would like to see how video cards play with those.

Now I get the idea of removing bottlenecks, but different GPUs will probably bottleneck differently.
 

ronnn

Diamond Member
May 22, 2003
3,918
0
71
Would like to see high end cards tested with high end resolutions and cpus and midrange with midrange monitors and cpus, etc. Also stay somewhere close to playable settings.
 

andrei3333

Senior member
Jan 31, 2008
449
0
0
-Test with mainstream CPU - you cannot possibly expect them to test with older cpus, and there are too many to test with, they cannot match you their rig to match every other persons rig out there... make it simple, max out all the other bottlenecks and test the performance of each card.... thats the best way

but another big one for me is this :::::::: Price for performance graph :
--lets say today i can get a 8800gt for 200 and a gts for 300 just for example...that gts is 100 dollars more (50% more cost) but does it yield 50% more performance ?

-- and in general i want to know how much better lets say a 9800GTX for 350 will be compared to a 8800GT512 for 200 since that card is still widely available, its a competitor, so if the gtx is double the performance then its worth it,..... i just want to see more cards in comparisons, not just SLI vs SLI or gt vs gts.. i want a broader field and i dont think its too hard to test that, just plug in a different card and run some benchmarks on the test rig
 

chizow

Diamond Member
Jun 26, 2001
9,537
2
0
I'd like to see:

1) Tests to reveal the true differences between parts and core architectures, like clock for clock comparisons to isolate performance differences between parts within the same family. Most recent examples being G80 GTS vs GTX, G92 GT vs GTS vs G94 etc. I think reviewers do a terrible job of informing consumers about what kind of differences they can expect when they're really the only people who can get a good working knowledge of the differences with hardware in-hand. Whether its because their hands are tied by the hardware vendors or they're indifferent, I don't know.

2) Follow-up or comparison benchmarking with updated drivers or updated parts. NV has a nasty habit of releasing drivers that aren't immediately compatible with all of their parts around an embargo. While I don't necessarily believe its due to any sinister motive other than lack of Q&A for older parts, comparing results with later drivers that support all parts would erase any suspicion while also giving an ongoing glimpse of performance improvements with driver updates. I know its difficult due to other components of the test bench being constantly changed as well, but it would be a useful comparison imo.

3) NV also has a nasty habit of letting board partners release multiple OC'd variants that often rival performance of higher priced SKUs. Many times these OC'd SKUs, although readily available on the market, will go completely unnoticed in reviews and instead, reference stock parts will be used. This imo creates an artificial performance delta between parts based solely on clockspeed and marketing when there is much less of a difference even if overclocking is taken out of the equation. Some reviews do an excellent job of comparing factory overclocked parts with stock parts that give very insightful hints at true performance differences (FiringSquad excels at this).

4) Additional testing and follow-up based on feedback. I know the writers here at AT often do reply to comments in their articles, but I'd like to see this taken a step further, like follow-up testing or investigating. Similar to 1), I just don't find much investigative benchmarking like there used to be, where hardware reviewers jumped at the opportunity to test and investigate any performance tweaks and gains. I'd like to see maybe a top 3 suggestions for follow-up testing performed within a month of the initial review. I'd also like to see any question marks or performance/stability issues mentioned in the initial review addressed in the follow-up.
 

MustangSVT

Lifer
Oct 7, 2000
11,554
12
81
I dont upgrade often but when I look at a review, esp videocard, I look for value by price and performance.

these are the things I look for:

1. Price
2. performance
3. Build quality and warranty
4. O/C ability
5. Quietness
6. Power consumption

and lots of WELL shot pictures. :)
 

JAG87

Diamond Member
Jan 3, 2006
3,921
3
76
A comparison with the largest amount possible of other cards. I hate reviews where they compare a card to 3 other cards and thats it.

I want page-long bar charts with 30 different graphic setups, including crossfire/sli setups, I don't care if it takes reviewers a month to do a review. Set up five identical platforms and have 5 people work at the benchmarks, in parallel it will get done faster.

I like to be aware of ALL my options before I buy something. And since reviewers don't seem to follow a standard for test setups, it's impossible to mix results between different reviews.

that's my 2 cents.
 

Keysplayr

Elite Member
Jan 16, 2003
21,211
50
91
Originally posted by: JAG87
A comparison with the largest amount possible of other cards. I hate reviews where they compare a card to 3 other cards and thats it.

I want page-long bar charts with 30 different graphic setups, including crossfire/sli setups, I don't care if it takes reviewers a month to do a review. Set up five identical platforms and have 5 people work at the benchmarks, in parallel it will get done faster.

I like to be aware of ALL my options before I buy something. And since reviewers don't seem to follow a standard for test setups, it's impossible to mix results between different reviews.

that's my 2 cents.

If it take you 30 days to do a review, you might as well hang it up. In that time, more drivers may come out rendering all the timely work, null and void. Ouch.
 

JAG87

Diamond Member
Jan 3, 2006
3,921
3
76
Originally posted by: keysplayr2003
Originally posted by: JAG87
A comparison with the largest amount possible of other cards. I hate reviews where they compare a card to 3 other cards and thats it.

I want page-long bar charts with 30 different graphic setups, including crossfire/sli setups, I don't care if it takes reviewers a month to do a review. Set up five identical platforms and have 5 people work at the benchmarks, in parallel it will get done faster.

I like to be aware of ALL my options before I buy something. And since reviewers don't seem to follow a standard for test setups, it's impossible to mix results between different reviews.

that's my 2 cents.

If it take you 30 days to do a review, you might as well hang it up. In that time, more drivers may come out rendering all the timely work, null and void. Ouch.



drivers dont increase performance that much, lets not kid ourselves. the only reason worth updating drivers for is new SLI profiles. otherwise the improvements are minimal.
 

SickBeast

Lifer
Jul 21, 2000
14,377
19
81
Why not use a 'standard testbed' for each generation of CPU? That way, a large number of GPUs can be benchmarked (along with other components) and compared in a database.

By that I mean, just use a C2D until intel comes out with something significantly better.