anyone else think most reviews have excessive fps and frame time benchmarks?

[Anarchist420]

i sure do mainly because both ihvs are always within a small fps range of each other, and you either find it enjoyable or tolerable/unplayable while there can be a huge difference between the two in feature set and IQ. and you should really know by the card's texture and pixel fill rates, the bandwidth, and the GFLOPS rating whether it's capable of giving the frame rate you want or at least on the high end. in other words, i think it is ridiculous to even bother benchmarking in a titan review (although performance benchmarks on the flagship nv and the flagship AMD with no other cards tested for individual game articles make sense).

and i do like the idle and load temps being measured. but testing 20 or so products of the same SKU and measuring the idle and load vid of each would be a good idea even though the buyer doesnt have a whole lot of control over that.

anyway, i just think it sucks that image quality and especially aa, compatibility, and feature set articles are few and far between. maybe reviewing each individual driver for compatibility/iq/bugs would be a good idea. and benchmarks measuring MSE compared to the microsoft reference rasterizer are good, but they're insufficient alone because the ref rasterizer doesnt always look the best (it's largely subjective because AMD generally had much lower MSEs on ante-DX10 hardware yet nvidia's general rasterization has always looked better to me).

anyone else reading this think that performance (frame times and fps) is overmeasured?

finally, i acknowledge that i am generally satisfied with 30fps (especially since it reduces input lag in most cases) and that i am out of the mainstream in that regard, but i think that the IHVs would be more motivated if there was more and deeper criticism and praise of their compatibility/iq and bugs, both drivers and hardware.

 

[VulgarDisplay]

I personally want more smoothness/mm2 benchmarks.

Now in all seriousness I kinda have to agree that benchmarking is in an odd place right now. Midrange cards are fully capable of maxing games at 1920x1080 at acceptable framerates.

I don't max games unless I'm playing single player which I don't really care that much about playability in single player. When it comes to MP though I push for 120fps @ 120hz and I reduce effects quality because it can often hurt visibility.

Benchmarks are pretty much pointless until 4K monitors are seeing widespread use and the majority of people replace their 1080p monitors.

 

[Anarchist420]

Benchmarks are pretty much pointless until 4K monitors are seeing widespread use and the majority of people replace their 1080p monitors.

thank you for your thoughtful and nice reply

anyway, i really hope that 4k doesnt become the norm for a very long time but im sure it will be rather mainstream no more than a year and a half from now... it's probably the worst technological advance up to date in my opinion. for movies it means more compression so more artifacts, less space for movies to be mastered and encoded in RGB8 rather than RGB6 and for video games it means less memory, bandwidth, and fill rate for high precision rendertargets and less memory for larger textures.

as for monitors and tvs, it is a huge waste there too as i think RGB10 displays and rgb led arrays without pwm are a lot more important than monitors with res greater than 2560x1600... many people will see the high pixel density of 4k monitors and tvs then forget that there is pwm flicker, backlight bleed, and/or low color gamut. it is made that much worse considering that HDMI 2.0, thanks to IP legislation, may be the standard for quite some time rather than the superior Display Port or whatever superior successor DisplayPort group makes.

i also dont think OLEDs are necessary... the problem is largely that the LCD panel makers dont use all the very best of what's already available like DP1.2, higher grade panels (i wish they didnt let quality vary that much for the same SKU or whatever), native RGB 10, DDM or less processing (like less than 3ms signal lag), RGB LED arrays (side/edge lit sucks), pwm only at very low brightness if at all, ips with faster response time done without processing, not trying to sell panels that wildly vary in contrast ratio as the same product, making sure frame skipping is minimal or non-existant regardless of input rate, maybe using better quality PCBs, all japan made caps, and better VRMs.

and while OLEDs have excellent response time and contrast, they have their issues as well and future IPS panels could easily get down to 5 ms typical without processing and some hit even more than 1k SCR already.

anyway, i just see OLEDs as pointless and i am rather cynical about the future.

 

[bystander36]

i sure do mainly because both ihvs are always within a small fps range of each other, and you either find it enjoyable or tolerable/unplayable while there can be a huge difference between the two in feature set and IQ. and you should really know by the card's texture and pixel fill rates, the bandwidth, and the GFLOPS rating whether it's capable of giving the frame rate you want or at least on the high end. in other words, i think it is ridiculous to even bother benchmarking in a titan review (although performance benchmarks on the flagship nv and the flagship AMD with no other cards tested for individual game articles make sense).

and i do like the idle and load temps being measured. but testing 20 or so products of the same SKU and measuring the idle and load vid of each would be a good idea even though the buyer doesnt have a whole lot of control over that.

anyway, i just think it sucks that image quality and especially aa, compatibility, and feature set articles are few and far between. maybe reviewing each individual driver for compatibility/iq/bugs would be a good idea. and benchmarks measuring MSE compared to the microsoft reference rasterizer are good, but they're insufficient alone because the ref rasterizer doesnt always look the best (it's largely subjective because AMD generally had much lower MSEs on ante-DX10 hardware yet nvidia's general rasterization has always looked better to me).

anyone else reading this think that performance (frame times and fps) is overmeasured?

finally, i acknowledge that i am generally satisfied with 30fps (especially since it reduces input lag in most cases) and that i am out of the mainstream in that regard, but i think that the IHVs would be more motivated if there was more and deeper criticism and praise of their compatibility/iq and bugs, both drivers and hardware.

I disagree that these measurements are at all useful to most people. To anyone really, as you need the benchmarks for a baseline to know how those may effect performance. Benchmarks have a place, but I do agree that other things also matter, and should be looked into. I also wish they'd benchmark at different quality settings more often. Some people like to game at 60 FPS, some are ok at 40 FPS and others want 80+ FPS, so it would be good to see what is achievable and what setup is need for those settings.

I also agree that bugs, stuttering (now being looked at a lot), features, image quality and other things should be looked at. However, these are much more difficult to compare, which is why they are probably rarely looked into. A lot of these things are more subjective, where a benchmark is not. As soon as reviews only look at these things, every fanboy is going to be screaming at every review that favors one brand or another.

 

[Enigmoid]

What would be nice is standard deviation/standard error measurments. It does not tell me much if card A is 50fps average of 3-5 runs if the score for each run was (47,52,55,48,48) especially with the newer non-constant boost clocks. In this case the average 50 fps with the standard deviation 3.4, standard error 1.5. Error bars would be helpful, especially in cases with throttling where performance decreases over time.

 

[BrightCandle]

Reviews these days aren't really designed for the first time PC builder. You don't really get told much about the difference in features between sets of GPUs, mainly because most reviews only focus on one card and its feature set and not how it compares. The main feature of a GPU however is its performance, its primary function is the rendering of the images and how many of those it can achieve a second.

One of the reason why they test so many games is that the performance of cards differs so much in different games. Gamegpu.ru tests hundreds of games by this point and everyone of them shows a different picture of how hardware compares.