Fermi's lead over Cypress shrinks w/ new drivers

[Skurge]

Far Cry 2
4870 1GB
Ultra settings, 1920x1080, no AA

Run 1
FPS
49 48 48 48 48 50 49 51 52 51 50 46 43 42 43 45 49 48 46 47 56 55 46 48 48 43 44 51 52 47 46 50 54 53 45 48 51 49 52 52 47 51 52 52 56 49 45 45 49 51 47 43 46 47 54 51 47 53 54 56

Frames, Time (ms), Min, Max, Avg
2939, 60000, 38, 61, 48.983

Run 2
FPS
50 47 47 47 48 49 50 52 52 48 45 40 43 41 45 43 45 49 47 46 45 49 41 44 43 56 46 49 51 66 63 49 53 49 52 53 48 48 47 52 51 50 49 61 55 48 46 47 51 47 51 50 54 51 48 46 52 52 53 54

Frames, Time (ms), Min, Max, Avg
2955, 60000, 38, 69, 49.250

Run 3
FPS
47 47 47 47 48 47 53 51 52 47 46 41 44 44 45 50 53 52 51 51 50 53 50 44 49 45 50 45 48 51 64 54 45 55 51 51 52 51 51 47 44 51 49 50 46 49 50 49 47 42 43 47 49 44 48 48 46 46 48 49

Frames, Time (ms), Min, Max, Avg
2915, 60000, 38, 68, 48.583

Thats my FC2 runs, tried to be as consistent as possible, but there was some variation in the runs

 

[happy medium]

Far Cry 2
4870 1GB
Ultra settings, 1920x1080, no AA

Run 1
FPS
49 48 48 48 48 50 49 51 52 51 50 46 43 42 43 45 49 48 46 47 56 55 46 48 48 43 44 51 52 47 46 50 54 53 45 48 51 49 52 52 47 51 52 52 56 49 45 45 49 51 47 43 46 47 54 51 47 53 54 56

Frames, Time (ms), Min, Max, Avg
2939, 60000, 38, 61, 48.983

Run 2
FPS
50 47 47 47 48 49 50 52 52 48 45 40 43 41 45 43 45 49 47 46 45 49 41 44 43 56 46 49 51 66 63 49 53 49 52 53 48 48 47 52 51 50 49 61 55 48 46 47 51 47 51 50 54 51 48 46 52 52 53 54

Frames, Time (ms), Min, Max, Avg
2955, 60000, 38, 69, 49.250

Run 3
FPS
47 47 47 47 48 47 53 51 52 47 46 41 44 44 45 50 53 52 51 51 50 53 50 44 49 45 50 45 48 51 64 54 45 55 51 51 52 51 51 47 44 51 49 50 46 49 50 49 47 42 43 47 49 44 48 48 46 46 48 49

Frames, Time (ms), Min, Max, Avg
2915, 60000, 38, 68, 48.583

Thats my FC2 runs, tried to be as consistent as possible, but there was some variation in the runs

I benched the part in the very beginning, after the long video intro,when he first gets out of the bed after the guy is talking to him and he is sick in bed.

When he gets up I push f11 and go outside and fight a whole bunch of guys, shit is blowing up and I'm getting my ass kicked.

 

[NoQuarter]

I think I got the dates mixed up,there were alot of benches in the folder, sorry.

Here are the 4 benches I have in the folder, I deleted the others so there is no confusion.

Frames, Time (ms), Min, Max, Avg
4412, 60000, 15, 241, 73.533

FPS 67 66 66 67 67 65 66 64 61 58 62 64 70 71 70 68 70 71 70 65 64 65 65 64 65 64 65 64 65 65 65 65 64 65 65 65 62 61 31 19 25 196 229 237 233 238
228 175 25 25 25 25 25 25 25 25 25 25 19 45


Frames, Time (ms), Min, Max, Avg
3397, 60000, 42, 87, 56.617

FPS 55 58 62 49 52 50 51 55 59 66 67 65 63 66 64 63 58 65 64 69 77 86 58 51 51 52 52 53 48 43 50 60 70 71 55 43 42 43 43 47 48 49 45 46 46 45 47 50 52 59 61 61 51 51 60 66 63 65 70 65

Frames, Time (ms), Min, Max, Avg
3557, 60000, 13, 89, 59.283

FPS 57 58 63 59 58 60 64 63 60 62 60 70 66 70 62 57 53 53 54 50 46 51 54 56 54 51 53 53 47 49 56 59 45 46 40 45 45 44 48 55 58 85 76 83 83 84 85 84 85 87 88 89 88 88 84 14 25 25 25 24

Frames, Time (ms), Min, Max, Avg
3235, 60000, 41, 68, 53.917

FPS 61 65 58 57 57 62 63 62 60 65 61 53 55 57 52 46 47 53 49 50 57 62 64 53 52 52 54 64 59 60 60 63 60 57 60 67 60 50 49 50 49 50 45 54 51 50 46 49 53 45 41 43 44 45 45 44 43 42 54 55

Well in the first bench you obviously were recording a loading screen or menu for 6 entries (200+ and 175), while the other 3 don't have anomalies, which gives us 56.606 FPS +/- 5%? I can totally live with that. I do feel there's some user induced discrepancy though. My tests a few months ago with BC2 where I played a whole checkpoint were very consistent results even though things never went exactly the same.

 

[Skurge]

I benched the part in the very beginning, after the long video intro,when he first gets out of the bed after the guy is talking to him and he is sick in bed.

When he gets up I push f11 and go outside and fight a whole bunch of guys, shit is blowing up and I'm getting my ass kicked.

I did the first camp you are sent to raid, where you are suppose to rescue that guy. I didn't play the game much further than that, it sucked. Glad I didn't pay full price, but I still feel ripped off though.

 

[happy medium]

Well in the first bench you obviously were recording a loading screen or menu for 6 entries (200+ and 175), while the other 3 don't have anomalies, which gives us 56.606 FPS +/- 5%? I can totally live with that. I do feel there's some user induced discrepancy though. My tests a few months ago with BC2 where I played a whole checkpoint were very consistent results even though things never went exactly the same.

You know what? I think I died , thats the one I didn't include the first time.

 

[GaiaHunter]

Thanks for clearing up the problem.


As you see, and while your benches are far from being constant, those 80 FPS of the included canned bench are nowhere to be seen.

If you were a reviewer and I read about the 5750 I would think it was a monster at that resolution. It is still quite good but maybe I would have gone to a 5770 to squeeze if you presented this data as you played.

Now what reviewers do is not as simple as just trying to be as close as possible - although as we can see you could easily comment on the playability of this card at this resolution - they can produce their own time demos.

I remember UT had a way of recording a time demo in game. I guess some other games do also. Dunno if there is a tool out there.

That gives more reproducibility without being open to "optimizations" from NVIDIA and ATi.

I asked before BFG10K in this thread how he creates his own time demos but he didn't reply!

 

[tincart]

With the proper data sets being posted, the results look fairly consistent and paint a much different picture from canned benchmarking.

The operative question is: Which method will better inform the reader about the performance they can expect from a given card?

 

[Skurge]

With the proper data sets being posted, the results look fairly consistent and paint a much different picture from canned benchmarking.

The operative question is: Which method will better inform the reader about the performance they can expect from a given card?

I thinks its safe to say that question has been answered with the above posts.

 

[happy medium]

Thanks for clearing up the problem.


As you see, and while your benches are far from being constant, those 80 FPS of the included canned bench are nowhere to be seen.

If you were a reviewer and I read about the 5750 I would think it was a monster at that resolution. It is still quite good but maybe I would have gone to a 5770 to squeeze if you presented this data as you played.Now what reviewers do is not as simple as just trying to be as close as possible - although as we can see you could easily comment on the playability of this card at this resolution - they can produce their own time demos.

I remember UT had a way of recording a time demo in game. I guess some other games do also. Dunno if there is a tool out there.

That gives more reproducibility without being open to "optimizations" from NVIDIA and ATi.

I asked before BFG10K in this thread how he creates his own time demos but he didn't reply!

truth is i had the overclock set at 900 core and 1350 mem, so it was as fast as a 5770. but I see your point.

I like the idea of time demo's myself, but its hard to replicate the same pattern to give a accurate IN GAME fps.

 

[GaiaHunter]

I like the idea of time demo's myself, but its hard to replicate the same pattern to give a accurate IN GAME fps.

Except if your time demo is like a canned one but without being public.

Look at the graphs of the original link - they all follow a similar pattern. Not like yours where you can see that the run is very inconsistent. The most surprising is that despite the inconsistent pattern of your run the averages are very similar in most of them.

 

[NoQuarter]

truth is i had the overclock set at 900 core and 1350 mem, so it was as fast as a 5770. but I see your point.

I like the idea of time demo's myself, but its hard to replicate the same pattern to give a accurate IN GAME fps.

I think it's easier to replicate if you set it to easy, and get a specific pattern of what you're going to do down. When I did BC2 I cut down the first couple guys, shot a barrel to get some explosions going, threw a grenade into a building for some destruction, went around the side and shot up the rest. It was always the same pattern, even though things were slightly different each time I was able to always look at the same things and cause the same explosions. If things went way off course or I got injured and had to take cover/do something I didn't before I'd just cancel the bench and start it over.

 

[BFG10K]

I asked before BFG10K in this thread how he creates his own time demos but he didn't reply!

I didn’t see that – sorry.

There’s no universal tool for this; every game is different, and requires research on how to make benchmarks for it. Also some games can’t make their own benchmarks.

 

[tannat]

I thinks its safe to say that question has been answered with the above posts.

It seems like it is very easy to get a significant result from actual playing of the game. Happy Medium seem to have shown how that, as long as you don't die, variation is very much less than the offset given by using canned benches or even the compromise of making your own timedemos suggested by BFG10K.

Of course based on the assumption that the around 30% offsets in the [H] article isn't an outlier.

What I don't get is why not more sites test like [H] then. All other test procedures seem to be totally swallowed by the unknown discrepancy between benches and gameplay. To be harsh: They don't tell us anything at all, they're just random noice.

 

[GaiaHunter]

I didn’t see that – sorry.

There’s no universal tool for this; every game is different, and requires research on how to make benchmarks for it. Also some games can’t make their own benchmarks.

No problem mate.

I know some games can do it through console command but I was hoping there was some universal tool.

Thanks for the info.