Halo's bench numbers too low

[BenSkywalker]

I spent a few hours or so going through the Halo bench and there are a few constant oddities that are showing framerates that are not only not a good indicator, but poor for comparing the gap between systems in terms of in game performance. The loading times for all the scene transitions is factored in to the bench, this drags everyone's scores down but since HDs and processors are much closer in performance terms then vid cards, the relative numbers are not showing close to the disparity that you will experience in game. The portions of the bench that are processor limited are those areas in scene transitions, none of the in game bench portions seem to be effected at all moving processor speed around considerably.

The level of performance in game is a decent amount higher then what is indicated by the bench, anyone who has the game is likely well aware of that, but for those of you that are hesitant on picking up the game- besides the bench focusing on some of the more graphics intensive segments of the game, it also is dragged down decently by the amount of time your rig will take to load up all the different segments. Bungie should get around to releasing a bench, or allow us the tools to do it ourselves, that focuses on one area. We would likely be seeing bench numbers considerably higher across the board and one that better shows the performance disparity.

I've been able to run back to back benches within 5FPS of each other where one was very clearly much faster then the other, in bench and playing with the same settings.

This is all based on the retail build of the game, pre and post patch.

Edit-

Forgot to mention about the optimizations, the PC port runs significantly faster then the XBox version of the game on my rig. What's more, I'm GPU limited and even dropping my Ti to below XBox levels the performance remains significantly faster then the original version of the game. It isn't close to screaming along like Quake3, but areas where my framerate would dip in to the single digits on the XBox are now in the low 20s on my PC. I'm actually quite surprised how well they handled this port, the performance is up and the visuals have been improved decently. Not sure what people are saying about the game not looking too good either, the visuals have been improved considerably over the XBox and it certainly is comparable to Unreal2(models aren't as nice, shader effects and texturing is actually quite a bit better).

 

[Jeff7181]

Link to the Halo benchmark? =)

 

[clicknext]

He just said that

Bungie should get around to releasing a bench, or allow us the tools to do it ourselves

-_-'

Anyway, thanks for that info skywalker, I think I'll actually get it and give it a try on my 8500LE.

 

[1ManArmY]

Quote

--------------------------------------------------------------------------------
- Gearbox, Bungie and Microsoft care about our customers and our products. We are committed to supporting Halo PC.

- We have noticed some people wondering about performance scores with Halo PC and we have some information and news about that.

- Halo PC includes a timedemo feature that was designed to help hardware enthusiasts do benchmark comparisons between different hardware configurations with Halo PC.

- While the feature is very precise, it?s not very accurate. In other words, the numbers are real, but they don't mean what you think they mean. The numbers during time demo are affected by debugging software that was used by developers to optimize memory consumption and rendering performance. The numbers are also affected by shortcuts in providing the feature, such as not discriminating between time spent rendering versus time spent loading each of the scenes in the time demo.

- Since the game?s launch, we?ve noticed that many users are not using the timedemo feature for comparative benchmarking, but as a gauge of their own system?s performance in terms of average frames per second performance of Halo.

- The numbers being looked at are therefore inaccurate for that purpose.

- So, we?ve decided to change the timedemo features so that it can be better used for the purpose of gauging performance.

- First, we removed per-frame memory checks that were hampering performance during timedemo testing. We no longer need these checks for memory optimization and they were only affecting your memory adversely.

- Second, we eliminated load times from the calculations as they were being counted towards the amount of time elapsing even though the game was halted and wasn?t rendering any frames.

- Finally, we improved the output messages to relay this information precisely and accurately.

- The end result for us is that in our configuration testing (limited to just a handful of video cards and computers), we saw anywhere from 30% to 60% increased in frames per second reported by time demo scores.

- These changes will be included in the next update (along with some other cool improvements).

- The next update should be released within a week or so, but please wait for an official release date to be announced before holding us to it

Thanks! Now get back to saving the universe!

This was found in a gearbox forum yesterday, the link is not working.

 

[Robor]

Thanks for the info 1ManArmY. I'm looking forward to a "real" benchmark. Really sad that Halo is getting such bad press because of the low FPS benchmarks. To be honest, when I was playing around with the FLT release my gameplay wasn't bad at all and my system only benchmarked at an average 39.something FPS. Hope the next release clears this all up and people will give Halo a try.

 

[BenSkywalker]

Thanks for posting the info 1Man, looks like we'll be seeing this rectified hopefully within the week. It almost seems like people who have the game are criticizing it because they score low on the bench, not due to in game performance issues. Didn't have a clue about the debugging either(although perhaps with the exact RAM utilization I should have figured it out).

- The end result for us is that in our configuration testing (limited to just a handful of video cards and computers), we saw anywhere from 30% to 60% increased in frames per second reported by time demo scores.

This would bring Robor in to the 60FPS ballpark, I'd say that's fairly decent given how heavy the shader useage in the game is, I wouldn't be quite that high running at the same settings but it would certainly be more reflective of the level of performance I see in game.

 

[rbV5]

It almost seems like people who have the game are criticizing it because they score low on the bench, not due to in game performance issues

A little of the "pile on" mentality that is all the rage these days as well I think. Folks whose opinions on gameplay I trust, are giving Halo PC the thumbs up.

 

[1ManArmY]

I was going to compare my 1.5 beta benchmarks to the retail benchmarks but my beta stopped working a couple of days ago so I just uninstalled it. I heard that there is a bug with catalyst 3.7, strange that it had been working fine since the 9-11-03 and the gameplay was acceptable for a beta. The game would not launch period. Don't know if I will encounter this anomaly with the retail as I have yet to install it. I'm told the the game works just fine with the omega driver's. Anyway the catalyst 3.8 are due out next week so we should see some inceased performance and maybe Gearbox, Bungie, and Microsoft will have their update out as well.

 

[Regs]

All this time I was thinking the game was simply not optimized for the pc. So a high end pc out performs the x-box with better details considerably.

^Bump, good info, thanks guys.

 

[jiffylube1024]

Yeah, Gearbox said they're patching the game to represent FPS more accurately, so it should look better than the ~35-45 fps nonsense everyone seems to be benching now.

 

[1ManArmY]

I would think that one would verify their current benchies with FRAPS (2.0) that supports DX 9.0 and figure out that the timedemo was not accurate. It seems that a lot of hardware reviewers are benching cut scenes in games and using FRAPS not sure if that is a good idea or not.