Ashes of the Singularity User Benchmarks Thread

[Despoiler]

One thing I'm surprised by is that the benchmark boils the numbers down to average framerate. Even in the logs they don't give you min or max framerate. Anyone that has used or benched Mantle knows that average FPS is higher more so due to higher min fps than max fps. That combined with the lower frametime creates a superior user experience. It's just plain smoother. I will be providing feedback to the devs to hopefully get better breakdowns. Without further ado.

Comp specs in signature, AMD 15.7.1 Drivers, Win10 Pro build 10525 (note this build has performance issues). Resolution 1904x1041(windowed). All video options maxed.

The default benchmark is GPU-bound 100%. All CPU cores more or less at 45-75%% usage though entire benchmark on DX12. Average around 55%. Less variance scene to scene. DX11 had noticeably wider variance scene to scene. 35-90%. The average probably would end up near the DX12, but that doesn't explain the experience difference.

DX11 Summary

Average Framerate (all batches) 16.9 FPS
Average Framerate (normal batches) 26.6 FPS
Average Framerate (medium batches) 18.1 FPS
Average Framerate (heavy batches) 11.8 FPS

DX12 Summary

Average Framerate (all batches) 23.9 FPS
Average Framerate (normal batches) 30.8 FPS
Average Framerate (medium batches) 23.9 FPS
Average Framerate (heavy batches) 19.5 FPS

DX12 gain/loss vs DX11 +41%, +15%, +32%, +65%

Solid gains here.
-----------------------

This mode assumes you have an infinitely powerful GPU ie CPU-bound. All cores loaded to 100% through entire benchmark.

DX11 CPU Test Summary

Average Framerate (all batches) 16.7 FPS
Average Framerate (normal batches) 25.9 FPS
Average Framerate (medium batches) 17.8 FPS
Average Framerate (heavy batches) 11.8 FPS

DX12 CPU Test Summary

Average Framerate (all batches) 59.2 FPS
Average Framerate (normal batches) 67.9 FPS
Average Framerate (medium batches) 62.6 FPS
Average Framerate (heavy batches) 50.0 FPS
Average CPU Framerate (all batches) 59.2
Percent GPU Bound (normal batches) 62.5%
Percent GPU Bound (medium batches) 67.6%
Percent GPU Bound (heavy batches) 70.3%

DX12 gain/loss vs DX11+354%, +262%, +351%, +423%

Absolutely monsterous gains.

 

[Phanuel]

I haven't looked into this title at all due to the genre, but can you explain what that second test is actually doing?

 

[Despoiler]

I haven't looked into this title at all due to the genre, but can you explain what that second test is actually doing?

The first test is GPU-bound. The second is CPU-bound.

 

[Phanuel]

Ok, you edited out what I was questioning. You had maxed settings noted for the 2nd tests which is what threw me for a loop.

Basically this game is hugely gpu bound.

 

[utahraptor]

Here are my results:

Direct X 11 GPU:

Direct X 11 CPU:

Direct X 12 GPU:

Direct X 12 CPU:

 

[AtenRa]

Direct X 12 CPU:

Could you disable HT in BIOS and run the same test again with only 4 cores ??

Also question, is your CPU at default or OC ??

 

[dogen1]

Interesting that Kepler doesn't seem to benefit at all from DX12 in the 100% GPU bound test, while GCN improves 40% on average. That's an incredible improvement.
Async compute related perhaps?

 

[utahraptor]

Could you disable HT in BIOS and run the same test again with only 4 cores ??

Also question, is your CPU at default or OC ??

Direct X 12 CPU With Hyper Threading Disabled:

CPU Settings:

What I find interesting is in the D 11 CPU test the cores only ran about about 50% utilization. In the D 12 CPU test with and without HT enabled they all ran full out at 100% utilization for the duration of the test. I want to see someone with 6 or more cores run it.

 

[dogen1]

What I find interesting is in the D 11 CPU test the cores only ran about about 50% utilization. In the D 12 CPU test with and without HT enabled they all ran full out at 100% utilization for the duration of the test. I want to see someone with 6 or more cores run it.

I've read that's how it's supposed to work with hyper-threading, since the 2 virtual cores share the resources of one. Not sure if it's true since I can't confirm it.

 

[shady28]

You've got a 680 (same as a 770) matching up with an R9 290 in those GPU tests, and the 680 was running at a higher resolution.

This game has a problem being able to run fullscreen from what I've read.

Looking at the benchmarks, it appears to me that DX12 takes a massive hit on performance if the game is windowed.

I noticed that pattern in the benchmarks posted in the other thread on this topic a few days ago.

Despoiler - can you get it to run fullscreen mode and re-run the DX12 benchmark on your 290?

If I'm right I bet your GPU scores go up 50% or more.

 

[dogen1]

You've got a 680 (same as a 770) matching up with an R9 290 in those GPU tests, and the 680 was running at a higher resolution.

This game has a problem being able to run fullscreen from what I've read.

Looking at the benchmarks, it appears to me that DX12 takes a massive hit on performance if the game is windowed.

I noticed that pattern in the benchmarks posted in the other thread on this topic a few days ago.

Despoiler - can you get it to run fullscreen mode and re-run the DX12 benchmark on your 290?

If I'm right I bet your GPU scores go up 50% or more.

The 680 guy wasn't using any multisampling, while OP said maxed, which I'm assuming is at least 4xMSAA.

 

[shady28]

The 680 guy wasn't using any multisampling, while OP said maxed, which I'm assuming is at least 4xMSAA.

The 680 guy's screenshot shows that he was (below). He was also running at higher resolution (1920x1200) and fullscreen mode while Despoiler (R9 290) was and I quote "Resolution 1904x1041(windowed). ".

Again, I think the systems running windowed mode are taking major hits in this game under DX12. And apparently there is a bug which hits certain cards or driver versions that prevents full-screen mode.

 

[dogen1]

The 680 guy's screenshot shows that he was (below). He was also running at higher resolution (1920x1200) and fullscreen mode while Despoiler (R9 290) was and I quote "Resolution 1904x1041(windowed). ".

Again, I think the systems running windowed mode are taking major hits in this game under DX12. And apparently there is a bug which hits certain cards or driver versions that prevents full-screen mode.

[/QUOTE]

I see 1x Multisampling..

 

[utahraptor]

I do not believe it allows you to adjust the other settings besides resolution and CPU mode.

 

[jj109]

Interesting that Kepler doesn't seem to benefit at all from DX12 in the 100% GPU bound test, while GCN improves 40% on average. That's an incredible improvement.
Async compute related perhaps?

Given that DX11 performance in GPU bound and CPU bound test for the GCN card was almost identical, I would speculate that both were CPU bound despite what the game claims.

 

[Despoiler]

You've got a 680 (same as a 770) matching up with an R9 290 in those GPU tests, and the 680 was running at a higher resolution.

This game has a problem being able to run fullscreen from what I've read.

Looking at the benchmarks, it appears to me that DX12 takes a massive hit on performance if the game is windowed.

I noticed that pattern in the benchmarks posted in the other thread on this topic a few days ago.

Despoiler - can you get it to run fullscreen mode and re-run the DX12 benchmark on your 290?

If I'm right I bet your GPU scores go up 50% or more.

The game can run in full screen mode. There is a bug that causes the game to crash if you alt+tab though. The interesting thing about this bug is that it's actually a MS feature. According to Stardock, Windows process scheduler is antiquated. It doesn't know how to deal with the threading that Stardock has done. Either way, yes I will rerun and pull the numbers from the log files. I'm going to be out of town for most of the weekend. I'll report back Sunday.

I do not believe it allows you to adjust the other settings besides resolution and CPU mode.

You can adjust everything that is in the options. MSAA 4x, temporal AA, etc..

 

[selni]

The game can run in full screen mode. There is a bug that causes the game to crash if you alt+tab though. The interesting thing about this bug is that it's actually a MS feature. According to Stardock, Windows process scheduler is antiquated. It doesn't know how to deal with the threading that Stardock has done. Either way, yes I will rerun and pull the numbers from the log files. I'm going to be out of town for most of the weekend. I'll report back Sunday.



You can adjust everything that is in the options. MSAA 4x, temporal AA, etc..

Blaming MS for your own bugs is pretty standard practice, I wouldn't take that seriously from stardock of all companies without a LOT more evidence - they don't exactly have a record of bug free software.

 

[Shivansps]

The CPU mode is even using the GPU? I think is using Software Rendering, kinda making it impossible for comparing AMD vs Nvidia in this way, if that the case its more a bench for comparing CPUs.

 

[Despoiler]

Blaming MS for your own bugs is pretty standard practice, I wouldn't take that seriously from stardock of all companies without a LOT more evidence - they don't exactly have a record of bug free software.

Stardock's team that wrote the Oxide engine is the same team that wrote all of the Civ 5 engines. Dan Baker is one of the handful of engine gurus in the entire world. They know exactly what's going on.

Counterpoint....MS is known for great, revolutionary code?

 

[utahraptor]

So now there is this: http://www.overclock.net/t/1569897/...singularity-dx12-benchmarks/400#post_24321843

It is kind of saying Nvidia put all their eggs in the DX 11 basket and not to expect great things for D 12. I am not so sure I want a 980 ti now >.<

 

[Glo.]

His posts are astonishing and give huge understanding of what is going on under the hood both on terms hardware and software in this particular situation.

It becomes more interesting with every moment.

 

[tential]

So now there is this: http://www.overclock.net/t/1569897/...singularity-dx12-benchmarks/400#post_24321843

It is kind of saying Nvidia put all their eggs in the DX 11 basket and not to expect great things for D 12. I am not so sure I want a 980 ti now >.<

It's been my feeling nvidia gives the best performance now, while amd usually has a more forward looking plan.
Boils down to how high your graphics card turnover rate is.

 

[Silverforce11]

So now there is this: http://www.overclock.net/t/1569897/...singularity-dx12-benchmarks/400#post_24321843

It is kind of saying Nvidia put all their eggs in the DX 11 basket and not to expect great things for D 12. I am not so sure I want a 980 ti now >.<

Gamedevs on b3d have been saying the same stuff for over a year since Maxwell 2's debut. The queues are fluff, since the pipeline is serial, in-order. As soon as you try to change the context of the in-order pipeline, it incurs a performance hit. GCN is all parallel pipelines, all out of order so it can handle parallelism very well, in fact, it was DESIGNED for a future API, Mantle and now DX12.

Two key things are holding back GCN performance now in DX11: 1) single threaded driver & overhead, 2) serial workload gimps GCN. Despite this, we get results like the 380 beating the 960 and beating it on power efficiency too (Latest review from Techspot) and Fury X stretches its legs a bit at 4K.

Once DX12 removes those limitations...

Pascal better be GCN-like, lots of parallel out-of-order pipelines and lots of compute units, or it's gonna be in a world of hurt in a DX12 battle.

Edit: @Glo, Techspot review of 950, latest drivers. http://www.techspot.com/review/1049-nvidia-geForce-gtx-950/ Look at the performance of 950, 960, 380 and look at the power usage. 380 is clear lead, using similar power to 960. GCN's maturing real well already on an API it wasn't designed to excel on. It'll fly on DX12!

 

[Glo.]

It is amazing, because right now the best value for money has R9 390X. Good performance right now, and amazing potential for the even future. For much less than Fury X and GTX 980 Ti.

Now it gets really interesting.

Edit. Silverforce, can you give me a link on PM to that review on Techspot? I don't know what are you referring to.

 

[AtenRa]

Direct X 12 CPU With Hyper Threading Disabled:

CPU Settings:

What I find interesting is in the D 11 CPU test the cores only ran about about 50% utilization. In the D 12 CPU test with and without HT enabled they all ran full out at 100% utilization for the duration of the test. I want to see someone with 6 or more cores run it.

Thanks for the tests, yea i would like to see more CPUs in that test as well