Flaws on Oxide

[Paul98]

Which is why it should have an LOD system.

Object so far away, its simply at dot = 1 draw call (assumption need someone with more experience to confirm),

Multiple identical objects= batched draw call,

Objects so far away = forget about motion blur.

It looks like oxide is running the same draws on each object despite them being identical objects that are very far away (like independently revolving turrets on a dot). Also objects seem to be rendered, even if they are occluded by an explosion.

Like I said magic, you haven't done any DX programming so you really have no clue about what you are talking about.

 

[blackened23]

Like I said magic, you haven't done any DX programming so you really have no clue about what you are talking about.

So I take it that you're a developer? Credentials?

 

[Spjut]

BF4 is. Repi's tweet says they are supporting it. Repi's slides say the only major roadblock is "performant drivers". Nvidia had the "performant" drivers problem solved since driver release 270, as shown by the sudden and confusing massive Civ V FPS improvement.

I find it sad that so many here think that developers trying to sell a platform will tell them the whole truth.

Having support is one thing, using it is another.

You are welcome to look at one of repi's twitters from two days ago, where he states that it is "fundamentally broken". That sounds like a far range from your claim about Nvidia solving it since the 270 drivers (which also were available before the release of BF3)

Repi could withhold parts of the truth, but his claim is further strengthened by the absence of DX11 games using multithreaded rendering.


It's also pretty naive to point to a benchmark where us normal folks have no idea where the performance limitation lies, and claim a difference exists because of feature X

 

[Gloomy]

Hi everyone –
There has been some confusion about Star Swarm, Mantle, performance on various hardware, and other topics that we’d like to clear up. Here are the best answers we can give to some of the most common questions and complaints we’ve seen since Star Swarm launched to the public yesterday:

Q. Where is the Mantle support you promised?
A. AMD has delayed the Mantle driver beta release to an unspecified date. As soon as it comes out, you can download it from AMD here: http://support.amd.com/en-us/kb-articles/Pages/latest-catalyst-windows-beta.aspx. Mantle support is built into Star Swarm, so the demo is ready to go as soon as the driver is available.

Q. What’s with registration for a free demo?
A. We’ve heard the complaints about the registration process and have decided to strip out the activation process. A new build of Star Swarm will be uploaded to Steam shortly, or very possibly will already be available by the time this message is published.

Q. Why are you calling this a benchmark? I get different results every time I run it.
A: Star Swarm is designed as a total game engine test, not just a graphics test. As such, we have complete AI, flocking, physics, etc. being simulated instead of a pre-canned demo. We believe this adds an important real-world component, and any data gathered from the stress test displays a true game result.
In addition, adding completely deterministic behavior to any engine will create artificial results and stress points, which is something we feel is important to avoid.
However, we are putting together a guide for people interested in getting more consistent numbers. Stay tuned. In the meantime, use the “Follow” scenario for benchmarking purposes – the “Attract” scenario was built to provide random views of the battle.

Q. This is just a marketing tool for AMD; you’ve obviously crippled the DirectX version!
A. We really haven’t; to be perfectly honest we’ve spent more time optimizing for DirectX than we have on Mantle. The fact is that DirectX is conceived and implemented as a single-threaded API, and so a lot of the more significant gains we see thanks to the Nitrous engine’s aggressive multithreading are badly limited by API overhead when we’re using it.
We obviously can’t prove this to the satisfaction of everyone on the Internet, but understand that our primary goal with Nitrous is to make the best engine we can so that we can open the door to the new kinds of games that we want to make (and play ourselves!). An awful lot of the gamers we hope to entertain don’t and won’t have access to Mantle-enabled hardware any time soon, so we’d be making a huge mistake as entertainers and as businesspeople by not supporting or poorly supporting non-Mantle hardware.

Q. Why is my CPU usage imbalanced when running Star Swarm on DirectX?
A. One of the issues we are trying to help Microsoft and their partners with DirectX work on is the serial nature of the DirectX API. The imbalance you are noticing is directly related to this issue. When running an API such as Mantle, which is designed to allow scalability across multiple cores, this issue will disappear. For a more in-depth answer, please see Oxide’s APU 13 presentation regarding this subject: http://www.youtube.com/watch?v=QIWyf8Hyjbg

Q. Why is my GPU utilization not at 100% on DirectX?
A. DirectX and the driver are responsible for interpreting the graphics input from the application and converting that to commands that the GPU can process. Under the current DirectX API, this can be a very CPU-intensive process. If the CPU cannot generate commands faster than the card can process them, then your GPU will not be fully utilized. For a more in-depth answer, please see Oxide’s APU 13 presentation regarding this subject: http://www.youtube.com/watch?v=QIWyf8Hyjbg

Q. Are games using Nitrous going to have the same DirectX performance as Star Swarm?
A. No. We are working closely with all partners involved with DirectX to improve its performance going forward. Star Swarm is intended as a stress test, to ensure that games built on Nitrous will have an opportunity to efficiently use all the hardware available and not be artificially limited by the platform it is running on. In addition, all games will tailor their graphics settings to allow for optimal performance on each platform. Since Star Swarm is a stress test, it makes no distinction between DirectX and other platforms.

Q. Is Star Swarm unfairly targeting DirectX?
A. No. In fact our performance under DirectX is quite good. With a high-end CPU with excellent single-core performance and a high-end GPU, Nitrous can push up to 30,000 batches at 30 frames per second. We believe this number to be quite competitive with any other engine out there. We have also made additional optimizations specific to the DirectX version which are not required on other platforms.

Q. What about deferred contexts? Don’t those help multithreaded applications in DirectX?
A. Until the very latest drivers, we saw equal or even worse performance using deferred contexts in Nitrous. The newest drivers, however, have seen performance gains and so the latest build of Star Swarm can be set to use deferred contexts. Users can enable deferred contexts by modifying the line EnableDeferredContexts=FALSE to =TRUE in their settings files (for more information on customizing settings, see the link below). We have seen some stability issues with deferred contexts enabled, so change the setting at your own risk.

Q. How can I customize Star Swarm with different settings?
A. See this post.

http://www.oxidegames.com/2014/01/31/star-swarm-faq/

 

[jj109]

You are welcome to look at one of repi's twitters from two days ago, where he states that it is "fundamentally broken". That sounds like a far range from your claim about Nvidia solving it since the 270 drivers (which also were available before the release of BF3)

Repi could withhold parts of the truth, but his claim is further strengthened by the absence of DX11 games using multithreaded rendering.

Metro LL, Civ V, BF4, Crysis 3, and AC3 (this one didn't need it lol) come to mind immediately. Why would a rushed console port need DX11 CL in the first place?

Hey a new update for Star Swarm:

Added DeferredContexts flag to ini files (disabled by default - use at your own risk)

It doesn't work unfortunately. Immediately crashes.

 

[Spjut]

Metro LL, Civ V, BF4, Crysis 3, and AC3 (this one didn't need it lol) come to mind immediately. Why would a rushed console port need DX11 CL in the first place?

Out of those games, the only ones confirmed using it are Civilization V and Assassins Creed 3.

I don't know why a console port would need it in the first place, I don't think the majority of the ones arguing about it in here do know it either.

Actual graphics programmers however, the guys who make their living making the engines, apparently do think a better solution is needed, and a few of them are even prepared to support a whole different API.

 

[Enigmoid]

Like I said magic, you haven't done any DX programming so you really have no clue about what you are talking about.

Please elaborate. I thought I had the gist of it (but again as I said and continue to say I'm only a novice if that).

But please, if you know then TELL me, rather than calling me clueless.

 

[Carfax83]

Actual graphics programmers however, the guys who make their living making the engines, apparently do think a better solution is needed, and a few of them are even prepared to support a whole different API.

The problem with DX11 multithreading, is that it's performance is wholly contingent on the drivers. It's the drivers that are responsible for deferred context thread creation and recording to the command list for upload to the immediate context..

Thing is though, this is a mammoth undertaking to implement. It took NVidia almost 2 years I think to finally implement it (and they're still refining it), and AMD still hasn't done so yet.

As the drivers improve, so will the DX11 multithreading performance. That link I posted on the previous page showed a GTX 660 getting roughly 55% faster frame rates than a 7850 in Project CARS, a game that uses driver command lists:

GTX 660 vs 7850 in Project CARS

55% is very significant. AC III used it as well, but the gains were less as the game isn't very CPU bound. But still, it allows NVidia to edge out AMD in that title:

These are GPU limited settings, but the most CPU limited card, the Gigabyte GTX 780 Ti GHz is 28% faster than the Asus 290x DirectCU II.

 

[james1701]

3770K stock settings, 7970ghz stock settings, 8gb Corsair 2133mhz memory.
Crossfire not working.

================================================== =========
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\james\Documents\Star Swarm\Output_14_01_31_2054.txt
Version 1.00
01/31/2014 20:54
================================================== =========

== Hardware Configuration =================================
GPU: AMD Radeon HD 7900 Series
CPU: GenuineIntel
Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
Physical Cores: 4
Logical Cores: 8
Physical Memory: 8530481152
Allocatable Memory: 140737488224256
================================================== =========


== Configuration ==========================================
API: DirectX
Scenario: ScenarioFollow.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Disabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
================================================== =========


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 10768

Average FPS: 29.91
Average Unit Count: 4050
Maximum Unit Count: 5606
Average Batches/MS: 530.76
Maximum Batches/MS: 1394.93
Average Batch Count: 20184
Maximum Batch Count: 159676
================================================== =========



Second Run
Average FPS: 35.83

Third Run
Average FPS: 33.46

Fourth Run
Average FPS: 46.16

Fifth Run
Average FPS: 28.30

Xfire Enabled without Profile
Run 1
Average FPS: 26.76

Run 2
Average FPS: 29.90

Some interesting observations during these runs. When single card was working, set for Xfire with profile, GPU usage during the heavy scenes was at 32% with CPU usage at 20%. That is the scenes with a kajillion lasers and ships and other stuff. During lighter action scenes, I got 100% GPU usage, and about 24% CPU usage.

When I enabled Xfire without profile, my GPU usage was 54% on both cards, and 25% on the CPU usage on light action scenes. When it came to the heavy stuff, I got 20% on both GPU's and 21% on the CPU.

Everything monitored with Aida 64 on G19

 

[Enigmoid]

Average FPS: 29.91

Second Run
Average FPS: 35.83

Third Run
Average FPS: 33.46

Fourth Run
Average FPS: 46.16

Fifth Run
Average FPS: 28.30

Are these identical runs? The variation is massive.

 

[james1701]

Yes. The benchmark is dynamic, so I don't think you can get fully reproducible results.

 

[jj109]

I don't understand why they tied the simulation rate to the FPS. There's a setting to use fixed time steps but it obviously doesn't work. When the scene pans over to the blob, everything slows down appreciably.

 

[Despoiler]

Yes. The benchmark is dynamic, so I don't think you can get fully reproducible results.

It's not a benchmark as people are running it, it's a stress test. The mode people are using is two AI opponents fighting each other trying to win a game.

 

[DaZeeMan]

I see that gloomy posted the FAQ from the Oxide forums. I didn't realize that they had added a forum to coincide with the demo release - good to know...


I fired off a PM to Brad Wardell this morning when I saw this thread (I'm a member over at the Stardock forums), letting him know about this thread. He sent me this reply, with permission to repost it here:

Frogboy
Sent Yesterday, 11:58 AM
I think some of them are not understanding the point of a stress test benchmark. If a 2014 stress test didn't push their hardware, what would be the point? It would be like people complaining about a 2003 stress test because the app used a lot of lighting and shadows, features that weren't common back then, and saying it shouldn't try to use those features.
The assets do have LOD and such in them. The reason it's slow on DirectX is because, by default, it has per-object motion blur (it's not a post processor effect like you get in a typical FPS). So it's literally rendering every object off-screen many times in order to create a more realistic scene.
AMD is scheduled to release Mantle imminently and what it brings to the table (and for us, it's the main thing) is that its platform is very multi-core friendly. The reason (and the main reason) Star Swarm demo is slow on DirectX is because even DirectX 11 isn't very good in dealing with a multi-core engine.
Now, on the games that are using Nitrous, those games are very aware of what the installed base has on them and as a result optimize their experience for that user base. But that's not what Star Swarm is supposed to do. It's *supposed* to hammer the user's system so that they can see, especially over time as Nvidia, Intel (and of course AMD's Mantle) improve their multi-core rendering support.
The goal here, imo, is that someone running Star Swarm today should go "eGads, that's slow!" and yet a year from now, when the drivers (not the hardware but the software that controls the hardware) are improved you'll be able to run the same demo and see a big benefit.

You can repost this, I won't have time to head over there.

I did see that a forum poster over there managed to get between roughly 13.58 and 32.16 Average FPS depending on the settings. His system:
Intel 3930k @ 4.6ghz (watercooled)
Rampage IV Extreme (watercooled)
16Gb Corsair Vengeance 1600mhz
2x Gigabyte GTX780Ti's
Corsair Force 3 SSD


By comparison, my circa 2007 laptop with integrated graphics has no hope of running this... maybe the next laptop.

 

[Gloomy]

I get 2-5 FPS on my laptop/tablet, get @ me dog

 

[krumme]

Are these identical runs? The variation is massive.

You post a thread with the title "Flaws on Oxide" - (And i guess you mean "Star Swarm" that is the engine, not Oxide that is the name of the company)

Yet you dont know the most basic about this test and engine, as this question shows. Next time take some time to try to understand and read about the subject, or post it as questions.

 

[Imouto]

DX is not messed up. How could anyone with any sense make a comment like that, when PC games have never looked or performed better!?

We have the Witcher 3, a game that will feature no loading screens at all, and have no chapters. It's completely open World from beginning to end.

And it's DX11 native. A game like that would not even be possible on DX9 without serious sacrifices. And your precious Mantle uses HLSL, a DX11 feature to work btw..

Now seriously, you're the most clueless guy I've ever talked to. What's the novelty on it? We had this 15 years ago with Legacy of Kain: Soul Reaver on the PSX.

Right, almost 50% is "minimal." But when Mantle claims 45% improvement it's all of a sudden spectacular!

Much maths? 50% ahead in a 7970Ghz vs GTX 780 which is already 20% faster overall? If we push the 7970Ghz a 20% to normalize things the GTX 780 would be 22% faster.

Anyway the fair comparison is the 7970Ghz vs the GTX 770 which is 15% faster.

If we compare the GTX 780 to the R9 290(nonX) the advantage shrinks to 9%.

So much for that glorious DX MT.

BTW, here's a taste of what's to come with Project CARS. A 7850 vs a GTX 660 test off, and the GTX 660 has well over 50% faster frame rates than the 7850, courtesy of DX11 multithreading.

You're talking like that game is finished or was finished almost a year ago when that vid was taken. Again no biggie, those Splinter Cell games prove it.

You're way behind the times there dude. Mantle aside, after multiple patch and driver updates, BF4 does not work better on AMD hardware. It actually runs faster on NVidia, especially in multiplayer where the CPU plays a bigger role in performance.

Read the latest reviews at GameGPU, HardOCP, PClabs.pl or any website that reviews using MP, and you'll see that NVidia now has the lead.

I'd wait for Mantle reviews, then we talk about who's living in the past.

You can think what you like, but Project CARS has tons of youtube channels devoted to it. Also, it's the biggest multiplatform racing game for next gen, that alone will make it a success.

Perfect, let's talk about it in Q3 this year m'kay?

 

[VulgarDisplay]

The problem with DX11 multithreading, is that it's performance is wholly contingent on the drivers. It's the drivers that are responsible for deferred context thread creation and recording to the command list for upload to the immediate context..

Thing is though, this is a mammoth undertaking to implement. It took NVidia almost 2 years I think to finally implement it (and they're still refining it), and AMD still hasn't done so yet.

As the drivers improve, so will the DX11 multithreading performance. That link I posted on the previous page showed a GTX 660 getting roughly 55% faster frame rates than a 7850 in Project CARS, a game that uses driver command lists:

GTX 660 vs 7850 in Project CARS

55% is very significant. AC III used it as well, but the gains were less as the game isn't very CPU bound. But still, it allows NVidia to edge out AMD in that title:

These are GPU limited settings, but the most CPU limited card, the Gigabyte GTX 780 Ti GHz is 28% faster than the Asus 290x DirectCU II.

It shows a bigger die, and much more expensive overclocked card winning by 28%.

The 290x when prices are not silly is more in league with the gtx780.

 

[OCGuy]

You might want to read up and or watch the interviews with the Oxide devs on what their goals were with their engine, how its engineered, and what is actually going on that you are looking at.

The short is that they were going for a render engine that is exactly like what Hollywood uses in movies only it's real time. Everything on the screen is an independent entity with full physics and AI were applicable.

http://forums.anandtech.com/showthread.php?t=2361756

https://www.youtube.com/watch?v=VC8RWntPRvI

Oh come on now...

I think the writing is on the wall at this point.

 

[richaron]

Oh come on now...

I think the writing is on the wall at this point.

You already have real benchtsets!!

 

[sontin]

Repi said yesterday that DCL is fundamentally broken in DX.

Yesterday.

And yet, at the same day the other Mantle party member updated their benchmark with Deferred Contexts.

And it improves performance on my nVidia card up to 100%:

With

== Results ================================================
Test Duration:            99 Seconds
Total Frames:            6072

Average FPS:            60.76
Average Unit Count:        3636
Maximum Unit Count:        5400
Average Batches/MS:        746.74
Maximum Batches/MS:        2130.22
Average Batch Count:        14166
Maximum Batch Count:        79900

Without:

Test Duration:            99 Seconds
Total Frames:            3237

Average FPS:            32.41
Average Unit Count:        3703
Maximum Unit Count:        5345
Average Batches/MS:        739.74
Maximum Batches/MS:        1500.26
Average Batch Count:        28048
Maximum Batch Count:        151054

It's not DC which is broken.

 

[BrightCandle]

I enabled deferred contexts (which on a pair of Nvidia 680's does work) and I came out with the following on otherwise identical settings with the latest version (that runs for 360 seconds not 120):

DeferredContexts enabled:

Test Duration:			360 Seconds
Total Frames:			15948

Average FPS:			44.30
Average Unit Count:		4418
Maximum Unit Count:		5952
Average Batches/MS:		838.57
Maximum Batches/MS:		4511.59
Average Batch Count:		20080
Maximum Batch Count:		132196

And with it disabled

Test Duration:			360 Seconds
Total Frames:			12145

Average FPS:			33.73
Average Unit Count:		4065
Maximum Unit Count:		5383
Average Batches/MS:		614.70
Maximum Batches/MS:		1786.21
Average Batch Count:		19289
Maximum Batch Count:		114370

Deferred contexts is at least showing for me a significant performance improvement, of around 50%. It did so on a test that had 10% more units on average and at peak and shows clear improvements in batch sizes and performance. Deferred contexts seem to have a similar benefit magnitude to Mantle, but without the need to completely change API. Interesting very interesting.

Ohhh snap someone just beat me to it!

 

[jj109]

You post a thread with the title "Flaws on Oxide" - (And i guess you mean "Star Swarm" that is the engine, not Oxide that is the name of the company)

Yet you dont know the most basic about this test and engine, as this question shows. Next time take some time to try to understand and read about the subject, or post it as questions.

Since you apparently know the basics about this test and engine, please explain why their "revolutionary" engine has inferior CPU utilization compared to CryEngine 3, Civ5, 4A Engine, and Frostbite 3 in CPU-bound situations. Hey look, they all use deferred context, token+replay, or command lists to make their engine efficient.

Edit:

How did you guys get DC working?

 

[sontin]

Since you apparently know the basics about this test and engine, please explain why their "revolutionary" engine has inferior CPU utilization compared to CryEngine 3, Civ5, 4A Engine, and Frostbite 3 in CPU-bound situations. Hey look, they all use deferred context, token+replay, or command lists to make their engine efficient.

Edit:

How did you guys get DC working?

You can enable it in the .ini files.

 

[Gloomy]

What's the performance like with DC, but without blur?

Also, someone with an AMD card should step up.