News NVidia Streamline to combine NVidia DLSS, Intel XeSS, and maybe AMD FSR in one interface

[Heartbreaker]

As soon as I saw Intel XeSS, and handful of supporting games, I was thinking we really need an abstraction layer so developers can write once, and support all the advanced Image scaling methods.

I was surprised that NVidia is the first to deliver this abstraction layer:

Nvidia Streamline Aims to Simplify Developer Support for All Upscaling Algorithms

Streamline is an open-source framework and Intel is already on board.

www.tomshardware.com

Thankfully, Streamline is both open-source and can accommodate super-resolution technologies from diverse hardware and game engine vendors. Intel is already on board, but we have no word from AMD on its plans.

...

"Intel believes strongly in the power of open interfaces," said Andre Bremer, VP of AXG and director of game engineering at Intel. "We are excited to support Streamline, an open, cross-IHV framework for new graphics effects. This will simplify game developers’ integration efforts and accelerate the adoption of new technology."

Nvidia says that Streamline is available today on GitHub supporting both DLSS and DLAA (Deep Learning Anti-Aliasing). NIS support is "coming soon." The framework is extensible beyond super-samplers, as evidenced by Nvidia including its Real-time Denoiser. Please note that Streamline supports DirectX 11 and DirectX 12 titles, but Vulkan compatibility is still in the beta testing stages.

 

[Aapje]

No it isn't because API's evolve. We have had decades of GPU APIs that every maker adhered to and evolved over time, and what we have now is three version of the same thing with the same inputs.

Except that GPU's didn't actually start out with a common interface (see GLIDE for example). That only came later and there were often proprietary extensions that only later become part of the standard.

They are both essentially modeled on DLSS, and taking the same inputs as DLSS, and attempting to create the same result.

It's purely an assumption on your part that they are using the same inputs. Just because they globally use the same inputs doesn't mean that they use the exact same inputs on a detail level. Again, two of these supersamplers haven't been released yet, so you are just making assumptions.

I first want to see someone actually make FSR 2 and XeSS work with Streamline without a performance penalty. Right now, Streamline only supports DLSS.

Given the massive lead DLSS already has over competing solutions, having a lesser version for other cards would likely kill all competing scaling alternatives.

This technology doesn't create all that much vendor-lock in, so I don't much room for any vendor to block a better solution.

 

[Heartbreaker]

This technology doesn't create all that much vendor-lock in, so I don't much room for any vendor to block a better solution.

That's funny because you also said:

My prediction is that DLSS is going to die and FSR 2 is going to take over, assuming that FSR 2 delivers on the promises.

You are assuming that FSR2 is going to kill DLSS because it's available for all brands. But if was DLSS available for all brands, that wouldn't happen. Why?

 

[Aapje]

You are assuming that FSR2 is going to kill DLSS because it's available for all brands. But if was DLSS available for all brands, that wouldn't happen. Why?

I don't think that FSR 2 is going to win due to vendor lock in, but because it is going to support all cards and is going to be good enough. I doubt that it is worth just supporting DLSS or supporting it in addition, even if DLSS is a bit better on some Nvidia cards.

And DLSS isn't available for all cards. As you noted yourself, Nvidia actually made it, but refused to release it (presumably to prevent AMD customers from benefiting, even though many Nvidia customers would also benefit). The very fact that they released Streamline, rather than a DLSS that works on all cards, suggests that they won't.

 

[Heartbreaker]

I don't think that FSR 2 is going to win due to vendor lock in, but because it is going to support all cards and is going to be good enough. I doubt that it is worth just supporting DLSS or supporting it in addition, even if DLSS is a bit better on some Nvidia cards.

And DLSS isn't available for all cards. As you noted yourself, Nvidia actually made it, but refused to release it (presumably to prevent AMD customers from benefiting, even though many Nvidia customers would also benefit). The very fact that they released Streamline, rather than a DLSS that works on all cards, suggests that they won't.

Obviously you didn't read the section of the post you replied to. Which was talking about would happen if NVidia made DLSS available for all cards...

 

[Stuka87]

Obviously you didn't read the section of the post you replied to. Which was talking about would happen if NVidia made DLSS available for all cards...

Not possible. They decided to tie it directly to specialized compute units that only some nVidia cards have.

nVidia has NEVER had a single proprietary system stay around long term. Ever. DLSS will join the dozen other things they have put out over the years. It doesn't matter if it has slightly better image quality, or maybe offers slightly better performance. Dev's are not going to waste their very valuable dev time to implement a redundant piece of technology that only works for some cards.

 

[Saylick]

Thought this more or less summarized a bunch of thoughts mentioned in this thread. Basically, FSR 2.0 adoption will be like gangbusters and Nvidia counters with Streamline to mitigate their potential losses. Intel signs up for Streamline because it could only help them with XeSS adoption as well.

 

[Heartbreaker]

Not possible. They decided to tie it directly to specialized compute units that only some nVidia cards have.

They had a version between 1.0 and 2.0 that didn't use specialized units and was just a traditional Shader program.

DLSS 1.0 This is when games needed training for each game, and it mostly sucked.
DLSS 1.5 in Control - was shader based, better than DLSS 1.0. Only Control got this version.
DLSS 2.0 New version using tensor cores, that didn't need training for each game.

DLSS 1.5 could be ported to other architectures easily. It's probably very similar to FSR 2.

 

[coercitiv]

DLSS 1.5 could be ported to other architectures easily.

Not according to Nvidia engineers:

This begs the question, why did Nvidia feel the need to go back to an AI model running on tensor cores for the latest version of DLSS? We asked Nvidia the question, and the answer was pretty straightforward: Nvidia's engineers felt that they had reached the limits with the shader version.

Apparently this implementation for Control required a lot of hand tuning and was found to not work well with other types of games, whereas DLSS 2.0 back on the tensor cores is more generalized and more easily applicable to a wide range of games without per-game training.

 

[Heartbreaker]

Not according to Nvidia engineers:

So, open source that can be easily verified, can't be trusted from NVidia.

But non-verifiable excuses, from NVidia for not doing a generic solution, are perfectly trustworthy?

IMO that was corporate dissembling, to justify not doing a generic solution, when they viewed DLSS as a competitive advantage, they wanted to keep exclusive.

They were obviously motivated to do a NVidia exclusive solution, and this was an excuse.

It's very unlikely that they reached the limits of what a shader solution could achieve in one stop gap implementation.

It would have been more reasonable to claim that a Deep Learning Solution could go further than a Shader based version, and they chose to continue with that instead, and abandon the lesser shader based solution. Instead they went the extra mile to make it sound like it was nearly impossible as a general solution, to forestall any calls for a non-RTX solution.

But the motivation against the generic solution changes a lot, when DLSS has competition like it does now.

NVidia could still have a shader based generic scaling in the works, as plan B if common interface fails. Call it DLSS lite/basic or something like that. For any game that supports DLSS, NVidia RTX cards get "True" DLSS , while other cards get DLSS lite. Now developers can choose DLSS as an option that works everywhere, it just works better on NVidia cards (same thing Intel is doing with XeSS).

Easily achievable, and as far as contingencies, a fairly obvious one.