News NVidia Streamline to combine NVidia DLSS, Intel XeSS, and maybe AMD FSR in one interface

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
As soon as I saw Intel XeSS, and handful of supporting games, I was thinking we really need an abstraction layer so developers can write once, and support all the advanced Image scaling methods.

I was surprised that NVidia is the first to deliver this abstraction layer:


Thankfully, Streamline is both open-source and can accommodate super-resolution technologies from diverse hardware and game engine vendors. Intel is already on board, but we have no word from AMD on its plans.

...

"Intel believes strongly in the power of open interfaces," said Andre Bremer, VP of AXG and director of game engineering at Intel. "We are excited to support Streamline, an open, cross-IHV framework for new graphics effects. This will simplify game developers’ integration efforts and accelerate the adoption of new technology."

Nvidia says that Streamline is available today on GitHub supporting both DLSS and DLAA (Deep Learning Anti-Aliasing). NIS support is "coming soon." The framework is extensible beyond super-samplers, as evidenced by Nvidia including its Real-time Denoiser. Please note that Streamline supports DirectX 11 and DirectX 12 titles, but Vulkan compatibility is still in the beta testing stages.
 
  • Wow
Reactions: igor_kavinski

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
Kyle Bennett from hardocp is not impressed

He's a raging NVidia hater, so that's not exactly a surprise.

Intel would probably be the biggest winner if this takes off, because they are facing far less support for XeSS than they other players are getting for their solutions.

These things have to start somewhere, and I don't care who starts it, as long as it gets done.

It's open source and eventually I expect it will be pulled in Vulcan as HW agnostic API, and perhaps Microsoft will have their own API eventually as well.

Then everyone can just do their own implementation of a common API. This is the best possible outcome, instead of piecemeal approach that leads to games supporting one solution or the other.
 
  • Like
Reactions: xpea and Leeea

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
He may be an NV hater, but he is 100% right.

We need this incorporated in independent industry standard, not in a piece of open-sourced SW "controlled" by any IHV

Spare me the paranoia.

Khronos group will likely take it over for Vulkan, much like they used NVidia's open source Vulkan implementation of Ray Tracing, as the basis for Vulkans Ray Tracing APIs.

It has to start somewhere with someone.
 
  • Like
Reactions: Tup3x

Tup3x

Senior member
Dec 31, 2016
940
922
136
Spare me the paranoia.

Khronos group will likely take it over for Vulkan, much like they used NVidia's open source Vulkan implementation of Ray Tracing, as the basis for Vulkans Ray Tracing APIs.

It has to start somewhere with someone.
I wouldn't be surprised if this is a start for new API feature. This is not a bad thing and probably will push things in the right direction.
 

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
Nvidia will allow Vendor #3 if FSR will be renamed to "DLSS Compatible" :tearsofjoy:

Apparently you don't know what open source means.

NVidia can't control anything. It's also under the MIT license which is extremely permissive (basically you can do anything you want with it).

Khronos group can just take the code, and incorporate it into Vulkan, and make any changes they want, and NVidia can't do anything to stop them.

Likewise AMD/Intel/Microsoft/anyone can just take the code and do anything they want with it.
 

Aapje

Golden Member
Mar 21, 2022
1,298
1,747
96
FSR 2 is a big threat to DLSS as the former will support all GPUs, so it's very tempting for developers to implement it, even if it is a little worse than DLSS.

By having a standard interface, Nvidia gets the benefit of their cards having the better SS implementation, assuming that this is actually true.
 
  • Like
Reactions: Tlh97

Bigos

Member
Jun 2, 2019
125
269
136
Apparently you don't know what open source means.

NVidia can't control anything. It's also under the MIT license which is extremely permissive (basically you can do anything you want with it).

Khronos group can just take the code, and incorporate it into Vulkan, and make any changes they want, and NVidia can't do anything to stop them.

Likewise AMD/Intel/Microsoft/anyone can just take the code and do anything they want with it.

This is as "open source" as GPUOpen initiatives are. The source is there, the license allows you to use it. But the development process is entirely in-house. NVidia is the owner of the repository and they can publish whatever they want there.

For a project which is meant to unify other vendors' solutions, giving it in the hands of one such vendor is problematic to say the least. There is no guarantee the end result will work well on competitor's hardware. You can certainly fork it and fix the problems yourself, but now we are back with multiple repos, one from each vendor... What is a game developer meant to use then?

It would make more sense for this to be governed by either a third party or a consortium. However, D3D or Vulkan are not correct places. The upscaling technologies are just algorithms, like shadow mapping, ambient occlusion, etc. There is no place in the graphics driver for such things.

A separate project governed by Khronos could fit the bill. While Khronos is not known for open source development practices (it is still a walled garden), at the very least it consists of all of the stake holders - IHVs, game developers, etc.

The most ideal solution would be for this to be a 100% open source project. With open development workflow, multiple parties contributing their changes, CI systems that monitor for performance regressions, etc. One can dream...
 

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
It would make more sense for this to be governed by either a third party or a consortium.

That's fine too, if anyone wants to step up and take over, they can simply take NVidia's project and fork it.

FSR 2 is a big threat to DLSS as the former will support all GPUs, so it's very tempting for developers to implement it, even if it is a little worse than DLSS.

By having a standard interface, Nvidia gets the benefit of their cards having the better SS implementation, assuming that this is actually true.

By having a standard interface, everyone benefits. Users benefit by having support for their particular card. IHV win by not having to negotiate/pay for their particular solution to be implemented on a per game basis. I guess you could argue game devs lose as they aren't as likely to get payments to implement a single specific solution.
 

GodisanAtheist

Diamond Member
Nov 16, 2006
6,685
6,945
136
By having a standard interface, everyone benefits. Users benefit by having support for their particular card. IHV win by not having to negotiate/pay for their particular solution to be implemented on a per game basis. I guess you could argue game devs lose as they aren't as likely to get payments to implement a single specific solution.

- I'm curious about that statement. NV is generally not one to hand out their stuff for free (to competitors) unless it gives them a leg up over the competition (See: entire Gameworks suite). They've reportedly even burned bridges with console manufacturers trying to play hardball with rights over certain proprietary implementations. As the market leader, something that helps everyone is really something that hurts them, so why would they be doing this? The status quo in theory should benefit them: they have the most mature tech, they have the better dev relation team, they have the lions share of the relevant GPU markets...

Playing devil's advocate as to why NV is doing this (i.e. how it benefits them):

- Its a trojan horse: NV's tech will fare the best simply "dropped in" to any game engine without any developer input thanks to the underlying nature of the technology and NV's head start in making sure the tech is mature. AMD/Intel's solutions might require the most "hand tuning" to dial in competitive visual quality and the Streamline solution will disincentivise that extra hand tuning step, resulting in FSR 2.0 and XeSS looking generally worse than the DLSS option.

- They fear DLSS being abandoned: If alternative solutions work on NV's hardware, but NV's solution doesn't work on the competition, then software devs are going to go with the options that have the broadest reach: FSR 2.0 and maybe XeSS. Streamline provides a backstop to ensures that DLSS comes along for the ride anytime upscaling techniques are incorporated into a game.
 

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
- I'm curious about that statement. NV is generally not one to hand out their stuff for free (to competitors)...

This is not really giving away anything for free. It's just a common API, not back end algorithms.


Playing devil's advocate as to why NV is doing this (i.e. how it benefits them):

- Its a trojan horse: NV's tech will fare the best simply "dropped in" to any game engine without any developer input

Seriously doubt any of them need individual tuning, outside of adjusting the sharpening level, and if they establish good defaults that probably doesn't matter.



- They fear DLSS being abandoned: If alternative solutions work on NV's hardware, but NV's solution doesn't work on the competition, then software devs are going to go with the options that have the broadest reach: FSR 2.0 and maybe XeSS. Streamline provides a backstop to ensures that DLSS comes along for the ride anytime upscaling techniques are incorporated into a game.

Probably closer to this. Though even if this is the motivation, it's still is beneficial to have a common API.

Common APIs will ensure the widest support, and allow users to choose the best option for their cards.

Common APIs are basic no-brainer. Imagine if for Ray Tracing, developers had to code it uniquely for NVidia, AMD and soon Intel, instead of just using one API, and letting each IHV write their own code to answer API calls. No one would think this is a good idea, people would be calling for a common API.

Now we have a potential sensible move to a common scaling API, and I get the sense people are against it, largely because they don't like the source.
 

Saylick

Diamond Member
Sep 10, 2012
3,082
6,172
136
- They fear DLSS being abandoned: If alternative solutions work on NV's hardware, but NV's solution doesn't work on the competition, then software devs are going to go with the options that have the broadest reach: FSR 2.0 and maybe XeSS. Streamline provides a backstop to ensures that DLSS comes along for the ride anytime upscaling techniques are incorporated into a game.
It's pretty much this. Developers are going to want to implement upscaling techniques that offer the widest adoption range as possible so that they can address the largest portion of the market as possible. More people who can take advantage of upscaling ---> more people who buy the game ---> more revenue for the game developer. If we assume this to be true, it means that the adoption of XeSS and FSR 2.0 will be prioritized over DLSS because DLSS only works for a subset of the entire market, albeit it is a large subset nonetheless. By introducing Streamline, Nvidia give developers a way to piggy back off of XeSS and FSR 2.0's potential success, therefore ensuring that DLSS never gets dropped by the way side. In theory, if XeSS and FSR 2.0 become good enough, there would be little incentive for developers to implement DLSS at all because, again, it only works on a subset of the market. Streamline is clearly a defensive play, and they must know that the above is true for them to even offer something like this so openly and so quickly, especially since XeSS and FSR 2.0 aren't even out yet.
 

GodisanAtheist

Diamond Member
Nov 16, 2006
6,685
6,945
136
Now we have a potential sensible move to a common scaling API, and I get the sense people are against it, largely because they don't like the source.

- Not sure about against it, just skeptical given the source.

We know AMD does open source stuff because they simply do not have the internal development framework and GPU first philosophy that NV does, so people are rarely skeptical of AMD's motives because it logically makes sense for them to provide open source stuff given their history as the underdog in all markets and lack of resources.

NV on the other hand has almost promoted itself as "too good" for open source plebian garbage and has been all about the NV ecosystem and walled garden, so when they release something seemingly from the goodness of their hearts, people have a right to be skeptical of the provided reasoning.

And I'm not some whizbang NV Dev or Marketing exec or whatever, so I'm extra skeptical because I recognize I may not have the brainpower or knowledge to figure NV's angle here. I just have a bunch of circumstantial evidence and smoke, but no fire.
 
  • Like
Reactions: Tlh97 and Saylick

Aapje

Golden Member
Mar 21, 2022
1,298
1,747
96
The decision of Intel to start making discrete video cards increases the incentive for Nvidia to open up the APIs, because otherwise Intel and AMD would logically work together to create an API and pool their budgets to promote it.

The rule in war and business is divide and conquer, not making your opponents work together.
 

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
The decision of Intel to start making discrete video cards increases the incentive for Nvidia to open up the APIs, because otherwise Intel and AMD would logically work together to create an API and pool their budgets to promote it.

No sign of that though, instead Intel went it's own way with XeSS, and obviously even paid a few devs to include it's solution.

It was seeing a small list of XeSS supporting games that really made me think that this is really a mess now with a third option joining the fight for developer attention, and someone should do an abstraction layer.

I really thought Intel would do it, because they are by far the underdog on this, and really the most gain if this shifts to one abstract API.
 

BFG10K

Lifer
Aug 14, 2000
22,656
2,778
126
This is as "open source" as GPUOpen initiatives are. The source is there, the license allows you to use it. But the development process is entirely in-house. NVidia is the owner of the repository and they can publish whatever they want there.
Exactly right. Anyone that thinks nVidia is doing this to help consumers is delusional. At the end it's all about control or locking something down.

This reminds me of the time nVidia "freely offered" hardware PhysX to AMD...by telling AMD they can spend their own resources to create a back-end to support a competitor's technology. o_O

Remember folks, this is the same company that disabled hardware PhysX on paying customers' cards if the drivers detected an AMD GPU in the system.
 

Stuka87

Diamond Member
Dec 10, 2010
6,240
2,559
136
Not sure why somebody is going out of their way to say how great this is for everybody. nVidia has a very documented history of doing this kind of thing to appear "open" while maintaining control.

This reeks of a way for nVidia to get developers to implement their code by claiming "oh, if you implement DLSS, and then use our man-in-the-middle API, it will still work on other manufactures cards!" with the small text being, at a much lower level of performance.

Sure, this may allow DLSS to run on an AMD or Intel card. But its going to be dog slow. Which will make other cards look really bad.

EDIT: For reference, they did this same thing with GameWorks. It was cross platform, at a cost.
 
Last edited:
  • Like
Reactions: Tlh97

Aapje

Golden Member
Mar 21, 2022
1,298
1,747
96
Sure, this may allow DLSS to run on an AMD or Intel card. But its going to be dog slow. Which will make other cards look really bad.

That's not what the software does...

It allows developers to program to a single API and yet support DLSS, FSR 2 and XeSS. DLSS is still only going to be available with an Nvidia GPU.

This will benefit Nvidia if DLSS is better than the other two and neutral if it is worse (assuming that the winner works just as well on Nvidia cards). Not doing this is only beneficial to Nvidia if they can get game developers to support DLSS, but not support FSR 2 or XeSS, which seems unlikely to me.
 
  • Like
Reactions: Dribble

Heartbreaker

Diamond Member
Apr 3, 2006
4,219
5,220
136
That's not what the software does...

It allows developers to program to a single API and yet support DLSS, FSR 2 and XeSS. DLSS is still only going to be available with an Nvidia GPU.

This will benefit Nvidia if DLSS is better than the other two and neutral if it is worse (assuming that the winner works just as well on Nvidia cards). Not doing this is only beneficial to Nvidia if they can get game developers to support DLSS, but not support FSR 2 or XeSS, which seems unlikely to me.

And it benefits everyone because unlike the status quo where you may get three games where:

Game A: Only supports DLSS.
Game B: Only supports XeSS.
Game C: Only supports FSR 2.

Instead you get:

Game A: Supports DLSS, XeSS and FSR 2.
Game B: Supports DLSS, XeSS and FSR 2.
Game C: Supports DLSS, XeSS and FSR 2.

The second situation is obviously better for consumers, regardless of which brand you buy, because they all get supported in all the games.
 

Aapje

Golden Member
Mar 21, 2022
1,298
1,747
96
And it benefits everyone [...]

The second situation is obviously better for consumers, regardless of which brand you buy, because they all get supported in all the games.

Consumers are not everyone. The best scenario for Nvidia is if consumers only get good SS with Nvidia cards and they can convince developers to build in specific support for it because it is so good. The second best situation is if DLSS is a bit better than the competition or roughly on par with FSR 2 and yet any game that has FSR 2 also has DLSS. The worst situation is if future games only support FSR 2, DLSS dies due to lack of adoption and AMD gets all the credit for supersampling, getting a ton of free advertising and gratitude from gamers.

This gambit is all about reducing the risk of that worst scenario at the expense of taking a shot at the best scenario. This suggests that Nvidia thinks that FSR 2 is very good and don't think that they can maintain the huge gap over FSR 1 that they now enjoy, which seems like a good assessment to me.