News NVidia Streamline to combine NVidia DLSS, Intel XeSS, and maybe AMD FSR in one interface

[Heartbreaker]

As soon as I saw Intel XeSS, and handful of supporting games, I was thinking we really need an abstraction layer so developers can write once, and support all the advanced Image scaling methods.

I was surprised that NVidia is the first to deliver this abstraction layer:

Nvidia Streamline Aims to Simplify Developer Support for All Upscaling Algorithms

Streamline is an open-source framework and Intel is already on board.

www.tomshardware.com

Thankfully, Streamline is both open-source and can accommodate super-resolution technologies from diverse hardware and game engine vendors. Intel is already on board, but we have no word from AMD on its plans.

...

"Intel believes strongly in the power of open interfaces," said Andre Bremer, VP of AXG and director of game engineering at Intel. "We are excited to support Streamline, an open, cross-IHV framework for new graphics effects. This will simplify game developers’ integration efforts and accelerate the adoption of new technology."

Nvidia says that Streamline is available today on GitHub supporting both DLSS and DLAA (Deep Learning Anti-Aliasing). NIS support is "coming soon." The framework is extensible beyond super-samplers, as evidenced by Nvidia including its Real-time Denoiser. Please note that Streamline supports DirectX 11 and DirectX 12 titles, but Vulkan compatibility is still in the beta testing stages.

 

[Heartbreaker]

Consumers are not everyone. The best scenario for Nvidia is if consumers only get good SS with Nvidia cards and they can convince developers to build in specific support for it because it is so good. The second best situation is if DLSS is a bit better than the competition or roughly on par with FSR 2 and yet any game that has FSR 2 also has DLSS. The worst situation is if future games only support FSR 2, DLSS dies due to lack of adoption and AMD gets all the credit for supersampling, getting a ton of free advertising and gratitude from gamers.

This gambit is all about reducing the risk of that worst scenario at the expense of taking a shot at the best scenario. This suggests that Nvidia thinks that FSR 2 is very good and don't think that they can maintain the huge gap over FSR 1 that they now enjoy, which seems like a good assessment to me.

So consumers don't matter, and they only thing that does matter is making sure NVidia doesn't get anything out of this.

Got it.

 

[Aapje]

So consumers don't matter, and they only thing that does matter is making sure NVidia doesn't get anything out of this.

Got it.

I'm just explaining what I think Nvidia's angle is here. If you think that Nvidia's interest aligns perfectly with the interest of their customers, I have a bridge to sell you.

 

[Saylick]

Consumers are not everyone. The best scenario for Nvidia is if consumers only get good SS with Nvidia cards and they can convince developers to build in specific support for it because it is so good. The second best situation is if DLSS is a bit better than the competition or roughly on par with FSR 2 and yet any game that has FSR 2 also has DLSS. The worst situation is if future games only support FSR 2, DLSS dies due to lack of adoption and AMD gets all the credit for supersampling, getting a ton of free advertising and gratitude from gamers.

This gambit is all about reducing the risk of that worst scenario at the expense of taking a shot at the best scenario. This suggests that Nvidia thinks that FSR 2 is very good and don't think that they can maintain the huge gap over FSR 1 that they now enjoy, which seems like a good assessment to me.

To add to this, the fact that Nvidia is rolling Streamline out now, i.e. before XeSS and FSR 2.0 are even out, is very telling. They must know that the adoption rate of those two technologies will be very fast, so offering Streamline now, rather than say a year after XeSS and FSR 2.0 are released, is very much intended to piggy back off of the potential success of XeSS and FSR 2.0, thus ensuring that DLSS adoption is roughly on par with the other two competing technologies. I mean, let's be real: if DLSS offered a significant benefit at the same cost over XeSS or FSR 2.0, Nvidia would NEVER offer Streamline. If Nvidia were truly doing something for all consumers, they'd open source DLSS for all vendors, but yet they won't.

 

[Saylick]

And it benefits everyone because unlike the status quo where you may get three games where:

Game A: Only supports DLSS.
Game B: Only supports XeSS.
Game C: Only supports FSR 2.

Instead you get:

Game A: Supports DLSS, XeSS and FSR 2.
Game B: Supports DLSS, XeSS and FSR 2.
Game C: Supports DLSS, XeSS and FSR 2.

The second situation is obviously better for consumers, regardless of which brand you buy, because they all get supported in all the games.

Without Streamline, the real situation is closer to this:
Game A: Supports XeSS and FSR 2.0.
Game B: Supports XeSS and FSR 2.0.
Game C: Supports XeSS, FSR 2.0, and DLSS.

And Nvidia want it to be:
Game A: Supports DLSS, XeSS and FSR 2.
Game B: Supports DLSS, XeSS and FSR 2.
Game C: Supports DLSS, XeSS and FSR 2.

 

[Heartbreaker]

I'm just explaining what I think Nvidia's angle is here. If you think that Nvidia's interest aligns perfectly with the interest of their customers, I have a bridge to sell you.

No but I think the consumer benefits, should outweighs concerns that NVidia might also benefit.

It really seems some people would cut off their nose to spite their face when comes to NVidia.

We could have a situation where all games, that support advanced scaling, would support everyones scaling system. But no, screw that, NVidia might get supported in all games too, so we can't have that.

So what if this reduces risk for NVidia. How is that a harm for anyone.

You want to deny better service to all consumers for what? A better chance for NVidia to fail?

 

[Frenetic Pony]

Nvidia's angle is probably pretty simple: XESS and FSR2 work with consoles, and FSR2 works with nigh everything, but DLSS does not. They want something that keeps DLSS in games; and making it easy to put those other more useful techniques into your title while oh look DLSS is right there too is a way to accomplish this.

 

[Aapje]

No but I think the consumer benefits, should outweighs concerns that NVidia might also benefit.

It really seems some people would cut off their nose to spite their face when comes to NVidia.

We could have a situation where all games, that support advanced scaling, would support everyones scaling system. But no, screw that, NVidia might get supported in all games too, so we can't have that.

So what if this reduces risk for NVidia. How is that a harm for anyone.

You want to deny better service to all consumers for what? A better chance for NVidia to fail?

I'm really confused by your aggression towards me. I never spoke out against this technology or its adoption; nor do I have any power to influence that. I'm just arguing that Nvidia's interests aren't the same as consumer interests (if they were, Nvidia wouldn't be making record profits right now) and that it thus doesn't make any sense to interpret Nvidia's motives purely from the perspective of consumer interests.

 

[Aapje]

Nvidia's angle is probably pretty simple: XESS and FSR2 work with consoles, and FSR2 works with nigh everything, but DLSS does not. They want something that keeps DLSS in games; and making it easy to put those other more useful techniques into your title while oh look DLSS is right there too is a way to accomplish this.

Yes, I think that consoles are a very significant factor here. Basically, console games will be 'required' to implement FSR 2 in the future, so when those games are ported, there is no reason for them not to keep that support. However, implementing DLSS is extra work, which will need to be justified. So far it is justified due to how good DLSS is compared to FSR 1, but I doubt that will remain the case once FSR 2 is released.

Of course, there is no particular incentive for AMD to put Streamline support in the console drivers, so Nvidia may get screwed over this time.

 

[Heartbreaker]

I'm just arguing that Nvidia's interests aren't the same as consumer interests (if they were, Nvidia wouldn't be making record profits right now) and that it thus doesn't make any sense to interpret Nvidia's motives purely from the perspective of consumer interests.

You seem to be arguing against a good idea, because you don't like the motivation behind it.

Newsflash: they are all in it for the money. AMD/Intel/NVidia are all about making as much money as possible. None of them is your friend, none of them are doing the work for you.

So, when it comes to business actions, outcomes matter much more than motives.

The outcome of this working, benefits all GPU consumers and that's what really matters.

 

[Aapje]

You seem to be arguing against a good idea, because you don't like the motivation behind it.

Can you point out where I argued against it? Because this seems to be something that you made up.

 

[Heartbreaker]

Can you point out where I argued against it? Because this seems to be something that you made up.

When you continuously brush aside the consumer benefit, to focus only on "What does NVidia get out of this", it certainly looks like you are arguing against it because NVidia might also benefit.

When I posted a clear example of how this benefits all consumers, your answer was:

"Consumers are not everyone."

Then right back to arguing about what NVidia might get out of it.

Everyone here is a consumer, and what consumers get out of it, matters much more than what NVidia gets out of it.

So why are you ignoring the benefit to the GPU consumer (essentially all of us), and deflecting every mention of benefit to what NVidia gets?

Judging by your posts, your priorities WRT to this API appear to be:

Obvious Consumer benefit: Irrelevant.
That NVidia might benefit: The only thing that matters.

 

[Aapje]

So why are you ignoring the benefit to the GPU consumer (essentially all of us), and deflecting every mention of benefit to what NVidia gets?

Because that's not what I was discussing. For me the potential benefit to consumers is so obvious that it's hardly worth mentioning, assuming that Streamline is actually vendor-neutral (which we cannot judge).

It's a bad habit to make assumptions about what people are against based on them not making a certain argument in favor. Sometimes people just want to discuss something else than what you'd like to talk about.

Keep in mind that I was simply responding to someone else and arguing about the topic they brought up, which was about Nvidia's motive. Again, I'm allowed to address that angle and if it doesn't interest you, you are allowed to ignore that and make a different argument instead. What is not OK is to start accusing people of making claims they never made.

 

[Paul98]

NVidia knows DLSS will get less adoption moving forward with both Intel and AMD having solutions as well as AMD being in consoles which will have a way higher adoption rate of FSR2.0 as consoles will get the most out of it. Plus since FSR2.0 also works on NVidia GPU's why even bother to integrate DLSS if they are close in quality.

Streamline is NVidia wanting to keep DLSS adoption up so that they can continue to develop and market it as premium feature of NVidia GPU's.

 

[Heartbreaker]

Because that's not what I was discussing. For me the potential benefit to consumers is so obvious that it's hardly worth mentioning, assuming that Streamline is actually vendor-neutral (which we cannot judge).

It's a bad habit to make assumptions about what people are against based on them not making a certain argument in favor. Sometimes people just want to discuss something else than what you'd like to talk about.

Keep in mind that I was simply responding to someone else and arguing about the topic they brought up, which was about Nvidia's motive. Again, I'm allowed to address that angle and if it doesn't interest you, you are allowed to ignore that and make a different argument instead. What is not OK is to start accusing people of making claims they never made.

Your holier-than-thou, diatribe doesn't ring true. If you thought the consumer benefit was so obvious, why belittle it with "Consumers are not everyone"?

 

[Stuka87]

The outcome of this working, benefits all GPU consumers and that's what really matters.

Except it doesn't.

nVidia dropping DLSS and switching to a free, open source, FSR2 would benefit all customers. nVidia's closed source, proprietary systems NEVER benefit the customer. They are nVidia's way of inducing vendor lock-in.

 

[Heartbreaker]

Except it doesn't.

nVidia dropping DLSS and switching to a free, open source, FSR2 would benefit all customers. nVidia's closed source, proprietary systems NEVER benefit the customer. They are nVidia's way of inducing vendor lock-in.

NVidia GPUs will automatically support FSR2(according to AMD), so there is no consumer benefit to drop another option that is potentially superior (DLSS).

I do wish it wasn't NVidia that had done the common API, so we could get past the strong anti-NVidia sentiments that many have.

A common API is the obvious way to go for anyone with any programming background.

This wasn't the case when it was DLSS vs FSR, because they didn't work the same way and required different inputs.

But With FSR2, DLSS, and XeSS, we now have three temporal scaling systems which all need the same inputs.

It's actually stupid to not have these in a common API now that everyone is doing temporal upscaling and requiring the same inputs.

Again, it's really too bad that it was NVidia that did this. Because instead of being happy that obviously correct thing is being done, instead we have people trying to rationalize why it's wrong, or delving for dark motives, because NVidia.

I guess the next best hope is that Microsoft and Khronos pull it in house.

 

[Thala]

I mean, let's be real: if DLSS offered a significant benefit at the same cost over XeSS or FSR 2.0, Nvidia would NEVER offer Streamline.

They offer streamline, because DLSS only support Nvidia GPUs while FSR is supporting all GPUs and therefore the adoption rate of FSR among game developers is going to be higher. It has not much to do with the question of which solution is better. At the moment FSR has a long way to go in order to be roughly on par with DLSS.
Would be a pity if Nvidia owners in the future would have to fall-back to FSR just because DLSS is not supported because the developer was playing cheap here.

 

[Aapje]

If you thought the consumer benefit was so obvious, why belittle it with "Consumers are not everyone"?

Because you replied to my comment about Nvidia's motives by giving the benefits to consumers, but Nvidia is motivated by the benefits to themselves. This overlaps to some extent with consumer interests, but not fully. For example, it clearly would have been in consumer interest for Nvidia to open source DLSS, yet they didn't. So if you assume that Nvidia is always doing what is in the customer interest, you will expect them to do things that they don't actually do.

Similarly, it is questionable whether AMD is actually going to adopt Streamline in consoles, as to encourage developers to make FSR 2-only ports, rather than also support DLSS. This is also not in the interest of gamers (assuming Streamline is actually vendor-neutral), but can be in the interest of AMD.

It is a very common mistake to ascribe your own motives/needs to others, who don't actually have those same motives/needs. The result of this is misjudgment, as that other party won't do what you predict them to do.

 

[GodisanAtheist]

@guidryp Outside of discussion NV's motivations for releasing the tool, what really is there to discuss about it? It's a forum of primarily armchair hobbyists, its not a developer forum, and Streamline isn't even that sexy of a tool. There just really isn't a lot to say about it except for "Alright, cool, I guess".

Hell with a bunch of people stuck on ancient hardware a lot of the actual upscaling techniques themselves are not available for most. I have an NV card and I haven't even seen Freesync/Gsync in action yet, let alone DLSS, FSR/RSR, etc.

So we're just discussing what is interesting to us about the tech: NV's motivations. Frankly I wish more people took such a skeptical attitude when their own pet corporation does something. Look at all the AMD fanboys screeching about "supporting AMD through the hard times" when AMD doesn't deliver whatever precious niche demand they have or prices their parts according to the performance they provide against their competitors. Those folks could do with some of the same skepticism.

 

[DrMrLordX]

FSR 2 is a big threat to DLSS as the former will support all GPUs, so it's very tempting for developers to implement it, even if it is a little worse than DLSS.

By having a standard interface, Nvidia gets the benefit of their cards having the better SS implementation, assuming that this is actually true.

From a developer's standpoint, why would you ever bother with DLSS or XeSS if FSR 2 hits your entire target audience?

Unless someone puts a money hat on your head?

nVidia dropping DLSS and switching to a free, open source, FSR2 would benefit all customers.

True but FSR 2 is currently an (admittedly open) AMD standard, so that sort of puts Intel and NV at their mercy. Unless AMD cedes development to a third party or something.

 

[Heartbreaker]

@guidryp Outside of discussion NV's motivations for releasing the tool, what really is there to discuss about it? It's a forum of primarily armchair hobbyists, its not a developer forum, and Streamline isn't even that sexy of a tool. There just really isn't a lot to say about it except for "Alright, cool, I guess".

Hell with a bunch of people stuck on ancient hardware a lot of the actual upscaling techniques themselves are not available for most. I have an NV card and I haven't even seen Freesync/Gsync in action yet, let alone DLSS, FSR/RSR, etc.

So we're just discussing what is interesting to us about the tech: NV's motivations. Frankly I wish more people took such a skeptical attitude when their own pet corporation does something. Look at all the AMD fanboys screeching about "supporting AMD through the hard times" when AMD doesn't deliver whatever precious niche demand they have or prices their parts according to the performance they provide against their competitors. Those folks could do with some of the same skepticism.

It's not even a tool. It's just a common interface.

For me it's more interesting what it enables, which is more universal support for advanced scaling, than why NVidia is doing it.

It was kind of inevitable that some party make the obvious move, and put a common Interface in front of the three advanced scaling systems, it's just what you would do in this situation.

We can speculate about NVidia motivation beyond the obvious common sense move, as most probably de-risking the potential to lose support if they stayed separate APIs as others have speculated. But beyond that whats to talk about on that front either?

My next speculation is what happens next. Does Microsoft/Khronos absorb it or something similar? Or does this get used as is.

 

[GodisanAtheist]

It's not even a tool. It's just a common interface.

For me it's more interesting what it enables, which is more universal support for advanced scaling, than why NVidia is doing it.

It was kind of inevitable that some party make the obvious move, and put a common Interface in front of the three advanced scaling systems, it's just what you would do in this situation.

We can speculate about NVidia motivation beyond the obvious common sense move, as most probably de-risking the potential to lose support if they stayed separate APIs as others have speculated. But beyond that whats to talk about on that front either?

My next speculation is what happens next. Does Microsoft/Khronos absorb it or something similar? Or does this get used as is.

- Frankly I think if MS/Khronos do something, it will be to come out with their own vendor agnostic upscaler and just package it into their next DX/Vulkan release and standardize it.

That seems to be the trend: GPU manufactuers push a bunch of fragmented software/driver level stuff and MS/Vulkan roll it all back into the next major DX/Vulkan release.

 

[Heartbreaker]

From a developer's standpoint, why would you ever bother with DLSS or XeSS if FSR 2 hits your entire target audience?

Unless someone puts a money hat on your head?

With a common interface, you hit the entire target audience with the same amount of effort, and you actually get to target the GPU makers preferred solution to all your customers on their specific brand of GPU.

You can claim XeSS, DLSS and FSR 2 support keeping all the GPU owners happy with their preferred solutions.

Even if NVida didn't create the common interface, someone would, it's so obvious, as to be inevitable.

 

[DrMrLordX]

With a common interface

But they've already got one. They can just ignore XeSS and DLSS and call it a day.

 

[Stuka87]

With a common interface, you hit the entire target audience with the same amount of effort, and you actually get to target the GPU makers preferred solution to all your customers on their specific brand of GPU.

You can claim XeSS, DLSS and FSR 2 support keeping all the GPU owners happy with their preferred solutions.

Even if NVida didn't create the common interface, someone would, it's so obvious, as to be inevitable.

Thats not how it works. Game Devs DO NOT WANT three different upscalers. Just like display manufacturers didn't want two adaptive sync systems.

You are making it out as though devs just need to push a button and bam they support three different upscaler methods. But the fact is, the devs still have to do work to implement them. Which is not something devs want to do. They do not want to spend precious resources implementing what is basically the same thing, three times. Especially when two of those will run on any GPU.