Question DLSS 2.0

[Dribble]

There is no thread on this, and it's worth one because it seems to finally be realising it's potential. There's many articles discussing it elsewhere - it's used by wolfenstein young blood, control and a few other games and it works really well. Allegedly it's easy to add - no per game training required. Gives a good performance increase and looks really sharp.

Nvidia article.
wccf article.
eurogamer article

The above articles have some good comparison screen shots that really demonstrate what it can do.

Discuss...

 

[Heartbreaker]

DLSS 2.0 is using Convolution and Inferencing to upscale images and do AA by using a Machine Learning trained model (Nvidia calls it Convolutional Autoencoder ) , that is exactly what they displayed in the DirectML video above.

They clearly say in the video that they are using a NVIDIA Model called "Autoencoder" (time of video 19.43) for their Super Resolution Neural Network. So either you havent seen the video or you talking about something else because what they showcased was exactly what DLSS 2.0 does.

Convolution is a standard mathematical operation. That's basically like saying: "They both use math, so they are the same".

I watched the video, and I also watched the video I posted that explains in greater detail how DLSS 2.0 works. Go watch that one.

It's clear that this old network NVidia provided to Microsoft, is just using standard scaling techniques and combining them with some AI based correction. But it is just working on the lower res output frames, plus training to generate higher resolution

That was close to how DLSS 1.0 worked.

But DLSS 2.0 does NOT work like that. DLSS 2.0 requires direct hooks into the game engine to build a rich data set. It works more like advanced AI driven checkerboard rendering. You have to do very specific setup. You adjust the Mip Bias to feed the higher resolution textures, more typical of the output resolution, than the input resolution. Then you set up a Halton pixel jitter pattern, to do mult-sampling of pixels, the bigger the upscale, the more jitter phases.

This is a very advanced multi sample spatial-temporal sampling, along with feeding in higher than normal quality textures. Now all of this data has what you need to reconstruct a higher resolution image, but writing an algorithm to actually reconstruct the image from this complex input data, would bury a human in endless corner cases. The AI is then trained to do the reconstruction from the this rich data.

DLSS 1.0 and that conference video, are simply doing scaling with AI corrections. Which has demonstrably mixed results.

DLSS 2.0 is doing advanced AI reconstruction from a rich data set.

They are nothing alike.

DLSS 1.0 is AI aided Scaling. DLSS 2.0 is AI aided advanced Image reconstruction.

A little logic as well, would tell you it's a lot more likely that in 2018 they would share a trained network closer DLSS 1.0 than DLSS 2.0.

 

[Heartbreaker]

At the 1:00 mark the whole scene appears brighter with DLSS2.0 and halo effects in many places, both probably due to sharpening but then at 2:10 mark you see texture details in her hair tie that aren't present in the native image but I don't know how much of that is just the sharpening feature as well.

Anyway, not gonna go through second by second but overall it's impressive tech. Hopefully Nvidia keeps improving on it to clean up the issues here and there.

Cyberpunk DLSS comparison is interesting because here it looks like they didn't apply much sharpening to the DLSS 2.0 output and instead, they sharpen the native presentation. If they weren't labeled I would have guessed that Native and DLSS quality mode were reversed given what we have seen in Control. Here it's in Native that I see sharpening artifacts.

Unfortunately he seems to spend most of his time on on "Ultra Performance DLSS" which is garbage, only really intented to get them to 8K. Maybe they should have locked it to 8K, to prevent people from focussing on it so much. Really only interested in Quality mode. It's the only one that really has a chance of competing with Native:

 

[cytg111]

This sounds like marketing BS to me.

Some law about thermo dynamics, no free lunch etc .. that sort of deal?

 

[Leeea]

Cyberpunk DLSS comparison is interesting because here it looks like they didn't apply much sharpening to the DLSS 2.0 output and instead, they sharpen the native presentation. If they weren't labeled I would have guessed that Native and DLSS quality mode were reversed given what we have seen in Control. Here it's in Native that I see sharpening artifacts.

Unfortunately he seems to spend most of his time on on "Ultra Performance DLSS" which is garbage, only really intented to get them to 8K. Maybe they should have locked it to 8K, to prevent people from focussing on it so much. Really only interested in Quality mode. It's the only one that really has a chance of competing with Native:

DLSS 2.0 was very impressive in cyber punk.

Part of it was cyberpunks anti-aliasing being a bit odd. But it added real value in cyberpunk.

If I had unobtanium capable of using it in cyberpunk I think I would.

 

[CakeMonster]

I haven't been able to play CyberPunk after the last update, is it true that sharpening is now forced for non-DLSS?

I've long been skeptical of DLSS because of what I've seen with my own eyes, but sharpening is way less tolerable to me and I really hope it doesn't get sneaked into games from now on without any easy way to disable it.

DLSS Quality in CP2077 looked like quite decent overall I must admit, but trees and vegetation look like "fake" props because they are disproportionally blurred compared to the rest, and the mirror image of the main character is also horribly blurred. I can live with that, but its still annoying.

 

[GoodRevrnd]

From the GN video I have mixed feelings on vegetation. In some scenes native looks over sharpened and DLSS looks good, in others native looks good and DLSS looks over blurred.

Texture quality is also a little odd sometimes. I generally feel in all cases the DLSS smears out too much detail, but many native textures do look over sharpened.

 

[Qwertilot]

Except in cases where its blatant, you're only really going to get sane judgements on this if people do properly double blinded trials. Which no one will.

Otherwise there's all sorts of conformation bias etc that make a rational judgement incredibly hard.

 

[Mopetar]

Except in cases where its blatant, you're only really going to get sane judgements on this if people do properly double blinded trials. Which no one will.

Otherwise there's all sorts of conformation bias etc that make a rational judgement incredibly hard.

It's not even just a matter of having it on/off and being able to tell the difference. There are times where the changes it make produce a better or worse aesthetic that can make the result preferable in terms of overall appearance.

Even still images are only half the picture since we don't sit around in games staring at a still image. You need to see it in motion as well since there are cases where you wouldn't notice the differences once it's being animated and other cases where DLSS can produce some weird artifacts that don't exist when it's off. There's also a question as to frame rate as even if the image quality is worse, being able to stay above 60 FPS may produce a better experience than something that dips well below that on occasion.

Add in any effects that are exacerbated due to certain types of displays and it's going to be impossible to test objectively without an extensive amount of work.

 

[Heartbreaker]

It's not even just a matter of having it on/off and being able to tell the difference. There are times where the changes it make produce a better or worse aesthetic that can make the result preferable in terms of overall appearance.

Even still images are only half the picture since we don't sit around in games staring at a still image. You need to see it in motion as well since there are cases where you wouldn't notice the differences once it's being animated and other cases where DLSS can produce some weird artifacts that don't exist when it's off.

You do need to see it in motion to see the real impact, where, on balance, DLSS Quality mode, is superior at fixing the worse Aliasing Motion artifacts. There was a similar result in Death Stranding. Watch this video clip at full screen/resolution (At 7:30 timestamp) will clearly demonstrate the difference. The native side is full of flashing/poping/crawling Aliasing artifacts, where the DLSS side is stable and clean:

That isn't to say that DLSS is perfect. It has trouble with particle effects, since they lack motion vectors, the leave motion trails, this can be seen most readily in death stranding.

I would rather have the particle effect anomalies, than aliasing motion artifacts.

Next is Detail, and again compared to native, there is a trading of blows. Since both TAA and DLSS can soften the image. This one is highly susceptible to tuning.

In "Control", DLSS 2.0 is a tad over sharpened for my taste, but for those less susceptible to sharpening artifices, it can look better than Native TAA softness.

In "Cyberpunk 2077" the situation is reversed. TAA is over-sharpened and has the sharpening artifacts, while DLSS is not. Leading some to claim DLSS 2.0 has less detail in this case, but in many scenes DLSS 2.0 is actually resolving more detail, but isn't sharpened like TAA.

IMO, on visuals alone, I consider DLSS 2.0 Quality mode the better overall presentation, before we even get to the bonus improvement in frame rates.

The one thing it could use right away though, is an exposed sharpness setting, to allow this to be set to personal taste. Sharpness is NOT one size fits all.

 

[Dribble]

The one thing it could use right away though, is an exposed sharpness setting, to allow this to be set to personal taste. Sharpness is NOT one size fits all.

Apparently you can set the texture lod bias to sharpen it if you wish: https://wccftech.com/heres-how-to-improve-cyberpunk-2077-texture-sharpness-when-using-nvidia-dlss/

 

[GoodRevrnd]

In "Cyberpunk 2077" the situation is reversed. TAA is over-sharpened and has the sharpening artifacts, while DLSS is not. Leading some to claim DLSS 2.0 has less detail in this case, but in many scenes DLSS 2.0 is actually resolving more detail, but isn't sharpened like TAA.

Can you turn TAA off in CP or is it forced on? Crossed my mind in a game like this they could be futzing w/ the TAA to make DLSS look better (comparatively).

 

[amenx]

Was never impressed with DLSS 2.0 until I tried it with sharpening @ 4k in cyberpunk. At 1080p its not compelling at all. At 4k its a god send for those trying to get higher fps with it.



Sorry dont have the before shots as it wasnt an intended comparison test, but took these because I was surprised at the result. Can do before/after shots at a later time if asked.
Also this was on a RTX 2080. Again, this at 4k. No ultra presets, just carefully selected settings where it mattered. FPS was in the 30s, but with DLSS it shot up to the 50s.

p.s. look at the detail on trunk of the palm tree on far left when expanded in 1st pic.

 

[Heartbreaker]

Can you turn TAA off in CP or is it forced on? Crossed my mind in a game like this they could be futzing w/ the TAA to make DLSS look better (comparatively).

More like they are doing the opposite.

Since they are sharpening TAA, and leaving DLSS unsharpened, this will look like "better" detail for TAA than DLSS, to most people.

 

[Gideon]

More like they are doing the opposite.

Since they are sharpening TAA, and leaving DLSS unsharpened, this will look like "better" detail for TAA than DLSS, to most people.

They only added it in v1.04 I'm not sure if this is still there on 1.05v hopefully they make it togglable

 

[Dribble]

Looks like someone is finally doing a quality version of DLSS with no scaling (called DLAA):

NVIDIA DLAA (Deep Learning Anti-Aliasing) to Debut in Elder Scrolls Online, Delivering Incredible AA Quality

NVIDIA DLAA (Deep Learning Anti-Aliasing), a quality-focused version of DLSS, will debut soon in Zenimax's MMORPG Elder Scrolls Online.

wccftech.com

 

[GodisanAtheist]

Looks like someone is finally doing a quality version of DLSS with no scaling (called DLAA):

NVIDIA DLAA (Deep Learning Anti-Aliasing) to Debut in Elder Scrolls Online, Delivering Incredible AA Quality

NVIDIA DLAA (Deep Learning Anti-Aliasing), a quality-focused version of DLSS, will debut soon in Zenimax's MMORPG Elder Scrolls Online.

wccftech.com

- Nice.

I am always pro-image quality, its incredible what even basic downscaling can do for graphical fidelity (like SSAA). I think it was @coercitiv that did a little thing with FSR where the game was over-rendered, then downscaled to native resolution by the FSR algo and the image quality was incredible.

Look forward to seeing DLSS pointed at image quality, not just reconstruction.

 

[coercitiv]

I am always pro-image quality, its incredible what even basic downscaling can do for graphical fidelity (like SSAA). I think it was @coercitiv that did a little thing with FSR where the game was over-rendered, then downscaled to native resolution by the FSR algo and the image quality was incredible.

I was quite critical of DLSS when it came out (so DLSS 1.0, not the actual tech), but even back then I made it very clear that I would have loved to see that Deep Learning technique being used as AA on native resolutions. That is why when FSR came out there were many people on Twitter exchanging results on what was essentially a different use case for FSR: post processing on native renders, with excellent results in terms of edge alias removal and detail enhancement.

I'm very curious to see how DLAA performs, the marketing finally allows it to break free of it's original packaging as the Ray Tracing Sidekick.