Discussion AMD Gaming Super Resolution GSR

[DisEnchantment]

New Patent came up today for AMD's FSR

Patent Public Search | USPTO

appft.uspto.gov

https://www.freepatentsonline.com/y2021/0150669.html

20210150669 GAMING SUPER RESOLUTION

Abstract
A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display

[0008] Conventional super-resolution techniques include a variety of conventional neural network architectures which perform super-resolution by upscaling images using linear functions. These linear functions do not, however, utilize the advantages of other types of information (e.g., non-linear information), which typically results in blurry and/or corrupted images. In addition, conventional neural network architectures are generalizable and trained to operate without significant knowledge of an immediate problem. Other conventional super-resolution techniques use deep learning approaches. The deep learning techniques do not, however, incorporate important aspects of the original image, resulting in lost color and lost detail information.

[0009] The present application provides devices and methods for efficiently super-resolving an image, which preserves the original information of the image while upscaling the image and improving fidelity. The devices and methods utilize linear and non-linear up-sampling in a wholly learned environment.

[0010] The devices and methods include a gaming super resolution (GSR) network architecture which efficiently super resolves images in a convolutional and generalizable manner. The GSR architecture employs image condensation and a combination of linear and nonlinear operations to accelerate the process to gaming viable levels. GSR renders images at a low quality scale to create high quality image approximations and achieve high framerates. High quality reference images are approximated by applying a specific configuration of convolutional layers and activation functions to a low quality reference image. The GSR network approximates more generalized problems more accurately and efficiently than conventional super resolution techniques by training the weights of the convolutional layers with a corpus of images.

[0011] A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display.

[0012] A processing device is provided which includes memory and a processor configured to receive an input image having a first resolution. The processor is also configured to generate a plurality of non-linear down-sampled versions of the input image via a non-linear upscaling network and generate one or more linear down-sampled versions of the input image via a linear upscaling network. The processor is also configured to combine the non-linear down-sampled versions and the one or more linear down-sampled versions to provide a plurality of combined down-sampled versions. The processor is also configured to convert the combined down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution by assigning, to each of a plurality of pixel blocks of the output image, a co-located pixel in each of the combined down-sampled versions and provide the output image for display.

[0013] A super resolution processing method is provided which improves processing performance. The method includes receiving an input image having a first resolution, generating linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generating non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The method also includes converting the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and providing the output image for display.

It uses Inferencing for upscaling. As will all ML models, how you assemble the layers, what kind of parameters you choose, which activation functions you choose etc, matters a lot, and the difference could be night and day in accuracy, performance and memory

 

[GodisanAtheist]

I figured something similar to UE5's TAAU was coming from AMD in FSR 2.0.

If AMD gets this thing to be game/engine/vendor agnostic (I mean it is, but is a standard) or even gets incorporated at the driver level (i've read that AMD could theoretically use vector data from game AF data as a substitute for getting exact vectors right from the engine) then it will be GG all other upscaling techniques.

To use a prior example, it will be VHS vs. Betamax all over again. Good enough but standardized and free will beat the technically superior but limited and proprietary solution every time.

 

[uzzi38]

The code will be open source next week too, which is surprisingly soon. Or at least, that's how I read this, although I'm not 100% sure.

Colin Riley 🎗 on Twitter: "Can finally announce the real content of my #GDC2022 session, where Thomas and I will delve into the algorithms, optimizations and integration aspects of #FSR2. It will also be available to all on @GPUOpen a day later. Looking forward to it!" / Twitter

 

[moinmoin]

The code will be open source next week too, which is surprisingly soon. Or at least, that's how I read this, although I'm not 100% sure.

Colin Riley 🎗 on Twitter: "Can finally announce the real content of my #GDC2022 session, where Thomas and I will delve into the algorithms, optimizations and integration aspects of #FSR2. It will also be available to all on @GPUOpen a day later. Looking forward to it!" / Twitter

No, the videos will be on GPUOpen, not the source.

 

[RnR_au]

 

[VirtualLarry]

AMD's "Gaming Super Resolution" on Elden Ring @ 4K.

Can someone riddle me this? This scales the games' output resolution to the (presumably higher) native display output resolution.

How is this any different to chosing from display device or GPU scaling? Isn't this just the same as choosing "GPU scaling"?

 

[Shamrock]

RSR is an absolute winner! The new drivers are also WHQL.

Mechwarrior online is an NV sponsored title (Even has Dawn, the fairy). I have a Sapphire rx 6700xt, paired with an AMD 5600x cpu.

Everything on high. 1440p, I could only manage 55fps.
Everything on high (except particles), 1080p, 128fps. (Thanks @Leeea for the tip about particles!)
Everything on high (except particles) 1080p upscaled to 1440p with RSR. 189fps!!!! and it looks better.

 

[RnR_au]

RSR is an absolute winner! The new drivers are also WHQL.

Mechwarrior online is an NV sponsored title (Even has Dawn, the fairy). I have a Sapphire rx 6700xt, paired with an AMD 5600x cpu.

Everything on high. 1440p, I could only manage 55fps.
Everything on high (except particles), 1080p, 128fps. (Thanks @Leeea for the tip about particles!)
Everything on high (except particles) 1080p upscaled to 1440p with RSR. 189fps!!!! and it looks better.

How does the UI look scaled up like that?

 

[Shamrock]

How does the UI look scaled up like that?

No different than 1080p. I know under normal use, the 1440p UI was tiny, but with it upscaled, its normal.

 

[AtenRa]

Techpowerup review,

AMD Radeon Super Resolution RSR Quality & Performance Review

Today, AMD released their Radeon Super Resolution Technology. RSR is the magic bullet to achieve higher framerates in ALL games; it works on the driver-level, so no special game is support needed, it just works, even on older titles. We've tested it in several games, and the image quality is...

www.techpowerup.com

 

[uzzi38]

No, the videos will be on GPUOpen, not the source.

That would make more sense lol

 

[RnR_au]

Some static slider images to compare DLSS, FSR2 and FSR1;

https://imgsli.com/MTAwNjA2
https://imgsli.com/MTAwNTk5/2/3

A good showing for FSR2. Just need to see motion now. To see if there are any cons to the pros.

 

[Hitman928]

Some static slider images to compare DLSS, FSR2 and FSR1;

https://imgsli.com/MTAwNjA2
https://imgsli.com/MTAwNTk5/2/3

A good showing for FSR2. Just need to see motion now. To see if there are any cons to the pros.

Quick comparison just from these, it looks to me like FSR2 is a big step up from FSR1. Compared to modern DLSS, it seems FSR2 is slightly less detailed, but also doesn't have some of the artifacts in the DLSS examples and has slightly less aliasing. This is really just one example though and, like you said, we need high quality video comparisons to really judge.

 

[biostud]

Any comparisons on 1440p displays?

The downside of the G9 is that it requires nearly as much GPU power as a 4K display, but upscaling on 1440p displays are seldom as good as on a 4k monitor.

 

[uzzi38]

Quick comparison just from these, it looks to me like FSR2 is a big step up from FSR1. Compared to modern DLSS, it seems FSR2 is slightly less detailed, but also doesn't have some of the artifacts in the DLSS examples and has slightly less aliasing. This is really just one example though and, like you said, we need high quality video comparisons to really judge.

EDIT: I was looking at this image to be specific, I saw it elsewhere and assumed it was being passed around here. It's from the same source: RgIIhFr.jpg (3840×2160) (imgur.com)

The DLSS 2.3 image looks completely rife with artifacts if you ask me personally. It's completely oversharpened for starters which hurts the stone wall on the left and fills the bottom half of the red roof in the back with oversharpening artifacts (that's the sort of grain on the flat red you can see). The stone wall on the left has some looks extremely aliased on all of the outside edges (take a look at the very top for example). One of the shadows on this wall are more aliased on FSR 2.0, one more aliased on DLSS 2.3, but I think FSR 2.0 did a worse job on these shadows because the one that is more aliased here also lost a bit of detail too. (I have a feeling that detail levels within shadows is going to be a recurring theme of weakness for FSR 2.0, because this is the second time I've noticed an oddity with shadows, although the first is not in this image).

The right hand side of that orange face in the background is an interesting one. DLSS 2.3 looks closer to native on this half of the face... but personally I think that actually looks worse? It just seems too aliased to me again. Same goes for the green face on the right. I think both FSR 2.0 and DLSS mess up the left hand side though. FSR 2.0 is too soft, DLSS 2.3 is oversharpened again.

I am going to have to point this out very clearly though: most of the issues I can see with DLSS are definitely over-sharpening ones, which is an issue of the game developers here - they've set a sharpening value too high. The only real issue with DLSS I can see is what it's done with the lines on the upper half of the red roof. It's added in some horizontal lines which would have looked great if there was only one of them. There's not, and the way it is in this image it actually stands out as an artifact IMO. That's the one issue I can see.

 

[coercitiv]

I am going to have to point this out very clearly though: most of the issues I can see with DLSS are definitely over-sharpening ones, which is an issue of the game developers here - they've set a sharpening value too high.

That was my exact reaction when I saw your (deleted) tweet, it was rather pointless to look for a "winner" considering the overdose of sharpening applied in one of the samples. Sharpening is a very misleading technique, it always has side-effects and tricks even the trained eye into an overdose.

That being said, sharpening was a problem for DLSS before, and I expect we'll see FSR 2.0 implementations suffering from the same "overdose" bias:

And when it doesn't, it feels like an old smartphone camera software took over: sharpening and edge contrast enhancement all over.

View attachment 30805

I really like the improvements in DLSS 2.0 and I think this type of tehnique may very well be the future, but as long as there's little will to openly discuss both pros and cons, I doubt threads like this will go anywhere. The flaws of DLSS 1.0 where an order of magnitude more obvious, yet it was just as fiercely defended.

 

[DiogoDX]

Founded the source. Seems like the guy printed the imges that AMD sent to HardwareUnboxed and compared with his game running DLSS2.3.

 

[Stuka87]

Founded the source. Seems like the guy printed the imges that AMD sent to HardwareUnboxed and compared with his game running DLSS2.3.

Printed them? As in, like, on paper??

 

[DiogoDX]

Printed them? As in, like, on paper??

I mean "print screen" or screenshot.

Brazilian here and "print" is an expression that we use for screenshot.

 

[ryanjagtap]

This video has FSR 2.0 preview but no framerate counter is shown.

 

[Stuka87]

I mean "print screen" or screenshot.

Brazilian here and "print" is an expression that we use for screenshot.

Ahh, ok. That makes much more sense

 

[biostud]

RSR is an absolute winner! The new drivers are also WHQL.

Mechwarrior online is an NV sponsored title (Even has Dawn, the fairy). I have a Sapphire rx 6700xt, paired with an AMD 5600x cpu.

Everything on high. 1440p, I could only manage 55fps.
Everything on high (except particles), 1080p, 128fps. (Thanks @Leeea for the tip about particles!)
Everything on high (except particles) 1080p upscaled to 1440p with RSR. 189fps!!!! and it looks better.

I guess you play on a 1440p monitor.

How does upscaling look to you when you upscale from 1080p, compared to native 1440p?

 

[biostud]

AMD Radeon Super Resolution is FSR-style upscaling for almost any game | Rock Paper Shotgun

Here are some samples where they also do 1080p to 1440p, definitely a technology I'll be interested in using.

 

[Shamrock]

I guess you play on a 1440p monitor.

How does upscaling look to you when you upscale from 1080p, compared to native 1440p?

Yes, I have an LG 32GK650-F 1440p

To me, 1080p upscaled to 1440p looks better than native 1080p. Obviously, native 1440p would still be better. But its still pretty hard to tell the difference. Especially on the heat of battle.

 

[biostud]

Yes, I have an LG 32GK650-F 1440p

To me, 1080p upscaled to 1440p looks better than native 1080p. Obviously, native 1440p would still be better. But its still pretty hard to tell the difference. Especially on the heat of battle.

1080p doesn't look good at all to me, but with my GTX 1070 I haven't used any of the new upscaling modes, so I'm really interested in anything that can increase my fps without turning it into a blurry image, as the G9 is pretty hard to drive.

 

[Shivansps]

I used a similar feature a lot in the year i expend playing with a 2200G as my main pc... where you would turn on "hardware gpu scaling" and wharever fullscreen game no matter the resolution, it would get upscaled to native monitor resolution.

I was a very usefull feature, the only problem is that i was forced to use fullscreen in games. This seems like an update of that tech.

It supports APUs? because this is very very usefull for apu gaming.