News AMD FSR 4 Redstone path tracing software uses ML2CODE framework to run on any GPU

[basix]

NRC is not part of Ray Regeneration, same applies for Nvidias Ray Reconstruction. NRC is a separate algorithm / tool and is complementary to RR.

NRC would get applied one step before doing RR. You would feed a less noisy image into the RR algorithm, which makes life easier for RR (resulting in enhanced image quality).
Or with NRC you could reduce the number of rays / RT samples per pixel with equal quality results, but improved performance.

 

[MrMPFR]

NRC is not part of Ray Regeneration, same applies for Nvidias Ray Reconstruction. NRC is a separate algorithm / tool and is complementary to RR.

NRC would get applied one step before doing RR. You would feed a less noisy image into the RR algorithm, which makes life easier for RR (resulting in enhanced image quality).
Or with NRC you could reduce the number of rays / RT samples per pixel with equal quality results, but improved performance.

TL;DR: Using ML to improve input for denoiser, while using ML to boost fidelity of output (frame) vs a standard analytical (hand-tuned) denoiser. At iso-quality it's a two step quadratic perf gain or at iso-ms a boosted fidelity^2 gain.

Just realized bad the current way of doing things (ReSTIR) is compared to all the publicly released advances in research pipeline: Neural importance sampling, Histogram Stratification for Spatio-Temporal Reservoir Sampling, Neural visibility cache (direct lighting), Neural radiance cache (indirect lighting), GATE (neural encoding > hash based solutions), Markov Chain Path Guiding, Neural appearance models, and likely many more I've missed.

While FSR Redstone reaching parity with NVIDIA is good, AMD better prepare for whatever comes next. RTX 60 series won't just launch production ready versions of Neural Materials, RTX Texture streaming, and Neural Texture Compression.
AMD has to stop playing catchup and begin to anticipate NVIDIA's next moves and make their own or nothing will ever change.

Edit: Read next reply before commenting. Was referring to Restir vs Restir + all the other stuff on top.

 

[soresu]

Just realized bad the current way of doing things (ReSTIR) is compared to all the publicly released advances in research pipeline

ReSTIR is not mutually exclusive to all other techniques, including path guiding, and there are different ways of doing it (Area ReSTIR, Conditional ReSTIR etc).

Not to mention that it isn't necessarily used across the entire rendering pipeline at the moment.

Subsurface scattering using ReSTIR only became available in 2024.

 

[adroc_thurston]

While FSR Redstone reaching parity with NVIDIA is good, AMD better prepare for whatever comes next. RTX 60 series won't just launch production ready versions of Neural Materials, RTX Texture streaming, and Neural Texture Compression.
AMD has to stop playing catchup and begin to anticipate NVIDIA's next moves and make their own or nothing will ever change.

Tons of buzzwords and none of that matters.
Whoever has the best gfx IP wins.

 

[MrMPFR]

ReSTIR is not mutually exclusive to all other techniques, including path guiding, and there are different ways of doing it (Area ReSTIR, Conditional ReSTIR etc).

Not to mention that it isn't necessarily used across the entire rendering pipeline at the moment.

Subsurface scattering using ReSTIR only became available in 2024.

Mentioned it because it's SOTA for RTRT in games (PT), even if adoption is limited to NVIDIA sponsored games. Indeed most of the other stuff is meant to enhance ReSTIR rather than replace it, and like you said it keeps getting updated.
Was comparing baseline ReSTIR + whatever new techniques baked into it versus that AND a full implementation on top tackling all parts of the PT pipeline with analytical and neural methods. As usual sorry for my poor wording.

Tons of buzzwords and none of that matters.
Whoever has the best gfx IP wins.

For short term no but it hurts their brand reputation, if there's anything left. Gives the impression of AMD always following the leader and being off brand GeForce.
But let's be real like you said the most important thing is GFX ip. That's unless NVIDIA has frame extrapolation, without artifacts and input of DLSS, paired with Reflex 2, while at the same time pushing for rapid mass adoption. Seems very unlikely, but it would be the next logical step to stay ahead of AMD.

 

[adroc_thurston]

For short term no but it hurts their brand reputation

You can't hurt what doesn't exist.

Gives the impression of AMD always following the leader and being off brand GeForce.

Because AMD hasn't shipped a real halo config since Fiji.
It's a market where either you win and take all, or you're the eternal loser.

 

[MrMPFR]

You can't hurt what doesn't exist.

Because AMD hasn't shipped a real halo config since Fiji.
It's a market where either you win and take all, or you're the eternal loser.

780 TI beat Fiji (290X) as with 3090 TI > 6950XT. But being first prob what counts so yeah AMD hasn't beaten NVIDIA since 2013.

Even if they managed to beat NVIDIA's halo tier nextgen (seems unlikely) I doubt anyone would take it seriously if it's a very incomplete package vs NVIDIA, but could be wrong. But it prob won't even come to that nextgen.

 

[adroc_thurston]

Even if they managed to beat NVIDIA's halo tier nextgen (seems unlikely) I doubt anyone would take it seriously if it's a very incomplete package vs NVIDIA

none of the bling ever matters.
All you gotta do is win.

 

[SolidQ]

780 TI beat Fiji (290X)

at start yes, but later drastically lose to RX290

 

[DaaQ]

Also Fiji was HBM packaging, Fury IIRC.
R290x was Hawaii. Which was faster than original 780 before Ti, although reviews blew it apart for blower noise.
EDIT: AND started mining craze.

 

[basix]

Just realized bad the current way of doing things (ReSTIR) is compared to all the publicly released advances in research pipeline: Neural importance sampling, Histogram Stratification for Spatio-Temporal Reservoir Sampling, Neural visibility cache (direct lighting), Neural radiance cache (indirect lighting), GATE (neural encoding > hash based solutions), Markov Chain Path Guiding, Neural appearance models, and likely many more I've missed.

That is true, indeed. ML is very good at augmenting algorithms and techniques. DLSS and FSR4 SR are very good examples of that (parameter prediction of an SR algorithm). Algorithm augmentation is much easier to control than having a 100% DNN blackbox.

That algorithm augmentation is missing from many parts of the rendering pipeline, like e.g. standard ReSTIR. But to address that issue, cooperative vectors have been "invented". We will see many more ML augmented algorithms in the future. Not only from GPU vendors like AMD and Nvidia, but also game developers themselves.

While FSR Redstone reaching parity with NVIDIA is good, AMD better prepare for whatever comes next. RTX 60 series won't just launch production ready versions of Neural Materials, RTX Texture streaming, and Neural Texture Compression.
AMD has to stop playing catchup and begin to anticipate NVIDIA's next moves and make their own or nothing will ever change.

I think the most important part will be having good matrix core acceleration together with good support for cooperative vectors. They will not leapfrog Nvidia with that, but at least not fall back as far as with RT when Nvidia introduces more neural rendering stuff.
If AMD really wants to kick Nvidias a$$:

• FSR Redstone SR brings better quality than DLSS 4 SR and runs faster
  • And it runs on all GPUs since Turing and RDNA1
• FSR Redstone RR gets on par with DLSS 4 RR
  • If PS5 Pro gets RR support as well (I expect that to happen), RR adoption will accelerate (RR can also be used without pathtracing) and AMD can get the merits for that
• FSR Redstone FG brings 4x MFG with better quality and higher performance than Nvidias solution
  • Ultimate FG leapfrog: Extrapolation with time-warp -> FG frames feel like real rendered frames. But that seems to be unlikely, it is maybe a thing for DLSS 5 and FSR for RDNA5 / NextGen consoles
• NRC solves the pathtracing weakness of RDNA4 (compared to Lovelace and Blackwell)
  • Less rays due to NRC help RDNA4 and RR makes a nice image out of it

An additional factor will be work graphs. That allows for very cool algorithm concepts. Not sure if also useful for cooperative vectors or stuff like FSR. RDNA5 will likely bring very strong support for work graphs.

 

[marees]

That is true, indeed. ML is very good at augmenting algorithms and techniques. DLSS and FSR4 SR are very good examples of that (parameter prediction of an SR algorithm). Algorithm augmentation is much easier to control than having a 100% DNN blackbox.

That algorithm augmentation is missing from many parts of the rendering pipeline, like e.g. standard ReSTIR. But to address that issue, cooperative vectors have been "invented". We will see many more ML augmented algorithms in the future. Not only from GPU vendors like AMD and Nvidia, but also game developers themselves.


I think the most important part will be having good matrix core acceleration together with good support for cooperative vectors. They will not leapfrog Nvidia with that, but at least not fall back as far as with RT when Nvidia introduces more neural rendering stuff.
If AMD really wants to kick Nvidias a$$:

• FSR Redstone SR brings better quality than DLSS 4 SR and runs faster
  • And it runs on all GPUs since Turing and RDNA1
• FSR Redstone RR gets on par with DLSS 4 RR
  • If PS5 Pro gets RR support as well (I expect that to happen), RR adoption will accelerate (RR can also be used without pathtracing) and AMD can get the merits for that
• FSR Redstone FG brings 4x MFG with better quality and higher performance than Nvidias solution
  • Ultimate FG leapfrog: Extrapolation with time-warp -> FG frames feel like real rendered frames. But that seems to be unlikely, it is maybe a thing for DLSS 5 and FSR for RDNA5 / NextGen consoles
• NRC solves the pathtracing weakness of RDNA4 (compared to Lovelace and Blackwell)
  • Less rays due to NRC help RDNA4 and RR makes a nice image out of it

An additional factor will be work graphs. That allows for very cool algorithm concepts. Not sure if also useful for cooperative vectors or stuff like FSR. RDNA5 will likely bring very strong support for work graphs.

Is this co-operative vector a directX (Microsoft) specific API ?

 

[MrMPFR]

I think the most important part will be having good matrix core acceleration together with good support for cooperative vectors. They will not leapfrog Nvidia with that, but at least not fall back as far as with RT when Nvidia introduces more neural rendering stuff.
If AMD really wants to kick Nvidias a$$:

• FSR Redstone SR brings better quality than DLSS 4 SR and runs faster
  • And it runs on all GPUs since Turing and RDNA1
• FSR Redstone RR gets on par with DLSS 4 RR
  • If PS5 Pro gets RR support as well (I expect that to happen), RR adoption will accelerate (RR can also be used without pathtracing) and AMD can get the merits for that
• FSR Redstone FG brings 4x MFG with better quality and higher performance than Nvidias solution
  • Ultimate FG leapfrog: Extrapolation with time-warp -> FG frames feel like real rendered frames. But that seems to be unlikely, it is maybe a thing for DLSS 5 and FSR for RDNA5 / NextGen consoles
• NRC solves the pathtracing weakness of RDNA4 (compared to Lovelace and Blackwell)
  • Less rays due to NRC help RDNA4 and RR makes a nice image out of it

Great summarization of everything @basix.
As outlined earlier extrapolated off @Kepler_L2 7PF figure for AT0, likely 2X raw and 2X from FP8 -> FP4 vs RDNA 4. Thus effectively (usable FP4 format like NVFP4) up to 4X gen-on-gen gain. Should be plenty and yeah unlike with RT with this ML foundation they can actually catch up later.

For sure FSR suite leapfrogging DLSS4 > matching esoteric nextgen NVIDIA MLP SDKs reserved for NVIDIA graphics showcases for the next many years.

FSR Redstone doing all that near EoY 2025 seems extremely unlikely given AMD's modus operandim, i.e reactive instead of proactive. But this would indeed be required to really make an impact on gaming SW side.
There's some other issues with those targets as well I would hope to be proven wrong but doing all this before EoY 2025 just seems like a very tall order. For example look at the perf impact of the leaked INT8 FSR4 on RDNA 2. Getting image quality not only to match the full FP8 based FSR4, but even exceed the superior DLSS 4 SR and getting it to run faster on newer cards and workable on older cards is extremely unlikely. RR matching DLSS4 RR prob not happening either, given how bad the RR looked when they unveiled the Toyshop demo near the end of February. As for PS5 Pro it'll prob get Lite INT8 based versions for SR and RR, but you're right this could drive widespread adoption of ML RR and should allow AMD to add RR in many games and help catch up with NVIDIA.
MFG might be an area where AMD can leapfrog, but yeah extrapolation with timewarp is very unlikely.

Perhaps a combination of NRC, the Intel histogram technique + the clever techniques used by Epic for MegaLights to reduce noise could be daoble. Realistically 0.5SPP could easily look as good as 2SPP with these changes. Still don't think NRC alone is enough. Sure NVIDIA can do ~15% speedup AND improved indirect lighting simultanously with ReSTIR+NRC and scaling it back to iso-quality should deliver much higher speedups, but the PT deficit CU vs SM is massive and prob too big to overcome with one SW side change.

An additional factor will be work graphs. That allows for very cool algorithm concepts. Not sure if also useful for cooperative vectors or stuff like FSR. RDNA5 will likely bring very strong support for work graphs.

Depends on how branchy the ML implementation is, but for the most part prob not. Looks like it's much more useful for PT, simulations, and procedural generation.

All based on patents, no concrete confirmations yet unfortunately, but would be a huge deal for the next gen consoles and RDNA 5 if actually implemented. Local launch + SE autonomous scheduling are clever HW optimizations for Workgraphs API and the later would help immensely with core scaling.

 

[MrMPFR]

Is this co-operative vector a directX (Microsoft) specific API ?

IIRC Unveiled by NVIDIA at CES 2025 to drive Neural shaders, later entered preview by MS at GDC 2025, recently pulled because it wasn't good enough. Fine for small MLPs, but everything else nope. New release coming at a later date

Shader Model 6.9 and The Future of Cooperative Vector - DirectX Developer Blog

Direct3D Cooperative Vector is evolving with new linear algebra capabilities in an upcoming Shader Model release.

devblogs.microsoft.com

 

[basix]

Is this co-operative vector a directX (Microsoft) specific API ?

"Cooperative vectors" as it was presented until now, yes. Is a part of DX12 Agility SDK. But there will be a Vulkan counterpart: https://docs.vulkan.org/features/latest/features/proposals/VK_NV_cooperative_vector.html

 

[marees]

"Cooperative vectors" as it was presented until now, yes. Is a part of DX12 Agility SDK. But there will be a Vulkan counterpart: https://docs.vulkan.org/features/latest/features/proposals/VK_NV_cooperative_vector.html

Ok. The Redstone version as of now has its own native implementation as there is a latency cost with DirectX api

 

[basix]

Nvidia has its own implementation as well. Used since ages within DLSS. The Vulkan link I shared above is also not a standard implementation, but a Nvidia specific extension (VK_NV_xxx)

FSR Redstone doing all that near EoY 2025 seems extremely unlikely given AMD's modus operandim, i.e reactive instead of proactive. But this would indeed be required to really make an impact on gaming SW side.
There's some other issues with those targets as well I would hope to be proven wrong but doing all this before EoY 2025 just seems like a very tall order. For example look at the perf impact of the leaked INT8 FSR4 on RDNA 2. Getting image quality not only to match the full FP8 based FSR4, but even exceed the superior DLSS 4 SR and getting it to run faster on newer cards and workable on older cards is extremely unlikely. RR matching DLSS4 RR prob not happening either, given how bad the RR looked when they unveiled the Toyshop demo near the end of February. As for PS5 Pro it'll prob get Lite INT8 based versions for SR and RR, but you're right this could drive widespread adoption of ML RR and should allow AMD to add RR in many games and help catch up with NVIDIA.
MFG might be an area where AMD can leapfrog, but yeah extrapolation with timewarp is very unlikely.

What I described is the ideal case of FSR Redstone surpassing DLSS4. Would be cool if that happens but as you say, achieving everything in one go is unlikely.

Regarding performance on older cards I don't see too many issues. I do not expect that AMD increases the size of their DNN model, but improves its quality. With that you do not get an additional performance hit.
On older cards it will run slower than FSR3 and RR might not be feasible outside of RDNA4 and PS5 Pro. But SR and FG should run with good enough performance. I would be fine with that.

NRC & RR is probably limited to RDNA4 and PS5 Pro. You could probably run the algorithms on other cards but I expect them to be too slow. But that is fine for me as well.
Heck, in an ideal case we could see first pathtraced games on PS5 Pro already (with some limitations). On the other hand, UE5 MegaLights is very close to pathtracing already.
Also MegaLights would benefit from NRC, RR and enhanced SR algorithms. The base PS5 might only use MegaLights and SR but would still look much better than games available today.

 

[MrMPFR]

What I described is the ideal case of FSR Redstone surpassing DLSS4. Would be cool if that happens but as you say, achieving everything in one go is unlikely.

Yeah I know just had to point out that achieving all this is very difficult.

Regarding performance on older cards I don't see too many issues. I do not expect that AMD increases the size of their DNN model, but improves its quality.

Hope that PSSR2 release can give some clarity here. I don't imagine porting FSR4 hybrid to INT8 is without sacrifices likely a massively scaled down transformer stage. CNN side already tapped out as per NVIDIA DLSS 2-3.7.

NRC & RR is probably limited to RDNA4 and PS5 Pro. You could probably run the algorithms on other cards but I expect them to be too slow. But that is fine for me as well.
Heck, in an ideal case we could see first pathtraced games on PS5 Pro already (with some limitations). On the other hand, UE5 MegaLights is very close to pathtracing already.
Also MegaLights would benefit from NRC, RR and enhanced SR algorithms. The base PS5 might only use MegaLights and SR but would still look much better than games available today.

Not sure about RR hybrid model even on PS5 Pro. Look at how DLSS RR TF runs on 20-30 series and they have much stronger FP16 performance than PS5 core per core. PS5 Pro has 67 TFLOPs FP16 dense and 300 TOPS INT8 dense. Compare with 2080 TI's ~110FLOPs FP16 dense. But perhaps a CNN based model or a scaled down version of full RDNA 4 version similar to INT8 port for SR is possible. IDK just stating this isn't going to be easy for AMD.

Sounds about right. MegaLights in beta as of UE 5.7 preview so prob still ~2 years from games shipping with it. Prob no earlier than late 2027, but that's probably when we'll start to see games lean heavily into MegaLights and/or Neural rendering to get past the ReSTIR only stage. In that process democratizing ReSTIR like visuals across a wide range of HW, even current gen consoles, and pushing well beyond that for newer HW and the nextgen consoles.

 

[basix]

300 TOPS INT8 should be fine. RDNA4 does not have much more (~400 TFLOPS FP8 on 9070XT and ~200 TFLOPS on 9060XT). A 5070 Ti is also on a similar level with ~400 TFLOPS FP8.

I always wondered, why PSSR2 takes so long when FSR4 is already available since March 2025. But now I think the main reason for the late PSSR2 release is FSR Redstone with all the new shiny features like updated SR and the new FG, RR and NRC.

 

[MrMPFR]

300 TOPS INT8 should be fine. RDNA4 does not have much more (~400 TFLOPS FP8 on 9070XT and ~200 TFLOPS on 9060XT). A 5070 Ti is also on a similar level with ~400 TFLOPS FP8.

I always wondered, why PSSR2 takes so long when FSR4 is already available since March 2025. But now I think the main reason for the late PSSR2 release is FSR Redstone with all the new shiny features like updated SR and the new FG, RR and NRC.

Some math just can't be translated. Transformers generally need FP8 or fallback (FP16), while CNNs pretty much exclusively mostly rely on INT8. But perhaps AMD can find some clever ways to scale down or completely rearchitect the transformer stage without impacting RR quality significantly.

That and changing the SR and RR algorithm to work with an architecture having less than a fourth the FP16 throughput vs INT8, unlike RDNA 4 which is 1:1. For RR that isn't even ready yet on consumer side, unlike FSR, that could be a reason for the vague PSSR2 release data of 2026. RR is tricky, but getting it to work with less FP throughput even more difficult.

FSR Redstone is likely the first major result (FSR4 still not on PS5 Pro) of Project Amethyst and Sony probably wants to go all out with a full release, not just SR, so yeah that certainly explains the delay.

Edit: Generalizing here based on what has been accomplished so far, but there are many areas of research as pointed out by @basix so INT8 might actually be good enough and here PS6 actually has a +12.5% lead per CU in dense INT8 vs RDNA 4. But it's a massive task port redo the entire pipeline to run almost exclusive with INT8, which explains the delay but it will be worth it prob even on newer cards. INT is much less heavy on cachemem than FP.
IIRC pretty much all transformer derived tech has relied heavily on floating points up till now, but that could change soon. Exciting stuff for PS5 Pro and massive ramifications here. TBH it might even get the full suite of PS6 neural rendering features and later FSR5 derivatives in PS6 enhanced crossgen titles, although prob somewhat scaled down version.

 

[basix]

Transformers don't care about the datatype. You can build transformer models with 1bit weights (see Bitnet). It is just easier to get good output quality with higher precision data types.

CNNs mostly use 3x3 convolutions (which get HW acceleration from PS5 Pro). Such convolutions get widely used in transformer models as well. You can build bigger kernels for transformers by stacking 3x3 convolutions (2 * 3x3 kernels are resulting in a 5x5 kernel and so on). What is missing is the self-attention block, but most of that is 1x1 / 3x3 / MLPs as well. Sony does support that by saying that the full FSR4 including transformer part can run on PS5 Pro (with some re-architecting). And there is research like Conv2Former which uses convolutions for the self-attention mechanism: https://arxiv.org/pdf/2211.11943

As RDNA4 has different HW for matrix acceleration, you might redesign some parts of the self-attention block to get optimal (performance) results.
Now the funny part: Add here some recent Nvidia research (PostNAS and JetBlock, https://arxiv.org/html/2508.15884v1) which optimizes for HW aware attention mechanisms and you are good to go

If you look at the recent paper from AMD (which shows their Ray Regeneration prototype), AMD is using a multi-branch U-Net architecture. UNet is the thingy which gets used inside PSSR. So this is the starting point. Add a transformer similar to FSR4 and use the PostNAS self-tuning mechanism for FSR Redstone, then you should get to a decent solution. Which run very well on both RDNA4 and PS5 Pro.

 

[MrMPFR]

Thanks for enlightening me. Didn't know this was even possible and made sure to edit my last comment.

Just assumed AMD would be held back by the same limitations as NVIDIA. Look at how bad FSR4 RR and SR TF runs on 20-30 series vs 40 series, which given no changes to INT side indicates a heavy FP workload. Seems like NVIDIA took the easy route with DLSS4 relying on tried and tested methods, while AMD is implementing bleeding edge methods to rely on INT8 for PSSR2 (PS5 Pro).
So RDNA 4 gets the early NVIDIA like FSR4 version (prob heavy reliance on FP), while PSSR2 is INT8 whenever possible. FP def easier than rearchitecting a lot of the pipeline based on new methods. Explains the delay.

If they can keep this up with the rest of the Project Amethyst stuff, no reason why PS5 Pro can't feel much closer to PS6 then PS5 during entire crossgen. Could take advantage of FSR updates and all the MLP based neural rendering tech. PS5 Pro looks like a testing ground in preperation for PS6. So in effect during entire crossgen PS5 Pro should probably be called PS6 Lite instead.

 

[marees]

PS5 Pro looks like a testing ground in preperation for PS6. So in effect during entire crossgen PS5 Pro should probably be called PS6 Lite instead.

I think of it like this:

PS6 = RDNA 5
PS5 pro = RDNA 3+
PS5 = RDNA 2

Redstone (path tracing) could potentially come to base PS5/Steam Deck, although there might be a performance hit

 

[adroc_thurston]

Redstone (path tracing) could potentially come to base PS5/Steam Deck, although there might be a performance hit

It ain't gonna run on the gabeboy, not enough frametime in the whole wide world for that.

 

[Saylick]

It ain't gonna run on the gabeboy, not enough frametime in the whole wide world for that.

Just need to aim for 240p internal res with 10 fps target.