The larger the context the longer the prefill phase takes. Prefill needs lots of low-precision compute but barely any bandwidth, so the big fat expensive HBM is sitting idle during prefill.
what kind of magic sauce is nvidia putting into these chips to 6-8x FP4 Perf while barely increasing area and power
That and using the xtor budget for bigger systolic arrays. Ain’t rocket science.
AT0 is a rasterization-able 512bit GDDR7 part, possibly close to reticle limit as well.
RDNA5 probably supports FP8 and FP4 as well.
The MiXXX series don't compete with rasterizers like this CPX.
Nothing.
Duhhhhh.
It's not made for inference.
It's driven by AMD.
Yeah. It’s becoming more and more evident that consumer Nvidia GPUs are trickle down enterprise parts now. Consumer Rubin GB202 sounds like we’ll get 20-30% faster pure raster due to same SM count but higher clocks, likely a bigger boost to RT workloads via 2x ray/triangle intersection, and an even bigger boost to tensor performance. Oh, and invent some new version of an AI feature that requires tapping into a ton of FP4 which coincidentally Rubin has.
Yeah, but at least some are purpose-build parts.
See the problem is that we're not really limited by box/tri testing for RTRT.
Understandably so. Nvidia are re-purposing server AI parts for consumers and AMD are re-purposing parts from other, higher volume markets for consumers. Makes sense in my mind.
Queue providing a new AI feature that leverages FP4 out the wazoo as a crutch for the lackluster RT performance uplift.
Large context inference market is the most lucrative. Dunno if AMD can afford to just give that away.
They aren't "giving it away". They just don't have an equivalent product to the CPX. For large scale inference, they'll provide MI450.
Why not just use AT0? Maybe it's not equivalent but if it is profitable to sell it for $3k for that then it can work easily as long as same code runs on it as well as MI400
Why would not it be? RDNA4 already got FP4 and it's use clearly going even bigger in next gen, it's pretty clear single direction of travel for "AI" stuff and looks like 4 bits is the floor for the time being, so can go all in on it just like Nvidia appears to do.
Cuz AT0 is only ~7 PF or so, AMD is not doing the same matrix cores on gaming dGPUs and DC GPUs like NVIDIA is doing with Rubin.
So that means AMD at least got good chance to catch up in perf and exceed in top end card for gaming?
Who knows where it will land in gaming performance. It won't touch GR202, that's for sure. Nvidia can afford to bring that die to consumers at reduced margin because it's a hand-me-down from the newly announced CPX line, essentially. AMD doesn't have that luxury.
I can get to ~6PF with the same rates as Blackwell, I doubt RDNA5 will go wider than that.
Whilst NV certainly should win with a 800mm^2ish die, I'm pretty sure AMD will have better gaming PPA across the stack with how much die area NV has put in systolic arrays and associated stuff like TMEM.
And what's the volume and margin expected for cloud gaming? Probably laughable compared to Nvidia's AI volumes.
AT0 should ship over a million units over its lifetime just in the cloud, and for several thousand a pop so very good margins.
Facebook
Twitter