Discussion Nvidia Blackwell with 512-bit memory bus in H2-2024?

[Tigerick]

With Kopite7kimi pretty much confirm the 512-bit memory bus interface of upcoming Blackwell GPU, here comes the thread for discussion of all future Blackwell GPU family, aka RTX 5000 series. We also know about codename of all 5 die sizes; namely GB202, GB203, GB205, GB206 and GB207. Below is the table with some speculations of my own, you guys are welcome to pitch in:- I also put in upcoming RDNA5 just for comparison sake...

https://twitter.com/i/web/status/1684497812921147392

Codename	Possible Model Number	Possible Memory Configuration	Possible Mobile GPU	Possible AMD's response
GB202	5090 Ti	512-bit 32GB
	5090	448-bit 28GB
GB203	5080 Ti	384-bit 24GB		N51 384-bit 24GB GDDR7
	5080	320-bit 20GB		N51 320-bit 20GB GDDR7
GB205	5070 Ti	256-bit 16GB	256-bit 16GB GDDR6	N52 256-bit 16GB GDDR7
GB206	5070	192-bit 12GB	192-bit 12GB GDDR6	N53 128-bit 12GB GDDR7
GB207	5060 Ti	128-bit 8/16 GB	128-bit 8GB GDDR6
	5060	128-bit 8GB

Even though we know nVidia will implement 512-bit memory bus, we don't know what types of memory they will choose, I try to list down all possibility of memory choices below:

	384-bit GDDR6(X)	384-bit GDDR7	512-bit GDDR7	512-bit GDDR6X
RTX 4090	24GB 21Gbps 1TB/s	24GB 32Gbps 1.5TB/s	32GB 32Gbps 2TB/s	32GB 24Gbps 1.5TB/s
		+ 50%	+ 100%	+ 50%
Pros		Lower TDP due to GDDR7's lower voltage Extra 50% bandwidth	Lower TDP due to GDDR7's lower voltage Bigger memory size Double the memory bandwidth	Bigger memory size Extra 50% bandwidth Cheaper to source
Cons		More expensive than GDDR6x	Highest BOM More layers PCB to accommodate 16 pcs memory chips	More layers PCB to accommodate 16 pcs memory chips Highest TDP
AMD Radeon RX 7900 XTX	24GB 20Gbps 960GB/s	24GB 32Gbps 1.5TB/s
		+ 60%

There was rumored about usage of 384-bit GDDR7 memory before, I believe nVidia is testing both 384-bit GDDR7 and 512-bit GDDR6x and then decide to use GDDR6x options. Both memories have same bandwidth even though inteface is different.

Hmm, it seems to me usage of GDDR6x make more sense compared to GDDR7, what do you think???

As for amount of CUDA cores, nVidia is changing the architecture of Blackwell GPU, so far no leaks about structure of new Shader Model...However, we should be expecting at least 50% performance improvement due to extra 50% memory bandwidth improvement...

Update 1:

Model	Codename	Die Size (mm2)	SM/CU	CUDA/	L2 Cache	Memory	Memory BW	BW +/-	TPU Perf
RTX 3090Ti	GA102	628	84	10752	6 MB	384-bit 24GB GDDR6X	1 TB/s
RTX 4090	AD102	609	128	16384	72 MB	384-bit 24GB GDDR6X	1 TB/s	+ 0%	+ 45%
RTX 5090(Ti)	GB202	?	?	?	128 MB	512-bit 32GB GDDR6X	1.5 TB/s	+ 50%	?
					?	384-bit 24/36 GB GDDR7	1.5 TB/s	+ 50%	?
RTX 3080Ti	GA102	628	80	10240	6 MB	384-bit 12GB GDDR6X	912.4 GB/s
RTX 4080Ti	AD102	609	?	?	?	320-bit 20GB GDDR6X	?
RTX 5080Ti	GB203	?	?	?	96 MB	384-bit 24GB GDDR6X	1152 GB/s (75% of N51)	+ 61%	?
RTX 4080	AD103	379	76	9728	64 MB	256-bit 16GB GDDR6X	716 GB/s
RTX 5080	GB203	?	?	?	80 MB	320-bit 20GB GDDR6X	960 GB/s (75% of N51)	+ 34%	?
RTX 3080	GA102	628	68	8704	5 MB	320-bit 10GB GDDR6X	760.3 GB/s
RTX 4070Ti	AD104	294	60	7680	48 MB	192-bit 12GB GDDR6X	504.2 GB/s	- 34%	+ 17%
RTX 5070Ti	GB205	?	?	?	64 MB	256-bit 16GB GDDR6X	768 GB/s (75% of N52)	+ 52%	?
RTX 3070Ti	GA104	392	48	6144	4 MB	256-bit 8GB GDDR6X	608.3 GB/s
RTX 4070	AD104	294	46	5888	36 MB	192-bit 12GB GDDR6X	504.2 GB/s	- 17%	+ 14%
RTX 5070	GB206	?	?	?	48 MB	192-bit 12GB GDDR6X	576 GB/s (12.5% > N53)	+ 14%	?
RTX 3060Ti	GA104	392	38	4864	4 MB	256-bit 8GB GDDR6	448 GB/s
RTX 4060Ti	AD106	188	34	4352	32 MB	128-bit 8/16 GB GDDR6	288 GB/s	- 36%	+ 11%
RTX 5060Ti	GB207	?	?	?	32 MB	128-bit 8/16 GB GDDR6X	384 GB/s	+ 33%	?
RTX 3060	GA106	276	28	3584	3 MB	192-bit 12GB GDDR6	360 GB/s
RTX 4060	AD107	159	24	3072	24 MB	128-bit 8GB GDDR6	272 GB/s	- 25%	+ 18%
RTX 5060	GB207	?	?	?	24 MB?	128-bit 8GB GDDR6	320 GB/s	+ 18%	?
RX 6950 XT	Navi 21	520	80	5120	128 MB	256-bit 16GB GDDR6	576 GB/s
RX 7900 XTX	Navi 31	529	96	6144	96 MB	384-bit 24GB GDDR6	960 GB/s	+ 67%	+ 36%
RX 8950 ?	Navi 51	?	?	?	?	384-bit 24GB GDDR7	1.5 TB/s	+ 60%	?
RX 6750 XT	Navi 22	335	40	2560	96 MB	192-bit 12GB GDDR6	432 GB/s
RX 7800 XT	Navi 32	346	60	3840	64 MB	256-bit 16GB GDDR6	620.8 GB/s	+ 44%	+ 40%
RX 8800 ?	Navi 52	?	?	?	?	256-bit 16GB GDDR7	1 TB/s	+ 65%	?
RX 7700 XT	Navi 32	346	54	3456	48 MB	192-bit 12GB GDDR6	432 GB/s
RX 8700 ?	Navi 53	?	?	?	?	128-bit 12GB GDDR7	512 GB/s	+ 18.5%	?

 

[Ajay]

With Kopite7kimi pretty much confirm the 512-bit memory bus interface of upcoming Blackwell GPU, here comes the thread for discussion of all future Blackwell GPU family, aka RTX 5000 series. We also know about codename of all 5 die sizes; namely GB202, GB203, GB205, GB206 and GB207. Below is the table with some speculations of my own, you guys are welcome to pitch in:- I also put in upcoming RDNA5 just for comparison sake...

https://twitter.com/i/web/status/1684497812921147392

Codename	Possible Model Number	Possible Memory Configuration	Possible Mobile GPU	Possible AMD's response
GB202	5090 Ti	512-bit 32GB
	5090	448-bit 28GB
GB203	5080 Ti	384-bit 24GB		N51 384-bit 24GB GDDR7
	5080	320-bit 20GB		N51 320-bit 20GB GDDR7
GB205	5070 Ti	256-bit 16GB	256-bit 16GB GDDR6	N52 256-bit 16GB GDDR7
GB206	5070	192-bit 12GB	192-bit 12GB GDDR6	N52 192-bit 12GB GDDR7
GB207	5060	128-bit 8/16 GB	128-bit 8GB GDDR6	N43/N44/N53 ???
	5050	128-bit 8GB

Even though we know nVidia will implement 512-bit memory bus, we don't know what types of memory they will choose, I try to list down all possibility of memory choices below:

	384-bit GDDR6(X)	384-bit GDDR7	512-bit GDDR7	512-bit GDDR6X
RTX 4090	24GB 21Gbps 1TB/s	24GB 32Gbps 1.5TB/s	32GB 32Gbps 2TB/s	32GB 24Gbps 1.5TB/s
		+ 50%	+ 100%	+ 50%
Pros		Lower TDP due to GDDR7's lower voltage Extra 50% bandwidth	Lower TDP due to GDDR7's lower voltage Bigger memory size Double the memory bandwidth	Bigger memory size Extra 50% bandwidth Cheaper to source
Cons		More expensive than GDDR6x	Highest BOM More layers PCB to accommodate 16 pcs memory chips	More layers PCB to accommodate 16 pcs memory chips Highest TDP
AMD Radeon RX 7900 XTX	24GB 20Gbps 960GB/s	24GB 32Gbps 1.5TB/s
		+ 60%

There was rumored about usage of 384-bit GDDR7 memory before, I believe nVidia is testing both 384-bit GDDR7 and 512-bit GDDR6x and then decide to use GDDR6x options. Both memories have same bandwidth even though inteface is different.

Hmm, it seems to me usage of GDDR6x make more sense compared to GDDR7, what do you think???

As for amount of CUDA cores, nVidia is changing the architecture of Blackwell GPU, so far no leaks about structure of new Shader Model...However, we should be expecting at least 50% performance improvement due to extra 50% memory bandwidth improvement...

Since the release date is somewhere in 2025 - I think GDDR7 is highly probable - at least for GB102 based GFX AIBs. Power consumption per Gb is a bit high on the current gen of GDDR7, but, at least with Samsung, there is a gen2 in the works that will use less power per Gb (not sure if this is from a process change, or an implementation change). As for everything else - who knows, well, who knows that can talk! No details on uarch changes, CCs, SMs, RT cores, etc. The one thing that's clear, given current trends, is that you better be prepared to sell one of your children to buy the top SKU!

 

[CakeMonster]

I don't feel qualified to guess at all, but right now I just hope for better memory bandwidth as well as a top card with 32GB, just for hobbyist AI stuff. If more AI progress is made, those will no doubt be useful in all sorts of cool stuff we can hardly imagine now. For gaming, I frankly have no idea, as various upscaling and frame generation technologies don't necessarily scale with the same specs that raster did before, and those are taking over so hardware support for features unknown as of now might be more impactful. I can't imagine there would be a gaming need for >24GB even if it releases in 2025, but I'll of course be happy to take the headroom if the price isn't more than a 4090..

Edit: Also, given CPU limitation rearing its head these days for those with 4090s, we might have different wish list specs for 50xx series depending on how CPU's perform when we're closer to release, as both AMD and Intel actual next gen (not just refresh) should be out by then.

 

[Mopetar]

If I had to guess it's unlikely that the consumer card will get the full bus. The (mostly) uncut cards are going into the data center. Consumer cards are the salvage parts and with a bus that big it's getting cut down.

 

[Ajay]

If I had to guess it's unlikely that the consumer card will get the full bus. The (mostly) uncut cards are going into the data center. Consumer cards are the salvage parts and with a bus that big it's getting cut down.

Datacenter? Hope not, that would be lame (though I guess some of that is currently happening with the 4090). Workstation cards will be built off the top two chips - so there could be a supply problem. We'll just have to see what Nvidia's wafer allotment is.

 

[MoogleW]

If I had to guess it's unlikely that the consumer card will get the full bus. The (mostly) uncut cards are going into the data center. Consumer cards are the salvage parts and with a bus that big it's getting cut down.

If by datacenter you mean Quadro then you're like right, although that doesn't matter since they typically seem to target same performace but Quadro relies on achieving the same with more hardware while consumer does so with higher clocks.

Like 4090 vs rtx A6000 Ada. 4090 often outperforms despite less hardware but A6000 ada is more efficient and in a smaller footprint

I have been wondering why there are dozens of pages of RDNA4 and CDNA3 but absolutely no discussion of GB100 and GB102/202

 

[Ajay]

I have been wondering why there are dozens of pages of RDNA4 and CDNA3 but absolutely no discussion of GB100 and GB102/202

Probably because Nvidia has been very tight lipped on this and is successful keeping info out of leakers hands for now.

 

[jpiniero]

I think the reason that GB202 has 512-bit is mainly for Quadro (ie: they want more memory capacity) but maybe they will do a Titan as well. It would not surprise me if they slash the L2 cache size because of N3E's lack of SRAM scaling... and really on the lower models it also wouldn't surprise me if they also cut the memory controller count too. GDDR7 is likely not going to be cheap either.

ie:

GB202 512-bit
GB203 320-bit
GB205 192-bit
GB206 128-bit
GB207 128-bit (or less)

 

[Ajay]

I think the reason that GB202 has 512-bit is mainly for Quadro (ie: they want more memory capacity) but maybe they will do a Titan as well. It would not surprise me if they slash the L2 cache size because of N3E's lack of SRAM scaling... and really on the lower models it also wouldn't surprise me if they also cut the memory controller count too. GDDR7 is likely not going to be cheap either.

Lack of SRAM scaling isn't a reason to cut it. It could be a reason for not increasing it. I'll just take up a larger portion of the GPU - though changes are coming even to consumer GPUs that will move 'uncore' stuff out of the rendering core. It's a necessity, if for no other reason than high NA EUV will reduce reticle sizes (and of course, improving yields).

 

[jpiniero]

Lack of SRAM scaling isn't a reason to cut it. It could be a reason for not increasing it. I'll just take up a larger portion of the GPU - though changes are coming even to consumer GPUs that will move 'uncore' stuff out of the rendering core. It's a necessity, if for no other reason than high NA EUV will reduce reticle sizes (and of course, improving yields).

I'm expecting die sizes to be cut (other than GB202) to compensate for the higher costs of N3E and GDDR7.

 

[Ajay]

I'm expecting die sizes to be cut (other than GB202) to compensate for the higher costs of N3E and GDDR7.

Fair enough as speculation. Unfortunately, crystal balls aren't real

 

[Tigerick]

I'm expecting die sizes to be cut (other than GB202) to compensate for the higher costs of N3E and GDDR7.

That's why I think NV will use higher-bus of GDDR6X instead of more expensive GDDR7 on Blackwell GPU to counter upcoming RDNA5 with GDDR7. With GDDR6X, NV might be able to offer GB203 and GB205 with performance increase while maintaining current pricing... Current RTX4080 with 256-bit GDDR6X offers similar raster performance than 7900XTX with 384-bit GDDR6. We might see another round of NV magic

 

[Ajay]

That's why I think NV will use higher-bus of GDDR6X instead of more expensive GDDR7 on Blackwell GPU to counter upcoming RDNA5 with GDDR7. With GDDR6X, NV might be able to offer GB203 and GB205 with performance increase while maintaining current pricing... Current RTX4080 with 256-bit GDDR6X offers similar raster performance than 7900XTX with 384-bit GDDR6. We might see another round of NV magic

Nvidia has always pushed the memory for its GPUs to the max. They won't bother with saving a bit of money. These are going to be very expensive GFX AIBs - no need to cut corners.

 

[jpiniero]

Nvidia has always pushed the memory for its GPUs to the max. They won't bother with saving a bit of money. These are going to be very expensive GFX AIBs - no need to cut corners.

nVidia is more likely to use less chips, esp assuming there's a capacity increase. Which will help defray the cost increase at the expense of reducing the bandwidth increase (of course).

At 3 GB/chip, 128-bit becomes 12 GB and 192-bit becomes 18 GB.

 

[Ajay]

nVidia is more likely to use less chips, esp assuming there's a capacity increase. Which will help defray the cost increase at the expense of reducing the bandwidth increase (of course).

At 3 GB/chip, 128-bit becomes 12 GB and 192-bit becomes 18 GB.

If that's the case GB202 512-bit and GB203 320-bit could have insane amounts of memory. So, probably just the GPUs with narrow busses?

 

[dr1337]

Where's the 2025 rumor coming from? Nvidia has had a strong 2 year cadence going for almost a decade and I'd be really surprised if they broke it now.

 

[Ajay]

Where's the 2025 rumor coming from? Nvidia has had a strong 2 year cadence going for almost a decade and I'd be really surprised if they broke it now.

I believe it was @adroc_thurston who mentioned it (for NV). Not sure why. Wafer shortage because of G(H)100/200 - though I thought the problem was packaging (CoWoS)??

 

[adroc_thurston]

Not sure why.

New SM hard.

 

[MoogleW]

Where's the 2025 rumor coming from? Nvidia has had a strong 2 year cadence going for almost a decade and I'd be really surprised if they broke it now.

According to RedGaminTech Nvidia is 'breaking tradition' with its SM design, he also mentioned an improved TPC design scaled down from the H100 design (DSMEM) that reduces trips to L2 cache (but was seemingly shelved for Lovelace or was always meant for rtx 50) so that could factor. Another could be bugs that require further efforts. Last but not least, Nvidia could be waiting for 3nm to mature a bit more for more performance or less cost

As for source of the leak, its an interpretation of a slide

Weiterleitungshinweis

 

[KompuKare]

Last but not least, Nvidia could be waiting for 3nm to mature a bit more for more performance or less cost

The interval between new nodes certainly has slowed down.
Now for Ada (aside from the xx90 models), that doesn't matter that much as all the rest are small enough that on the present node they have plenty of scope not just for a refresh but actually a second generation. Hence the performance stagnation at most tiers.

What was surprising about AD102 was that it was so big. Better for sales this generation, but even +20% above 3090 Ti would have sold. 600mm² doesnn't leave them much of option but to move to a new node. BW102 maybe 450mm² 3nm, with an BW101 at around 600mm² but with no intention to launch a gaming card with the latter? That might make sense (for Nvidia).

My feeling is that from now on node progression well be trickled out. Well, unless the calculation is that a rival might use that to get ahead, so something like a big BW101 die might be their answer.

 

[jpiniero]

If that CoWoS shortage talk is legit, it wouldn't surprise me if GB202's main purpose is to be a GDDR7 compute product. May not be suitable for gaming. But if it is, it's going to be very expensive (ie: Titan only)

 

[Mopetar]

What was surprising about AD102 was that it was so big. Better for sales this generation, but even +20% above 3090 Ti would have sold. 600mm² doesnn't leave them much of option but to move to a new node.

Reticle limit is 858 mm^2 so there's room to grow. We've seen this in the past where NVidia has two generations on the same node. Smaller designs in the first generation mean better yields and room to grow. By the time the second generation goes into production the process should be mature enough to support the bigger dies.

The big die might be restricted to professional and data center users. Even the 4090 is pretty cut down and since AMD is t offering anything that can match NVidia on performance there's not much incentive to release another card above that. It's possible we get more resources on a slightly smaller die for a 5090 just because it isn't as cut down.

NVidia has options, but a lot of what gets sold and under what label depends on what the alternatives look like at the time or what NVidia is predicting the competition to have.

 

[jpiniero]

Reticle limit is 858 mm^2 so there's room to grow. We've seen this in the past where NVidia has two generations on the same node. Smaller designs in the first generation mean better yields and room to grow. By the time the second generation goes into production the process should be mature enough to support the bigger dies.

I had kind of wondered if nVidia would do a respin of Ada to N4P (?) and throw in GDDR7. Wouldn't be any smaller but perhaps slightly higher clock speeds and any performance gains they can get from the higher bandwidth.

 

[Mopetar]

I think there's some confusion over their process node. They've decided to call their custom set of stuff 4N, but it's not the same as TSMC's N4 nodes.

I could see them changing anyway even if it doesn't get them any extra density. The power or performance boost is usually worth it.

 

[jpiniero]

I think there's some confusion over their process node. They've decided to call their custom set of stuff 4N, but it's not the same as TSMC's N4 nodes.

I could see them changing anyway even if it doesn't get them any extra density. The power or performance boost is usually worth it.

It doesn't appear they are doing any sort of Ada refresh.