Discussion Zen 5 Speculation (EPYC Turin and Strix Point/Granite Ridge - Ryzen 9000)

[DisEnchantment]

Speculate at will

 

[Joe NYC]

No, it looks like a 16c CCX and two 4c CCX. The big questions:

• Are these really just 3 CCX on a custom die or 2 different types of CCD connected?
• Is the 16c CCD essentially Bergamo?
• Will Bergamo get rid of IFoP just as MI300 does?
• Will this be the little to Zen5's BIG?

View attachment 76892
Just the picture - annotations by AMD.

Funny how AMD is releasing info by small drips. Lisa's speech has a bunch, and confirmation of things people speculated about.

Some people were not convinced about MLID leak of large (300-350 mm2) base dies, with compute chiplets stacked on top. So, Lisa confirmed this arrangement.

As far as CPU cores, there are 3 chiplets on top of the last base die. My prediction is 2 x 12 core chiplets + a mystery (non-CPU) chiplet.

Even though the SH5 socket will be loaded with electrical I/O, since Lisa mentioned need for Optical I/O, I wonder if the mystery chip might be that...

 

[Joe NYC]

The 4x4 block just doesn't look anything like a CPU whereas, as @HurleyBird mentioned, the top and bottom sections look like 8 core CCXs. I realize it's not a true to life render, but the 4x4 section is not even really close to a CPU looking structure. Who knows though, I'm happy waiting until AMD actually reveals more details.

I think 24 CPU cores on Mi300 came from one of AMD's presentations (I don't remember which one).

As @DisEnchantment mentioned, Mike Clark has been talking about potentially more cores per chiplet, and as I mentioned before, this has to be a brand new, ground up new Zen 4 chiplet, which gets all of its power and IO from TSVs, it may just as well have the 12 cores.

 

[uzzi38]

I think 24 CPU cores on Mi300 came from one of AMD's presentations (I don't remember which one).

As @DisEnchantment mentioned, Mike Clark has been talking about potentially more cores per chiplet, and as I mentioned before, this has to be a brand new, ground up new Zen 4 chiplet, which gets all of its power and IO from TSVs, it may just as well have the 12 cores.

Zen 4c is also more cores per chiplet mind you.

It has them split out across 2 CCXs, but is still double the cores per CCD.

 

[scineram]

Zen 4c is also more cores per chiplet mind you.

It has them split out across 2 CCXs, but is still double the cores per CCD.

The key point is the interconnect, which is completely different in MI300 than Epyc. Also Clark said more cores per L3 complex. So not 4c.

 

[BorisTheBlade82]

Some napkin measurements, since Locuza has retired 😁

• based on Locuza's measurements of a de-lidded MI300 and AMD's own rendering
• assuming, that at least proportions and aspect ratio on the render are accurate

Width for what I suspect is a 16c CCD is 9.2mm, Height is 7.9mm, area is 73.0mm2.

Quite coincidentally what AMD has been saying about Bergamo: Double the cores, same area 😉

 

[Joe NYC]

Some napkin measurements, since Locuza has retired 😁

• based on Locuza's measurements of a de-lidded MI300 and AMD's own rendering
• assuming, that at least proportions and aspect ratio on the render are accurate

Width for what I suspect is a 16c CCD is 9.2mm, Height is 7.9mm, area is 73.0mm2.

Quite coincidentally what AMD has been saying about Bergamo: Double the cores, same area 😉

View attachment 76944

View attachment 76945

Hmm... Could those 2 smaller chiplets fit 12 cores each? Assuming those chiplets would not have L3 cache or GMI links...

 

[BorisTheBlade82]

Hmm... Could those 2 smaller chiplets fit 12 cores each? Assuming those chiplets would not have L3 cache or GMI links...

Why would they? AMD already stated 24c in total for MI300. And sizes for the two 2x2 blocks matches as well.
Maybe EFB between the 16c and the smaller ones, maybe InFo-R. Maybe, the smaller ones house I/O and/or HBM controllers as well?

 

[DisEnchantment]

However I noticed AMD has been working a lot on PIM, just a handful of what I found

I am wondering if the feature they had with Xilinx Virtex Ultrascale+ with Samsung Aquabolt XL for PIM will make it to Zen 5 DC parts with HBM (MI300 type parts), usually recurring patents and provisional patents are good candidates for making it to a product.

Samsung Brings In-Memory Processing Power to Wider Range of Applications

Integration of HBM-PIM with the Xilinx Alveo AI accelerator system will boost overall system performance by 2.5X while reducing energy consumption by more than 60% PIM architecture will be broadly deployed beyond HBM, to include mainstream DRAM modules and mobile memory

news.samsung.com

Well to nobody's surprise, patent trail is right again on the PIM. And with Samsung indeed. They started with Xilinx and extended to CPU. Posted patents again for context.

PROVIDING ATOMICITY FOR COMPLEX OPERATIONS USING NEAR-MEMORY COMPUTING
<https://www.freepatentsonline.com/y2022/0413849.html>
APPROACH FOR REDUCING SIDE EFFECTS OF COMPUTATION OFFLOAD TO MEMORY
<https://www.freepatentsonline.com/y2023/0004491.html>
ERROR CHECKING DATA USED IN OFFLOADED OPERATIONS
<https://www.freepatentsonline.com/y2022/0318089.html>
DETECTING EXECUTION HAZARDS IN OFFLOADED OPERATIONS
<https://www.freepatentsonline.com/y2022/0318085.html>
Processing-in-memory concurrent processing system and method
<https://www.freepatentsonline.com/11468001.html>
OFFLOADING COMPUTATIONS FROM A PROCESSOR TO REMOTE EXECUTION LOGIC
<https://www.freepatentsonline.com/y2022/0206855.html>
MEMORY ALLOCATION FOR PROCESSING-IN-MEMORY OPERATIONS
<https://www.freepatentsonline.com/y2021/0303355.html>
Command throughput in PIM-enabled memory using available data bus bandwidth
<https://www.freepatentsonline.com/11262949.html>
HARDWARE-SOFTWARE COLLABORATIVE ADDRESS MAPPING SCHEME FOR EFFICIENT PROCESSING-IN-MEMORY SYSTEMS
<https://www.freepatentsonline.com/y2022/0066662.html>
PROCESSOR-GUIDED EXECUTION OF OFFLOADED INSTRUCTIONS USING FIXED FUNCTION OPERATIONS
<https://www.freepatentsonline.com/y2022/0188117.html>
REUSING REMOTE REGISTERS IN PROCESSING IN MEMORY
<https://www.freepatentsonline.com/y2022/0206685.html>
PRESERVING MEMORY ORDERING BETWEEN OFFLOADED INSTRUCTIONS AND NON-OFFLOADED INSTRUCTIONS
<https://www.freepatentsonline.com/y2022/0206817.html>
Providing host-based error detection capabilities in a remote execution device
<https://www.freepatentsonline.com/11409608.html>
VIRTUALIZING RESOURCES OF A MEMORY-BASED EXECUTION DEVICE
<https://www.freepatentsonline.com/y2022/0206869.html>
MANAGING CACHED DATA USED BY PROCESSING-IN-MEMORY INSTRUCTIONS
<https://www.freepatentsonline.com/y2022/0188233.html>
DYNAMICALLY COALESCING ATOMIC MEMORY OPERATIONS FOR MEMORY-LOCAL COMPUTING
<https://www.freepatentsonline.com/y2022/0414013.html>

And on the optical interconnect seems to match patent again. Great potential on the using this for the next gen XGMI. Nice long reach interconnect for the unified memory architecture.

OPTICAL BRIDGE INTERCONNECT UNIT FOR ADJACENT PROCESSORS
<https://www.freepatentsonline.com/y2022/0318174.html>
FANOUT MODULE INTEGRATING A PHOTONIC INTEGRATED CIRCUIT
<https://www.freepatentsonline.com/y2022/0342165.html>
OPTICAL DIE-LAST WAFER-LEVEL FANOUT PACKAGE WITH FIBER ATTACH CAPABILITY
<https://www.freepatentsonline.com/y2022/0206221.html>

I am curious whether PIM will come in MI300A instead of Zen 5.

 

[DisEnchantment]

Some napkin measurements, since Locuza has retired 😁

• based on Locuza's measurements of a de-lidded MI300 and AMD's own rendering
• assuming, that at least proportions and aspect ratio on the render are accurate

Width for what I suspect is a 16c CCD is 9.2mm, Height is 7.9mm, area is 73.0mm2.

Quite coincidentally what AMD has been saying about Bergamo: Double the cores, same area 😉

View attachment 76944

View attachment 76945

I think AMD rendering is not precise or purposefully vague. There should be some debug circuitry like the Debug circuity on the current CCD on the CCX top die in MI300.
This is the only way to verify a good die before stacking. Or they could stack first then test the KGSD. Again, whatever Naffziger wrote all the way from 2021

 

[BorisTheBlade82]

@DisEnchantment
Yes, I know. They simply photoshopped some "idealised" structure patterns together. But somehow I am under the impression that at least proportions and aspect ratio are the real deal - everything just adds up too nicely...

 

[DisEnchantment]

@DisEnchantment
Yes, I know. They simply photoshopped some "idealised" structure patterns together. But somehow I am under the impression that at least proportions and aspect ratio are the real deal - everything just adds up too nicely...

I am trying to perform this exercise regardless of how futile it may be. From Locuza, he measured 365mm2 for the base die.
And for this specific chiplet on top, I estimated around ~95mm2. The chiplet in the middle is ~158mm2. There are fanout mold/filler, power rails and interconnects on the sides of the base die facing each other.

But 365mm2 is too big for 24 Cores. The one in the middle is definitely not CPU. Which leaves the one on top and below as CPU chiplets with L2/L3 structures. So 2x 96mm2 12c chiplets which is more plausible. But I expect some Debug block too and SMU. So the render is not totally correct but depicting just the high level blocks.
Question is what is that in the middle. I think it is some XDNA block.






At least now we can remove the assumption that there are cores in the base die. There is only Infinity cache and IF and L3 is still on the CPU chiplet.
This design seems scalable and conservative with incremental risks. The IF/GMI can be hosted in a bridge chiplet for regular CCDs or in this specific case it is there in the base die itself.

So for now I would summarize my speculation as below based on what is revealed so far.

• 2x 12c chiplets, one in the top and one below (CPU chiplet is just indicative of the block not really some actual floorplan)
• L2/L3 in the CCD including the SMU and debug
• Base die has IF (as shown in slide). A slice of this can be taken out and used to connect to IOD or put the IOD function on in the base die itself.
  • This give some flexibility in case the CCD is stacked on the base die directly or a slice of the IF can be connected to an actual IOD with fanout links
  • This give some flexibility when scaling a larger amount of CCDs to a single IOD.
  • It is very likely that the SerDes is wider and uses less energy when actual route is via active Silicon with repeaters.
• The bigger chiplet in the middle (150mm2-160mm2) is likely XDNA. It is very uniform and looks like an actual FPGA.

 

[Vattila]

Is the 16c CCD essentially Bergamo?

The "new theory of Vattila" is congealing around the idea that the top and bottom chiplets on the lower right base die are indeed 16-core "Zen 4c" CCDs.

It would make sense from a reuse perspective. The die illustration in the presentation slide may simply be outdated. Perhaps they started with two 8-core "Zen 4" CCDs for 16 cores total, but then upgraded the design to 16-core "Zen 4c" CCDs later for 24 cores total (32 - 8 disabled). The reason for disabling 4 of the cores on each CCD may be down to limitations in power budget, heat, TSV yield, or other.

If so, it will be interesting to see how they are able to reuse the "Zen 4c" CCD in such — presumably — very different packaging solutions ("Bergamo" and MI300).

The bigger chiplet in the middle (150mm2-160mm2) is likely XDNA.

If that is the case, it is strange that they just label it part of the CPU.

Instead, could it be just gobs of cache? Then they would have a little bit of cache on each "Zen 4c" CCD, with more underneath in the base die, and even more in that big middle die.

 

[BorisTheBlade82]

@DisEnchantment & @Vattila
While I am still holding my stance, I am very thankful for your contributions. Most of the aspects you point out, I generally agree with.

Some questions:

• @DisEnchantment Can you point me to the error in my calculation that could lead to such big differences to your numbers? I arrive at 14px/mm for the render (the higher res one from CES I used), which results in the numbers I posted.
• Also, what program were you using? I'd like to give it a try.
• AMD confirmed this whole sector to be the CPU part, they also confirmed 24 cores and there are 24 identical structures looking much like idealised cores. Isn't that a lot of evidence in and of itself? (edit: "indication" might be more appropriate)
• The middle die will not need structures like an iMC for Bergamo, because it has some form of IOD. MI300 has no IOD, that is why the custom die on top and bottom has something that looks like that.
• The idea of an intermediate die for Bergamo IFoP connection occurred to me as well. But having another die that needs to be connected via advanced packaging would quite likely be more expensive than going full throttle InFo-R (they don't need these wide traces that N31 has for the CCD).

 

[BorisTheBlade82]

@DisEnchantment
I just spotted an error in your calculation: You assume 365mm2 for the CPU block from Locuza. But he measured the structural silicon, which goes all the way from HBM to HBM. On the render there is a lot of area around the CPU structure - so you are overestimating the 4x4 structure and everything else. Please have a look again at the two pictures I posted for reference.

 

[DisEnchantment]

@DisEnchantment
I just spotted an error in your calculation: You assume 365mm2 for the CPU block from Locuza. But he measured the structural silicon, which goes all the way from HBM to HBM. On the render there is a lot of area around the CPU structure - so you are overestimating the 4x4 structure and everything else. Please have a look again at the two pictures I posted for reference.

You are right.

I am using ImageJ

Download

imagej.nih.gov

You can set the scale using a known die length or width to the pixel count. then it will calculate everything.


Unsurprisingly, the base die in the actual pic of the package is not aligned with the PR graphic, so I resized the image to get the aspect ratio. Now it somewhat lines up, but the central chiplet is quite odd, there is no L3. and why 4 core chiplets?

 

[Joe NYC]

Well to nobody's surprise, patent trail is right again on the PIM. And with Samsung indeed. They started with Xilinx and extended to CPU. Posted patents again for context.

View attachment 76947


And on the optical interconnect seems to match patent again. Great potential on the using this for the next gen XGMI. Nice long reach interconnect for the unified memory architecture.

View attachment 76948



I am curious whether PIM will come in MI300A instead of Zen 5.

The PIM is nice, but I think it is a short run solution (which may lack some flexibility).

AMD already signaled (IMO) a long-term solution - HBM stacked on compute.

Well, not exactly, but imagine one of the quadrants of Mi300 with base die + 2 HBM stacks. Imagine enlarging the base die and stacking the HBM stacks on top of it.

So Mi300 maybe the first leg of realizing power savings going from:
DIMMs - 12 pj/bit
HBM with 2.5D, 2 jumps over microbumps = 3.5 pj/bit

Better than 3x improvement.

Suppose Mi400 has this arrangement of stacking HBM stacks on top of base die, the energy needed falls to 0.2 pj/bit. That being 60x improvement, or if 2 trips over Hybrid Bond are 0.2 each, then 0.4, and that would be exactly the 30x improvement in power that AMD has proposed over ~5 years.

 

[Joe NYC]

• AMD confirmed this whole sector to be the CPU part, they also confirmed 24 cores and there are 24 identical structures looking much like idealised cores. Isn't that a lot of evidence in and of itself? (edit: "indication" might be more appropriate)

There are similar squares on the GPU chiplets, which lead @Vattila to speculate that those chiplets had 4 CPU cores each. With that in mind, it would be less of an indication, and more of AMD just drawing some pattern on the chiplets.

[*]The middle die will not need structures like an iMC for Bergamo, because it has some form of IOD. MI300 has no IOD, that is why the custom die on top and bottom has something that looks like that.

My guess would be that each base die has some IO, mostly PCIe lanes, 1/4 of the total, and maybe one day (we can hope) the 1/4 or 1/2 of the total Mi300 will get its socket for desktop or HEDT.

 

[maddie]

The PIM is nice, but I think it is a short run solution (which may lack some flexibility).

AMD already signaled (IMO) a long-term solution - HBM stacked on compute.

Well, not exactly, but imagine one of the quadrants of Mi300 with base die + 2 HBM stacks. Imagine enlarging the base die and stacking the HBM stacks on top of it.

So Mi300 maybe the first leg of realizing power savings going from:
DIMMs - 12 pj/bit
HBM with 2.5D, 2 jumps over microbumps = 3.5 pj/bit

Better than 3x improvement.

Suppose Mi400 has this arrangement of stacking HBM stacks on top of base die, the energy needed falls to 0.2 pj/bit. That being 60x improvement, or if 2 trips over Hybrid Bond are 0.2 each, then 0.4, and that would be exactly the 30x improvement in power that AMD has proposed over ~5 years.

Do you see PIM and HBM as exclusive choices? It's not, and allows further power reductions irrespective of the memory used.

 

[DisEnchantment]

PIM will be a big thing going forward. Not only on package but off package as well.
I am just imagining doing matrix multiplication for huge vectors of specific dimensions laid out in memory without engaging the CPU ALUs or any memory transaction from CPU. Very simple ops, just addition/multiplication with memory offsets is enough to offload huge computation for matrix operations. Possibilities are quite vast.
Would be interesting if they program the logic on the PIM module as well.

It is there for both DIMMs and HBM from Samsung. Hynix is also making GDDR6 PIM modules.


Wondering if this can be extended to CXL memory

 

[scineram]

So, I have alse been thinking about how AMD could go about increasing core counts but more conservatively, without a massive overhaul. Looking at the CCD sizes so far, the shape more specifically, they have all been rectangular that is quite far from a square. So they could increase the number of cores in the columns on both sides to 6. Maybe combined with a slight process shrink looks like this 12 core CCD could fit on existing packages even with the same IOD. It would probably also not require a significant change in the floor plan. The L3 complex would also be expanded vertically of course. Now with the breakdown of SRAM scaling it's possible they couldn't fit similar 4 MiB blocks in front of each core. Then these could shrink to 3 or 3.5 MiB, for an overall size of 36 or 42 instead of 48. That could still be a big win overall, especially if the private L2 increased again by 50-100%.

 

[q52]

now that the 7950X3D has launched, is that the end of the road for updates to consumer desktop Zen 4? And is it clear yet what kind of spec increases are on the horizon for Zen 5? I am looking for ways to fit the most capacity (CPU cores + memory) in a small form factor (mITX), and it seems like 7950X(3D) + 96GB (two-slot mITX) might be the best available for the next year or more. I am trying to figure out if we are expecting any Zen 5 offerings to significantly exceed that kind of capacity in the desktop space? More cores and/or more memory support?

 

[Joe NYC]

now that the 7950X3D has launched, is that the end of the road for updates to consumer desktop Zen 4? And is it clear yet what kind of spec increases are on the horizon for Zen 5? I am looking for ways to fit the most capacity (CPU cores + memory) in a small form factor (mITX), and it seems like 7950X(3D) + 96GB (two-slot mITX) might be the best available for the next year or more. I am trying to figure out if we are expecting any Zen 5 offerings to significantly exceed that kind of capacity in the desktop space? More cores and/or more memory support?

I think we will see more than 1 layer of V-Cache and maybe improved tolerances for the V-Cache so that the it cam be clocked closer to non-V-Cache Hopefully before Christmas of this year.

On a low probability end of things, AMD may have a 12 core Zen Core CCD that can be stocked on top of base die. In theory, AMD could reuse this CCD and release a new client base die.

But probably there is not enough time left for Zen 4, if Zen 5 is released roughly a year from now...

 

[DisEnchantment]

I think we will see more than 1 layer of V-Cache and maybe improved tolerances for the V-Cache so that the it cam be clocked closer to non-V-Cache Hopefully before Christmas of this year.

Not very likely I think, diminishing returns for DT especially with faster DDR5.

On a low probability end of things, AMD may have a 12 core Zen Core CCD that can be stocked on top of base die. In theory, AMD could reuse this CCD and release a new client base die.

I also would like to see a 12 core CCD on N4P. But...the CCD is going to be large.

Totally guessing based on Mike's statement's

getting more cores in a sharing L3 environment

12C per L3

Then the L2 - if your L2 is bigger then you can cut back some on your L3 as well.

L2 1MiB --> 1.5MiB
L3 4MiB/Slice --> 3MiB/Slice = 36MiB/CCD

GMI SerDes PHY replaced with a lower power PHY with small beach head.
A ring with quadrants would probably be suitable to cut latency across 12 cores. Additionally the faster interconnect should help when snooping the other CCX.

Even with this, CCD seems a bit too big for AMD, ~100mm2 is quite big. Unless, they manage to use the perf gains from N4P and claw back some density tradeoffs to reach at least ~90mm2.
On the other hand N4/5 supply is plenty in 2024, 180k+ wpm.
Additionally, the IOD will be totally new. RDNA3+ and AIE for DT as well.

Too much unknowns but interesting for me to ponder about this, as usual.

 

[Joe NYC]

Not very likely I think, diminishing returns for DT especially with faster DDR5.

It would probably be 2 additional layer of 64 MB to get the same performance increase as the first one.

7950x3d presents a great platform for testing.

For example, fix frequency to 4 GHz, run the task on V-Cache CCD, then run the same task at same frequency on the other CCD, to get the precise increase from V-Cache.

As far as I know, none of the testers did that,

I also would like to see a 12 core CCD on N4P. But...the CCD is going to be large.
View attachment 77369
Totally guessing based on Mike's statement's

12C per L3

L2 1MiB --> 1.5MiB
L3 4MiB/Slice --> 3MiB/Slice = 36MiB/CCD

That's really not so bad. 100 mm2 is the sweat spot.

But I don't think AMD would just design an N4P CCD with 12 cores as a replacement for current Zen 4 CCD.

I was thinking about reusing the Zen 4 CCD from Mi300 (so no extra cost for masks) and changing IOD to be the "Base Die", on which this re-used 12 core Zen 4 CCD would be stacked.

So based on what the Mi300 CCD looks like, it could mean no L3 and no GMI, but more TSVs.

So minus 27 mm2 for L3, minus 5mm2 for GMI and back to 70mm2.

The base / IO die would grow the N6 area, with mostly SRAM for shared L3, and have room to stack 2 of the 12 core CCDs, which would share the L3.

All of this (idle) speculation stems from the fact that AMD likes to reuse chiplets. and this would be a nice way to reuse the Mi300 chiplet for desktop, and also address several bottleneck in the client design.

Most of the of the 7950X3D shenanigans are tied to the fact that:
- L3 on CCD and stacked L3 caches are hard to share between chiplets
- which reduces 16 core gaming CPU to 8 core gaming CPU
- SRAM on stacked top reduces clock headroom

So these ideas from Mi300 are more likely going to make it to Zen 5. Extremely low probability for Zen 4 and the Mi300 Zen 4 CCD will most likely have no re-use in other products. But it is still a lego piece to build a reconfigurable Mi300.

GMI SerDes PHY replaced with a lower power PHY with small beach head.
A ring with quadrants would probably be suitable to cut latency across 12 cores. Additionally the faster interconnect should help when snooping the other CCX.

Even with this, CCD seems a bit too big for AMD, ~100mm2 is quite big. Unless, they manage to use the perf gains from N4P and claw back some density tradeoffs to reach at least ~90mm2.
On the other hand N4/5 supply is plenty in 2024, 180k+ wpm.
Additionally, the IOD will be totally new. RDNA3+ and AIE for DT as well.

Too much unknowns but interesting for me to ponder about this, as usual.

The question for the RDNA3+ and AIE in desktop is if they are going to graduate, in Zen5, to be standalone chiplets or integrated in other chiplets...

 

[A///]

Not to be that one guy, but can I get a summary of the last 27 pages?