Speculation: AMD's APU with HBM

[PeterScott]

While I often see someone claim HBM is higher latency, I googled around a lot for that, and most of my research indicates HBM is lower latency, at least lower than GDDR:

AMD Presentation talking about Latency improving features:
http://www.legitreviews.com/amd-high-bandwidth-memory-detailed-with-joe-macri_163855
"The PHY’s are laid out on the edges of the chips to keep the latency to a minimum. HBM does decrease the latency as data is for the most part no longer being moved horizontally to a central part of the die like it was with GDDR5. With HBM it is pushed down vertically and that really helps reduce the latency. HBM has more channels and banks and that helps improve pseudo random access, which is critical to reducing latency for the HPC market."

Answer from a guy who works at Micron:
https://www.quora.com/In-how-many-clock-cycles-is-Samsung-s-High-Bandwidth-Memory-HBM2-faster-than-current-GDDR5-on-current-GPU-chipsets
"HBM latency is lower than GDDR primarily because HBM is not required to have a 7GHz interface. HBM statistically has low latency for random workloads because it has fewer bank group restrictions.
TLDR; HBM is lower latency than GDDR. Calculating how much is workload dependent."

I can't really find anything technical that indicates any kind of HBM latency issue, and really I don't expect one.

The tradeoff for HBM is cost, and flexibility. You pretty much need to be on a silicon interposer next to the device you are supplying.

So if you want to connect it to a CPU you have fixed your amount of memory and created a VERY expensive part.

I don't expect system ram packaged like this in the foreseeable future for general purpose PCs.

But I could see this as the basis for next generation Game Consoles (Sony PS5 and XBX2XLX3...).

In fact I give it high odds that at least one of the next generation consoles uses HBM exclusively for its system and graphics memory.

 

[whm1974]

While I often see someone claim HBM is higher latency, I googled around a lot for that, and most of my research indicates HBM is lower latency, at least lower than GDDR:

AMD Presentation talking about Latency improving features:
http://www.legitreviews.com/amd-high-bandwidth-memory-detailed-with-joe-macri_163855
"The PHY’s are laid out on the edges of the chips to keep the latency to a minimum. HBM does decrease the latency as data is for the most part no longer being moved horizontally to a central part of the die like it was with GDDR5. With HBM it is pushed down vertically and that really helps reduce the latency. HBM has more channels and banks and that helps improve pseudo random access, which is critical to reducing latency for the HPC market."

Answer from a guy who works at Micron:
https://www.quora.com/In-how-many-clock-cycles-is-Samsung-s-High-Bandwidth-Memory-HBM2-faster-than-current-GDDR5-on-current-GPU-chipsets
"HBM latency is lower than GDDR primarily because HBM is not required to have a 7GHz interface. HBM statistically has low latency for random workloads because it has fewer bank group restrictions.
TLDR; HBM is lower latency than GDDR. Calculating how much is workload dependent."

I can't really find anything technical that indicates any kind of HBM latency issue, and really I don't expect one.

The tradeoff for HBM is cost, and flexibility. You pretty much need to be on a silicon interposer next to the device you are supplying.

So if you want to connect it to a CPU you have fixed your amount of memory and created a VERY expensive part.

I don't expect system ram packaged like this in the foreseeable future for general purpose PCs.

But I could see this as the basis for next generation Game Consoles (Sony PS5 and XBX2XLX3...).

In fact I give it high odds that at least one of the next generation consoles uses HBM exclusively for its system and graphics memory.

I should clarify that what I meant by high latency is that HBM has more latency then normal desktop memory such as DDR4. Of course if am wrong about that, then I will accept being corrected.

 

[PeterScott]

I should clarify that what I meant by high latency is that HBM has more latency then normal desktop memory such as DDR4. Of course if am wrong about that, then I will accept being corrected.

I figured that is what you meant, but I could only find comparisons discussing GDDR.

But given the lack of any negative findings on HBM latency, and how it directly connections to device, I am thinking the current indicators are that it really doesn't have any latency issues. But I will likewise adjust my view according to any new evidence.

It does make for a compelling product idea, when you get an APU containing CPU/GPU packaged with all the system memory under once heat spreader. But really seems like it will be cost prohibitive for some time to come.

That and you have married your system ram to your CPU/APU.

If you want to upgrade your CPU, or GPU, or RAM, you have to ditch your old CPU, GPU and your old RAM and buy it all again.

Works great in a console which is treated like a sealed box, but not so great in a traditional PC.

 

[whm1974]

I figured that is what you meant, but I could only find comparisons discussing GDDR.

But given the lack of any negative findings on HBM latency, and how it directly connections to device, I am thinking the current indicators are that it really doesn't have any latency issues. But I will likewise adjust my view according to any new evidence.

It does make for a compelling product idea, when you get an APU containing CPU/GPU packaged with all the system memory under once heat spreader. But really seems like it will be cost prohibitive for some time to come.

That and you have married your system ram to your CPU/APU.

If you want to upgrade your CPU, or GPU, or RAM, you have to ditch your old CPU, GPU and your old RAM and buy it all again.

Works great in a console which is treated like a sealed box, but not so great in a traditional PC.

Considering that games will and are becoming more detailed as time passes, most gamers will prefer and need both the CPU and dGPU to each have their own separate pools of memory.

 

[PeterScott]

Considering that games will and are becoming more detailed as time passes, most gamers will prefer and need both the CPU and dGPU to each have their own separate pools of memory.

I agree, there are still issues beyond cost. What segment does it target?

Not low end because you will never justify the cost of unneeded HBM.

Not High end, because your die size/power usage would be off the charts.

So some kind of mid range solution that you can't upgrade, so laptop or speciality SFF. And even that seems to niche and expensive.

 

[whm1974]

I agree, there are still issues beyond cost. What segment does it target?

Not low end because you will never justify the cost of unneeded HBM.

Not High end, because your die size/power usage would be off the charts.

So some kind of mid range solution that you can't upgrade, so laptop or speciality SFF. And even that seems to niche and expensive.

The only suitable place I could see for an APU with HBM or even something like the Kaby-G is a game console or something similar.

 

[beginner99]

I agree, there are still issues beyond cost. What segment does it target?

Not low end because you will never justify the cost of unneeded HBM.

Not High end, because your die size/power usage would be off the charts.

So some kind of mid range solution that you can't upgrade, so laptop or speciality SFF. And even that seems to niche and expensive.

Right now it wouldn't target anything but ultimately this is also a software issue. APUs / iGPUs will only really be useful when they are used for "GPGPU" transparent to the user (user = programmer). But given current trends (SIMD, AVX-512) I doubt this will ever become reality and HSA sadly seems to essential be dead as we haven't heard about it in a long time. (Here we see the financial constraints of AMD, pouring everything into Zen thereby crippling their GPU and fusion efforts) EDIT: And that is probably why Raja went to Intel, besides internal political struggles.

Current iGPUs is just so highly inefficient. For most that actually use the iGPU, it's way, way overkill. Some facebook, office and email doesn't need much at all. For anybody that needs it, iGPUs are still anemic. So if you have the money you will buy a dGPU. This integration stopped half-way. They might be on the same die but else? CPU and GPU are still completely separate.

 

[MarkizSchnitzel]

Mobile 2500U is strong enough for very light gaming, which is fine for many (f.e. I just need Civ6 low 30FPS), can be put in thin and light laptops, and current models are significantly cheaper than i5+MX150. F.e. Zenbook 13 with i5+mx150+8+512 is 1200€, Envy x360with 2500U+8+512 AND touchscreen AND 2-in-1 AND better build is 900€. Best thing ever that my Zenbook 13 broke on the very first day. And it was GPU that broke.

So I very much see much point in APUs in laptops.

 

[whm1974]

Right now it wouldn't target anything but ultimately this is also a software issue. APUs / iGPUs will only really be useful when they are used for "GPGPU" transparent to the user (user = programmer). But given current trends (SIMD, AVX-512) I doubt this will ever become reality and HSA sadly seems to essential be dead as we haven't heard about it in a long time. (Here we see the financial constraints of AMD, pouring everything into Zen thereby crippling their GPU and fusion efforts) EDIT: And that is probably why Raja went to Intel, besides internal political struggles.

Current iGPUs is just so highly inefficient. For most that actually use the iGPU, it's way, way overkill. Some facebook, office and email doesn't need much at all. For anybody that needs it, iGPUs are still anemic. So if you have the money you will buy a dGPU. This integration stopped half-way. They might be on the same die but else? CPU and GPU are still completely separate.

Realistically I think that AMD figured out that with HSA and Fusion the drawbacks outweighed any benefits and getting them to work properly would take more resources then they could afford to spend.

 

[beginner99]

Realistically I think that AMD figured out that with HSA and Fusion the drawbacks outweighed any benefits and getting them to work properly would take more resources then they could afford to spend.

And what do you mean by drawbacks?

I think as said that the money went into Zen which makes sense. They need marketshare first or else no one will adopt HSA into their software. Albeit only the APU will profit for now. Therefore it would as PeterScott says make sense if Desktop zen also had a iGPU.

It's always the same with AMD. Good ideas but then lack the financial punch to go through with it.

 

[LightningZ71]

With so many low end mobile devices like laptops and 2 in 1s being hard or impossible to have a user replace/upgrade the RAM on, I wonder if the bottom end of the market would benefit from just having the APU and HBM all integrated in one unit? The product doesn't really require that any of it be upgradeable, and the vendor would rather they just bought a new one anyway. Honestly, with Windows 10 and Chrome OS, 4GB of DRAM is usable in a "light use" situation (and is a very typical configuration in atom and celeron laptops with their own iGPUs). If you went with an 8GB HBM stack, even if it was a low cost version, then it would still have packaging and throughput advantages over a regular DDR solution. With the package located to one end of the product, all the hot components would be located very close to the edge and no heat piping would be needed. Since the RAM is on package, there's no need for board area for running the traces for DRAM, it's just I/O that needs to get places. That also leaves more room for a big battery on the other end.

All that really needs to happen is that HBM itself and emib/mcm solutions come down in cost. With volume and experience, both will happen.

 

[PeterScott]

All that really needs to happen is that HBM itself and emib/mcm solutions come down in cost. With volume and experience, both will happen.

HBM may come down in price, but it will always be more complex and thus expensive than simple DDR. On the low end it's a total waste of the HBM bandwidth. There is no need/benefit for HBM, so no one is going to pay the premium.

HBM will remain a Mid-range and above solution, but you are back to needing a relatively big and expensive supersize APU to go with it, making the whole package very expensive.

Still I expect at least one of the next generation consoles will go this route, this will be the first place we could expect to see such a design around 2020.

It's a lot harder to make the case for trying to build such a product for a general purpose PC/Laptop. You need to be guaranteed you are going to sell many millions of them, but really it's a niche product.

 

[whm1974]

HBM may come down in price, but it will always be more complex and thus expensive than simple DDR. On the low end it's a total waste of the HBM bandwidth. There is no need/benefit for HBM, so no one is going to pay the premium.

HBM will remain a Mid-range and above solution, but you are back to needing a relatively big and expensive supersize APU to go with it, making the whole package very expensive.

Still I expect at least one of the next generation consoles will go this route, this will be the first place we could expect to see such a design around 2020.

It's a lot harder to make the case for trying to build such a product for a general purpose PC/Laptop. You need to be guaranteed you are going to sell many millions of them, but really it's a niche product.

As far as general purpose PC/Laptops go, I think that is something only Apple could manage to sell enough of to bother doing.

 

[William Gaatjes]

HBM may come down in price, but it will always be more complex and thus expensive than simple DDR. On the low end it's a total waste of the HBM bandwidth. There is no need/benefit for HBM, so no one is going to pay the premium.

HBM will remain a Mid-range and above solution, but you are back to needing a relatively big and expensive supersize APU to go with it, making the whole package very expensive.

Still I expect at least one of the next generation consoles will go this route, this will be the first place we could expect to see such a design around 2020.

It's a lot harder to make the case for trying to build such a product for a general purpose PC/Laptop. You need to be guaranteed you are going to sell many millions of them, but really it's a niche product.

I sometimes get the impression you have something against apu's and HSA in general and against HBM.
HBM2 has pseudochannels that with a good memory controller can alleviate a lot of the issues first gen HBM has.
And the cost is an issue, yes. It would be nice to see a SOC with hbm(2). It makes sense that the consoles get there first.
However, a laptop or 2 in 1 would be nice, but what is needed a killer application that can make proper use of HSA.
Maybe a in the near future surface pro with some software from the kitchen from microsoft could help it.
But there must be a killer application or it will be deemed useless. Just for gaming will not be sufficient.
Although a 2 in1 laptop that can connect to a gaming console with similar hardware might help in augmented reality.
But the killer application and support from a big player is what makes hardware shine.
Augmented reality or some neural network software that makes the devices work together nicely and very cooperative.
At least 7nm is needed for that.
And then telephones. Low power, Package on package is possible because of the lower power consumption.

 

[LightningZ71]

My business case is that such a product can extend across several market segments. The APU/HBM package can be cost optimized in tiers, from the weakest APU with a small, low cost HBM 4GB package to a high end CPU, on package dgpu and 2 HBM sacks (one for main ram, and one dedicated to the dgpu. All that's needed is a common package size. The platform itself needs only have one motherboard that just handles I/O, which, in volume is manageable in cost. Thermal solutions can be passive on the low end, to larger and dual fan on the high end, as needed. The chassis can have a standardized frame with different levels of quality to the outer shell depending on the market. It will make the lowest end of the products slightly more expensive, but as the platform can be shared with the high end, it makes that less expensive as volume reduces the unit cost. It's up to AMD to offer a family of APU to CPU/dGPU packages to that market.

There's a way to do this with efficient cost, it just takes a big picture view. In the end, everyone gets a better product for it. The only drawback that I see is that memory configuration stays fixed. This will be undesirable for enthusiast parts, but they make up a vanishingly small part of the market.

This post is referencing the idea that a case can be made for a CPU/GPU/HBM package that has all it's RAM on the package and no user addable module slots on the motherboard. You save cost elsewhere to move it into the package. This makes for a consistent motherboard between models (with some headers that may go unused in different models) and saves on development costs.

 

[PeterScott]

I sometimes get the impression you have something against apu's and HSA in general and against HBM.

You are lumping three things that you think I am against, they aren't all the same so I will address them individually:

APUs: Far from it. I actually love APUs, and consider them the biggest, most important part of the market, and with Ryzen/Vega, AMD has the potential to build great APUs. But, I don't let my desires of what I would like to see built, influence my views on what is most likely actually going to be built. IOW, I fight hard to stay away from wishful thinking. As much as I wanted to see an APU with 8 cores, and/or a iGPU that would obsolete a GTX 1050, I had no expectation of such, because there are problem with the cost of the larger die, and with memory bandwidth. In short: I Love APUs, and would Love such powerful APUs. But I just don't see a business case outside of Consoles.

HBM: Again, not even close. I think HBM is great stuff from technical standpoint, but it is very expensive to implement (definitive fact) and much like GDDR never got down to DDR prices, HBM won't either. So again Love HBM, and HBM-Super APU system would be awesome, but again, I can't see a business case outside of Consoles.

HSA: Here I am very skeptical of the hype surround HSA. It looks more like a far future possibility. Especially if this is about having compilers that seamlessly shifts workloads between CPU/GPU portions of chip. AMD was hyping this more years ago. I didn't hear anything about HSA with the Raven Ridge launch. I think AMD is realizing this kind of sharing is more of a "maybe, someday" than anything significant in the near term.

 

[whm1974]

Speaking of APUs, I think at best AMD might produce a 6c/12t with 8MB of L3 cache and with an iGPU halfway between the 1030 and 1050. But that will most likely would require both DDR5 and more efencent use of memory bandwidth.

 

[William Gaatjes]

You are lumping three things that you think I am against, they aren't all the same so I will address them individually:

APUs: Far from it. I actually love APUs, and consider them the biggest, most important part of the market, and with Ryzen/Vega, AMD has the potential to build great APUs. But, I don't let my desires of what I would like to see built, influence my views on what is most likely actually going to be built. IOW, I fight hard to stay away from wishful thinking. As much as I wanted to see an APU with 8 cores, and/or a iGPU that would obsolete a GTX 1050, I had no expectation of such, because there are problem with the cost of the larger die, and with memory bandwidth. In short: I Love APUs, and would Love such powerful APUs. But I just don't see a business case outside of Consoles.

HBM: Again, not even close. I think HBM is great stuff from technical standpoint, but it is very expensive to implement (definitive fact) and much like GDDR never got down to DDR prices, HBM won't either. So again Love HBM, and HBM-Super APU system would be awesome, but again, I can't see a business case outside of Consoles.

HSA: Here I am very skeptical of the hype surround HSA. It looks more like a far future possibility. Especially if this is about having compilers that seamlessly shifts workloads between CPU/GPU portions of chip. AMD was hyping this more years ago. I didn't hear anything about HSA with the Raven Ridge launch. I think AMD is realizing this kind of sharing is more of a "maybe, someday" than anything significant in the near term.

HSA is supported by ARM as well. And most modern graphic cards, even discrete ones are able to do at least one feature of HSA , and that is zero copy. Nvidia cards of course as well.
The VEGA gpu memory controller can page in and out virtual memory just like a cpu can.
That is another step closer to an fully HSA compliant system even with discrete cards.
Although apus have lower latency since the memory is shared and cpu and gpu are on the same die. There are a few modern telephones socs that can do HSA as well.
Because of having multiple cpu on die and a gpu with shared memory.
But yeah, os support and a killer application is needed.

Nice link (yes, 5 year old) :
http://www.electronicdesign.com/microprocessors/heterogeneous-system-architecture-changes-cpugpu-software

 

[PeterScott]

Speaking of APUs, I think at best AMD might produce a 6c/12t with 8MB of L3 cache and with an iGPU halfway between the 1030 and 1050. But that will most likely would require both DDR5 and more efencent use of memory bandwidth.

Of course, by the time it gets implemented, it will be somewhere behind an 1130, and games will have become more demanding again, so in the end you made no progress relative to the market. This is the never-ending issue with mainstream APUs. '

6 cores would be great though, but I wonder if AMD is going to do a 6 core CCX, or is it just going to end up adding more CCX units for higher core counts in Ryzen. If it's the later, it will either be 4 core or 8 cores APU's.

 

[whm1974]

Of course, by the time it gets implemented, it will be somewhere behind an 1130, and games will have become more demanding again, so in the end you made no progress relative to the market. This is the never-ending issue with mainstream APUs. '

6 cores would be great though, but I wonder if AMD is going to do a 6 core CCX, or is it just going to end up adding more CCX units for higher core counts in Ryzen. If it's the later, it will either be 4 core or 8 cores APU's.

Come to to think of it, considering that AMD sells both dGPU and APUs/CPUs, it is probably not in their best interest to design a powerful APU as that would be expensive to produce and they would make more money with CPUs with higher core counts and dGPUs anyway.

 

[Cerb]

Come to to think of it, considering that AMD sells both dGPU and APUs/CPUs, it is probably not in their best interest to design a powerful APU as that would be expensive to produce and they would make more money with CPUs with higher core counts and dGPUs anyway.

It would make sense for NUC-like form factors and notebooks. They could likely sell many of them to OEMs at lower costs than cards, and make bank...if the relative cost of an interposer and HBM is low enough. It's not the same kind of comparison on a CPU socket as it is compared to an entire video card. OTOH, they could also try pulling an Intel w/ L4, using GF's eDRAM, though I don't know what the leakiness is like, these days. But, again, cost.

 

[PeterScott]

It would make sense for NUC-like form factors and notebooks. They could likely sell many of them to OEMs at lower costs than cards, and make bank...if the relative cost of an interposer and HBM is low enough. It's not the same kind of comparison on a CPU socket as it is compared to an entire video card. OTOH, they could also try pulling an Intel w/ L4, using GF's eDRAM, though I don't know what the leakiness is like, these days. But, again, cost.

Many, like dozens? This game is all about design wins.

OEMs are looking to cost reduce where ever possible. These are the people who routinely only equip one channel of DDR to ship at lower cost. Do you you think you are going to get design wins with needlessly expensive part? System memory doesn't need the benefits of HBM. They are just wasted expensive overkill for that task.

HBM is NEVER going to get down to basic DDR prices. Using it for system memory is going to be a wasted expense in the PC market.

HBM as system memory is just not in the foreseeable future in the PC market.

At best HBM can be a replacement for also expensive GDDR for GPUs. It's still more expensive that GDDR, but at least for GPU memory it offers significant benefits (higher BW, lower latency, lower power) that make a difference.

But even then, it definitely is costing AMD/partners more to build their GPU cards than NVidia and HBM is a part of that, so there is less profit in AMD GPUs.

Bottom line even if you build a Super APU with HBM, the HBM will be for the iGPU, and it will use DDR for system memory. That would be the most cost effective option, and allow for memory upgrades.

 

[Cerb]

Many, like dozens? This game is all about design wins.

If it's only dozens, I wouldn't imagine the hypothetical competition to be in the cages of Best Buys (MSI and Asus gaming notebooks).

OEMs are looking to cost reduce where ever possible. These are the people who routinely only equip one channel of DDR to ship at lower cost. Do you you think you are going to get design wins with needlessly expensive part? System memory doesn't need the benefits of HBM. They are just wasted expensive overkill for that task.

System memory can't compete with HBM or GDDR5, and if they are looking to reduce costs, which is going to be cheaper: a CPU package and a GPU+GDDR5/HBM on that package, or just a CPU package? If it's cheap enough to add accessible fast memory to the CPU package, that could be cheaper. I'm not sure if it would be, or not. I also don't see how anything is needless, though, unless you already know just how much the relative costs are.

But even then, it definitely is costing AMD/partners more to build their GPU cards than NVidia and HBM is a part of that, so there is less profit in AMD GPUs.

Both companies use HBM and DDR, and both are making new GPUs to support either, as well. AMD's popular gaming models don't use HBM, generally, right now. HBM's primary market being GPUs, with little else driving demand, is a negative for its long-term prices, though. Unless something unpredictable happens with new GDDR, or if HMC takes off, or unless Hynix can lower the cost to help improve adoption...not sure that part will change too much. But, RAM prices as they are, they could also just compress and get close enough to each other, over time.

 

[whm1974]

System memory can't compete with HBM or GDDR5, and if they are looking to reduce costs, which is going to be cheaper: a CPU package and a GPU+GDDR5/HBM on that package, or just a CPU package? If it's cheap enough to add accessible fast memory to the CPU package, that could be cheaper. I'm not sure if it would be, or not. I also don't see how anything is needless, though, unless you already know just how much the relative costs are.

In order for HBM to be low cost enough to replace DDRx for system memory, they will need to find a way to get rid of the interposer(can't spell) that HBM and be able to just put the memory on the CPU package itself. Even then you will still need to have enough memory available for consumers to consider buying systems using this SuperAPU.

This is assuming that the current separate CPU, dGPU, and system memory model isn't still going to be prefer by consumers.

 

[PeterScott]

System memory can't compete with HBM or GDDR5, and if they are looking to reduce costs, which is going to be cheaper: a CPU package and a GPU+GDDR5/HBM on that package, or just a CPU package? If it's cheap enough to add accessible fast memory to the CPU package, that could be cheaper. I'm not sure if it would be, or not. I also don't see how anything is needless, though, unless you already know just how much the relative costs are.

System memory CPU performance impact, is already at the point of diminishing return with DDR, adding HBM is just overkill and thus: Pointless/Needless overspending. It's VERY POOR bang/buck.

For price, it's DDR<GDDR<HBM, and will most likely always be that way, though the delta may shrink.

The rumors thus far are that NVidia is going to GDDR6 for it's next generation GPUs, which implies that HBM will still be significantly more expensive than even GDDR6, because this is the one area where HBM would have a potentially significantly performance benefit and it still isn't used. If there is one place HBM can deliver bang/buck, this should be it. But it really isn't even delivering enough bang/buck here yet.

IMO AMD uses HBM on Vega, because it is behind in both performance and power usage. HBM was a Hail Mary to try and catch NVidia on their top cards, that didn't quite deliver.

HBM will dominate ALL GPUs, midrange and above, LONG before anyone would even consider using it as system memory in a PC. This is the First logical migration for HBM.

But it looks like we are many years before HBM dominates GPUs, so system memory isn't really in the foreseeable future.