Intel releases their own "mesh" version of AMD's Infinity Fabric

[swilli89]

http://www.tomshardware.com/news/intel-mesh-architecture-skylake-x-hedt,34806.html

Looks like Intel is debuting a mesh version of their own similar to AMD's innovative and first to market Infinity Fabric.

However upon further inspection of the die it looks to still appear as a ring and is in conflict with the block diagram Intel provided.

Is it possible Intel's marketing folks scrambled to try to compete with AMD's Infinity Fabric by just rebranding the ring bus? Die shots indicate this could be true and that this is merely just a new version of the ring bus that is different enough to brand it as intel mesh.

What do you guys think about the intel mesh?

 

[tamz_msc]

I think it is a stretch to call Intel's solution similar to AMD's Infinity Fabric. It has a name: MoDe-X: Modular Decoupled Crossbar, and it debuted in KL. Somebody with access to the paper can read it and share the details.

EDIT: KL = Knight's Landing

 

[swilli89]

I think it is a stretch to call Intel's solution similar to AMD's Infinity Fabric. It has a name: MoDe-X: Modular Decoupled Crossbar, and it debuted in KL. Somebody with access to the paper can read it and share the details.

Seeing as KL had identical IPC to skylake does it mean that it is really intended for the higher core counts? Why would Intel upgrade a 4-core die with this if the performance is the same?

I'm finding core to core interconnects very intriguing.

 

[tamz_msc]

Seeing as KL had identical IPC to skylake does it mean that it is really intended for the higher core counts? Why would Intel upgrade a 4-core die with this if the performance is the same?

I'm finding core to core interconnects very intriguing.

Whoops 'KL' means Knight's Landing in this case.

 

[swilli89]

Whoops 'KL' means Knight's Landing in this case.

Oh well in that case it makes much more sense

 

[JoeRambo]

I think it is a stretch to call Intel's solution similar to AMD's Infinity Fabric. It has a name: MoDe-X: Modular Decoupled Crossbar, and it debuted in KL. Somebody with access to the paper can read it and share the details.

Thanks, that is a real good paper on topic.

 

[moinmoin]

What do you guys think about the intel mesh?

Certainly an interesting development.

The advantages of IF are mainly two: scalability wrt core count and memory speed, and scalability wrt combining multiple ICs. Intel's ring bus introduced a latency that increased with its size and couldn't profit of higher memory speed, the decoupled crossbar mesh appears to tackle that issue. IF's latter advantage doesn't appear to be touched by this.

Now the question is if Intel even wants to tackle the latter disadvantage. While it's a huge TTM advantage for AMD and its limited work force and R&D budget, Intel may be perfectly content to continue designing many different custom dies and use the resulting market segregation. That approach may be negative for propagating microarchitecture and process node improvements across the whole product portfolio, but we'll have to see how fast Intel and AMD are at doing that once 10nm respectively 7LP arrive.

 

[swilli89]

Certainly an interesting development.

The advantages of IF are mainly two: scalability wrt core count and memory speed, and scalability wrt combining multiple ICs. Intel's ring bus introduced a latency that increased with its size and couldn't profit of higher memory speed, the decoupled crossbar mesh appears to tackle that issue. IF's latter advantage doesn't appear to be touched by this.

Now the question is if Intel even wants to tackle the latter disadvantage. While it's a huge TTM advantage for AMD and its limited work force and R&D budget, Intel may be perfectly content to continue designing many different custom dies and use the resulting market segregation. That approach may be negative for propagating microarchitecture and process node improvements across the whole product portfolio, but we'll have to see how fast Intel and AMD are at doing that once 10nm respectively 7LP arrive.

Intel is definitely at the point where they don't know how to allocate their massive R&D budget, there is surely a point of diminishing returns. I think this is evidenced by the fact that they were willing to spend billions on trying to force their way into the mobile market, while having the world's "best" fabrication process and yet still somehow failing.

 

[tamz_msc]

Intel is definitely at the point where they don't know how to allocate their massive R&D budget, there is surely a point of diminishing returns. I think this is evidenced by the fact that they were willing to spend billions on trying to force their way into the mobile market, while having the world's "best" fabrication process and yet still somehow failing.

Well they had to go with something other than ring bus because it was beginning to hurt them in latency in 22+ core Broadwell-E.

 

[moinmoin]

Intel is definitely at the point where they don't know how to allocate their massive R&D budget, there is surely a point of diminishing returns. I think this is evidenced by the fact that they were willing to spend billions on trying to force their way into the mobile market, while having the world's "best" fabrication process and yet still somehow failing.

To be fair Intel trying to force itself into other markets is a whole different story. They depend way too much on their (up to now) near monopoly in the x86 market as their main source of income so they keep trying to expand into other markets. But those efforts are regularly and predictably cut short since there's no hope they manage to reach the insane margin they have in their core market. Maybe now AMD can actually help Intel dropping their margin far enough that efforts in other markets are actually seen as potentially fruitful by the stakeholders (lol).

 

[LaikaSpaceCat]

http://www.tomshardware.com/news/intel-mesh-architecture-skylake-x-hedt,34806.html

Looks like Intel is debuting a mesh version of their own similar to AMD's innovative and first to market Infinity Fabric.

However upon further inspection of the die it looks to still appear as a ring and is in conflict with the block diagram Intel provided.

Is it possible Intel's marketing folks scrambled to try to compete with AMD's Infinity Fabric by just rebranding the ring bus? Die shots indicate this could be true and that this is merely just a new version of the ring bus that is different enough to brand it as intel mesh.

What do you guys think about the intel mesh?

Considering Knight's Landing had this mesh, I doubt Skylake-X's mesh is just marketing.
There are many structures there I can see that could act as horizontal connections, which would already be very different from ring bus. There's also how the memory controllers are on the left and right instead of being together at the bottom, which is something you would expect from the mesh.

And honestly, the mesh and Infinity Fabric appear to be very different solutions to the same problem. IF is more akin to the love child of ring bus and mesh together rather than either by itself.




Stop creating new accounts. You have one account and one account only.


esquared
Anandtech Forum Director

 

[Phynaz]

http://www.tomshardware.com/news/intel-mesh-architecture-skylake-x-hedt,34806.html

Looks like Intel is debuting a mesh version of their own similar to AMD's innovative and first to market Infinity Fabric.

However upon further inspection of the die it looks to still appear as a ring and is in conflict with the block diagram Intel provided.

Is it possible Intel's marketing folks scrambled to try to compete with AMD's Infinity Fabric by just rebranding the ring bus? Die shots indicate this could be true and that this is merely just a new version of the ring bus that is different enough to brand it as intel mesh.

What do you guys think about the intel mesh?

I think you are missing two points:

1. Infinity Fabric is at its most basic is Hyper Transport 4. It's not magic.
2. Intel has had a mesh interconnect for years. It's a logical expansion of their ring bus.

Edit:
Mesh interconnect used in the MIT Tile Processor in 2007.
https://www.princeton.edu/~wentzlaf/documents/Wentzlaff.2007.IEEE_Micro.Tilera.pdf

Mesh Interconnect used in the Intel TeraFlops processor, 2007:
http://www.springer.com/cda/content...1902627-c1.pdf?SGWID=0-0-45-793318-p173905509

 

[maddie]

I think you are missing two points:

1. Infinity Fabric is at its most basic is Hyper Transport 4. It's not magic.
2. Intel has had a mesh interconnect for years. It's a logical expansion of their ring bus.

Edit:
Mesh interconnect used in the MIT Tile Processor in 2007.
https://www.princeton.edu/~wentzlaf/documents/Wentzlaff.2007.IEEE_Micro.Tilera.pdf

Mesh Interconnect used in the Intel TeraFlops processor, 2007:
http://www.springer.com/cda/content...1902627-c1.pdf?SGWID=0-0-45-793318-p173905509

Does this new Intel mesh connect across multiple die and also additionally across sockets like IF? They seem to have quite different attributes to me, one being a sort of subset of the other.

 

[itsmydamnation]

I think you are missing two points:

1. Infinity Fabric is at its most basic is Hyper Transport 4. It's not magic.
2. Intel has had a mesh interconnect for years. It's a logical expansion of their ring bus.

Edit:
Mesh interconnect used in the MIT Tile Processor in 2007.
https://www.princeton.edu/~wentzlaf/documents/Wentzlaff.2007.IEEE_Micro.Tilera.pdf

Mesh Interconnect used in the Intel TeraFlops processor, 2007:
http://www.springer.com/cda/content...1902627-c1.pdf?SGWID=0-0-45-793318-p173905509

Your overstating one and understating the other, IF is far more then just HT4.0, its decoupled control and data, its not just point to point like HT was, amd have stated IF can and does (in vega) scale to 100's of nodes, on naples it looks like its around 100 . What it does build on from HT is the coherency model (MOESI), directory cache and packet format.

I personally dont think one is better then the other they are just different. There is no doubt that Intels mesh likely has insane amounts of bandwidth at good latency that will be great for things like HPC. AMD's fabric allows them to build something once and then interconnect it to anything else that they built once. AMD have already stated IF takes interconnect changes down from months of effort to hours. That is what is going to allow them to compete across almost every x86 and GPU market with 1/10th of the man power.

I think what most people miss is that the very vast majority of servers dont need the things intel is bringing, cloud, web farms, vm farms, dont really care about avx-512 or massive inter core bandwidths ( you try to keep core counts low per VM to reduce scheduling contention) and you tend to be memory throughput bound not cache/core sharing bound. AMD is targeting the bread and butter, the general 80% of the market, given how close skylake and zen are in IPC, its going to be a Cores/clock speed/sku race.

At the place im currently doing work for per server the single biggest lifetime cost is our hypervisor/"hyperscale" licencing. We could get more memory bandwidth and the same compute performance in 1P with naples which would 1/2 that cost while also having a reduced platform cost or we could go the other way, and over double our memory/compute foot print at the same licencing cost.

We wont be multi vendoring our farm's but it is food for thought.

 

[maddie]

AMD has taken the Pareto rule to an extreme.

 

[IntelUser2000]

The two are very different. Infinity Fabric will be easier to scale and get new products out, but the mesh fabric on Skylake-X and -SP will have better performance. The mesh fabric is more customized meaning design cycles are longer and its not as portable. But they seem to be willing to use diverged structures for PC and Server, so if it works for them, they will continue to do so.

-Performance
-TTM
-Cost

You always get to choose only two of them. Of course you can really optimize for one area so you get just one. But you will never get three.

 

[w3rd]

I think you are missing two points:

1. Infinity Fabric is at its most basic is Hyper Transport 4. It's not magic.
2. Intel has had a mesh interconnect for years. It's a logical expansion of their ring bus.

Edit:
Mesh interconnect used in the MIT Tile Processor in 2007.
https://www.princeton.edu/~wentzlaf/documents/Wentzlaff.2007.IEEE_Micro.Tilera.pdf

Mesh Interconnect used in the Intel TeraFlops processor, 2007:
http://www.springer.com/cda/content...1902627-c1.pdf?SGWID=0-0-45-793318-p173905509

All the same, right..?


Not to be rude, but I really do think you are the one missing the point. HSA is something AMD has been investing in, and moving towards for seven years now. This "fabric" is just not something thought up, it has been in the making for a long time, with a lot of forward thinking engineering going on. And I think a few patents too..

We are starting to see the first iterations of AMD's "infinity fabric" being laid out across it's entire portfolio and CPU & GPU, etc. (This is gen 1).




I see Intel's "mesh"... as a new universal interconnect. Acting as a bus for cross-talk, & back-talk I/O & IOPS. To overcome some known limitation in it's current designs, etc.

I am more than looking forward to seeing how this reduces system latencies, etc. And how it offers greater internal bandwidth, etc.

 

[CHADBOGA]

All the same, right..?


Not to be rude, but I really do think you are the one missing the point. HSA is something AMD has been investing in, and moving towards for seven years now. This "fabric" is just not something thought up, it has been in the making for a long time, with a lot of forward thinking engineering going on. And I think a few patents too...

Interesting you mention HSA, I have been thinking for a few weeks now that Infinity Fabric is the "new HSA".

 

[ub4ty]

All the same, right..?


Not to be rude, but I really do think you are the one missing the point. HSA is something AMD has been investing in, and moving towards for seven years now. This "fabric" is just not something thought up, it has been in the making for a long time, with a lot of forward thinking engineering going on. And I think a few patents too..

We are starting to see the first iterations of AMD's "infinity fabric" being laid out across it's entire portfolio and CPU & GPU, etc. (This is gen 1).

Good reflections w.r.t to relating infinity fabric to their longer term HSA plans. This is indeed the connection and the path towards it.
Infinity Fabric is a far more elegant and wide ranging solution to multi and many core solutions. Infinity fabric is an elegant pipe from one place to another. Once a transmission falls out of the big pipe (infinity fabric), then you can decide on how to more effectively and even proprietarily pipe it locally. As such, (infinity fabric) can be used across AMD's CPU/GPU line and it can now also open it up to third party venders. Imagine a future in which AMD exposes infinity fabric externally and you now have memory/accelerators directly connected to the CPU as if it were another CPU socket as in 2-socket naples...

Not to mention, people are ignoring that this design allows for (local) piping. Local CPU cores to a CCX incur ~40ns latency. Whereas the latency from core to core is ~100ns with intel. Also, since AMD is running the infinity fabric in relation to DRAM speeds, the latency of infinity fabric communication drops down from 140ns to 110/120ns with higher speed ram. So, in Ryzen 8 Core/16thread, you're incurring 40ns latency for 4cores/8threads. You only hit 110/120ns when going across the fabric interconnect. Not to mention that they're likely using less power to do so. A fully meshed design that doesn't use an aggregation pipe is hella power intensive.

I see Intel's "mesh"... as a new universal interconnect. Acting as a bus for cross-talk, & back-talk I/O & IOPS. To overcome some known limitation in it's current designs, etc.

I am more than looking forward to seeing how this reduces system latencies, etc. And how it offers greater internal bandwidth, etc.

I too look forward to more details coming out on both. I find both approaches to be very interesting/intriguing. However, I do note, like you that AMD is playing a longer term game here and is doing so quite exceptionally even on present-day performance vs. mesh.

I have been doing some digging and it seems few people except for HPC really dig into micro-architectures and link them to real world performance. It will be interesting to read a detailed writeup on how these unique micro-architectural designs map all the way up to the real-time performance one sees. What people often ignore nor detail in enthusiasts benchmarks is the complexity at runtime w.r.t to all sorts of latencies that factor in to the picture that way out-scale 100ns. Thread data lock latency, thread starvation, context switching, cache misses, kernel call, user space to kernel space switching, etc are all real-world phenomena when you're running an OS and an application within it... 200 processes with 1000 threads isn't unheard of for an 'idle' system. Those threads and processes don't just magically and instantly begin running on the CPU and there's latency involved when they do (tremendous latency)