Question Arm-based Alibaba Cloud T-Head Yitian 710 Crushes SPECrate2017_int_base

[mikegg]

Arm-based Alibaba Cloud T-Head Yitian 710 Crushes SPECrate2017_int_base

The Arm-based Alibaba Cloud T-Head Yitian 710 128 core DDR5 and PCIe Gen5 CPU submitted an official top 1P SPECrate2017_int_base score

www.servethehome.com

One reason why I'm bearish on AMD and Intel long-term. Bullish on Amazon, Ampere, Apple, Qualcomm and others who adopt ARM chips. However, I'm actually bearish on ARM, the company itself.

 

[IntelUser2000]

Arm-based Alibaba Cloud T-Head Yitian 710 Crushes SPECrate2017_int_base

The Arm-based Alibaba Cloud T-Head Yitian 710 128 core DDR5 and PCIe Gen5 CPU submitted an official top 1P SPECrate2017_int_base score

www.servethehome.com

One reason why I'm bearish on AMD and Intel long-term. Bullish on Amazon, Ampere, Apple, Qualcomm and others who adopt ARM chips. However, I'm actually bearish on ARM, the company itself.

It only has 16% advantage over AMD Milan and in just a few days will be replaced by Genoa that's 75% faster than the predecessor. This is despite only having 96 cores so the single thread advantage is significant which makes it a better general purpose server CPU. Bergamo will be a better competitor to ARM chips because it'll offer 2x the performance by having 128 cores, but each core will perform less than Genoa does.

 

[mikegg]

It only has 16% advantage over AMD Milan and in just a few days will be replaced by Genoa that's 75% faster than the predecessor. This is despite only having 96 cores so the single thread advantage is significant which makes it a better general purpose server CPU. Bergamo will be a better competitor to ARM chips because it'll offer 2x the performance by having 128 cores, but each core will perform less than Genoa does.

Performance/core doesn't matter much.

In data centers, it's minimum acceptable ST speed (which ARM designs meet), then it's all about perf/watt, perf/socket, perf/cost.

ARM is currently winning in all 3 categories.

Take for example, a top binned 64-core Zen4 Epyc will cost $7k+. When hyperscalers like AWS and Alibaba manufacture their own chips based on mostly ARM core designs, they're cutting out the middle man. AMD's gross margins are 48%. For their cloud chips, their gross margins are probably 80%+. If AWS and Alibaba can save 80% on their chips it's no wonder they're pushing so hard to skip AMD and Intel altogether. Not to mention ARM chips run at roughly 100w while Epyc runs at 280w.

AWS's Graviton2 accounted for 50% of all new EC2 instances back in 2020. Graviton3 must be close to 80% by now. The market will be shrinking fast for x86 cloud CPUs in the next 2-3 years, just like how it shrank in the personal computer space.

I'm a software engineer. All my cloud instances are currently running on AWS Graviton2 or Graviton3 because they meet my performance standards, and they're cheaper than equivalent Intel/AMD based instances.

There are a lot of AMD [redacted] here because no one here has built a custom master race PC with an ARM chip yet. Everyone here is just trying to justify their AMD/Intel chips to make themselves feel better. It was the same ordeal back when we were discussing the rumored first-generation Apple mac chips in 2019/2020. A ton of AMD/Intel/x86 [redacted] here were denying that Apple could pull it off.

The use of "fanboys" or other insults are not allowed in the tech forums.

AT Mod Usandthem

You also are not allowed to edit out moderator comments/warnings either.

AT Mod Usandthem

 

[gdansk]

ARM has so many more options. This variety of designs means their ISA is destined to win the server space. But only if they don't shoot themselves in the foot with lawsuits.
AMD is doing great with x64 but they'll eventually settle on being just another ARM vendor when the market switches over.
Unfortunately for AMD and Intel the multi-trillion-dollar vertical monopolists demand there must be no 3rd party taking a margin.

 

[mikegg]

ARM has so many more options. This variety of designs means their ISA is destined to win the server space. But only if they don't shoot themselves in the foot with lawsuits.
AMD is doing great with x64 but they'll eventually settle on being just another ARM vendor when the market switches over.
Unfortunately for AMD and Intel the multi-trillion-dollar vertical monopolists demand there must be no 3rd party taking a margin.

Or cloud companies are tired of the duopoly of AMD/Intel. At any given time, one of them owns the monopoly over x86 cloud chips. Currently, AMD is on its way to becoming the x86 cloud monopoly. If Intel climbs back and wins again, it'll be Intel. ARM gives hyperscalers and smaller cloud companies many more options.

AWS controls its own destiny with Graviton. Other hyperscalers are seemingly doing the same thing such as Chinese hyperscalers. Microsoft and Google are behind but they also have plans for custom ARM chips.

 

[gdansk]

Or cloud companies are tired of the duopoly of AMD/Intel. At any given time, one of them owns the monopoly over x86 cloud chips. Currently, it's AMD. If Intel climbs back and wins again, it'll be Intel. ARM gives hyperscalers and smaller cloud companies many more options.

We'll see. There's a place for specialized companies designing server chips. But ARM made server cores a commodity. You can't win when all your biggest customers having the option to buy things at cost. So I think all but a few companies focusing on designing server CPUs will be acquired or bankrupted by vertical monopolies over the years. With our present market rules serious competition is temporary.

 

[mikegg]

We'll see. There's a place for specialized companies designing server chips. But ARM made server cores a commodity. You can't win when all your biggest customers having the option to buy things at cost. So I think all but a few companies focusing on designing server CPUs will be acquired or bankrupted by vertical monopolies over the years. With our present market rules serious competition is temporary.

Hence, I named Qualcomm as a possible future winner because they're designing custom ARM cores via Nuvia, rather than just relying on stock ARM designs. Ampere is also designing custom cores. Obviously, Apple is too but for the consumer market.

I can see AWS/Alibaba designing custom cores to further differentiate.

I believe competition is here to stay. But instead of AMD vs Intel, it'll be AMD vs Intel vs AWS vs Alibaba vs Tencent vs Microsoft vs Google vs Ampere vs Nvidia vs Apple vs ARM vs Mediatek vs Qualcomm.

I think this forum needs to change its mindset that it's still only AMD vs Intel like it was in 2006.

 

[gdansk]

Hence, I named Qualcomm as a possible future winner because they're designing custom ARM cores via Nuvia, rather than just relying on stock ARM designs. Ampere is also designing custom cores. Obviously, Apple is too but for the consumer market.

I can see AWS/Alibaba designing custom cores to further differentiate.

I believe competition is here to stay. But instead of AMD vs Intel, it'll be AMD vs Intel vs AWS vs Alibaba vs Tencent vs Microsoft vs Google vs Ampere vs Nvidia vs Apple vs ARM vs Mediatek vs Qualcomm.

I think this forum needs to change its mindset that it's still only AMD vs Intel like it was in 2006.

I think you're way too optimistic. Only a few will survive in the server space outside of China.

And a lot of us cannot be bothered to care. Seldom do they offer products we can actually buy. It is entirely understandable that when all they offer is the option to be a serf renting a lot that they don't attract much attention among the "DIY community".

 

[mikegg]

I think you're way too optimistic. Only a few will survive in the server space outside of China.

And a lot of us cannot be bothered to care. Seldom do they offer products we can actually buy. It is entirely understandable that when all they offer is the option to be a serf renting a lot that they don't attract much attention among the "DIY community".

You'll have to be more specific about my over optimism.

The "DIY" mindset is exactly what needs to change here in order to take ARM more seriously. Apple Silicon made a huge impact here. It'll happen in server CPUs too.

 

[gdansk]

You'll have to be more specific about my over optimism.

The "DIY" mindset is exactly what needs to change here in order to take ARM more seriously. Apple Silicon made a huge impact here. It'll happen in server CPUs too.

That list of companies is a very temporary state of affairs. There is nothing that capitalists/investors abhor more than actual competition.

And no, the DIY attitude doesn't need to change at all. The first one to offer a compelling ARM product we can actually own, install the software and secondary hardware how we want is who we will reward, I hope.

Maybe RPi will scale up to that level at some point. It is certainly more interesting to me than for-rent, newage big iron.

Posted from my M1 Pro MacBook.

 

[mikegg]

That list of companies is a very temporary state of affairs. There is nothing that capitalists/investors abhor more than actual competition.

Sure. Maybe one company, such as Qualcomm or Ampere, will create an ARM chip that is so good, AWS and Alibaba will have to buy from them. That could happen. No denying that.

As a software engineer, I don't directly use x86 anymore. My computer is an M1 Pro Macbook. My phone uses ARM. My AWS server instances are using Graviton. My watch/tablet are also ARM.

 

[gdansk]

Sure. Maybe one company, such as Qualcomm or Ampere, will create an ARM chip that is so good, AWS and Alibaba will have to buy from them. That could happen. No denying that.

I'm sure you're aware of the ongoing Qualcomm-ARM lawsuit wherein ARM demands that Qualcomm cease working on Nuvia-derived IP and destroy it.

As a software engineer, I don't directly use x86 anymore. My computer is an M1 Pro Macbook. My phone uses ARM. My AWS server instances are using Graviton. My watch/tablet are also ARM.

Another software engineer with opinions on hardware? Me too! I do use x64 every day and while I recognize that is temporary it doesn't mean I'm going to start liking CPUs that are beyond mere mortal ownership. Instead I'll wait until someone offers it in an open, standardized PC-like way before becoming a cheerleader.

 

[moinmoin]

Or cloud companies are tired of the duopoly of AMD/Intel.

Timeline wise it actually looks like cloud companies were tired of Intel's slow monopoly. As soon as AMD rejoined the server competition efforts at cloud companies other than AWS slowed down considerably. I guess AWS was the farthest with its efforts and that's why it widely uses and constantly updates Graviton now. For all others the pain point is much less urgent now with the x86 duopoly, available ARM and increasingly realistic Risc-V competition.

 

[eek2121]

Wait until you find out how many companies don’t use big cloud vendors or have moved off/are not operating on “the cloud”.

Oh and saying DIY, a multibillion dollar market, needs to change? Yeah okay. How about ARM vendors start putting out DIY friendly, or even OEM friendly designs? Shoot, that approach doesn’t even help smaller cloud providers. I saved more money by leaving AWS for another (smaller) cloud provider than I would have by switching to Graviton instances.

The fact is none of these companies can churn out competitive designs that scale outside of “cloud” providers.

I 100% support ARM or RISC-V based competition, but the reason it hasn’t really happened yet is x86 combines the best of performance, perf/$, perf/watt, flexibility, accessibility, and compatibility all into one single package. A company may manage to beat x86 in 1 or two of the above, but they can never seem to beat them all.

 

[NostaSeronx]

Should be noted that Alibaba Cloud via T-Head Semiconductor is part of Beijing's Special Interest Group for RISC-V. For next-era Government/Defense/National Business/Academia; Datacenter, Cloud, Education, etc.


C910 => Performance


C920 => AI Acceleration/Enhancement // I am assuming C920 is the next performance core. It should also have RVV where the C910 removed it:

https://img.102.alibaba.com/1627958461165/49652c9412c41cb6f39b36fed1244e6e.pdf

While C908 the most recent core does have RVV:

https://img.102.alibaba.com/1667299425242/cc9eb98f11e41a47bb298ead4915e0c6.pdf

They also have a Vendor Specific ISA with RISC-V, but not with ARM: https://github.com/T-head-Semi/thead-extension-spec
Also, potential hint for Server ambition? => https://github.com/T-head-Semi/thea...ster/xtheadsync.adoc#xtheadsync-insns-sync-is

I'll be more interested in 2023/2024 if AC/T-head releases on SMIC 7nm:
Yitian with 128 C920 cores (~Yitian 710 complementary processor)
Wujian with 8 C920 cores + 4 C910 Cores (TH1520 successor)

 

[mikegg]

Wait until you find out how many companies don’t use big cloud vendors or have moved off/are not operating on “the cloud”.

There's no need to make up a fact. We have plenty of cloud market share data available. The vast majority use big cloud vendors.

Infographic: Amazon Maintains Cloud Lead as Microsoft Edges Closer

This chart shows worldwide market share of leading cloud infrastructure service providers in Q4 2023.

www.statista.com

Oh and saying DIY, a multibillion dollar market, needs to change? Yeah okay. How about ARM vendors start putting out DIY friendly, or even OEM friendly designs? Shoot, that approach doesn’t even help smaller cloud providers. I saved more money by leaving AWS for another (smaller) cloud provider than I would have by switching to Graviton instances.

You can save money by buying hardware directly or going to a small cloud that does not offer the same scaling and features as AWS. No one ever disputed this. However, the vast majority are still on big cloud. And in AWS, Graviton instances beat the pants out of Epyc and Xeon instances in value.

I 100% support ARM or RISC-V based competition, but the reason it hasn’t really happened yet is x86 combines the best of performance, perf/$, perf/watt, flexibility, accessibility, and compatibility all into one single package. A company may manage to beat x86 in 1 or two of the above, but they can never seem to beat them all.

Except that ARM has already beaten x86 in perf/$, perf/watt. Compatibility, it's almost there for cloud loads. Most popular cloud software is already ARM compatible. It's much easier to transition to ARM in the server space than in the consumer space.

Take for example, Postgres ARM on Graviton2:

While ARM-based instance is 25 percent cheaper, it is able to show a 15-20% performance gain in most of the tests over the corresponding x86 based instances. So ARM-based instances are giving conclusively better price-performance in all aspects. We should expect more and more cloud providers to provide ARM-based instances in the future. Please let us know if you wish to see any different type of benchmark tests.

PostgreSQL on ARM-based AWS EC2 Instances: Is It Any Good?

With the second gen of Graviton2 instances announced, we decided to take a look at the price/performance from the standpoint of running PostgreSQL.

www.percona.com