Review Raspberry PI (ARM A72) vs EPYC for DC study. Interesting results.

Page 5 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,478
14,434
136
OK, now this is a very specific test, but as much as they are disparate, I don't think other scenarios would be that different.

So I got an 8 gig Raspberry PI (8 gig, 32 gig HD, A72 v8 4 core 1.5 ghz). Its does the Open Pandenics covid 19 WCG unit, 4 cores times 7.5 hours

My EPYC 7742 does 128 of these same units at 2 ghz in 2.5 hours.

So it would take 32 PI's to do the same work, but 3 times slower. An 32 PIs at 5 watts each is 160 watts, vs the 7742 @250 watts. (motherboard, memory and all) So more power and 3 times longer for the same work is 2 times the power usage. (1200 watts total vs 625)

Cost... 32 PIs at about $60 each (including power supplies and cables, probably more) would be at least $1920.

The EPYC is about $4000+480+580 or $5060. (I paid $3000 less, that retail on ebay)

So the EPYC effectively less because it cost 2 times less electricity for the same work.

This seems WAY different than I am seeing when people talk about ARM and efficiency and power usage. Please tell me where I am mistaken in my math., But don't be a jerk if you find the errors of my ways. The run times are real.. The PI is at 7.5 hours and hasn't actually finished a unit yes, no other tasks running on the PI. And I am looking at several units on the 7742, one is at 97% in 2:21.


Edit: The first unit finished in 7:50, almost 8 hours. The next 2 units are at 8 hours and still running at 96 and 98%.

And my comment on "wins by a large margin" has to do with total power usage, which in data centers, and for me is a big deal.
 
Last edited:

DrMrLordX

Lifer
Apr 27, 2000
21,582
10,785
136
Thats why ARM Macs are dangerous to x86. Suddenly you have a readily available probably rather powerful ARM dev environment. As said in a previous comment, normal PCs being x86 is what made it successful as it triggered more and more x86 software.

I don't know that they're dangerous. Also, Apple's designs may not mimic the performance of ARM server cores well enough to do good performance profiling using them (that is to say, I think the Apple cores may excel in some areas where server ARM cores won't). You can run the same code (presumably) on an A14 as you can A76 or A77-based server hardware, but I'd rather have a reference board from ARM or at least something based on fairly standard A76/A77 cores.
 

Shivansps

Diamond Member
Sep 11, 2013
3,835
1,514
136
MacOS is a completely diferent software ecosystem, and more importantly, completely closed. I dont think it will have any effect in ARM adoption outside the closed black box it is.

ARM needs to get to the mass consumer market outside phones to get anywhere, but to be honest if they are having no market in server it is because the performance vs power is just not good enoght regardless of syntetic benchmarks.

Nvidia buying ARM may change a lot of things, Nvidia already do ARM64 Linux drivers for their gpus, but they do not do Windows ARM drivers, that will likely change.
If Nvidia work with other companies to fix the Windows ARM driver support issue and start making both pc motherboards and notebooks in the mainstream market, ARM will start eating into x86 market... still im not even sure if it worth it... ARM has the phone market thats a lot bigger.
 

LightningZ71

Golden Member
Mar 10, 2017
1,627
1,898
136
If Nvidia wanted to, they could make a fairly convincing and useful laptop based on the same setup specs that they use on the Jetson Xavier boards. Yes, form factor and thermals wold need work, but the foundation is there. There's just no market.
 

Shivansps

Diamond Member
Sep 11, 2013
3,835
1,514
136
To go back to the start, @Markfw what OS did you use on the Raspberry PI? Because Raspbian is still 32 bit, the RPI 4 and all A72 cpus have a large (in some cases more than 100%) performance running arm64 mode. You need to use Manjaro or Ubuntu Mate arm64 in the RPI 4 to use all the performance (and the 8GB of the ram).
 

sdifox

No Lifer
Sep 30, 2005
94,667
14,929
126
To go back to the start, @Markfw what OS did you use on the Raspberry PI? Because Raspbian is still 32 bit, the RPI 4 and all A72 cpus have a large (in some cases more than 100%) performance running arm64 mode. You need to use Manjaro or Ubuntu Mate arm64 in the RPI 4 to use all the performance (and the 8GB of the ram).


RPI OS 64 is in beta.
 
  • Like
Reactions: piokos

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,478
14,434
136
To go back to the start, @Markfw what OS did you use on the Raspberry PI? Because Raspbian is still 32 bit, the RPI 4 and all A72 cpus have a large (in some cases more than 100%) performance running arm64 mode. You need to use Manjaro or Ubuntu Mate arm64 in the RPI 4 to use all the performance (and the 8GB of the ram).
Interesting... I just used what it came with. I will have to look into that. Does someone have a link to download the correct version ?
 
  • Like
Reactions: Drazick

Shivansps

Diamond Member
Sep 11, 2013
3,835
1,514
136
Interesting... I just used what it came with. I will have to look into that. Does someone have a link to download the correct version ?

From the distro official website, manjaro also have a link on his website.

Then you use a distro burner like etcher to create the sd card
 

piokos

Senior member
Nov 2, 2018
554
206
86
MacOS is a completely diferent software ecosystem, and more importantly, completely closed. I dont think it will have any effect in ARM adoption outside the closed black box it is.
I'm not sure where such opinions come from. I don't think you know much about the platform. ;)

MacOS ecosystem is just as "closed" as Windows.

And when it comes to developing software for servers/cloud (anything Linux, really), it's actually easier on Macs, since the kernel provides native support of Linux workflows. On Windows it's all emulated in VMs - at least for now.
ARM needs to get to the mass consumer market outside phones to get anywhere
If Apple goes ARM, it IS a mass market.
This is what people on gaming forums don't get - they think it's some small company with expensive, fashionable products that forces trends in the market.
It's not.
Apple sells almost 20 mln Macs yearly.
By comparison, gaming GPU sales from Nvidia and AMD are somewhere around 40 mln.

Apple move to ARM will make it mainstream and will force some changes on the market - for better or worse.
At very least, it will mean some software popular among Mac users (Adobe, MS Office) will get compatible version very soon.
And it may result in version for Windows as well, so suddenly getting an ARM Windows PC will become an feasible option for a few million people.

I'm not a huge fan of this shift to ARM. But if all the stuff I need is available, I won't really mind having to buy into ARM. And I may be forced to - simply because at some point x86 may be squashed to "premium" (high performance) segment and all the sub $1000 laptops are ARM.
but to be honest if they are having no market in server it is because the performance vs power is just not good enoght regardless of syntetic benchmarks.
They have to market in servers because the platform is new. Few years from now half of AWS may be running on ARM. Other clouds as well.
And you don't really know what's underneath the cloud service you're using. It's completely transparent. That's an important pillar of cloud computing.
 

dacostafilipe

Senior member
Oct 10, 2013
771
244
116
Apple move to ARM will make it mainstream and will force some changes on the market - for better or worse.
At very least, it will mean some software popular among Mac users (Adobe, MS Office) will get compatible version very soon.
And it may result in version for Windows as well, so suddenly getting an ARM Windows PC will become an feasible option for a few million people.

Missing an important point, ARM compatibility.

Compiling code to an ISA version (v7, v8, v9) is easy, even today. But adding SVE or custom extensions will make it a lot harder as you need to port your code the given SoCs. Apple will certainly release SDKs so accelerate the implementation, but what about other SoC vendors?
 

piokos

Senior member
Nov 2, 2018
554
206
86
Missing an important point, ARM compatibility.
I'm perfectly aware of that. In fact, I've been mocked on this forum for mentioning this. So be careful. ;)

Still, the first big step is migrating x86 software to ARM (of any "smell"). It'll be much easier to spread it further.
But adding SVE or custom extensions will make it a lot harder as you need to port your code the given SoCs. Apple will certainly release SDKs so accelerate the implementation, but what about other SoC vendors?
Absolutely. Software created for base ARM version will probably suck compared to well optimized x86.
So developers will have to optimize for each platform. And ARM gives a lot of freedom when it comes to choosing and tweaking extensions.
Which means that software written for Macs probably won't automatically work on other ARM devices (and if it does, it may be very slow).

I've been mentioning this earlier. Just think about AVX-512 and how much controversy and pointless discussions it spurred over the years. Or the mess around Matlab being optimized for Intel.
When we switch to ARM, this will become an every-day reality.
And since Apple is leading the transition, we should expect most software to be optimized for their SoCs, then quickly (badly) ported to everything else and maybe optimized later on.

Software written specifically for Macs will also use Metal (low-level GPGPU API) and any other hardware acceleration Apple decides to provide.
Because if, say, 50% of MacBook Pro buyers use them for photo/video editing, why wouldn't Apple include a tiny ASIC that massively speeds up sharpening or color grading?

It could be even worse. Lets say some software has a big user share on Macs. Again: photo/video editing suites could be a good example.
It will be much easier for software studio to optimize for a unified Apple ARM that runs in 10 mln devices than for very fragmented ARM implementations running on 100 mln PCs.
So, my big worry is that software may start to remind the mobile accessory market - where many (especially high-end) companies focus just on the iPhone or iPad - simply because a single design has a massive reach.

So I do hope that if we go ARM, these chips will be made by a handful companies: Intel, AMD, maybe Microsoft or Qualcomm. Because if every OEM starts to make their own custom chips, it will become a mess. And at that point I'll probably just get a Mac...
 
Last edited:

Richie Rich

Senior member
Jul 28, 2019
470
229
76
Missing an important point, ARM compatibility.

Compiling code to an ISA version (v7, v8, v9) is easy, even today. But adding SVE or custom extensions will make it a lot harder as you need to port your code the given SoCs. Apple will certainly release SDKs so accelerate the implementation, but what about other SoC vendors?
There are some mandatory requirements in ARM ISA. For example ARMv8.6 in combination with SVE makes matrix multiplication and Bfloat16 options mandatory. So it gets better in time as selecting the most useful instructions as new standard. Same is expected for ARMv9 (probably ARMv8.6 + SVE2 + TME + some new instructions like for memory security as AMD and Intel has).

Compare it to AVX512 with 17 optional subsets which is total mess. There is no Intel CPU with full support until today. Intel Cooper Lake supports Bfloat16 but ICL and TGL does't. SVE2 has AFAIK only 1 or 2 options only.

ARMv7 as 32-bit is obsolete same way as i386 and useful maybe for Raspberry Pi SBC devices due to that Cortex A72 uarch is based on 32-bit A15. Apple doesn't support 32-bit mode anymore in order to maximize performance. There is always some kind of trade of in CPU design.
 
  • Like
Reactions: NTMBK

piokos

Senior member
Nov 2, 2018
554
206
86
Well, with all the optimism in your post, it certainly did not seem (to me) like you are aware of it.
Well, I'm not trying to spread my worries in every sentence. But I actually expanded that post before you answered, so it may seem less "optimistic" now. :p

But personally, I'm honestly worried. As a user, but especially as a dev. Because it may just happen that one day (probably in 2021), I'll start getting tickets like "this code sucks on my ARM Mac, make it faster". I imagine this has been 90% of dev work at Adobe lately. :)

The silver lining here is that I'm mostly writing stuff for cloud, so this won't be very serious. But of course we develop and test locally, so yeah... teams with mixed architectures will become a real problem.
 

dacostafilipe

Senior member
Oct 10, 2013
771
244
116
There are some mandatory requirements in ARM ISA. For example ARMv8.6 in combination with SVE makes matrix multiplication and Bfloat16 options mandatory. So it gets better in time as selecting the most useful instructions as new standard. Same is expected for ARMv9 (probably ARMv8.6 + SVE2 + TME + some new instructions like for memory security as AMD and Intel has).

Compare it to AVX512 with 17 optional subsets which is total mess. There is no Intel CPU with full support until today. Intel Cooper Lake supports Bfloat16 but ICL and TGL does't. SVE2 has AFAIK only 1 or 2 options only.

Last time I checked, SVE was a "flexible" spec. It was up to the vendors do implement the width in hardware. So yeah, not as bad as AVX, but the performance can still wary a lot. Specially for that part of code that can't scale down (256 -> 2x128).

There is always some kind of trade of in CPU design.

Yes, and having more vendors, with different goals (Mobile, Server, Desktop, IoT,...) will not help in solidifying the platform.

Whatever ARM adds to their portfolio, you will have those companies that can do better on their own because they have to money to do it, and you have those companies that do not. It works great on mobile, as it's a product by product implementation, but for desktop it's a different story.

Apple will not care about other vendors when implementing hardware features into Apple Silicon and Amazon will not care about others when adding hardware features to the next Graviton. So, for people that use Windows/Linux Desktops, this really is an issue.

I'm not trying to defend the x86 duopoly, but people need to take this issue seriously and start working on a solution, or in other words: Microsoft should wake up and support the ARM platform better!
 

dacostafilipe

Senior member
Oct 10, 2013
771
244
116
But personally, I'm honestly worried. As a user, but especially as a dev. Because it may just happen that one day (probably in 2021), I'll start getting tickets like "this code sucks on my ARM Mac, make it faster". I imagine this has been 90% of dev work at Adobe lately. :)

We have clients talking about AI in some native apps but the lack of dedicated hardware and/or the difference in hardware make this really hard to do, ergo expensive.

The silver lining here is that I'm mostly writing stuff for cloud, so this won't be very serious. But of course we develop and test locally, so yeah... teams with mixed architectures will become a real problem.

Same, I don't care about ARM/x86/PPC/RISC-V as long as my code performances as it should and the client is happy, but thinking about replacing all our macs or having a mixed x86/ARM ecosystem does really not make me happy :p
 

Richie Rich

Senior member
Jul 28, 2019
470
229
76
Maybe from PC/cloud/phone perspective.
Quite a few embedded devices still use ARMv7. You see them in cheaper NASes, in a lot of IoT stuff.
I agree. Embedded is still using 8-bit CPUs so it's kind of different world with higher priorities than raw performance. But ultra cheap ESP32 (32-bit dual core CPU with wifi antenna on-board for just 4$ is incredible) moved IoT to 32-bit so maybe we will see 64-bit devices soon. Maybe ESP will eat ARM's market share in IoT same way as ARM will eat x86 server and desktop market share. That's the rule of growing business. You grow or die.
 

piokos

Senior member
Nov 2, 2018
554
206
86
Same, I don't care about ARM/x86/PPC/RISC-V as long as my code performances as it should and the client is happy, but thinking about replacing all our macs or having a mixed x86/ARM ecosystem does really not make me happy :p
Yup. Soon to get into your dream project, you won't just have to know all the necessary technologies. You'll also need a dev laptop with the optimal (least problematic) CPU architecture...

I still don't know what will happen to Docker on Macs. Will it be emulated? Because many images on Docker Hub are x86 only and nothing will force owners to build for ARM. Frankly, some popular images aren't even actively managed anymore.
Which will probably lead to multiple badly managed forks...
 

xblax

Member
Feb 20, 2017
54
70
61
Interesting discussion! But it should absolutely be possible to run a benchmark on a modern Android Arm smartphone that can give a good estimate on ARM power efficiency vs X86. A C/C++ command line benchmarking tool could be cross-compiled just as for any other Linux distribution (ignoring Android completely) - Android is still Linux in the end.

The phone would probably need to bee rooted to avoid that the process is killed by android. Maybe some cooling would be required for the phone during a long benchmark run.

But I don't think ARM can beat modern EPYC systems that are running at their perf/power sweetspot - maybe in the future but not today. If you look at https://www.spec.org/power_ssj2008/results/ the leading systems are all AMD Epyc based (I didn't find in any ARM based benchmark result). If any of the available ARM server system could achieve notable results I'm pretty sure they would be published.
 
  • Like
Reactions: Tlh97 and moinmoin

piokos

Senior member
Nov 2, 2018
554
206
86
But I don't think ARM can beat modern EPYC systems that are running at their perf/power sweetspot - maybe in the future but not today. If you look at https://www.spec.org/power_ssj2008/results/ the leading systems are all AMD Epyc based (I didn't find in any ARM based benchmark result). If any of the available ARM server system could achieve notable results I'm pretty sure they would be published.
SPECpower requires physical access to the server - you have to use an external power analyzer.

The only ARM server platform used for large-scale production today is AWS Graviton - which means cloud.
The other option is Marvell ThunderX2, but that's more of a dev kit than a production-ready solution. Anyway, it's so new and scarce I don't even know anyone who has seen one.
 

xblax

Member
Feb 20, 2017
54
70
61
SPECpower requires physical access to the server - you have to use an external power analyzer.

Sure but I mean manufacturers would publish these kind of benchmark results themselves if their claims were true. Ampere computing published nice charts showing "estimated" performance and power metrics of their ARM Altra Server platform in March: https://www.nextplatform.com/2020/03/03/ampere-aims-for-the-clouds-with-altra-arm-server-chip/

Why don't they back it up witch actual measured results? Some of these systems must exist somewhere. They claim it's already being shipped around the world. https://www.phoenicselectronics.com/ampere.html

The only reason I can think of is that the results are too underwhelming to be openly published.
 

piokos

Senior member
Nov 2, 2018
554
206
86
You mean ThunderX3. ThunderX2 is quite old by now.

I actually though X2 launched last year, not in 2018. Which is even worse because I've honestly never heard anyone even mention considering them. Maybe they just aren't easy to get here. Or no one is interested.
Because seriously, who buys experimental stuff like that? You either really need a local server - in which case you probably want something that can be quickly fixed/replaced... or you go cloud (ARM or not).
 

Hitman928

Diamond Member
Apr 15, 2012
5,177
7,628
136
And when it comes to developing software for servers/cloud (anything Linux, really), it's actually easier on Macs, since the kernel provides native support of Linux workflows.

Are you sure about this? MacOS and Linux don't share a common source or kernel, not sure how you would get Linux support on Mac without some kind of translation layer, but then that wouldn't really be native. . .