servethehome:Cavium ThunderX2 Review and Benchmarks a Real Arm Server Option

csbin

Senior member
Feb 4, 2013
838
351
136
https://www.servethehome.com/cavium-thunderx2-review-benchmarks-real-arm-server-option/



Cavium ThunderX2 Power Consumption

Before we get into this section, we wanted to make a few notes clear. First off, our silicon was an early version that was not screened and binned for power. We were told that our power consumption results may be 10-15% higher than shipping parts. Second, TDP does not equal power consumption. There are very strange comparisons that happen based on TDP. Each vendor calculates TDP differently so seeing what total power consumption is at the system level is relevant for your TCO calculations.

Our Gigabyte/ Cavium ThunderX2 Saber development platform hit a peak of 823W at 100% load. We think that there are likely optimizations that can occur at the system’s firmware level, and by using GA power binned chips. At first, we thought that these numbers were way out of line so we discussed them with Cavium and that is when we were told that the ~800W range was correct for our system and pre-production chips. The company also told us that the production systems will have firmware that is better power optimized. As a result, we are not going to publish a direct comparison until we can get a mature Cavium ThunderX2 platform with production chips and system firmware. This may take some time, but publishing a comparison using the Saber platform and the unbinned silicon is disingenuous.

Note these results were taken using a 208V Schneider Electric / APC PDU at 17.8C and 71% RH. Our testing window shown here had a +/- 0.3C and +/- 2% RH variance. These are great power consumption figures. These are certainly solid results for this system and a large amount of the power is not used by the CPU and instead by the RAM and peripherals.





5af386fc553e7.jpg


5af3875da4ffb.jpg


5af3875da6899.jpg





SPECrate2017_int_peak

5af384c3e0477.jpg


5af384c3e05d3.jpg







STREAM Triad Memory Bandwidth


5af384c3e0509.jpg


5af384c3e05fb.jpg







Linpack Performance

5af384c3e0421.jpg







OpenSSL Performance
5af3889fb42ef.jpg


5af3889fb7950.jpg






c-ray 8K Performance

5af389dd09bfd.jpg






Compression Performance

5af38a2e621d1.jpg






UnixBench dhrystone 2 performance(multi-threaded)

5af38a6cb4d21.jpg







UnixBench dhrystone 2 performance(single-threaded)
5af38a8fc2721.jpg






UnixBench whetstone performance(multi-threaded)

5af38baa9e8d0.jpg







UnixBench whetstone performance(single-threaded)
5af38baaa12de.jpg







Storage Performance

5af38c86d5e71.jpg


5af38c86d7620.jpg


5af38c86d8ed4.jpg


5af38c86dad35.jpg
 
  • Like
Reactions: Hitman928

Hitman928

Diamond Member
Apr 15, 2012
5,262
7,882
136
Better performance than I expected if I'm being honest. Integer performance looks very good, but that power consumption. . . ouch. Even taking into account the early silicon and unrefined firmware, I think that's a hard sell to get worse or equal performance with less IO and significantly more power draw. We'll see what they can do in the refinement stages.
 

LTC8K6

Lifer
Mar 10, 2004
28,520
1,575
126
They said power consumption was great:

These are great power consumption figures. These are certainly solid results for this system and a large amount of the power is not used by the CPU and instead by the RAM and peripherals.
 

Hitman928

Diamond Member
Apr 15, 2012
5,262
7,882
136
That's hard to say since they didn't include any power comparisons, just that it was high.

They actually do give a power test result of 823 W at load. They don't do a comparison at this point because it is early silicon/firmware and they want to give them a chance to bring that number down to a more competitive level.

They said power consumption was great:

Great is completely subjective, especially since they didn't say what it was great in comparison to. However, if you look at their recent tests of AMD and intel 2P systems, they use around 500W at load (non-AVX for intel). So if you use that as your comparison, then no, over 800 W with less IO capability is not great.

I'm not trashing on the product, it looks more than capable, but I'm also trying to be realistic because businesses aren't going to switch to ARM just because it's trending. AMD has a hard enough time taking server share from intel and they don't have to contend with the ISA barrier. It will be even more difficult for an ARM platform to break into the market. Hopefully with some tweaking they can get performance up, power down, and get their toe in the door enough to build off of, it would be fun to see.
 

XavierMace

Diamond Member
Apr 20, 2013
4,307
450
126
They actually do give a power test result of 823 W at load. They don't do a comparison at this point because it is early silicon/firmware and they want to give them a chance to bring that number down to a more competitive level.

You can't claim it's "significantly more power draw" without having a number to compare to was my point. I've had 4 socket servers that were pulling 1400w. I've had newer 2 socket servers that are only pulling 400w. STH's Dual 6238 system clocked in at 232w at full load with 6 Dimms, a single Intel SSD, and a SATADOM. Their ARM system has 16 Dimms, 4 PCIe cards, and an unspecified amount of drives. That's obviously not going to account for 600w, but it's not a drop in the bucket either.
 

Hitman928

Diamond Member
Apr 15, 2012
5,262
7,882
136
You can't claim it's "significantly more power draw" without having a number to compare to was my point. I've had 4 socket servers that were pulling 1400w. I've had newer 2 socket servers that are only pulling 400w. STH's Dual 6238 system clocked in at 232w at full load with 6 Dimms, a single Intel SSD, and a SATADOM. Their ARM system has 16 Dimms, 4 PCIe cards, and an unspecified amount of drives. That's obviously not going to account for 600w, but it's not a drop in the bucket either.

I agree, that's why in the second part of my post I gave the numbers I was using to substantiate the "significantly more power" comment. Those numbers came from their recent tests of AMD and intel machines with large amounts of memory (12-16 DIMMs), storage, etc. It's not a direct comparison but when it's ~830 W versus ~500 W, it's pretty hard to see that difference being overcome through some firmware tweaks and maybe a few less system components.
 

ksec

Senior member
Mar 5, 2010
420
117
116
I have a hard time wrapping my head around 4 way SMT. I wonder how well it would do on my Web Apps. 256 Threads!

I am thinking if may be Cloudflare will switch to that If Qualcomm really decide to abandon their Centriq.
 

itsmydamnation

Platinum Member
Feb 6, 2011
2,769
3,139
136
I have a hard time wrapping my head around 4 way SMT. I wonder how well it would do on my Web Apps. 256 Threads!

I am thinking if may be Cloudflare will switch to that If Qualcomm really decide to abandon their Centriq.
Really probably only makes sense if your I/O latency (but not throughput) bound. It makes more sense for power because their single core is basically split in two all the way down the center.
 

LTC8K6

Lifer
Mar 10, 2004
28,520
1,575
126
I have a hard time wrapping my head around 4 way SMT. I wonder how well it would do on my Web Apps. 256 Threads!

I am thinking if may be Cloudflare will switch to that If Qualcomm really decide to abandon their Centriq.
Intel PHI has always had 4 threads per core. 72 cores and 288 threads for the top model.