Looking at relatively low TDP of 2690v4 at 135W, I am wondering what would be the total power consumption of your setup while bench-marking (with all cores hitting 100%)? How about 2696v4?
Under all-core non-AVX load, the dual 2690v4 box pulls about >350 W, and the dual 2696v4 box about <400 W. These are approximate numbers because I usually have only one power meter in front of several computers and some other devices. For the next few days I don't want to replug because I don't want to shutdown and restore their current jobs.
I don't recall AVX power consumption exactly; it's somewhat higher.
While on turbo, do all the cores hit 3.5GHz (and 3.6)?
Under all-core load, X10DAX (and also an Asus X99 board which I have with single 2696v4) maintain only the all-core turbo. More than that, with socket 2011-3 Xeons at least, is only
possible with microcode hacks on E5v3 Xeons.
For reference, here are turbo bins of the 14-core 2690v4, the 22-core 2696v4, and its non-OEM sibling 2699v4:
Code:
base non-AVX turbo offset in 100 MHz, while n cores are used
TDP clock 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
-----------------------------------------------------------------------------------
2699v4 145 W 2.2 GHz 14 14 12 11 10 9 8 7 6 6 6 6 6 6 6 6 6 6 6 6 6 6
2696v4 150 W 2.2 GHz 15 15 13 12 11 10 9 8 7 6 6 6 6 6 6 6 6 6 6 6 6 6
2690v4 135 W 2.6 GHz 9 9 7 6 6 6 6 6 6 6 6 6 6 6 - - - - - - - -
AVX base AVX turbo offset in 100 MHz, while n cores are used
TDP clock 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
-----------------------------------------------------------------------------------
2699v4 145 W 1.8 GHz 18 18 16 15 14 13 12 11 10 9 8 8 8 8 8 8 8 8 8 8 8 8
2696v4 150 W 1.8 GHz ? ... 8
2690v4 135 W 2.1 GHz 14 14 12 11 10 9 8 8 8 8 8 8 8 8 - - - - - - - -
I.e. in my usage with many-core workloads, the relevant processor clocks are 2.8/2.6 GHz (2696v4 non-AVX and AVX) and 3.2/2.9 GHz (2690v4 non-AVX and AVX).
How is the noise level on the fans?
IMO very satisfyingly with the 2690v4, but less so with the 2696v4.
So far I am using one of the built-in fan profiles of the BIOS; I don't remember which one. Subjectively, this works good enough for me with the 2690v4, but not as well with the 2696v4. That's because the maximum package temperatures are defined very differently in these two. Linux' sensors tool says:
2690v4: high = 93 °C, crit = 103 °C
2696v4: high = 83 °C, crit = 85 °C
Under the current all-core non-AVX load and cosy ~26 °C room temperature, I get
2690v4: 60...63 °C CPU temp, ~750 RPM case fans, ~810 RPM cooler fans
2696v4: 49...51 °C CPU temp, ~920 RPM case fans, ~900 RPM cooler fans
I.e. the BIOS's fan curve is aligned with the lower "high" temperature of the 2696v4 and drives its fans faster than on the 2690v4. (Or maybe the fan speed curve is aligned with TDP, but I doubt that.)
With more restrictive cases, I wouldn't like the fans speeds even of the 2690v4. But it's fine for me with the Corsair 400Q. (Front fascia is permanently off, but front dust filter is attached.) I fact I had this thing running 2 or 3 meters from my bed at some time and it didn't disturb my sleep at all.
The 900+ RPM of the 140 mm fans in the 2696v4 box causes more air noise which I wouldn't want under my desk, or in a bedroom. I haven't checked in the BIOS yet whether there is a slower fan profile, nor have I tried low noise adapters on the fans yet, as I had no pressing need for better acoustics so far.
My points for comparison are a Fractal Design R5 PCGH edition ( = no top vents, no side vent), in which I prefer 140 mm case fans and CPU cooler fan to stay below 650 RPM, and a Thermaltake Core case with 420 mm radiator at the top whose 140 mm radiator fans I preferred to stay below 700 RPM IIRC. (Both these setups are more restrictive than the 400Q, and the R5 also adds an irritating tonality at a certain level of air speed.)
By the way, as you may have noticed, my setup differs from yours regarding cooling config by lack of a high-wattage GPU. Therefore, both CPUs draw air almost at room temperature, not from the exhaust of the cooler of another high-powered device.
Furthermore, the front CPU is cooler than the back CPU by a few °C, typically by 2 °C, due to marginally better air supply of the CPU cooler in the front.
Also, did you use any of SuperMicro's overclocking features on the board?
I only enabled "Hyper-Turbo" which I presume helps with running the CPUs at turbo rather than base under all loads. I haven't experimented yet what would happen if I switch that off.
I did not enable BCLK overclocking yet, which Supermicro call "Hyper-Speed". It would overclock memory and PCIe too. I also kept core Voltage offset disabled so far.
Originally, I chose X10DAX for the 2690v4 based build (two builds actually) because I was interested in having at least all-core turbo applied at all time during many-core loads, and these being my first dual-processor builds ever, I was not sure whether other boards would give me that. I stuck with X10DAX with the 2696v4 build about a year later (actually two builds too) simply out of resistance against trying anything new. Clock speeds weren't that much of a criterion with the 2696v4 build for me anymore, but rather performance per socket and performance per Watt.
- all core turbo is around 2.8 Ghz I believe. 
Facebook
Twitter