What controls Turbo Core in Xeons?

Page 59 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

Cata40

Member
Mar 2, 2017
156
6
81
You can buy a eeprom reader/writer adapter Minpro100 and some W25Q128FIQ chips to wtire a origenal BIOS in that for spare.
My MOD BIOS all had been test,if you not sure your CPU can work -70mv,you can use the -50mv one.
I order eeprom reader/writer adapter Minpro100 For the time being, in a few days comes the second bios-chip from the asrock, I was suggested to make a save of bios, but of the total, but I do not know how to do it.
It would be interesting how to make a total and professional rescue
I have to learn with the eeprom to rewrite the bios, if it can on another original from my asrock, let's say it would be damage
I think for the first time to use the old dufus method, with sticks on the stick, to find out what suits the board and processor.
Then set me up at the best voltage and write it
I can handle a bios backup chip for now, if that's the case I'm buying on ebay
 

traderjay

Senior member
Sep 24, 2015
220
165
116
Can anyone please post a hacked BIOS for Supermicro x10DAX or x10DAi ?
Really appreciate it, cant figure out the process of modifying it.

I am building a Dual E5 2699 V4 system soon with the same motherboard. Hopefully the genius here can help unlock more performance :D
 

Cata40

Member
Mar 2, 2017
156
6
81
Only way to recovering mb bios to using usb eeprom reader/writer adapter on another working computer.
Like this;
https://www.ebay.com/p/CH-341-a-24-25-Series-EEPROM-Flash-Bios-USB-Programmer/1431400734
these are good all for me to rewrite bios?
https://www.ebay.com/sch/i.html?_od...EPROM-Flash-Bios-USB-Programmer&_sacat=181904
well, I mean those without the SOIC8 Clip On-Board, because you said the bios-chip from the asrock x99 itx ac is mobile, it can get over there and put on the programmer, right?
That programmer is with driver and software, but where does he take the image of a complete bios, even with non-rewriting parts? I have to save a complete bios asrock x99 itx ac or programmer already has this?
 

custom90gt

Member
Feb 9, 2017
38
7
41
So I just changed from an X99M Extreme 4 motherboard (running the v3.efi) to a gigabyte x99 ultra gaming (running a modified bios) and now I notice that my CPU won't go into C1-C3 states. Anyone else having an issue with the modified bios? I may try putting the stock bios on and seeing if it will enter power saving states.
 

Dufus

Senior member
Sep 20, 2010
675
119
101
C1 is a halt state and an actual opcode of the CPU so cannot be turned off. The OS would have to deliberately stop using the hlt opcode.

For myself PC6 is problematic but PC2 appears to work okay although for some reason this option is not given in the Asrock BIOS, doesn't seem to make a big difference in power draw anyway. CC3 and CC6 also appear to work okay although CC3 is by default disabled in the BIOS so perhaps there are issues.
 

Dan.R

Junior Member
Jul 16, 2017
2
0
66
C1 is a halt state and an actual opcode of the CPU so cannot be turned off. The OS would have to deliberately stop using the hlt opcode.

For myself PC6 is problematic but PC2 appears to work okay although for some reason this option is not given in the Asrock BIOS, doesn't seem to make a big difference in power draw anyway. CC3 and CC6 also appear to work okay although CC3 is by default disabled in the BIOS so perhaps there are issues.

I have the same motherboard, but a different CPU, I've noticed the same as you. I've been following this thread closely for ~6 months now, but I can't recall, what was the issue you had with PC6? Do you know if it's Haswell-E specific or Mobo specific?
 

custom90gt

Member
Feb 9, 2017
38
7
41
C1 is a halt state and an actual opcode of the CPU so cannot be turned off. The OS would have to deliberately stop using the hlt opcode.

For myself PC6 is problematic but PC2 appears to work okay although for some reason this option is not given in the Asrock BIOS, doesn't seem to make a big difference in power draw anyway. CC3 and CC6 also appear to work okay although CC3 is by default disabled in the BIOS so perhaps there are issues.

Yeah I'm not sure what is going on. C3 worked great with my Asrock board and lowered power consumption by ~30watts. Checking throttle stop it shows 0% c state percent for states 2-7. This is a fresh install of server 2016 with all the drivers installed. I've checked bios and forced C3 to enabled, but it still doesn't work. I may try flashing the stock bios and seeing what happens.
 

Dufus

Senior member
Sep 20, 2010
675
119
101
Do you know if it's Haswell-E specific or Mobo specific?

If I run default BIOS PC6 works and power from the wall drops ~10W at idle vs just using CC6. With a modified BIOS when CPU is idle and presumably PC6 is being initiated then the system freezes. Maybe something needs fixing with ucode early on or maybe I'm missing something. I had hoped to get some feedback if any of the other systems actually have PC6 working with the mod but nothing so far.
 

kjboughton

Senior member
Dec 19, 2007
330
118
116
If I run default BIOS PC6 works and power from the wall drops ~10W at idle vs just using CC6. With a modified BIOS when CPU is idle and presumably PC6 is being initiated then the system freezes. Maybe something needs fixing with ucode early on or maybe I'm missing something. I had hoped to get some feedback if any of the other systems actually have PC6 working with the mod but nothing so far.

What's the question/request?

Also: you'll like this...
Dual_E5-2696v3_4GHz_FCG.jpg
 
  • Like
Reactions: Cata40 and Dufus

Dufus

Senior member
Sep 20, 2010
675
119
101
@kjboughton IMHO things like that should not be hidden behind spoilers :D

What happens if the other 2 cores are enabled? Do you see any WHEA corrections at 105MHz BCLK? Can check under sensors of HWiNFO.

The question was regarding package C-States, in particular C6. Can be checked under HWiNFO under package C6 residency IIRC or using throttlestop and I seem to recall an Intel app but don't remember it's name.

Also do you see RAPL measurements for DRAM? Without the mod my board shows nothing but with the mod it appears to work. Might be linked somehow with PC6 problem and DRAM setup?


It certainly would have been interesting to see how these CPU's performed unlocked (E5-1691v3?) but perhaps segmentation, profit and Broadwell got in the way.
 
  • Like
Reactions: custom90gt

kjboughton

Senior member
Dec 19, 2007
330
118
116
@kjboughton

What happens if the other 2 cores are enabled?
I am working on a theory which should allow me to quite accurately model core power under a variety of scenarios.

As others have found, there is no readily-available means in which to defeat the processor's internal power control circuitry... at least as far as I am aware.

I disabled two cores (per CPU) for a couple of reasons...

Empirical testing showed more x264 encoding performance (i.e. higher average fps during conversion) than with all cores enabled. Presumably this is due to the higher overall sustained core frequencies with two fewer cores to power. As you know, circuit power consumption is a function of both frequency and voltage (which, for the group, is why we reduce Vcc_in) and with increasing frequency there is a reduction in the power/performance ratio (incrementally more power required to operate at diminishing frequency increases). Now, those that OC without a power ceiling don't care as they're looking to maximize performance independent power draw. Here we are limited and so strive for the highest power/performance point in order to maximum computational throughput.

If you look at a functional block diagram of the HCC HASWELL-EP/EX die you'll note it's broken into two rings with an uneven number of cores per ring (8 cores in Ring1, 10 cores in Ring2):

intel-xeon-e5-v3-block-diagram-detailed.jpg

So what I found was maximum power/performance, and thus performance, came when I reduced core count from 18 to 16 per CPU. (As an aside, with 32 real cores I saw no need to overtax the Windows scheduler by doubling this to 64 and so elected to keep HTT disabled.)
I could run the additional cores for effect but there would be no benefit (in fact, a reduction in performance). As well, in order to allow all cores to work a single instance of x264 I have disabled NUMA forming a single super-set of 32 cores instead of two 16-core nodes. This would be the equivalent of running a ThreadRipper in "Content Creation" mode. For gaming, I find there no need to switch configurations as six or eight (or whatever the game utilizes) cores run just fine up at 4GHz :)

Do you see any WHEA corrections at 105MHz BCLK?

Increased BCLK is not a problem. Too high a frequency is a problem. Eventually you simply need a higher core voltage to keep climbing.

Can check under sensors of HWiNFO.

Something funky with my system leads to freezes when I try to read sensors on my board whether that be in AIDA64, HWiNFO64, etc. Not sure what to make of this thus far... ask me again later.

The question was regarding package C-States, in particular C6. Can be checked under HWiNFO under package C6 residency IIRC or using throttlestop and I seem to recall an Intel app but don't remember it's name.

I believe you may referring to Intel XTU (Extreme Tuning Utility?) which I've never used. As far as I can tell, C6 is enabled and working properly with my system.
I am running Haswell microcode 0x1F. I regularly see a significant portion of core time spent in C6 (vs. C0/C1) when the system is idle.

Also do you see RAPL measurements for DRAM? Without the mod my board shows nothing but with the mod it appears to work. Might be linked somehow with PC6 problem and DRAM setup?

I do recall seeing RAPL figures before using the EFI Driver with your PowerCutTM feature built-in. I now no longer see this data (in addition to the problems I am having in general reading some sensor data). If I am remembering correctly it was on order of 5-7W per stick of 8GB DDR4-2133R SR memory.

It certainly would have been interesting to see how these CPU's performed unlocked (E5-1691v3?) but perhaps segmentation, profit and Broadwell got in the way.

OK. So allow me to now spill my guts on this. I think it's odd how I always end up settling into the ultimate system no matter the latest generation release. This time is no different.
Some months ago (not sure if it's still true today due to price and demand) I purchased a pair of used E5-2696 v3 (OEM only, mind you) Xeons off eBay for approximately $800 total ($400 each).
A refurbished ASUS Z10PE-D8 put me out another $400. Throw in another $800 for 64GB of DDR4 (new) from a reputable company and you have a barebones that can run the pants off both the i9-7980XE (unreleased) and the RYZEN Threadripper 1950X with room to spare. Now that's a $2,000 spending spree but what's a i9-7980XE due to cost alone? $1,999? I think we know who's going to win that one...

My recommendation for any who needs to CRUNCH is to go this route. Disable NUMA: this creates a single set of cores that can be thrown at even the most NUMA-unaware application out there. If it's parallel, like x264 encoding, disable NUMA... sure you lose some compute efficiency as on average 50% of the memory operations will be to far memory, the other operations being to near memory. That aside, double the number of cores will drown out any effect here (unless we're talking latency-sensitive applications... we're not; we're talking bandwidth-sensitive, if that). The point being, there is no replacement for displacement; at least not when comparing v3 Xeons to "v5" Xeons. With this hack applied my dual 2696 v3 system effectively performs at the level of a 2696 v4 (hrmp, imagine that) AND I get 3.8 (or ~4GHz) turbo (depending on if I want to push BCLK or not, typically not). Given that the i9-7980XE is essentially a ~HCC "v5" Xeon, in this case two top-end "v4" Xeons can certainly outpace a single mid-range "v5" Xeon and there you are. Winner, three-generation-old dual Xeon system with parts that are just absolutely spilling into the market as those bringing up the rear finally ditch their v3 Xeons to scoop up used v4 Xeons being rapidly discarded by large enterprise as they fulfill orders for the new Xeon Scalable Processor series.

By the way, AMD's use of "Content Creation" mode, which disables NUMA, is precisely what I am recommending to those that may build a system similar to mine. And unless you're talking bench-marking and other synthetic focus, there's zero reason to reboot to go between modes (i.e. enable NUMA)... and I'm talking Xeon here... as 3.8 (or 4GHz) is plenty fast, especially with 32 REAL cores on tap, to play even the most demanding of today's games.

TL;DR
Don't buy i9 or Threadripper, pick up a pair of E5-2696 v3's and beat the snot out of both
 
Last edited:
  • Like
Reactions: MooNiSLe

kjboughton

Senior member
Dec 19, 2007
330
118
116
hi, kjboughton
16 cores at 3.8ghz under heavy loads like cinebench ?

No. Under heavy loads, like Cinebench, cores flip between 3.4 and 3.5Ghz with most remaining at the higher speed.
So call it 3.45GHz. 3.45GHz x 32 cores = 110.4 total GHz :eek:
Now apply 105 BCLK and that jumps 5% to ~116 total GHZ o_O

Here's my Cinebench scores... best being 5374

HAS-Cinebench.jpg
 
  • Like
Reactions: custom90gt

custom90gt

Member
Feb 9, 2017
38
7
41
No. Under heavy loads, like Cinebench, cores flip between 3.4 and 3.5Ghz with most remaining at the higher speed.
So call it 3.45GHz. 3.45GHz x 32 cores = 110.4 total GHz :eek:
Now apply 105 BCLK and that jumps 5% to ~116 total GHZ o_O

Here's my Cinebench scores... best being 5374

Where did you get your 2696's from?
 

SZAO6

Junior Member
Feb 19, 2017
13
1
51
Can someone upload a randir v3x2_50_vcc.efi download link ?
I have searched for a couple of time, and not found.
I want ot test it with my new 2686v3. On C-Payne ffs I only get 3,0GHz, what is not enough for me.
 

SZAO6

Junior Member
Feb 19, 2017
13
1
51
downloaded and installed v3x2, but still only 3,0GHz on my 2686v3. I have a QGN8 QS version.
is it possible to get more than only 3,0GHz on all cores with this processor on a Asrock X99m 3.1 killer? I'm not satisfied with that I get for now. I though I will be able to get more, as I read here.
 

MooNiSLe

Junior Member
Apr 23, 2017
4
1
81

@kjboughton Great lecture! And what I am wondering is which two cores you choose to disable? Is that core 16 and core 17? I'm going to try this but there are 18 cores in Acitve Cores Management function named core 0 to 17 :smile:
 

Dufus

Senior member
Sep 20, 2010
675
119
101
@kjboughton that does not tell me if you are experiencing WHEA corrections or not. The correction is what enables things to continue so you may be getting them and not know it if you do not specifically check.

Unfortunately I only have 14 cores with 30x multi and they all run at top turbo unless AVX2 instructions are present then fully loaded with HT enabled results in dropping 1 bin. Might have shot myself in the foot for an 18 core chip by spilling the beans to early. :(

PC6 and CC6 are two types of C6 C-state, a picture would be very helpful to ensure the question isn't misunderstood. No, I did not refer to XTU, there was some other software which for the life of me I cannot remember but was stopped in development but could still be used. There is also Intel PCM and I think some others to. You could also use RWEverything with the residency MSR selected but be careful with this software as exception handling seems lacking and blue screens are easy to be had doing the wrong thing. In fact any MSR util would probably do.
 

kjboughton

Senior member
Dec 19, 2007
330
118
116
In Asrock x99 bios exist to disable Numa?
i don't find any node interleaving in memory settings!

NUMA pertains to multi-socket systems only. Single-CPU systems have no action.

EDIT: Exception to that rule, of course, being Threadripper which operates naively as two (2) 8-core NUMA nodes. This is similiar to setting Xeon v3+ Snoop Mode to Cluster-on-Die (COD). Meaning, if I set this on my system with a pair of E5-2696 v3 the result being four (4) 8-core NUMA nodes (in the case I disable two cores for a total of 16 per CPU).
 
Last edited:
  • Like
Reactions: Cata40

kjboughton

Senior member
Dec 19, 2007
330
118
116
@kjboughton Great lecture! And what I am wondering is which two cores you choose to disable?

Control is not that fine. Your BIOS should have an option to set the number of active cores up to total core count. I would imagine the PCU makes the decision as to what cores to disable. I couldnt tell you precisely which ones in the die are disabled. I suspect Ring2 with the 10 cores would become 8 cores like Ring1. Confirmation could probably be made but would require some thinking on how exactly to go about doing so.