What controls Turbo Core in Xeons?

Page 86 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

XDrego

Member
May 5, 2017
28
2
41
use v3x2_50_39_vcc still the best works fine for 2 cpu but disables all power limits(watch cooling) but balanced plan goes to lowest multiplier fine
 

rottwag

Member
Apr 2, 2017
77
11
41
Hi all,

I am back to this amazing Thread again :) ... I dropped in on page 20 or so with my retail "Xeon E5 2618L V3" 8 Core CPU having it running rock stable for 1 year now @104,9x33 =3,46Ghz on all Cores on a ASUS X99-A !!! (Actually it is a LOW Power CPU with a TDP of 75W... but single Core Multi is 34! LOL)

I tried several microcodes one year ago, but the "v3x2_50_39" works best (all core Multi of 33 - but 34 would be the max possible single core, but never managed to reach that)

NOW I got a XEON E5 2695 V3 ES (Stepping 2) that has a Single Core Turbo of 35x ?! (retail Version only as 33x)

#1 Could I simply replace the CPU without any BIOS reset and see if it works with my new CPU?
#2 Do you have any recommendations of microcode/EFI Drivers that I should try as in the meanwhile a lot has happended here?
#3 Is there anything else to be considered regarding BIOS or Win10 Settings that appeared to be helpfull?
#4 Is there a new simpler way to modify the BIOS for my ASUS that it was 1 year ago?

PS: In Cinebench R15 I get 1401 (8C/16T) and 149 for Singe core - quiet good for only 8 Cores, no?

Thanks for you help!
 

evilr00t

Member
Nov 5, 2013
29
8
81
NOW I got a XEON E5 2695 V3 ES (Stepping 2) that has a Single Core Turbo of 35x ?! (retail Version only as 33x)

#1 Could I simply replace the CPU without any BIOS reset and see if it works with my new CPU?
#2 Do you have any recommendations of microcode/EFI Drivers that I should try as in the meanwhile a lot has happended here?
#3 Is there anything else to be considered regarding BIOS or Win10 Settings that appeared to be helpfull?
#4 Is there a new simpler way to modify the BIOS for my ASUS that it was 1 year ago?

PS: In Cinebench R15 I get 1401 (8C/16T) and 149 for Singe core - quiet good for only 8 Cores, no?

Thanks for you help!

If you have one of the cpuid=0x306f2 QFxx C0-step CPUs, congratulations, your chip runs faster than the official ones, although I wouldn't run PCIe gen3 on them! If, on the other hand, you have a cpuid=0x306f1 B-step CPU, you'll probably not be able to apply this hack as the chips usually need microcode loaded early.
 

rottwag

Member
Apr 2, 2017
77
11
41
yes, it is a 306f2 QFQG !! see screenshot..ultra cool! i actually did not know, that such versions exist??
my gtx 1070 runs with PCIe 2.0 anyway as it only runs 1% slower than PCIe 3.0 and my board switches to 2.0 automatically when BCLK is higher 100.

Any comments of the rest of my questions?
https://photos.app.goo.gl/99nNylzADl5Lo3ww2

99nNylzADl5Lo3ww2
 

traderjay

Senior member
Sep 24, 2015
220
165
116
Hey Guys - I am a bit lost on how to mod my bios to unlock the turbo on my E5 2696V3. Can one of you kindly show me step-by-step? This is what James said but I am so lost lol:

Regarding of loading EFI file before OS, you first need to have a flash drive and format it as FAT32. Then copy all files within the folder that I uploaded (Make sure to have startup.nsh file (you can edit it using notepad) and EFI folder to be at the root of the drive).

And, you will need to change boot options to UEFI or dual. I have my OS installed with UEFI so I changed my boot order to be UEFI: flash drive to be 1st and then UEFI: Windows boot manager to be the 2nd. This way, EFI loaded and then go to Windows boot on every boot.

If you want to load it manually, you may Press F11 (boot menu) at the boot time. And, select UEFI: [your flash drive]. By default, it will try to execute the startup.nsh script automatically, if you want to stop it, just press ESC when you have option and then you can execute your own commands.

Can someone help me out lol...Happy 2018!
 

kjboughton

Senior member
Dec 19, 2007
330
118
116
That would be great, but where did you get that from? Any source? Shouldn't there be a list anywhere with affected CPUs?

And I was informed incorrectly...
https://security-center.intel.com/advisory.aspx?intelid=INTEL-SA-00088&languageid=en-fr

Of course, here's what *not* being shared by Intel:
SIMPLY DISABLING MANAGEMENT ENGINE (ME) IS ALL THAT IS NEEDED TO DEFEAT THIS THREAT
https://github.com/corna/me_cleaner
  • The network stack (partition NFTP) is removed
  • The PAVP (partition MDMV, module JCOM) is removed

To summarize: Intel would rather force a performance loss on you, the duped consumer, than they disable "Management Engine" in their fix and give up their back-door to your system.

Take your system back and DISABLE Management Engine!

ME ENABLED:
ME_enabled1.jpg


All but critical ME services DISABLED (only 'SiEn' or Silicon Enable is ENABLED):
ME_disabled1.jpg
 
Last edited:

Dinkydau

Member
Apr 1, 2012
50
5
71
After hours of trying settings I have the modification working on an e5 4667 v3.
ixm8Qnl.png


Here are some of the problems that I encountered and how I solved them. Maybe it's useful for someone else:

0. Does the motherboard work with this CPU? My motherboard is an ASRock X99 Extreme4. The used CPU does not appear on the CPU support list while other ASRock motherboards do support it. Strange. Maybe support is not official but it does actually work. I guess all the E5 2xxx v3 and 4xxx v3 CPUs work just fine.

1. There's an excellent step-by-step guide on page 9. Anyone wondering what is going on in this thread should just go to page 9 and read that guide. Every step is explained in detail.

2. Not all bios versions seem to work. I choose bios 2.10 to mod at first because I read a report that some settings were removed in later versions. It didn't work. I had to recover by using the backup bios chip. Also modding version 3.60 didn't work. Then mysteriously version 3.30 worked. There's nothing that I did differently to mod the bios. In case it helps anyone, here is the modded version 3.30:
https://mega.nz/#!H1dhFCoD!eRO-XC-CfxVaxNFBYEXoGjVn7fan9o1OAt3ykyRtA1E
Again this is for the "ASRock X99 Extreme4". I explicitly mention it's not for the Extreme4/3.1 version.

3. To auto-load the EFI driver from the system drive it needs to be GPT formatted but mine is MBR formatted. As a workaround I am now leaving the USB-device with the driver connected permanently so it can be auto-loaded from there. I actually recommend doing this at first because it allows to change the EFI file very easily (from within the OS) later on. Placing the EFI on the system drive is more suitable as a permanent solution. I remember reading some complaints about the driver not loading every time. I noticed that the driver seems to load if and only if the USB drive containing it was ejected (but not physically removed) before the last shutdown. I'm probably going to try to automate that some time.

4. AVX workloads were still restricting the CPU to normal speed! I experimented with other microcode and EFI driver versions and found the following combination of settings to work best:
4.1 Driver:
The exact one I'm using is v3_705020vcc1.8.EFI. This one magically solves the AVX problem... but not on it's own, because:
4.2 Based on sciff's idea that AVX speed may somehow depend on the BCLK I increased the BCLK slightly, to 102. A very small change such as 100.2 didn't work and 103 made my system unstable. Only after increasing the BCLK did the AVX multiplier go up.
4.3 Microcode: Version 27 supposedly gives the best speeds but my multiplier under AVX load actually went up from 28 to 29 when I updated the microcode to 39.
I'm now getting the maximum turbo frequency on all cores with prime95 and other programs that use AVX so there's nothing more that can be done do to improve the performance of this CPU.

5. I read a rumor that some programs that change the MSRs stop the mod from working, such as throttlestop. I'm actually using throttlestop and many other programs including speedfan, openhardwaremonitor, hwinfo64, cpu-z and realtemp without any problems.

6. Was there ever any doubt about virtualization working? Everything is working here. There are some engineering sample (ES) CPUs, however, that don't support virtualization. You can see this in HWiNFO64: if the VMX feature is green or red, you have virtualization. If it's grey, you don't.
 

sciff

Member
Mar 6, 2017
136
52
71
Can someone help me out lol...Happy 2018!
I WOULD NOT recommend unlocking dual 2696v3 on a Supermicro board. These boards usually have mediocre power delivery to the CPUs and poor cooling of VRMs. Your particular one (X10DRL-i) is no exception - one of the sockets doesn't even have a heatsink on its VRMs. Just bear in mind that one 2696v3 can consume up to 200W and sometimes even higher (in AVX2 software, for example). Most of the Supermicro motherboards are designed for up to 145W per CPU, but not higher than that.

However, if you insist and you're prepared to take the risk, I can make you a modified BIOS and we can go through the unlock process step-by-step.

0_165ba2_9e0e387f_orig.jpg


And I was informed incorrectly...
https://security-center.intel.com/advisory.aspx?intelid=INTEL-SA-00088&languageid=en-fr

Of course, here's what *not* being shared by Intel:
SIMPLY DISABLING MANAGEMENT ENGINE (ME) IS ALL THAT IS NEEDED TO DEFEAT THIS THREAT
https://github.com/corna/me_cleaner

To summarize: Intel would rather force a performance loss on you, the duped consumer, than they disable "Management Engine" in their fix and give up their back-door to your system. Take your system back and DISABLE ME!
Wow, thanks for the info, man! I have the same motherboard and I will disable it too. By the way, do you know why my system randomly freezes when I am in Windows with "C6 report" enabled in BIOS? C3 is disabled and C0/C1 is chosen for the first option. Boot logging etc. and many other options are set as you recommended. But I still can't find a way around "C6 freeze" issue.
 
Last edited:
  • Like
Reactions: kjboughton

kjboughton

Senior member
Dec 19, 2007
330
118
116
On the asus x99 boards, i don't find this intelRCsetup

Order is important... me_cleaner *must* be run *before* mod'ing BIOS to remove uCode

1. Download 'virgin' BIOS
2. De-encapsulate using UEFITool
3. Run me_cleaner (requires Python)
4. Modify to allow changes to SEC/CORE
5. Remove Haswell-E Microcode using UBU
6. Flash using external SPI device
7. Install BIOS chip
8. Enjoy no ME
 

kjboughton

Senior member
Dec 19, 2007
330
118
116
By the way, do you know why my system randomly freezes when I am in Windows with "C6 report" enabled in BIOS? C3 is disabled and C0/C1 is chosen for the first option. Boot logging etc. and many other options are set as you recommended. But I still can't find a way around "C6 freeze" issue.

C6 Report enabled with the earlier uCode *will* cause hang-ups (catastrophic error reported by MB HW indicator if you have one for each socket). This is a known "feature" of the CPU under light load (let me guess, you usually go away and come back to a frozen screen) that was corrected in later uCode but you lose the higher AVX multipler.... so... ENABLE C0/C1, ENABLE C3, DISABLE C6. Have found enabling C3 vice C6 gives the same benefit (lower power usage at idle) AND allows for higher AVX multi AND is perfectly stable. I've been up for months now without a reboot.
 

sciff

Member
Mar 6, 2017
136
52
71
C6 Report enabled with the earlier uCode *will* cause hang-ups (catastrophic error reported by MB HW indicator if you have one for each socket).
This is a known "feature" of the CPU under light load (let me guess, you usually go away and come back to a frozen screen) that was corrected in later uCode but you lose the higher AVX multipler.... so... ENABLE C0/C1, ENABLE C3, DISABLE C6.
Have found enabling C3 vice C6 gives the same benefit (lower power usage at idle) AND allows for higher AVX multi AND is perfectly stable. I've been up for months now without a reboot.
I don't use any µcode (reason why will be below).

No, I sit in front of my computer, but yes, the system is under light load or in idle when that happens. I don't remember it ever happening under load and I've read on some forum that it has to do with the inability of the cores to go into C6 power saving state (basically powering off) when negative voltage offsets are applied to them.

By the way, I get the same thing with µCU versions 1F, 27, 39 and 3A applied.

Thanks, I'm going to do that and then see if the system freezes!

3. Run me_cleaner (requires Python)
4. Modify to allow changes to SEC/CORE
5. Remove Haswell-E Microcode using UBU
6. Flash using external SPI device
Are you saying that ASUS BIOS Flashback will not accept the BIOS with ME disabled?

_____________________________

kjboughton, I've started compiling your UEFI DXE RC9 driver (I need my own offsets for my E5-2696s which are different - one CPU is Prod. unit (SR1XK), the other is Eng. sample (QGN7)), but I have an issue with that and a couple of questions.

1. I can't manage to compile the code without commenting out the following rows:



If I don't do that, then the build fails like this:



What is happening and why? I only have some knowledge of programming as I had it as a subject in a university, but I am no expert to really understand what's the issue here. So what is this about and what should I do? By the way, commenting out that small part of the code doesn't affect the driver's functionality in any way, I'm actually posting this from my workstation with this driver applied.

2. What should happen if I set CPU_SET_OC_LOCK constant as TRUE? What is this option for?

3. I always have double start when I power on my PC. It doesn't happen on reset and on warm boot, but it ALWAYS happens at cold boot. What can be causing this? Even changing BIOS settings and flashing an older version doesn't help.

4. Since I need maximum performance in Corona Render, I use µcode revision 1F. That way I used to get the same turbo boost in AVX2 workloads as in all other workloads (on my X99 ASUS board).

But here, on this motherboard with two CPUs I have a very strange issue. In regular workloads both CPUs boost well and the difference between them is minimal, like the one that I observed on my single socket ASUS mobo (where SR1XK is 100 MHz better than QGN7, x34 as opposed to x33). But the results that I get, say, in Cinebench, are less than I expected (a little above 5200 points, but nothing near 2*2800=5600 that I would want).

But the most trobling issue is the following. When I run Corona Benchmark or Corona Render, the SR1XK's cores all drop to x27 (consuming significantly less power and therefore heating up to a smaller degree), whereas QGN7's cores fluctuate around x32-x33 all the time. Enabling C3 and disabling C6 doesn't change anything. Two questions arise: a) why does this drop happen in AVX2 workload at all (even though with 3A µcode I get x29), and b) why does it seem like it's some other issue unrelated to the TDP? Like I said, the SR1XK cores are very stable at x27, there's no fluctuation at all, as if their frequency got artificially limited or cut by some feature, whereas QGN7 seems to actually reach it's TDP limit, hence the constant fluctuation (from uneven workload).

I tried swapping the CPUs, but it doesn't make any difference. Using your original pre-compiled RC9 drivers doesn't change anything either (powercut versions included). The SR1XK still boosts very poorly in AVX2. I even tried to remove QGN7 altogether and use SR1XK on its own in this motherboard - I got the same thing, x27 all-core in AVX2 with 1F µcode.

Therefore, for now, I use the system with no µCU loaded. I know it's against your recommendations (you said it's not optimal because 'bare' silicon has unfixed bugs in it and consumes more power), but I wanted to stress the system fully for test purposes.

By the way, in HWiNFO to the right from the Configurable TDP Level 1 (Down) wattages it says 1900 MHz for SR1XK and 2000 MHz for QGN7. What does this mean and where does this difference come from? Can it be relevant somehow?

 
Last edited:

sciff

Member
Mar 6, 2017
136
52
71
0. Does the motherboard work with this CPU? My motherboard is an ASRock X99 Extreme4. The used CPU does not appear on the CPU support list while other ASRock motherboards do support it. Strange. Maybe support is not official but it does actually work. I guess all the E5 2xxx v3 and 4xxx v3 CPUs work just fine.

I use server memory by Kingston that in not officially supported neither by ASUS X99 motherboards nor by ASUS C612-based boards. Not to even mention ASRock boards which have even less complete memory support lists as far as RDIMM memory is concerned.

However, it worked fine on my ASUS X99-E WS/USB 3.1 motherboard and works fine now on my new ASUS Z10PE-D8 WS.

You also won't find E5-2696v3 on any CPU support list at all, if we are talking about retail market motherboards. However, this CPU, largely similar to E5-2699v3 (but not the same and not even identified as the same model by the BIOS), will run on almost EVERY 2011-v3 motherboard that you can find!

Support lists for the CPUs and RAM are incomplete and if a type of CPU (given the same stepping, of course) or memory runs fine on a particular motherboard (i.e. it works with E5 v3 Xeons as such and with RDIMM memory), then there's a high chance that other similar models will work too.

3. To auto-load the EFI driver from the system drive it needs to be GPT formatted but mine is MBR formatted. As a workaround I am now leaving the USB-device with the driver connected permanently so it can be auto-loaded from there. I actually recommend doing this at first because it allows to change the EFI file very easily (from within the OS) later on. Placing the EFI on the system drive is more suitable as a permanent solution.

My recent setup with an ASUS X99 board had a system SSD formatted with MBR (hence, Windows was installed in legacy mode). Before I knew the unlock was possible on ASUS X99 boards without an EFI driver, I used one of the drivers and stored it on a hidden system partition which I could access either by running a .bat file that I created (and then it would open up in Explorer) or by temporarily assigning letter to that partition. I'm saying that GPT is not required.

Now I use a dual-CPU motherboard which doesn't allow the voltage offsets to be set manually in BIOS. Therefore, I use an EFI driver which I store on one of the partitions of my HDD that I use for storage (it's GPT, by the way). I decided so because I can easily disable the loading of the EFI driver just by disconnecting this drive and then even connect it back after the Windows fully loads! I would recommend you to do the same thing!

4.2 Based on sciff's idea that AVX speed may somehow depend on the BCLK I increased the BCLK slightly, to 102. A very small change such as 100.2 didn't work and 103 made my system unstable. Only after increasing the BCLK did the AVX multiplier go up.

I don't recall expressing such idea. I use 100.1 MHz BCLK merely for aesthetic reasons—I like the numbers that I get from it :)
(not 2998, 3298, 3397, etc., but 3003, 3304, 3404 instead)
 
Last edited:
  • Like
Reactions: Dinkydau

traderjay

Senior member
Sep 24, 2015
220
165
116
I WOULD NOT recommend unlocking dual 2696v3 on a Supermicro board. These boards usually have mediocre power delivery to the CPUs and poor cooling of VRMs. Your particular one (X10DRL-i) is no exception - one of the sockets doesn't even have a heatsink on its VRMs. Just bear in mind that one 2696v3 can consume up to 200W and sometimes even higher (in AVX2 software, for example). Most of the Supermicro motherboards are designed for up to 145W per CPU, but not higher than that.

However, if you insist and you're prepared to take the risk, I can make you a modified BIOS and we can go through the unlock process step-by-step.

0_165ba2_9e0e387f_orig.jpg



Wow, thanks for the info, man! I have the same motherboard and I will disable it too. By the way, do you know why my system randomly freezes when I am in Windows with "C6 report" enabled in BIOS? C3 is disabled and C0/C1 is chosen for the first option. Boot logging etc. and many other options are set as you recommended. But I still can't find a way around "C6 freeze" issue.

Hi Sciff - thanks for the reply. My mobo is actually a single socket X10SRL-F. Can you please walk me through the unlock process?
 

ajschot

Member
Feb 16, 2017
65
2
16
Order is important... me_cleaner *must* be run *before* mod'ing BIOS to remove uCode

1. Download 'virgin' BIOS
2. De-encapsulate using UEFITool
3. Run me_cleaner (requires Python)
4. Modify to allow changes to SEC/CORE
5. Remove Haswell-E Microcode using UBU
6. Flash using external SPI device
7. Install BIOS chip
8. Enjoy no ME

Sorry I am maybe stupid.... i try to edit my bios Asus ROG Strix X99 Gaming
steps 1-3 are done.... then i have a problem i don't understand step 4
Also... i need to remove MSR 0xE2 (for macOS) so i want to do a inbetweenstep between 4 and 5 (i think..) so Remove MSR 0xE2 with UEFIPatch.

Do you think this is the right order?

Also do i need to build my own EFI driver for E5-2696v3? Until now my E5-2683v3 on cheapest MSI board gives me almost same geekbench scores multicore then my E5-2696v3 on my ASUS RoG x99 Gaming (with v3x2_cup27.efi)

Also how do i get it back into a CAP file? I used BRenamerl but it just renames the file to .CAP but it is still a rom file not a AMI Aptio capsule
 
Last edited:

rottwag

Member
Apr 2, 2017
77
11
41
Order is important... me_cleaner *must* be run *before* mod'ing BIOS to remove uCode

1. Download 'virgin' BIOS
2. De-encapsulate using UEFITool
3. Run me_cleaner (requires Python)
4. Modify to allow changes to SEC/CORE
5. Remove Haswell-E Microcode using UBU
6. Flash using external SPI device
7. Install BIOS chip
8. Enjoy no ME


hmmm okay - but as far as I know Windows will be patched automatically via auto-update function to close this Memory leak thing, correct? If this is the case, then RAM-intense applications will slow down - it then does not make a difference if I have ME activated / deavtication in BIOS, no?

Can anybody confirm? Or did I get it wrong? :)
 

ajschot

Member
Feb 16, 2017
65
2
16
hmmm okay - but as far as I know Windows will be patched automatically via auto-update function to close this Memory leak thing, correct? If this is the case, then RAM-intense applications will slow down - it then does not make a difference if I have ME activated / deavtication in BIOS, no?

Can anybody confirm? Or did I get it wrong? :)
depends... maybe a microcode update in windows? then it is not a problem at all ;-) otherwise it will be slow again indeed
 

Cata40

Member
Mar 2, 2017
156
6
81
Order is important... me_cleaner *must* be run *before* mod'ing BIOS to remove uCode

1. Download 'virgin' BIOS
2. De-encapsulate using UEFITool
3. Run me_cleaner (requires Python)
4. Modify to allow changes to SEC/CORE
5. Remove Haswell-E Microcode using UBU
6. Flash using external SPI device
7. Install BIOS chip
8. Enjoy no ME
I have problems to intall python, in my windows, error
 

kjboughton

Senior member
Dec 19, 2007
330
118
116
Yeah apparently they are affected.
On a sidenote, some mobo manufacturers are pushing out bios updates, presumably with microcode patches. If we remove the microcode, wouldn't that be technically rendering the fix null?

Negative. You can run any ucode you want available for your processor type.
Remove ucode from BIOS is simply to allow for hack... following, any ucode or no ucode at all can be applied during OS boot process.
Hack simply requires that the ucode containing the patch to prevent this exploit not be in place at time of programming.
 
  • Like
Reactions: Cata40