What controls Turbo Core in Xeons?

[kjboughton]

@Dufus
I have seen WHEA error corrections reported in the past. Ive also seen BSOD with umcorrectable WHEA errors as the culprit. I assume some errors are being properly cured as I am running Registered ECC memory. Does that answer the question?

 

[Dufus]

Partially. Never mind, I see you are enjoying your setup and these questions are probably getting in the way of that.

FWIW

The WHEA errors I saw were on the PCIE bus which has an NVMe attached and had nothing to do with DRAM. So either a bclk reduction or link speed change for me was required to stop them.

On Asrock it is possible to individually select which cores are disabled. Each core is given an APIC ID. I don't think this will tell the relation to each core on the die but at least it makes an opportunity to disable the worse cores if one wants to do that.

Looks like for E5 v3 DRAM power readings may be grossly inaccurate when using software such as HWMonitor or HWiNFO so perhaps it was a bad idea for me to suggest HWiNFO to check.

 

[XeKToReX]

Haven't managed to get this to work on my X99E ITX, modding the bios to remove the microcode, everything seems to go fine, BIOS writes to the board and then it just won't boot, fans spin up but blank screen. I've done everything from scratch twice now and same thing.

Could it be because I'm using an ES/QS Xeon?

 

[Dufus]

If it's 306f2 then shouldn't be a problem. I think @CANONKONG made a bunch of modified BIOS, have you checked?

 

[XeKToReX]

Thanks Dufus, I've found the dl links, will give it a shot now and see how it goes!

Edit: The BIOS flashed OK which is good, now to get the driver loaded and the windows changes made and see how it goes! Thanks again

 

[kjboughton]

So either a bclk reduction or link speed change for me was required to stop them.

So, I'll stop tap dancing around the question then...
106MHz is no POST
105MHz is on the ragged edge for heavy workloads
104MHz is quasi-stable
103MHz and below operates like stock (I run -50mV here)
100MHz is cleanest from a specs-manship standpoint (and what a great reason to run Xeons) and allows for -70mV

4GHz is mainly for flash appeal. 3.8GHz top turbo bin is more than enough and is completely stable.

 

[wiseguyuk]

US-based eBay seller

@kjboughton Hi, What cooling system are you using ?

can you please post your modified bios ?

 

[wiseguyuk]

Hi All, I want to build a new quiet system with a DUAL E5-2696V3 with 128GB ram . Can you please suggest

• The best board & link me to the modified BIOS ? It is for 128GB RAM.
• Would I need ECC memory modules & would it need cooling as well ?
• Which would be the best case for less noise ?
• What cooling system should I use for running it for long duration at the max frequency ?

I already have a Dell Precision T7910 with Dual E5-2696V3 with liquid cooling. Will I be able to modify the dell bios ?

http://www.dell.com/support/home/us/en/04/drivers/driversdetails?driverId=5MRMV&fileId=1961672443&osCode=W764&productCode=precision-t7910-workstation&languageCode=en&categoryId=BI

Thank you very much in advance

 

[MOF]

I already have a Dell Precision T7910 with Dual E5-2696V3 with liquid cooling. Will I be able to modify the dell bios ?

Branded barebone systems are very sensitive to user modification.

 

[SZAO6]

how to change the tdp power limits? I have it changed in bios, but it will not affect when the system starts. in the hwinfo I still have the standard values. I nelive this is the problem to get my 2686v3 on the 3,5 all core turbo. should I mod the bios to get higher tdp values? how to do this? I have read something about to change the tdp values but I still don't know how I should do this and which tool I have to use.

 

[MOF]

Haven't managed to get this to work on my X99E ITX, modding the bios to remove the microcode, everything seems to go fine, BIOS writes to the board and then it just won't boot, fans spin up but blank screen. I've done everything from scratch twice now and same thing.

Could it be because I'm using an ES/QS Xeon?

Same here.
2696V3 ES (306f2) & Asrock Rock EP2C612D16

 

[custom90gt]

I'm not able to get C3 (or any powerstates besides C1) to work on my Gigabyte X99 Extreme Gaming motherboard. I went back to the stock bios and they still don't work, not sure if anyone here has this same board. Kind of ticks me off because I had zero issues with my Asrock X99m...

 

[Dufus]

Same here.2696V3 ES (306f2) & Asrock Rock EP2C612D16

how to change the tdp power limits? I have it changed in bios, but it will not affect when the system starts.

I'm not able to get C3 (or any powerstates besides C1) to work on my Gigabyte X99 Extreme Gaming motherboard.

Again, this is why a proper BIOS mod is IMHO the preferred method. Some BIOS may refuse to continue if no microcode update happens, maybe could try changing the update version dword to zero so if it's a simple compare after trigger it would pass or better still change the underlying code.

 

[kjboughton]

@Dufus

I have disabled Intel ME for my system... see below for in-BIOS status results (before & after).

Modifications were based on instructions provided in the following thread:
https://forums.anandtech.com/threads/disabling-intels-management-engine-me.2517366/

The system boots and operates the same as if ME were enabled. I see no difference in performance with respect to max power limits applied.
Undoubtedly, further testing and investigation is warranted for which I am happy to assist.

For those that wish to attempt the same... some hints from my success:

1. The Python script provided (me_cleaner.py) did not work on my modified BIOS. I was required to start with a new BIOS, modify it first using the method provided, edit the resulting ROM to remove the SEC Core write-protection (using UEFI Tool), and finally remove the offending Haswell-E microcode using UBU.
2. Running NO microcode with this modification results in all cores locked at HFM (2.3GHz for 2696v3). Turbo'ing to frequencies above HFM (up to and including max Turbo) are possible and works as expected; however, I simply do not see cores falling back to 1.2GHz (LFM) with Windows in "High Performance" mode. Changing this back to "Balanced" results in idle at LFM. And of course, "Power Saver" keeps all cores at LFM regardless of workload. I suspect this has to do with the ICC module this is not loaded...hmmmm

BEFORE... Operational (No Error)

• SiEn (Silicon Enable)
• NM (Non-volatile Memory)
• PECI Proxy (a.k.a. HECI)
• ICC (Integrated Clock Controller)
• PM (Power Management?)

AFTER... Recovery (No Error)

• 
  
  
  • SiEn (Silicon Enable) only

 

[wiseguyuk]

@kjboughton. Please post your modified BIOS & EFI files. I am trying to build a system with the same board & it will be helpful. Thank you very much in advance.

Have you enabled all the 36 cores ? It would be great if you can make a guide for this board. You will be my hero

 

[kjboughton]

ASUS Z10PE-D8 WS modified UEFI BIOS (Haswell-E microcode removed)

EFI Driver Collection

Collection of various Haswell-E microcode (in .DAT file format)

Water-cooling recommended:

• 2x EK Supremacy EVO CPU water blocks
• XPSC RX360 Triple Radiator V3 (120mm)
• Mayhems Ultra Clear Tubing 1/2" x 3/4" (13/19mm)
  
• EK-XTOP with dual D5 12V pumps
• Reservoir of your choice
• 6x 120mm fans of your choice
• Fan controller of your choice (optional)

I have operated the system in the past with all cores enabled but generally find better performance when reduced to 16 cores/CPU.

NUMA = disabled
Hyper Threading = disabled
Snoop Mode = Early Snoop

These are the big points. I will put up a BIOS setting guide shortly.

 

[wiseguyuk]

Thank you very much. You are a star.

I am using Corsair Hydro Series™ H80i v2 High Performance Liquid CPU Coolers. two of them, one for each CPU.
http://www.corsair.com/en-gb/hydro-series-h80i-v2-high-performance-liquid-cpu-cooler

Do you think they are not good enough ?

Thank you for the collection of files. Which specific EFI & Microcode files should I be using ? If you please explain a little , it will be useful.

Why should NUMA be disabled ?

 

[Dufus]

@kjboughton

ICC in this case is Integrated Clock Controller

Not sure why you have directed this at me but personally I haven't had reason to warrant disabling ME or MME (Mystery Management Engine). Why mystery, because not many people understand exactly what it is doing, including myself. In fact a well known ASUS rep a few years back on the P67 platform claimed it was impossible to update the ME Firmware and supposedly this came from ASUS themselves. (Users had heard it reported the ME updates fixed some issues that people were having and requested ASUS for it). Well one or two days later a user posted how to do it. Unfortunately that big ASUS BIOS thread on HF has been taken down some time ago so cannot provide a link. For that particular platform the updates were reported to help with issues users were experiencing so it has its uses.

 

[kjboughton]

@Dufus

The idea being that removing or entirely disabling portions of ME or maybe modifying other settings may allow for higher performance (i.e. higher or unbound power limits). The alternative being that this functionality is instantiated in silicon and therefore cannot be changed. Perhaps there are fuses that blown to set a TDP limit. Perhaps it is an internal hidden register. Maybe its based on the PM module operating within ME. I simply look at this as one more attack vector worth exploring further. I doubt the persons that did the research up to this point were too concerned with overclocking during their experiments. We are.

 

[Dufus]

Ahh, okay. Perhaps if ME were modified the correct way? I have no idea how to do that.

Are you suffering TDP throttling?

 

[kjboughton]

Well, I'll put it to you this way. Lighter loads, all cores will reach and sustain top turbo bin (3.8GHz); sustained full loads top turbo drops to 3.5GHz. I have reason to believe this is a power/current-related limit.

 

[Dufus]

Should be able to check CPU throttling, HWiNFO? Perhaps also it could be mainboard VRM?

 

[MOF]

Ahh, okay. Perhaps if ME were modified the correct way? I have no idea how to do that.

You can use Intel Flash Image Tool (fit.exe) for editing ME firmware values. (for Server MB firmware you need a different version of Flash Image Tool, spsfit.exe)


http://www.win-raid.com/t596f39-Intel-Management-Engine-Drivers-Firmware-amp-System-Tools.html

 

[kjboughton]

@MOF

I have been wholly unsuccessful in locating an SPS FIT. FIT packages for consumer boards can be found anywhere. Server Platform Services... not so much.

 

[MOF]

@kjboughton

Yes, it's not easy. I only found two versions of spsfit(3.0.5 & 4.0.3) on internet and neither of them is suitable for my motherboard.