What controls Turbo Core in Xeons?

[Dufus]

Yes, you got it: 95 and 90...

Okay, as DTS stays the same and software uses Tjmax to calculate temperature (Tjmax - DTS) then with 95C vs 90C readings will be reported 5C different even though they are the same. My CPU default is 95C and depending on BIOS used can show either 92C or 90C so presumably Tjmax can be changed by BIOS which seems to kind of defeat it's purpose. Also without ucode the "Thermal Margin" register doesn't work, at least on my CPU. Thermal Margin offers feedback for cooling solutions but I'm not sure if it's widely used or even used at all.

No. Supermicro boards do not have the option to ignore TDP limit.

Have you tried setting power limits? perhaps via the RU utility that was introduced in this thread by 'The Stilt'. Maybe your board can work even better than some X99 boards

 

[Saren]

Okay, as DTS stays the same and software uses Tjmax to calculate temperature (Tjmax - DTS) then with 95C vs 90C readings will be reported 5C different even though they are the same. My CPU default is 95C and depending on BIOS used can show either 92C or 90C so presumably Tjmax can be changed by BIOS which seems to kind of defeat it's purpose. Also without ucode the "Thermal Margin" register doesn't work, at least on my CPU. Thermal Margin offers feedback for cooling solutions but I'm not sure if it's widely used or even used at all.

Have you tried setting power limits? perhaps via the RU utility that was introduced in this thread by 'The Stilt'. Maybe your board can work even better than some X99 boards

No, supermicro boards do not have any access to voltage settings, only CPU states at most.

 

[Micrornd]

Did start a BIOS mod myself but with the increase in CPU prices that has come to a stop and remains in stasis. I personally can no longer justify buying a 2696 and some other chips to continue for now. Perhaps some time later on when prices come down if I have not already sold everything by then.

Well, all I know for sure is that since this thread started, the 2696 v3s I've sold have risen in price (auction and buy now) by over $300US and are still climbing.

 

[Saren]

Well, all I know for sure is that since this thread started, the 2696 v3s I've sold have risen in price (auction and buy now) by over $300US and are still climbing.

IMO 2696 no longer has value if you are not going to use it in dual socket systems. Just buy 1950x instead (unless you do gpu passthrough...).

 

[kjboughton]

I have added PowerCut to my EFI driver. Tell me what you want and I will build it included. You need to be aware that this feature requires extra attention in that it survives a warm AND cold boot. The only way to remove it to allow a fresh re-application (including new VCCIN) is with a complete power cycle at the PSU.

Again, tell me what you want specifically and i will build it. My driver has about 8 different options to be set not including another three voltages PER CPU.

option choices:

// 0 for auto-max Core turbo multiplier, not to exceed fused limit, no less than MFM
const UINTN SET_MAX_TURBO_RATIO = 0;

// 0 for auto-max Uncore multiplier, not to exceed fused limit, no less than 12
const UINTN SET_MAX_UNCORE_RATIO = 0;

// disable Uncore low power state, i.e. force frequency to remain at max
const BOOLEAN DIS_UNCORE_PWR_MAN = FALSE;

// disable FIVR Faults
const BOOLEAN DIS_FIVR_FAULTS = TRUE;

// disable FIVR Efficiency Management
const BOOLEAN DIS_FIVR_EFF_MAN = TRUE;

// disable auto VCCIN and program fixed value, not recommended if can be set in BIOS, required if using PowerCut
const BOOLEAN SET_SVID_FIXED_VCCIN = FALSE;

// disable FIVR SVID Control ("PowerCut"), forces SET_SVID_FIXED_VCCIN = TRUE
const BOOLEAN DIS_FIVR_SVID_BUS = FALSE;

// set Overlocking Lock Bit at completion of programming (recommended)
const BOOLEAN SET_OC_LOCK = TRUE;

I also need to know what Core and Uncore voltage offsets (in mV) that you would like, if any.

If you use PowerCut, you also need to give me your desired VCCIN.

 

[Saren]

I have added PowerCut to my EFI driver. Tell me what you want and i will build it included. You meed to be aware that that feature requires extra attention in that it survives a warm AND cold boot. The only way to remove it to allow a fresh reapplication (including new VCCIN) is with a complete power cycle at the PSU.

Again, tell me what you want specifically and i will build it. My driver has about 8 different options to be set not including another three voltages PER CPU.

Hi, I am afraid I need a PowerCut driver with no voltage reduction since my silicon is a piece of poop.
By the way, what's the difference between v3x4_50_50_1pt650pc.efi and v3x4_50_50_1pt950pc.efi? Which one is more stable? Thanks.

 

[Cata40]

I have added PowerCut to my EFI driver. Tell me what you want and i will build it included. You meed to be aware that that feature requires extra attention in that it survives a warm AND cold boot. The only way to remove it to allow a fresh reapplication (including new VCCIN) is with a complete power cycle at the PSU.

Again, tell me what you want specifically and i will build it. My driver has about 8 different options to be set not including another three voltages PER CPU.

I need a driver with -50 cpu cache volatage offset, -65 Vcore Voltage additional ofsset and -71 at system agent voltage offset

 

[kjboughton]

1pt950 = 1 point 950 = 1.950 (VCCIN)

Which one is more stable? you tell me...

 

[kjboughton]

I need a driver with -50 cpu cache volatage offset, -65 Vcore Voltage additional ofsset and -71 at system agent voltage offset

no PowerCut, yes PowerCut?

 

[kjboughton]

Hi, I am afraid I need a PowerCut driver with no voltage reduction since my silicon is a piece of poop.
By the way, what's the difference between v3x4_50_50_1pt650pc.efi and v3x4_50_50_1pt950pc.efi? Which one is more stable? Thanks.

latest RC8 build
SET_MAX_TURBO_RATIO = 0 (auto max)
SET_MAX_UNCORE_RATIO = 0 (auto max)
DIS_UNCORE_PWR_MAN = FALSE
DIS_FIVR_FAULTS = TRUE
DIS_FIVR_EFF_MAN = TRUE
SET_SVID_FIXED_VCCIN = TRUE
DIS_FIVR_SVID_BUS = TRUE >>>> "PowerCut" ENABLED <<<<
SET_OC_LOCK = TRUE

fixed VCCIN set to 1.825V
no changes to Vcore or CLR voltages

download here

 

[Saren]

latest RC8 build
SET_MAX_TURBO_RATIO = 0 (auto max)
SET_MAX_UNCORE_RATIO = 0 (auto max)
DIS_UNCORE_PWR_MAN = FALSE
DIS_FIVR_FAULTS = TRUE
DIS_FIVR_EFF_MAN = TRUE
SET_SVID_FIXED_VCCIN = TRUE
DIS_FIVR_SVID_BUS = TRUE >>>> "PowerCut" ENABLED <<<<
SET_OC_LOCK = TRUE

fixed VCCIN set to 1.825V
no changes to Vcore or CLR voltages

download here

I got 2.7 normal full turbo and 2.5 avx2... what am I missing out there? Is it because of 0x3A microcode? I will test it again with recommended microcode later.

Thanks btw.

 

[Cata40]

no PowerCut, yes PowerCut?

No power cut, at these value, works very stable

 

[kjboughton]

I need a driver with -50 cpu cache volatage offset, -65 Vcore Voltage additional ofsset and -71 at system agent voltage offset

download here

 

[wiseguyuk]

@kjboughton , I am almost there I think. I successfully flashed your modified bios.I removed dll file before flashing bios.

When I load any efi driver, it shows it is succesfully loaded. Even your latest powercut one.
I have tried a lot of efi drivers in vain.

Windows starts, the loading icon runs for a few seconds & it freezes. what am I doing wrong?

 

[kjboughton]

@kjboughton , I am almost there I think. I successfully flashed your modified bios.I removed dll file before flashing bios.

When I load any efi driver, it shows it is succesfully loaded. Even your latest powercut one.
I have tried a lot of efi drivers in vain.

Windows starts, the loading icon runs for a few seconds & it freezes. what am I doing wrong?

disable C3, disable C6, package limit C0/C1
MAKE SURE BOOT LOGGING IS ENABLED... for whatever reason, this halts Windows load every time if disabled.... I am in the middle of some stress testing right now... let me know if you don't know what I mean and I'll get the exact setting for you.

So, HWiNFO64 shows no uCode loaded then with the Windows file removed/renamed?

 

[kjboughton]

I got 2.7 normal full turbo and 2.5 avx2... what am I missing out there? Is it because of 0x3A microcode?

uCode 27 (0x1F) is the last-known freely-available uCode with no AVX2 clock reduction...

Also, looks like you might not have removed the microcode as intended. What CPU is this? Stock E5-2696 v3 is 2.8GHz all-core turbo...

 

[wiseguyuk]

Hi, I got all the C state settings done. I am not able to locate boot logging in BIOS. Can you please highlight where I can find this ? is this in Windows ?

 

[kjboughton]

Hi, I got all the C state settings done. I am not able to locate boot logging in BIOS. Can you please highlight where I can find this ? is this in Windows ?

in BIOS:
Event Logs
Change Smbios Event Log Setting
Smbios Event Log => ENABLED
Log System Boot Event => ENABLED

 

[wiseguyuk]

in BIOS:
Event Logs
Change Smbios Event Log Setting
Smbios Event Log => ENABLED
Log System Boot Event => ENABLED

Wow.. You really know this inside out I can't thank you enough. Yes, It is working for me , first step.

I used your latest efi driver & 1F microcode file. I get only 2993 Mhz now small improvement.
What should I do get atleast 3500 Mhz ?

 

[kjboughton]

Excellent work.

What are you using to stress you system in that screenshot? I suggest using CPU-Z as it's quick and effective and a good way to compare apples to apples as max frequency is definitely a function of load.

How many cores enabled per CPU? 18? I recommend 16. This can also be set in BIOS.

Disable VMX if you're not using Virtualization. This saves more power for the cores.

Now start lowering the FVID offsets... I suggest you try a 50x2 EFI (-50mV to Cores, -50mV to Uncores) and see where that gets you... replace existing EFI driver with new and restart. Be sure to power all the way down and flip rocker switch on PSU. Watch all lights go out on MB (you do have the side of the case open right?) and count to 10. Then start 'er back up.

https://github.com/freecableguy/v3x4/releases/tag/v3x4-0.10b-i306f2-rc7
Grab "v3x4-0.10b-i306f2-rc7_50x2.efi"

Ultimately, I feel there's a strong chance we can get you to -90mV Core and -70mV Uncore which if your cooling holds up should get you to that magical 3.5 or 3.6Ghz. Most heavy workload programs will see max sustained 3.4GHz. Can't complain.

Also, that 2993 is really 3GHz. I see your BCLK is reading a little low @ 99.8MHz. I bet you have Spread Spectrum enabled and its downclocking a little to keep the max below 100MHz. Disabled this from the main page in your BIOS.

EDIT: Wow, 128GB of LRDIMMs... what did you pay for those? But I see you are only running dual-channel on each processor... you know going to quad channel will essentially double your peak BW? You NEED to do that with so many cores.

 

[wiseguyuk]

Excellent work.

EDIT: Wow, 128GB of LRDIMMs... what did you pay for those? But I see you are only running dual-channel on each processor... you know going to quad channel will essentially double your peak BW. You NEED to do that with so many cores.

I bought them on ebay.co.uk for £750.

http://www.ebay.co.uk/itm/128GB-4x3...e=STRK:MEBIDX:IT&_trksid=p2057872.m2749.l2649

 

[kjboughton]

I bought them on ebay.co.uk for £750.

http://www.ebay.co.uk/itm/128GB-4x32GB-Samsung-ECC-RAM-DDR4-2133MHz-CL15-4DRx4-PC4-2133P/182785738584?ssPageName=STRK:MEBIDX:IT&_trksid=p2057872.m2749.l2649

Get a second set. Must also be LRDIMMs (I see that detail is missing from that listing) and same Rank configuration. Are those SR, DR, quad?

 

[custom90gt]

I have an odd issue maybe someone has seen.
I haven't been running any ucode on my 2696 and decided I'd try version 27. However after installing it with cpumcupdate2.1 I noticed my cpu is throttling.

Looking at throttle stop for limit reasons it says:
Power budget
EDP current
max turbo
multi turbo
core P1


Any ideas? I'm going to try version 39 and see what it does.
Thanks!

 

[KW8cSgF53G]

download here

Turely sorry for bothering you again!

I also wanna test a -50mV, -50mV, -50mV with IA, LLC, SA voltages offset on newest RCs (like:v3x4-0.10b-i306f2-rc8_50_50_50) build,
casue it has "vary uncore frequency" feature since RC5, may lowering the idle or full loading status temperature more and more than haven't it!

Really great work!

(Sorry for my bad english.)

 

[wiseguyuk]

Excellent work.

What are you using to stress you system in that screenshot? I suggest using CPU-Z as it's quick and effective and a good way to compare apples to apples as max frequency is definitely a function of load.

How many cores enabled per CPU? 18? I recommend 16. This can also be set in BIOS.

Disable VMX if you're not using Virtualization. This saves more power for the cores.

Now start lowering the FVID offsets... I suggest you try a 50x2 EFI (-50mV to Cores, -50mV to Uncores) and see where that gets you... replace existing EFI driver with new and restart. Be sure to power all the way down and flip rocker switch on PSU. Watch all lights go out on MB (you do have the side of the case open right?) and count to 10. Then start 'er back up.

https://github.com/freecableguy/v3x4/releases/tag/v3x4-0.10b-i306f2-rc7
Grab "v3x4-0.10b-i306f2-rc7_50x2.efi"

Ultimately, I feel there's a strong chance we can get you to -90mV Core and -70mV Uncore which if your cooling holds up should get you to that magical 3.5 or 3.6Ghz. Most heavy workload programs will see max sustained 3.4GHz. Can't complain.

Also, that 2993 is really 3GHz. I see your BCLK is reading a little low @ 99.8MHz. I bet you have Spread Spectrum enabled and its downclocking a little to keep the max below 100MHz. Disabled this from the main page in your BIOS.

EDIT: Wow, 128GB of LRDIMMs... what did you pay for those? But I see you are only running dual-channel on each processor... you know going to quad channel will essentially double your peak BW? You NEED to do that with so many cores.

• Loaded the 50x2 EFI
• I enabled only 16 cores per CPU, total of 32 now.
• Disabled VMX (although I will enable it later as I run a lot of VMware images, hence the need for 128gb memory)
• Disabled Spread Spectrum

I have H80i v2 cpu coolers & they are holding up very well. I use a chess program to stress the CPUs.

For 50,50 I get 3300 Mhz and for 90,70. I get 3400 Mhz sustained performance. The highest temperature recorded was 60'C.

This is for 90,70