The Threadripper with 128 GB RAM (!) BUILDERS thread

[DrMrLordX]

Sure, here it is:

That's b-die alright. Good find.

 

[cerosone]

Hi guys,
I am about to finish my system with Taichi x399 ,1950x and Enermax Liqtech TR4 360.
I'm just missing the 128 Gb DDR4 (>=3000 and <=CL16 ).
Can you please share /make a list with the supported nonECC/ECC DDR4 that you know / tested and are compatible (besides the QVL from https://www.asrock.com/mb/AMD/X399 Taichi/index.asp#Memory).
Thanks !

 

[DrMrLordX]

A good place to start would be to see what people are using in this thread.

 

[The Stilt]

Hi guys,
I am about to finish my system with Taichi x399 ,1950x and Enermax Liqtech TR4 360.
I'm just missing the 128 Gb DDR4 (>=3000 and <=CL16 ).
Can you please share /make a list with the supported nonECC/ECC DDR4 that you know / tested and are compatible (besides the QVL from https://www.asrock.com/mb/AMD/X399 Taichi/index.asp#Memory).
Thanks !

Unless you're willing to run the memory at 2400MHz or risk your sanity, you'll need B-die based modules.
Regardless what you purchase, don't expect to be able to hit high speeds with 2 DPC DR memory config on Threadripper.
2933 - 3000MHz is probably the best case scenario, even with B-die.

This is the cheapest 128GB B-die (guaranteed) kit from Newegg: https://www.newegg.com/Product/Product.aspx?Item=N82E16820236224

The specs are modest, but generally Corsair is sandbagging with their specs when it comes to B-die.
I've tested three Corsair 3333MHz CL16-18-18 rated 64GB (4x16G) kits and they all did 3200MHz 14-14-14 with =< 1.350V.

 

[Markfw]

Well, I have one system with
G.SKILL TridentZ Series 16GB (2 x 8GB) 288-Pin DDR4 SDRAM DDR4 3600 (PC4 28800) Intel Z170 / Z270 / Z370 / X299 Desktop Memory Model F4-3600C16D-16GTZ

Running 3466 totally stable, and another set coming.

I got this set, since it was 3200 cas 14,14,14,34, thinking it was samsung b-die, but it won't post over 2133 in my TR:
G.SKILL Ripjaws V Series 16GB (2 x 8GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model F4-3200C14D-16GVK

But that same memory runs in my 1800x@3200 cas 14 !! And both are Taichi motherboards, but one X370 (I think thats it) and one X399 !!

And threadripper number 3 is running this at 3200 cl14 (2 kits 8x4 for 32 gig)
G.SKILL TridentZ Series 16GB (2 x 8GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model F4-3200C14D-16GTZ

So it really does depend.

 

[Charlie22911]

This is the RAM (I got 2x 64GB kits) installed in my TaiChi, which is in the QVL. Never had an issue with it, but I am pretty sure it's not B Die memory.

https://www.newegg.com/Product/Product.aspx?Item=N82E16820232549

 

[bshor]

I purchased a 128Gb Corsair Vengeance LPX 3000MHz kit from Amazon for my 1950X plus Zenith Extreme build:

https://www.amazon.com/gp/product/B019X5RN84/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1

The problem is that I can only boot without problems either with 64Gb of RAM installed at 3000 MHz (in the ASUS' recommended slot configuration) or 128Gb at 2666 MHz. And Prime95 Torture Test detected errors at 2666 MHz, but 0 errors at 2400 MHz even over 3 days of nonstop testing. I have updated the BIOS to the 0902 version dated December 2017.

I'm doing scientific computing, not gaming, with this build. I wonder how much does memory speed matter? Some of the links I read suggested not too big of a punch for higher speed memory.

I need to keep the price to be approximately the same ($1650). Good news is Amazon's pretty good with returns. Should I return this set and try for another of the same kind, or another?

Here are the details from Thaiphoon.

 

[DrMrLordX]

Those are Hynix ICs . . .

 

[bshor]

Yeah -- didn't know as much as I now do -- plus wanted to order from Amazon.

So what's the consequence of only getting 2400 Mhz as opposed to say 3000 in scientific workloads? Is it worth the bother to send this back, etc?

I do see that the Vengeance LPX 3600 MHz suggested above is about the same price as the Hynix one I purchased...

https://www.amazon.com/Corsair-Vengeance-8x16GB-PC4-28800-CMK128GX4M8X3600C18/dp/B073CFGWRB

Those are Hynix ICs . . .

 

[DrMrLordX]

If you have the opportunity to return the kit, I would highly recommend you do so in favor of the LPX kit.

 

[The Stilt]

So what's the consequence of only getting 2400 Mhz as opposed to say 3000 in scientific workloads?

What kind of scientific workloads to be specific? Any example?

 

[scannall]

Yeah -- didn't know as much as I now do -- plus wanted to order from Amazon.

So what's the consequence of only getting 2400 Mhz as opposed to say 3000 in scientific workloads? Is it worth the bother to send this back, etc?

I do see that the Vengeance LPX 3600 MHz suggested above is about the same price as the Hynix one I purchased...

https://www.amazon.com/Corsair-Vengeance-8x16GB-PC4-28800-CMK128GX4M8X3600C18/dp/B073CFGWRB

I have to wonder why you aren't using ECC memory for that kind of work.

 

[bshor]

I'm doing exactly that. The price difference between the 3000 Ghz Hynix LPX that I have and the 3600 Mhz Samsung LPX that the Stilt suggested is $50. Easy.

I wonder, too, if memory prevented me from overclocking my 1950X. I kept getting errors in Prime95 with any setting over stock, even with pumping up the voltage a bit. I have a Liqtech 360 TR4 so I know I had adequate cooling (and temperatures never got above 60c anyway).

If you have the opportunity to return the kit, I would highly recommend you do so in favor of the LPX kit.

 

[bshor]

Lots of Monte Carlo simulations that I do with R and a variety of packages written by academics.

What kind of scientific workloads to be specific? Any example?

 

[bshor]

Nobody does it in my line of work (social science).

I have to wonder why you aren't using ECC memory for that kind of work.

 

[The Stilt]

Lots of Monte Carlo simulations that I do with R and a variety of packages written by academics.

The two MC workloads (Black Scholes and a raytracer) included in my benchmark suite at least don't seem to care about the MEMCLK.
Obviously that not might be the case for all MC workloads, however I think you'll be fine at 2400MHz, especially if your workloads are NUMA aware.

 

[eek2121]

I'm doing exactly that. The price difference between the 3000 Ghz Hynix LPX that I have and the 3600 Mhz Samsung LPX that the Stilt suggested is $50. Easy.

I wonder, too, if memory prevented me from overclocking my 1950X. I kept getting errors in Prime95 with any setting over stock, even with pumping up the voltage a bit. I have a Liqtech 360 TR4 so I know I had adequate cooling (and temperatures never got above 60c anyway).

Well, what has been shown is that overclocked memory (that is, anything above 2400MHz technically) raises power consumption. As I found out the hard way, boards can only push so much power through that socket. However, I have yet to have my memory set to 3200 MHz force me to lower clocks, usually other factors come into play long before that.

That being said, overclocking isn't simply setting a frequency and praying. You have to tweak the voltages manually, and LLC as well. If you don't, vdroop will always result in a failed overclock. If your system boots and idles/performs light tasks fine, chances are you were a victim of vdroop. If your system doesn't boot when overclocked at 4 GHz or under, something is wrong with your hardware. I'd start by making sure your PS is up to snuff and go from there. I've had Threadripper at full load pulling a good 500 watts out of my system, and that is with an IDLE GPU.

Unfortunately, LLC is different for every model. For instance, on my MSI, Mode 1 is the most aggressive (and can fry your chip unless your board has safeguards in place), while mode 8 is the least aggressive. On gigabyte boards I've been told that it is the other way around. On my board I use mode 3 along with a 1.25V for 4.0 GHz or 1.35V for 4.1 GHz.. I can actually go slightly lower and still be stable, but I do it out of caution. I used to be able to do 4.0 @ 1.2 at one point at LLC3 but the chip has degraded some because I've pushed it to 1.5-1.6 volts at times to play with clock speeds and study the intricacies of the chip under certain workloads. Oh, one more thing, LLC3 on my board apparently appears to eliminate vdroop completely, as the voltage does not change at all whether the system is running prime95 or not. Why do they need LLC and just do this if it's possible? I don't know enough about the LLC stuff unfortunately to understand all the intricacies, but I'm told that newer Intel chips don't even have LLC settings.

 

[Markfw]

Well, what has been shown is that overclocked memory (that is, anything above 2400MHz technically) raises power consumption. As I found out the hard way, boards can only push so much power through that socket. However, I have yet to have my memory set to 3200 MHz force me to lower clocks, usually other factors come into play long before that.

That being said, overclocking isn't simply setting a frequency and praying. You have to tweak the voltages manually, and LLC as well. If you don't, vdroop will always result in a failed overclock. If your system boots and idles/performs light tasks fine, chances are you were a victim of vdroop. If your system doesn't boot when overclocked at 4 GHz or under, something is wrong with your hardware. I'd start by making sure your PS is up to snuff and go from there. I've had Threadripper at full load pulling a good 500 watts out of my system, and that is with an IDLE GPU.

Unfortunately, LLC is different for every model. For instance, on my MSI, Mode 1 is the most aggressive (and can fry your chip unless your board has safeguards in place), while mode 8 is the least aggressive. On gigabyte boards I've been told that it is the other way around. On my board I use mode 3 along with a 1.25V for 4.0 GHz or 1.35V for 4.1 GHz.. I can actually go slightly lower and still be stable, but I do it out of caution. I used to be able to do 4.0 @ 1.2 at one point at LLC3 but the chip has degraded some because I've pushed it to 1.5-1.6 volts at times to play with clock speeds and study the intricacies of the chip under certain workloads. Oh, one more thing, LLC3 on my board apparently appears to eliminate vdroop completely, as the voltage does not change at all whether the system is running prime95 or not. Why do they need LLC and just do this if it's possible? I don't know enough about the LLC stuff unfortunately to understand all the intricacies, but I'm told that newer Intel chips don't even have LLC settings.

with the old memory I had that would only run at 2133, with everything@100% load it was 702 watts. Now with the 3600 its 705 watts.

 

[Charlie22911]

Well, what has been shown is that overclocked memory (that is, anything above 2400MHz technically) raises power consumption. As I found out the hard way, boards can only push so much power through that socket. However, I have yet to have my memory set to 3200 MHz force me to lower clocks, usually other factors come into play long before that.

That being said, overclocking isn't simply setting a frequency and praying. You have to tweak the voltages manually, and LLC as well. If you don't, vdroop will always result in a failed overclock. If your system boots and idles/performs light tasks fine, chances are you were a victim of vdroop. If your system doesn't boot when overclocked at 4 GHz or under, something is wrong with your hardware. I'd start by making sure your PS is up to snuff and go from there. I've had Threadripper at full load pulling a good 500 watts out of my system, and that is with an IDLE GPU.

Unfortunately, LLC is different for every model. For instance, on my MSI, Mode 1 is the most aggressive (and can fry your chip unless your board has safeguards in place), while mode 8 is the least aggressive. On gigabyte boards I've been told that it is the other way around. On my board I use mode 3 along with a 1.25V for 4.0 GHz or 1.35V for 4.1 GHz.. I can actually go slightly lower and still be stable, but I do it out of caution. I used to be able to do 4.0 @ 1.2 at one point at LLC3 but the chip has degraded some because I've pushed it to 1.5-1.6 volts at times to play with clock speeds and study the intricacies of the chip under certain workloads. Oh, one more thing, LLC3 on my board apparently appears to eliminate vdroop completely, as the voltage does not change at all whether the system is running prime95 or not. Why do they need LLC and just do this if it's possible? I don't know enough about the LLC stuff unfortunately to understand all the intricacies, but I'm told that newer Intel chips don't even have LLC settings.

GamersNexus did a very good video on load line calibration.

https://youtube.com/watch?v=NMIh8dTdJwI

 

[eek2121]

GamersNexus did a very good video on load line calibration.

https://youtube.com/watch?v=NMIh8dTdJwI

The issue isn't LLC itself, but rather, there is no 'standard' for LLC. My mode 1 is another board's mode 8, and some boards even label modes as percentages instead. I will take a look at the video, however, but I find GN content, while not being bad by any means, to not be...the best. They fall into the same trap that most reviewers do. They go for the earliest sample of a product they can get, then rush to review it before NDA...and then return it as soon as possible. EVERY SINGLE REVIEWER that reviewed Ryzen did it on a suboptimal configuration. Not a single reviewer bothered to use faster RAM, etc. Not a single reviewer got creative enough to play around, overclock, etc. (mostly because they are bound by NDA). Did you know that Threadripper had some thermal issues, and that they weren't allowed to talk about it thanks to the NDA? This is one of the reasons I'm going to launch a site. No NDAs, Open Source Benchmarks, etc. It's a direction that EVERY site should take. Unfortunately the top 3 are owned by purch, so it's bottom dollar, budget, paid for by advertising nonsense. It took me a day to run over 100 benchmarks on my 1950X, why does it take Anandtech a year to even update a single review? I don't fault them btw, Ian and the rest of the team are amazing, but something is clearly wrong. Even in AT Bench, there are flawed results (example: Rocket League scores are far lower than what I've EVER gotten in 300 hours of play, and chromium compile times are half of what other sites get) If I didn't know any better, I'd say that the folks at AT get less than 2 hours a week to work on an article. Even Linus @ LTT does a better job with coverage. I would love to shadow a few folks at ATT, because for me, I can hook up 4 different computers and bench 4 different CPUs in over 100 benchmarks while simultaneously working a full time job as a developer. Not only that, but I can easily spring for enough RAM, the same SSDs, etc. to have the same hardware across all 4 computers...and I'm not 'rich' by any means. I make an upper middle class salary, but I am the sole income provider for a family of 5 and an extended family of 9.

Anyway, sorry for the rant, I just hope I can use (and afford) 128GB of high speed ram soon, and that the 29xxx TR chips support it.

 

[Abwx]

Why do they need LLC and just do this if it's possible? I don't know enough about the LLC stuff unfortunately to understand all the intricacies, but I'm told that newer Intel chips don't even have LLC settings.

There is significant voltages drops in the MB copper lines as well as within the CPU supply lines, voltage is measured within the CPU at strategic places where the drops are of the higher amplitude and reported to the MB voltage regulation, LLC is even more needed given that traces resistance is variable from a MB model to another.

Think about it, there s something like 130A amperes that are drained by a fully loaded TR and running in the MB copper traces and then this current is distributed within the CPU in supply lines that are also not resistance free.

Peak currents required by the CPU should be well over 200A, at this point the voltage regulation must compensate for the quasi doubled voltage drop in the MB traces as well as in the most power hungry parts of the CPU that cause those current surges.

Without LLC efficency would be much lower as the voltage should be kept permanently at the highest value to ensure stability should the CPU require peak currents that exceed the nominal value, it s not required in a CPU like Raven Ridge when used as 15W notebook part because it was designed to handle roughly 95W and will have minimal internal (and external) voltage drops at low powers, hence, a sophisticated voltage management is not necessary...

 

[Drazick]

Anyone tried ThreadRipper with MATLAB and compared to Intel?

MATLAB relies heavily on Intel MKL and Intel IPP and I wonder if those packages discriminate AMD Zen based CPU's.

 

[bshor]

I returned the Hynix-based Corsair Vengeance LPX and ordered the LPX version recommended above by the stilt (they are both almost exactly the same price, about $1650).

Here's the Thaiphoon summary:

So this is indeed Samsung B-die. And I was immediately able to boot at 3066Mhz whereas before the only frequency that would boot was 2600 MHz. Now I'll try playing with it and see if I can't make it go higher (it cycled endlessly and eventually failed @ 3200 MHz).

Update 1: Prime95 Blend Test with all the default settings, gave me an error at 3066Mhz (though the system seems stable).

That's disappointing. Any ideas? What settings should I try on my Zenith Extreme?

 

[DrMrLordX]

Have you tried upping vDIMM or vSoC?

 

[bshor]

I pushed up vDIMM (AB and CD) to 1.5v, and vSoC to 1.35v, but still the same Prime95 errors @ 3200 Mhz. Trying at lower speeds now.

Update: lowered vSoC to 1.2v (but kept vDIMM at 1.5v) after reading warnings not to push that too high. Prime95 appears to have no errors @ 2900 Mhz. A far cry so far from 3466 that others have achieved, but certainly better than the 2400 I achieved with the old Hynix RAM. Perhaps more tinkering will get me higher.

BTW I'm doing this in the BIOS but should I be doing in Ryzen Master instead?