Threadripper BUILDERS thread

[dnavas]

A slim 360 mm radiator can be used to cool a 250 W heat source, if you accept high fan speeds but still rather high water temperature. From the latter follow low boost clocks of Threadripper.

Let's be precise, otherwise we're going to be arguing what "high speeds" and "high temps" mean. Let's grab a very specific thin 360mm: https://www.xtremerigs.net/2015/07/03/ek-coolstream-se-360mm-radiator-review/4/ (totally at random, I have no idea where this one hits, it's just one I've got a link handy for) At 1300rpm I see about 230W push only and reasonable flow rates for 10C. If we want the coolant stable under 60C, and we assume an ambient at a toasty 30C, that's way more than 500W. Even at 750rpm I'm seeing enough room to almost claim 500W. Leastwise, that's how I read those numbers. What am I missing?

No, it does not.

That doesn't square with what I've read. Obviously "book learning" isn't exactly the be-all/end-all. You tell me, what's the minimum I can expect / get away with?

Well, you may still need to remove dust from radiator fins occasionally.

ed: perhaps I should have asked, *how* do I achieve a loop that doesn't require a yearly drain and refill. Maybe I've just been reading the wrong info?

 

[Markfw]

Well, I got my Noctura U14TR4 HSF's today, 3 of them to replace all my Enermax coolers. I can't say I am impressed. @ pure stock (3.4 ghz) its up to 64c in an air conditioned house@75F and still rising.

Somebody needs to make a good air cooler for TR4. Or an affordable AIO that does not self-destruct.

Yes, I have seen all sorts of ones coming out. I want to buy one NOW !

Edit: 67.5 still rising
Edit 2: steady @ 67.5

 

[Viper GTS]

Did you also order a second fan for them for push/pull? They can take another fan with no issues using the provided clips.

They're never going to do what the AIO can but they will at least work and not self destruct. The Wraith Ripper is six weeks out still and doesn't appear to be a big improvement.

Viper GTS

 

[Markfw]

Did you also order a second fan for them for push/pull? They can take another fan with no issues using the provided clips.

They're never going to do what the AIO can but they will at least work and not self destruct. The Wraith Ripper is six weeks out still and doesn't appear to be a big improvement.

Viper GTS

I will get the wraith ripper and see. It sure looks better.

 

[Viper GTS]

Phoronix is the only early data I have seen - Better, but not dramatically so:

https://www.phoronix.com/scan.php?page=article&item=amd-2990wx-cooling&num=1

Viper GTS

 

[Markfw]

So question. What about temperature monitoring software for linux ! For a 1950x ?? I can't find it. I tried psensor, and it can't find the CPU temp.

 

[dlerious]

So question. What about temperature monitoring software for linux ! For a 1950x ?? I can't find it. I tried psensor, and it can't find the CPU temp.

https://www.ostechnix.com/view-cpu-temperature-linux/

 

[guskline]

Markfw, did any of your custom water cooling parts come it yet?

 

[StefanR5R]

Markfw, did any of your custom water cooling parts come it yet?

You should keep an eye on the other thread. :-)

--------

yet I also like the temperatures/thermal headroom of a 500W+ rated AIO

These 500+ W ratings are either meant as a joke, or deception.

A slim 360 mm radiator can be used to cool a 250 W heat source, if you accept high fan speeds but still rather high water temperature. From the latter follow low boost clocks of Threadripper.

Let's be precise, otherwise we're going to be arguing what "high speeds" and "high temps" mean. Let's grab a very specific thin 360mm: https://www.xtremerigs.net/2015/07/03/ek-coolstream-se-360mm-radiator-review/4/ (totally at random, I have no idea where this one hits, it's just one I've got a link handy for) At 1300rpm I see about 230W push only and reasonable flow rates for 10C. If we want the coolant stable under 60C, and we assume an ambient at a toasty 30C, that's way more than 500W. Even at 750rpm I'm seeing enough room to almost claim 500W. Leastwise, that's how I read those numbers. What am I missing?

I am not a Threadripper owner so far. Also, I never had an AIO with slim radiator myself, only a modular AIO from EKWB with 360 mm radiator with medium thickness, which I used to cool a 140 W Broadwell-E and, for a brief while, a 180 W GPU in addition. I don't have this modular AIO anymore, but I am using custom loops with 2...3 250 W GPUs (and assorted CPUs with much less wattage) as GPGPU compute nodes.

From what I remember with the Broadwell-E (which is a 14 nm CPU too, and soldered to the heat spreader similar to Threadripper), and from experience with the direct-die cooled GPUs that I have now, 60 °C water temperature are far too much for continuous operation.

Going back specifically to TR4 AIOs, the 500 W rating would translate to an overclocked 2990WX. I don't know what temperatures the Threadripper dies would have if the water temperatures is 60 °C. I strongly suspect the die temperatures would be so high that the clocks achieved wouldn't be... practical.

As a point of comparison, frequency response over wattage of 2950X and 2990WX when under different cooling systems, including a slim 240 AIO (Enermax LiqTech 240 TR4, with the same silly 500+ W TDP rating by Enermax), has been measured already:
https://www.tomshw.de/2018/08/13/am...st-echter-fortschritt-mit-bis-zu-32-kernen/7/
An English translation of this source:
https://www.tomshardware.com/reviews/amd-ryzen-threadripper-2-2990wx-2950x,5725-12.html
(Caveat with the custom loop and chilled loop results in this source: The waterblock was an adapted socket 105x/2011/2066/AM4 waterblock, not a dedicated TR4/SP3 waterblock.)

Meanwhile, custom water needs maintenance every year.

No, it does not.

That doesn't square with what I've read. Obviously "book learning" isn't exactly the be-all/end-all. You tell me, what's the minimum I can expect / get away with?

If you build your custom loop at least as well as AIO manufacturers build closed loops, your maintenance interval will be at least as long as the maintenance interval that the AIO vendors tell their customers. (And I say this even though I criticized Enermax' specs of their TR4 AIOs in this very post.) However, this is off-topic in this thread and subforum, which is why I kept my earlier statement on this as brief.

 

[Paratus]

Well, I got my Noctura U14TR4 HSF's today, 3 of them to replace all my Enermax coolers. I can't say I am impressed. @ pure stock (3.4 ghz) its up to 64c in an air conditioned house@75F and still rising.

Somebody needs to make a good air cooler for TR4. Or an affordable AIO that does not self-destruct.

Yes, I have seen all sorts of ones coming out. I want to buy one NOW !

Edit: 67.5 still rising
Edit 2: steady @ 67.5

That’s about what I’ve seen temp wise, (although with only 8 cores I see higher core clocks).

The real issue is to get the most out of a Threadripper you need fast RAM and a good cooler. With the Enermax I didn’t have to worry about OCing. XFR basically had it running nearly 4.0Ghz or above for most loads.

That being said the Noctua doesn’t have the issues the Enermax does. So I’m not sitting here worrying about it overheating or leaking out.

I’m working the RMA with Enermax and they said they’ll send a new one. What I’m waiting to hear back from them is have they made any design change to deal with the corrosion problems. Simply changing the fluid to include more anti-corrosives would probably do it. Otherwise it’s just gonna fail again in a couple of months.

 

[Face2Face]

Hello people,

I was one of the winners of the Intel 8086K competition they had awhile back. I was also one of the 40 that was selected to exchange the Intel CPU for an TR 1950X. I just sent the Intel CPU out today. Anyway, I currently have a Fractal Design S36 on my 5820K, and I’m going to be moving it over to the 1950X. Anyone here have any experience with this AIO and the 1950X?

 

[Markfw]

Hello people,

I was one of the winners of the Intel 8086K competition they had awhile back. I was also one of the 40 that was selected to exchange the Intel CPU for an TR 1950X. I just sent the Intel CPU out today. Anyway, I currently have a Fractal Design S36 on my 5820K, and I’m going to be moving it over to the 1950X. Anyone here have any experience with this AIO and the 1950X?

There is a bracket that comes with the 1950X. If it works, you are OK> Note this cooler does not have fill contact with the TR4 CPU, so cooling will be lower than the Enermax.

 

[dnavas]

I really appreciate the response and education, thanks!

An English translation of this source:
https://www.tomshardware.com/reviews/amd-ryzen-threadripper-2-2990wx-2950x,5725-12.html

Yeah, I read https://www.hardocp.com/article/2018/01/26/enermax_liqtech_tr4_280_aio_liquid_cpu_cooler_review/4 which admittedly is on the 1950X, but is likely to be applicable for the 2950x. Given it's being compared to a 420mm Raystorm, and it's running a slightly lighter fan rpm, it does surprisingly well. That said, I'm worried about the cooling my VRMs will get (I rather suspect that's where hardocp went wrong as well -- what kind of airflow are the VRMs getting under that plexiglass?). Running a front mount is going to guarantee fresh air for the CPU/rad, and leave room for a push pull (if the slim radiator will even benefit from that) but the VRMs are going to roast.

If you build your custom loop at least as well as AIO manufacturers build closed loops, your maintenance interval will be at least as long as the maintenance interval that the AIO vendors tell their customers.

And therein lies the rub. I can probably do a decent job of picking a radiator and a block, but the assembly.... This is not the build I'd like to learn on.

 

[Markfw]

The replacement TR4 AIO kit from Enermax appears to be the old one, I got mine today. Its great that I got it, and fairly soon, its just that I am worried about using it. And I have 3 more with possibly the same issue. (not symptomatic right now)

 

[dnavas]

The replacement TR4 AIO kit from Enermax appears to be the old one, I got mine today. Its great that I got it, and fairly soon, its just that I am worried about using it. And I have 3 more with possibly the same issue. (not symptomatic right now)

Do let us know if they respond to your concerns about the anti-corrosives they're using. At least the 360 model II is on sale. I'm still waiting to see an ELC-LTTRTO280-TBP show up in a store somewhere!

As long as I'm asking -- anyone know of a VRM water-cooling option for the taichi board? [Aside from the EK monoblock which still isn't getting great reviews.] I'd love if Watercool had something, but looks like they only have coolers for ASUS mbs

 

[TheGiant]

Anyone with 256 or 512 GB ECC RAM on TR platform?

which one to choose, board, memory, coolerz, PSU

I wanna use the ability to OC 2950X to 4GHz and ECC memory for CFD calculations.

Thanks

 

[StefanR5R]

That said, I'm worried about the cooling my VRMs will get

I hung one or more 80 mm fans into my watercooled boxes for this purpose, because I don't like hot board components either, even though I don't overclock CPUs. Comments somewhere in the forum pointed out that the VRMs themselves can stand quite high temperatures, but nearby components, notably capacitors, are a concern.

(low-maintenance watercooling)

And therein lies the rub. I can probably do a decent job of picking a radiator and a block, but the assembly.... This is not the build I'd like to learn on.

I head a bumpy learning curve myself. And while an air-cooled build can be put together in a hurry if necessary, a custom watercooled build takes notably more time. This can also become a big nuisance for later modifications, such as swapping add-in cards.

--------

Anyone with 256 or 512 GB ECC RAM on TR platform?

which one to choose, board, memory, coolerz, PSU

I wanna use the ability to OC 2950X to 4GHz and ECC memory for CFD calculations.

This amount of RAM requires registered DIMMs. I am not aware of any TR4 boards which support those, neither available nor announced.

Have you considered an EPYC 7551P? Extrapolating from Ian Cutress' 2990WX/2950X review, the 7551P will be about as fast as an overclocked TR 2950X in well-parallelized workloads which are not constrained by memory bandwidth, and faster in loads which are. All the while you don't have the headache of the cooling requirements of an overclocked Threadripper. Downsides of the 7551P are obviously the lower performance in workloads with pronounced serial sections, and about triple the CPU purchase price. Cheapest SP3 boards cost as much as mid-range TR4 boards.

 

[TheGiant]

Well I know about EPYC, or Xeon. But both are non overclockable thus not very high ST/low core count performance.

Today I have 2 machines, i5/6600k@4,5GHz for precalculations and other stuff and the working machine, BDW Xeon 14C with 256GB ECC RAM. But I need a second machine and I wanted to buy just one not 2. TR seems to be perfect at like 4GHz, because it has BDW IPC and not so high but acceptable frequency.

There is also Intel Xeon W-2195 but the price.....

The memory cost is the highest but the price difference for Xeon/EPYC is high and I don't wanna use any other features of the server class CPU except the ECC memory

 

[Face2Face]

There is a bracket that comes with the 1950X. If it works, you are OK> Note this cooler does not have fill contact with the TR4 CPU, so cooling will be lower than the Enermax.

It appears the bracket will work with the S36 AIO. Thanks!

Also, due to high memory prices, I'm going to stick with the 3000Mhz memory from my current system. Have you seen any benchmarks showing a large difference between 3000 vs. 3600 MHz memory using TR?

 

[Paratus]

It appears the bracket will work with the S36 AIO. Thanks!

Also, due to high memory prices, I'm going to stick with the 3000Mhz memory from my current system. Have you seen any benchmarks showing a large difference between 3000 vs. 3600 MHz memory using TR?

https://forums.anandtech.com/threads/my-threadripper-1900x-semi-formal-review-part-1.2544112/

In my review of my 1900X I varied the Mem speeds from 2133 to 3333 across several benchmarks and found generally single digit increases at the most.

 

[VirtualLarry]

https://www.newegg.com/Product/Product.aspx?Item=N82E16819113448

1920X @ Newegg is now $419.99 + $0.99 shipping.

New MSRP is supposed to be $400, so Newegg is still gouging... a little. But close enough for me.

Now I just have to sell off my older stuff to get a TR(1).

Also have to worry about cooling, though not quite as much as a 2995X or whatever the 32C/64T CPU is. 1920X is listed as 180W.

Recs. on mobos, with an eye to the future? Are most sTR4 boards, E-ATX? I bought a DIY-PC chassis with I think 8 expansion slots, it fairly big, tempered glass window, RGB, nice features (can mount a 240mm AIO rad. internal to case.) Going to look that up so I can determine mobo compatibility.

Would like 10GbE, if I'm going to splash out on a next-gen board, thinking of potential FIOS upgrades in the next five years, want to be ready for them, and contemplating a 10GbE-T LAN in the next year or two for my PCs. Or maybe just a new NAS with 10GbE, with a home-run between my TR rig and my mega-NAS unit. (Looking at QNAP 951X.)

Thinking strongly of this board. Apparently, it's just ATX, and not E-ATX, somehow?
10GbE-T, 2x 1GbE-T Intel, 3x PCI-E 3.0 x4 M.2 NVMe sockets, 4x PCI-E 3.0 x16 slots, 8 DDR4 DIMM slots, decent layout, SoundBlaster Cinema. And of course, sTR4.

https://www.newegg.com/Product/Product.aspx?Item=N82E16813157784

 

[Markfw]

https://www.newegg.com/Product/Product.aspx?Item=N82E16819113448

1920X @ Newegg is now $419.99 + $0.99 shipping.

New MSRP is supposed to be $400, so Newegg is still gouging... a little. But close enough for me.

Now I just have to sell off my older stuff to get a TR(1).

Also have to worry about cooling, though not quite as much as a 2995X or whatever the 32C/64T CPU is. 1920X is listed as 180W.

Recs. on mobos, with an eye to the future? Are most sTR4 boards, E-ATX? I bought a DIY-PC chassis with I think 8 expansion slots, it fairly big, tempered glass window, RGB, nice features (can mount a 240mm AIO rad. internal to case.) Going to look that up so I can determine mobo compatibility.

Would like 10GbE, if I'm going to splash out on a next-gen board, thinking of potential FIOS upgrades in the next five years, want to be ready for them, and contemplating a 10GbE-T LAN in the next year or two for my PCs. Or maybe just a new NAS with 10GbE, with a home-run between my TR rig and my mega-NAS unit. (Looking at QNAP 951X.)

Thinking strongly of this board. Apparently, it's just ATX, and not E-ATX, somehow?
10GbE-T, 2x 1GbE-T Intel, 3x PCI-E 3.0 x4 M.2 NVMe sockets, 4x PCI-E 3.0 x16 slots, 8 DDR4 DIMM slots, decent layout, SoundBlaster Cinema. And of course, sTR4.

https://www.newegg.com/Product/Product.aspx?Item=N82E16813157784

Just get the Taichi, its cheaper and just as good.And for HSF, the Noctura is the only good one right now.

 

[dnavas]

Recs. on mobos, with an eye to the future? Are most sTR4 boards, E-ATX?

Yes, and as the case I bought is ATX only, that's a problem for me, but the Taichi board seems popular and is ATX. My real concern is the rumors about x499 mobos and the thought that 39x0 CPUs may not be (fully?) backwards compat. Otherwise I assume we're just talking pci 4.0, which I don't think I care about (especially with 5.0 right on its heels). I don't see DDR5 being ready in 2019 for consumers, so while it might make sense for TR, my larger concern is a change in the memory bandwidth for TR boards via additional channels.

Anyway, Newegg just had a sale on the taichi at $290 (ended Thursday) and they have a $40 off mail-in coupon (otherwise known as waste-of-a-stamp, unless you've actually ever received a check on one of these things?). You could grab the Fatal1ty if you want the 10G option, or just buy an adapter. The price difference is about the same as the cost difference.

contemplating a 10GbE-T LAN in the next year or two for my PCs. Or maybe just a new NAS with 10GbE, with a home-run between my TR rig and my mega-NAS unit. (Looking at QNAP 951X.)

I've got an ASUS XG-C100C 10G adapter on my 1800X talking directly to my QNAP 873 with one of their 10G adapters in it. The ASUS adapter is the same chip as the Fatal1ty board has on it, I believe, and it works just fine. The QNAP adapter gets hot, and the fan setup on the older AMD processor in the 873 is a piece of junk ... and custom ... so I pulled the USB3.1 adapter out to give the ethernet card a little more room to breath, setup the rear fans to run steady at a slightly higher setting, and let the CPU fan do whatever qnap has it doing. As far as I can tell, QNAP's approach to fan control is "off, ineffective, loud" with step functions between those settings (curves? pah!), so, it was a thing to make it play nice.

A five bay isn't going to give you 10G throughput. If you're running RAID 5, max throughput, aside from caching on those four SSDs, is going to be 4 x ~150MB (depending on the drives you put in there -- probably the higher capacity drives are higher throughput each as well). I'm running RAID6 on 8 4TB drives, so I'm not maxing out my link either. I'm also running with two internal ssds, but qnap doesn't support nvme, so they're stuck at SATA speeds. I run those in RAID0 as a cache, but obviously that only affects read speeds. Not sure they were worth the money :shrug:

I hope that was helpful! My 2950X arrives Tuesday. Build will probably happen a couple of weeks later -- next weekend is birthday weekend for my wife and me, so I expect I'll be otherwise engaged :>

 

[CuriousMike]

2990x owners - if you haven't seen this video, it might be worth your 10 minutes.
The guy describes how there may be an issue with Windows and moar cores - he shows how disabling a single core from a particular benchmark dramatically improves the score.
https://www.youtube.com/watch?v=WSSAFqzbKgg

 

[Markfw]

2990x owners - if you haven't seen this video, it might be worth your 10 minutes.
The guy describes how there may be an issue with Windows and moar cores - he shows how disabling a single core from a particular benchmark dramatically improves the score.
https://www.youtube.com/watch?v=WSSAFqzbKgg

You should have put this in the 2990wx builders thread, but there are only 2 of us that I know, and I run linux