Threadripper BUILDERS thread

[ddogg]

Would really love to get another 960 evo and use it in raid 0. But the hassle of reinstalling windows and my programs again is putting me off. Curious to see what kind of performance people get with it

 

[Fir]

I don't mind doing a fresh install.
I have 10 virtual machines that have to be backed up though!
At least with nvme and having a 10Gb NAS it won't take that long.

 

[MarkPost]

Some air (Noctua) vs AIO (Kraken) comparison:

Noctua clearly wins, specially keeping in mind fans rpm

 

[Markfw]

Some air (Noctua) vs AIO (Kraken) comparison:

Noctua clearly wins, specially keeping in mind fans rpm

If I didn't have a clearance problem, I would have one in my cart right now.

 

[MarkPost]

If I didn't have a clearance problem, I would have one in my cart right now.

yeah, it seems clear that Noctua solution (specially U14 but even the U12) is a great option for TR, and just better than any Asetek "reused" AIO.

Only the Enermax TR AIO outperforms Noctua coolers (specially the 360 variant), besides proper TR custom loop of course.

 

[Fir]

nvme raid results are very disappointing. It's a gimmick unless you NEED 4+ GB/S sustained. iOPS are much lower. In other words you're trading off random io (what you want a SSD for in the first place!) for sequential...

There is no real configuration to adjust as well so no idea of stripe size, etc.

If you have the RAM, it's far better to use a caching software along with non raid nvme drives to have the best of both worlds.

 

[tamz_msc]

nvme raid results are very disappointing. It's a gimmick unless you NEED 4+ GB/S sustained. iOPS are much lower. In other words you're trading off random io (what you want a SSD for in the first place!) for sequential...

There is no real configuration to adjust as well so no idea of stripe size, etc.

If you have the RAM, it's far better to use a caching software along with non raid nvme drives to have the best of both worlds.

That doesn't seem to line up with PCper's testing with Intel VROC on X299 where IOPS remained constant.

 

[Fir]

That doesn't seem to line up with PCper's testing with Intel VROC on X299 where IOPS remained constant.

This is not an Intel system.

 

[ddogg]

Debauer went all crazy with 8x 950 pros in his TR setup. Was getting close to 27GB speeds.
But yeah I agree, for most people raid NVMe drives are completely overkill. I still want one

 

[Fir]

Those were 4X add on cards.
And yes the STR increases are nice.
But iOPS are extremely important as well.
I would at least expect them to stay close to that of a single drive not be cut by more than HALF!
I suppose if you were copying terabyte files (!) all day it would pay to have that kind of set up.
And active cooling as well as these drives get HOT when pushed continuously.
On the card with thermal pads touching the chips and controller well sunked to a nice frame it would do much better. Similar to the Intel PCI-E drives. I wonder if those can be raided with this driver!

 

[tamz_msc]

This is not an Intel system.

Random IOPS read or write? I rechecked the PCper article, they tested only 4K random reads. Too much focus on Optane means they simply didn't include any write results.

 

[mattiasnyc]

nvme raid results are very disappointing. It's a gimmick unless you NEED 4+ GB/S sustained. iOPS are much lower. In other words you're trading off random io (what you want a SSD for in the first place!) for sequential...

There is no real configuration to adjust as well so no idea of stripe size, etc.

If you have the RAM, it's far better to use a caching software along with non raid nvme drives to have the best of both worlds.

 

[mattiasnyc]

nvme raid results are very disappointing. It's a gimmick unless you NEED 4+ GB/S sustained. iOPS are much lower. In other words you're trading off random io (what you want a SSD for in the first place!) for sequential...

"very disappointing" seems a bit strong though. This is a high performance workstation platform though, so I would expect the bandwidth to be pretty nice to have working on 8k footage for example. I think you'll be saturating an x4 connection below 8k, so some sort of raid is required.

 

[Atari2600]

nvme raid results are very disappointing. It's a gimmick unless you NEED 4+ GB/S sustained. iOPS are much lower. In other words you're trading off random io (what you want a SSD for in the first place!) for sequential...

Hmmm, I'm thinking FEA scratch area for big models. Sustained writes are the name of the game.

 

[Charlie22911]

Got the parts in and got everything setup, with previous unRAID goodies transferred.
First thing I've noticed is this platform (or at least this board) is picky about what PCIe cards it will boot with, I have a Fresco Logic USB 3 card and Intel PRO/1000 VT intended for pass through that will hang the board at POST on latest BIOS. I have an Intel PRO/1000 PT I'm passing through instead for the moment, for now I'm experimenting with pfSense without messing with my dedicated box.
AMD has a survey up dealing with boot and pass through issues, I've already filled in and submitted for my configuration.
GPU pass through is also pretty broken; I already knew this though since it has been an issue since Ryzen was first released. In fact pass through on AMD was an issue even with the AM3+ and FX8350 system this replaced.

Overall the above aren't deal breakers for me and I am very satisfied with the machine, it is very nice to see AMD competitive again. Word of advice to anyone eyeing the Cosmos II however, it is huge and HEAVY; it's not a negative for me though.

Specs:
Threadrippper 1950X
Asrock x399 Taichi
128GB G.Skill Fortis 2400
AMD 6670 for host (GT 1030 on the way)
LSI Dual SAS with IT firmware
MSI Geforce 1060 6GB (unused for now)
Intel PRO/1000 PT quad port
EVGA Supernova 1000 T2 Power Supply
5x WD Red 4TB (2 are parity)
3x WD Green 4TB
3x Segate 4TB
2x Intel 180GB SSDs
2x Kingston 250GB SSDs (in BTRFS RAID 0 with Intel SSDs)

 

[Fir]

"very disappointing" seems a bit strong though. This is a high performance workstation platform though, so I would expect the bandwidth to be pretty nice to have working on 8k footage for example. I think you'll be saturating an x4 connection below 8k, so some sort of raid is required.

The high loss of iops is bad as I've said in the past going with dedicated iOP hardware iOPs remained unchanged whereas STR scales nicely with additional drives. It could be an immature driver.

8k video isn't close to stressing a single 960 unless you're talking about uncompressed stuff. In that case most professionals aren't using gaming motherboards! LOL

Random IOPS read or write? I rechecked the PCper article, they tested only 4K random reads. Too much focus on Optane means they simply didn't include any write results.

Both equally affected.

 

[MarkPost]

Got the parts in and got everything setup, with previous unRAID goodies transferred.
First thing I've noticed is this platform (or at least this board) is picky about what PCIe cards it will boot with, I have a Fresco Logic USB 3 card and Intel PRO/1000 VT intended for pass through that will hang the board at POST on latest BIOS. I have an Intel PRO/1000 PT I'm passing through instead for the moment, for now I'm experimenting with pfSense without messing with my dedicated box.
AMD has a survey up dealing with boot and pass through issues, I've already filled in and submitted for my configuration.
GPU pass through is also pretty broken; I already knew this though since it has been an issue since Ryzen was first released. In fact pass through on AMD was an issue even with the AM3+ and FX8350 system this replaced.

Overall the above aren't deal breakers for me and I am very satisfied with the machine, it is very nice to see AMD competitive again. Word of advice to anyone eyeing the Cosmos II however, it is huge and HEAVY; it's not a negative for me though.

Specs:
Threadrippper 1950X
Asrock x399 Taichi
128GB G.Skill Fortis 2400
AMD 6670 for host (GT 1030 on the way)
LSI Dual SAS with IT firmware
MSI Geforce 1060 6GB (unused for now)
Intel PRO/1000 PT quad port
EVGA Supernova 1000 T2 Power Supply
5x WD Red 4TB (2 are parity)
3x WD Green 4TB
3x Segate 4TB
2x Intel 180GB SSDs
2x Kingston 250GB SSDs (in BTRFS RAID 0 with Intel SSDs)

wow some storage there

 

[Markfw]

OK< so I can't user the Noctura air cooler for TR4, as its too big with my 2 video cards (that can't be moved), so I got this:

https://www.newegg.com/Product/Product.aspx?Item=N82E16835214080

I heard its the only thing NOT custom water that beats the Noctura. I will let you know in a week or so....
It will be replacing a Arctic cooling AIO 360 rad with 6 fans...

 

[Fir]

I would be curious how it compares and if it would be worth getting over the H115i I'm using now. I know if I push 4GHz I hit throttling which is no good.

 

[ddogg]

Got the parts in and got everything setup, with previous unRAID goodies transferred.
First thing I've noticed is this platform (or at least this board) is picky about what PCIe cards it will boot with, I have a Fresco Logic USB 3 card and Intel PRO/1000 VT intended for pass through that will hang the board at POST on latest BIOS. I have an Intel PRO/1000 PT I'm passing through instead for the moment, for now I'm experimenting with pfSense without messing with my dedicated box.
AMD has a survey up dealing with boot and pass through issues, I've already filled in and submitted for my configuration.
GPU pass through is also pretty broken; I already knew this though since it has been an issue since Ryzen was first released. In fact pass through on AMD was an issue even with the AM3+ and FX8350 system this replaced.

Overall the above aren't deal breakers for me and I am very satisfied with the machine, it is very nice to see AMD competitive again. Word of advice to anyone eyeing the Cosmos II however, it is huge and HEAVY; it's not a negative for me though.

Specs:
Threadrippper 1950X
Asrock x399 Taichi
128GB G.Skill Fortis 2400
AMD 6670 for host (GT 1030 on the way)
LSI Dual SAS with IT firmware
MSI Geforce 1060 6GB (unused for now)
Intel PRO/1000 PT quad port
EVGA Supernova 1000 T2 Power Supply
5x WD Red 4TB (2 are parity)
3x WD Green 4TB
3x Segate 4TB
2x Intel 180GB SSDs
2x Kingston 250GB SSDs (in BTRFS RAID 0 with Intel SSDs)

Nice build. That case definitely deserves some watercooling love though. Huge is also relative, as my case easily dwarfs that

 

[Chaotic42]

Here's how the system ended up. This is my first windowed case, but I haven't taken off the protective covering.

The Corsair H115i keeps this thing nice and cool even under total load.

 

[mattiasnyc]

The high loss of iops is bad as I've said in the past going with dedicated iOP hardware iOPs remained unchanged whereas STR scales nicely with additional drives. It could be an immature driver.

8k video isn't close to stressing a single 960 unless you're talking about uncompressed stuff. In that case most professionals aren't using gaming motherboards! LOL

Some have more than one video on the timeline when editing though.

But anyway, are these really gaming motherboards? What is a "gaming motherboard" and how is it different from a workstation motherboard? Why would a gamer buy a Threadripper?

Everything I've read yells "productivity" when I look at Threadripper, especially nvme raid and 60 PCIe lanes.

 

[Chaotic42]

Some have more than one video on the timeline when editing though.

But anyway, are these really gaming motherboards? What is a "gaming motherboard" and how is it different from a workstation motherboard? Why would a gamer buy a Threadripper?

Everything I've read yells "productivity" when I look at Threadripper, especially nvme raid and 60 PCIe lanes.

As best I can tell it's the overclocking emphasis, the gaudy RGB, and the early releases. Workstation customers wouldn't take these early adopter issues (like my 128GB of memory problem) very well.

 

[Fir]

It's a gaming motherboard for sure.
Workstation boards typically use more than one socket, full ECC RAM support (and a lot sometimes 1TB or more PER socket!) and other goodies like integrated 10Gbps NICs, out of band management and hardware security features. And the ability to support lots of storage or computational cards.

That said, this board I'm using (Zenith) seems to have the most robust power delivery of the bunch. The latest BIOS and firmware for the OLED panel makes it feel much more mature and ready for doing work.

Never had an issue with my 128GB RAM as well.

The multi socket EPYC boards will certainly be good workstation candidates.

 

[mattiasnyc]

The workstation category includes single-socket motherboards for sure, especially in media production. And in a lot of media production I think the balance of x399-type platforms is 'perfect'. More cores than consumer lines, but slower, fewer cores than EPYC etc, but faster.....

From what I can see the only items that would make these "gaming motherboards" don't make them not workstation motherboards.