Discussion EPYC builders thread

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
So now that I have 2 boxes, I thought I would start a builders thread, and also pass along what I have learned.

First, They are are cheap, and they are a different animal to build (see below)

OK, so first, I got these 2 7551 ES chips for $300 each, I thought, well that is cheap for a 32 core CPU, and with 8 channel memory they won't be handicapped like my 2990wx's.

Now to the bad... First they were only compatible with ONE motherboard that was available to buy, and with a BIOS that I had to fight with Gigabyte to provide me with ! (not available online)
So I tried several other motherboards and different BIOS's since I could not even find this one online. FINALLY I found the board at $470. I updated the BIOS (odd way, boot into motherboards default stream environment (or something like that) and do a DOS bios update. Well, it worked. Then while trying to install linux mint 19.2, no internet ???? Oh, I did not read the fine print. No ethernet that goes outside the local lan. Only SFP+ (whatever that is), so after some reasearch, I find the adapter for $13 more.

OK fast forward to when I now have an install and updated with ethernet. This 32 core chip only runs turbo all core at 1.6 ghz !!! So I spent almost $500 on the motherboard, and for 128 gig of ECC registered (the cheapest I could find@64 gig on 8 sticks or more) at another $530 and a $300 CPU I have $1330 invested in a 1600 mhz 32 core box... Picture:
JXvDun7.jpg


And before I got this working, one of the motherboards I tried (dual socket) that would not work with these ES chips, I found 2 7601 used retail chips for $1750 and a $650 motherboard. Now this one needed 16 sticks so that was $1100. So now I am in this one $3500 with no SSD, case or PSU. Well, I get it running, and it only boosts all core to 2.6 ghz. But at least thats a far cry to 1600 mhz.

Here:
5Gr0BgB.jpg


Bottom line ? The new 3960x and 3970x are not overpriced. At the same speed they are almost twice as fast. And don't venture into EPYC chips unless you have wads of cash to blow and learn. I bought another motherboard for my 2nd 7551 ES chip, so now I have to build that box too !

But I will soon have 3 server grade ECC memory chip boxes with 128 cores and 256 threads for almost $6000. I can't even get video cards to fit in these, as the memory and heatsinks are in the way ! I have to use the on-board video.
 
Last edited:

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
New learning's on new hardware for me at least. First, I could not get Ethernet on the first board in the picture, or sata drives ! It uses 2 new technologies I am not familiar with. The first is SFTP+ that requires the adapter at the bottom. I think its support 10gig Ethernet.

Second, is Slimline SAS SATA. This adapter is required to connect sata drives. With 4 of these ports, it looks like it supports at least 16 sata drives ! $28 for that little cable ????

kO6Ia7z.jpg
 

Topweasel

Diamond Member
Oct 19, 2000
5,436
1,654
136
OK, I got 2 64 core EPYC Rome CPUs coming. And a motherboard for the other 7551 ES. So in the next week or 2 I will have 4 EPYC boxes with 6 CPUs and 256 cores with 512 threads.


Stay tuned.....
Awesome..... Somebody donating them to you? Otherwise why not wait till the 3990x?
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
Awesome..... Somebody donating them to you? Otherwise why not wait till the 3990x?
No, I had to buy them,. But they are ES samples. They only all-core to about 2.2 per the seller, but the 7702 I think only does like 2,6-2.8 all-core. I got them for $800 each. vs $5000-7500 retail (ebay vs newegg or amazon)

When I hit the lottery, I will try the retail ones.....
 

cellarnoise

Senior member
Mar 22, 2017
709
394
136
Thank you for this build thread. Great stuff and about the only on the web from home users, especially with ES samples.

Though a few weeks late with you having posted previous teasers... :) Kidding!
 

rickxross

Junior Member
Mar 6, 2010
18
8
81
Yea ES are a major PITA to get running and they often have performance crippling bugs if they're early versions. Server grade stuff is so expensive too! But at least you have the onboard BMC, that can be pretty convenient and let you run them headless.
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
Well, bad news.... my H11-DSI-NT rev 2.0, bios 2.0 that is supposed to run 7002 series CPUs won't even post. I asked if they were 2P capable, and I think he said yes. Anyway, they are most likely going back. (the 64 core ES chips)

Oh, and I tried to post with only CPU1 in, in case that was the problem (not sure if you can even do that). No post that way either.
 

zir_blazer

Golden Member
Jun 6, 2013
1,160
400
136
New learning's on new hardware for me at least. First, I could not get Ethernet on the first board in the picture, or sata drives ! It uses 2 new technologies I am not familiar with. The first is SFTP+ that requires the adapter at the bottom. I think its support 10gig Ethernet.

Second, is Slimline SAS SATA. This adapter is required to connect sata drives. With 4 of these ports, it looks like it supports at least 16 sata drives ! $28 for that little cable ????

The SFP+ Ports gives you a lot of freedom since it allows you to use a sort of adapter so that you could select the physical connection that you want. For example, you could plug this SFP+ unit to have standard 10GBASE-T Ethernet via a RJ45 Port. You could also use a fiber optic transceiver. In many cases fiber transceivers and cables seems to be cheaper, have higher range, and lower power consumption than standard copper ones, assuming that you're willing to spend money in a Router with SFP+ Ports, too. Besides, as fiber cables are not electrical conductors, they are safe to lightning strikes. There was also a SFP based VDSL Modem. I find that they should be a lot more fun than standard builtin RJ45-only Ports assuming that you have the money to burn in different adapters types. Note that in many cases the Motherboard Firmware may work on a whitelist-only basis and refuse to POST/work with transceivers that are not of a specific brand.

The fun of the SAS or OCuLink Ports is that in some cases the same Port can support both 4x PCIe lanes or 4 SATA, so by changing the cable you get the flexibility of supporting either 4 PCIe SSDs or 16 SATA, or any combination. Note that this depends on whenever the Motherboard manufacturer decided to use the muxed Lanes that can be setup for either mode (And precisely, one of the reasons why I hate what AMD did with Socket AM4. The Zen die could literally have two OCuLink Ports to use as either 2 4x PCIe or 8x SATA. And OCuLink cables are cheaper than SAS type ones).


Well, bad news.... my H11-DSI-NT rev 2.0, bios 2.0 that is supposed to run 7002 series CPUs won't even post. I asked if they were 2P capable, and I think he said yes. Anyway, they are most likely going back. (the 64 core ES chips)

Oh, and I tried to post with only CPU1 in, in case that was the problem (not sure if you can even do that). No post that way either.
Engineering Samples are hard to get running because they only work in specific Motherboards with specific BIOS versions, unless you want to spend a lot of time doing BIOS modding to see if using older Microcodes could get them to POST. May want to ask in specialized BIOS modding forums like Win-RAID.
 
  • Like
Reactions: lightmanek

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
The SFP+ Ports gives you a lot of freedom since it allows you to use a sort of adapter so that you could select the physical connection that you want. For example, you could plug this SFP+ unit to have standard 10GBASE-T Ethernet via a RJ45 Port. You could also use a fiber optic transceiver. In many cases fiber transceivers and cables seems to be cheaper, have higher range, and lower power consumption than standard copper ones, assuming that you're willing to spend money in a Router with SFP+ Ports, too. Besides, as fiber cables are not electrical conductors, they are safe to lightning strikes. There was also a SFP based VDSL Modem. I find that they should be a lot more fun than standard builtin RJ45-only Ports assuming that you have the money to burn in different adapters types. Note that in many cases the Motherboard Firmware may work on a whitelist-only basis and refuse to POST/work with transceivers that are not of a specific brand.

The fun of the SAS or OCuLink Ports is that in some cases the same Port can support both 4x PCIe lanes or 4 SATA, so by changing the cable you get the flexibility of supporting either 4 PCIe SSDs or 16 SATA, or any combination. Note that this depends on whenever the Motherboard manufacturer decided to use the muxed Lanes that can be setup for either mode (And precisely, one of the reasons why I hate what AMD did with Socket AM4. The Zen die could literally have two OCuLink Ports to use as either 2 4x PCIe or 8x SATA. And OCuLink cables are cheaper than SAS type ones).



Engineering Samples are hard to get running because they only work in specific Motherboards with specific BIOS versions, unless you want to spend a lot of time doing BIOS modding to see if using older Microcodes could get them to POST. May want to ask in specialized BIOS modding forums like Win-RAID.
Do you happen to know if all ES models are only 1P ? What happens if you put a 1P chip in cpu1 slot of a 2P motherboard ?
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
Well, I now have tried those chips on a new 2p and a new 1p, and no post either place. So I am returning them. And the seller has not answered any of my questions.

So NOW I won a 7742 retail EPYC, the fastest 64 core Rome chip made ! Got it for $2700, a steal. Coming friday the 31st (maybe). I have the memory, the heatsing, and will replace a 1950x with it, so all I need is an NVME and it will be ready to install !
 

scannall

Golden Member
Jan 1, 2012
1,944
1,638
136
Well, I now have tried those chips on a new 2p and a new 1p, and no post either place. So I am returning them. And the seller has not answered any of my questions.

So NOW I won a 7742 retail EPYC, the fastest 64 core Rome chip made ! Got it for $2700, a steal. Coming friday the 31st (maybe). I have the memory, the heatsing, and will replace a 1950x with it, so all I need is an NVME and it will be ready to install !
I figured you'd get one at some point. Great deal. Let us know how well it runs.
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
I figured you'd get one at some point. Great deal. Let us know how well it runs.
I have the motherboard, heaksink and memory. Just take a 1950x off-lien and use that case/PSU. ETA from Fedex, says Wed. But I have to sign for it. And since I am deaf, I can't hear the doorbell.
 

scannall

Golden Member
Jan 1, 2012
1,944
1,638
136
I have the motherboard, heaksink and memory. Just take a 1950x off-lien and use that case/PSU. ETA from Fedex, says Wed. But I have to sign for it. And since I am deaf, I can't hear the doorbell.
They do make doorbells for deaf people. Replaces the 'bell' with flashing strobes you scatter around your home. Also a shaker that attaches to the leg of your bed, for when you are sleeping.
 
  • Like
Reactions: prtskg

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136
They do make doorbells for deaf people. Replaces the 'bell' with flashing strobes you scatter around your home. Also a shaker that attaches to the leg of your bed, for when you are sleeping.
I have looked ! I have only been deaf a year, so I have not worked out all the details. If you know of one, link me is appreciated !

Edit, looked and found one for $15 ! coming wed. But thats the day I need it/
 
Last edited:

scannall

Golden Member
Jan 1, 2012
1,944
1,638
136
I have looked ! I have only been deaf a year, so I have not worked out all the details. If you know of one, link me is appreciated !

Edit, looked and found one for $15 ! coming wed. But thats the day I need it/
Home Depot carries them. As I recall you are in the Portland area? The Beaverton store has 4 in stock.
 

cellarnoise

Senior member
Mar 22, 2017
709
394
136
All these unique home processor tests, but little testing of Boinc project production.

Maybe too much focus on getting the little bits to run fast?

Please keep updates coming with stats at some point :)

Still best thread on the webs.
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,482
14,434
136