Build for desktop node in a supercomputing cluster

[Dutshar]

Am building 7 computers to serve as nodes in a supercomputing cluster.

For our use case, the biggest bottle neck is usually the quantity of of RAM. Don't need any bells and whistles like HDMI, optical drive, etc... Also the speed of RAM is not even that big or an issue. I did choose Quad cores for the processors, but I chose slow ones because we usually are not CPU bottle-necked.

Was hoping to get some opinions from the community to see if I am planning this correctly or if I am making any incorrect assumptions.

Case:Logisys CS308RD
Mother board: ASRock 970 PRO3 R2.0 AM3+ AMD 970 SATA 6Gb/s USB 3.0 ATX AMD Motherboard with UEFI BIOS
CPU: AMD FX-4100 Zambezi 3.6GHz (3.8GHz Turbo) Socket AM3+ 95W Quad-Core Desktop Processor FD4100WMGUSBX
RAM: Corsair Vengeance 16GB (2x8GB) DDR3 1600 MHz (PC3 12800) Desktop Memory (CMZ16GX3M2A1600C10)
Hard drive: SanDisk Extreme SSD 240 GB SATA 6.0 Gb-s 2.5-Inch Solid State Drive SDSSDX-240G-G25

The biggest concerns that I have:

• Is this enough to complete the build? Am I missing something stupid like a case fan, CPU heat sink, thermal compound, wirres, etc..
• Does the mother board match the processor?
• Does the case match the mother board?
• Does the RAM match the mother board?
• Is the power supply enough to power this unit?
• Was hoping to screw attach everything together, plug it in and be able to install Linux Centos. Is this a reasonable expectation or might I have to screw around with the BIOS or something?

More minor questions I had:

• There seemed to be several "socket" types for AMD processors that matched my criteria (Quad-core). FM1, FM2 AM3, AM3+. What are the differences between these? As fas I could tell the lowest seemed to be AM3, then FM1, then FM2, and AM3+ seemed to be the most advanced processors. I went with a slower AM3+ processor because it and its mother board were about the same price as some of the other stuff. Is there more things that I should be considering with this decisions?
• I liked the mother board I chose because its expandable to 32gb of RAM, however, it seems that since it only had 4 slots, the RAM chips would have to be 16GB each. Single 16GB RAM modules do not seem to be readily available, is this correct?

Anything else I should be thinking about or considering?

Thanks so much!!!

 

[Ayah]

What exactly are you computing? 16GB per node isn't much these days.

Budget?

If you're memory per node limited and not cpu constrained at all, why not go old school or server?

5086B-TRF can give you 2TB of memory per node.
G34 based systems can give you 256GB per socket per node to a maximum of 4 sockets and are relatively cheap to boot.

If you have to go low end, 32GB over 4 sticks = 8GB per stick, which are commonly available.

Edit: less obviously implied -> use less nodes with more memory per node

 

[Dutshar]

Thanks for the input, the G34 looks like a really good option.

As far as the 2TB 5086B-TRF is concerned, the cluster is meant to be horizontally scalable, so the idea is to build it from multiple smaller nodes. This initial build is almost a test case to make sure the system works across multiple machines, some of the aspects of the cluster do not yet need to be on more than one machine, but we are doing it that way anyway so that we know we have set it up correctly to expand later.

Ultimately, we'll probably go with the weaker desktops than the G34 for now just to keep the cost low on the initial build of the cluster, but with your advice, we may be adding some G34 based machines later, so thanks very much.

What exactly are you computing? 16GB per node isn't much these days.

Budget?

If you're memory per node limited and not cpu constrained at all, why not go old school or server?

5086B-TRF can give you 2TB of memory per node.
G34 based systems can give you 256GB per socket per node to a maximum of 4 sockets and are relatively cheap to boot.

If you have to go low end, 32GB over 4 sticks = 8GB per stick, which are commonly available.

Edit: less obviously implied -> use less nodes with more memory per node

 

[Gunbuster]

I know this is nitpicking but you are not building a "supercomputing cluster" with a handfull of $99 FX CPU's

Cluster: Yes
Supercomputing: No

"A supercomputer is a computer at the frontline of contemporary processing capacity--particularly speed of calculation."

 

[Cerb]

Since any 4-slot board today can handle 32GB, and any 2-slot 16GB, I still don't get what you're after, if 16GB is enough.

Are you trying to minimize the budget?

Are you needing quads for decent testing?

There seemed to be several "socket" types for AMD processors that matched my criteria (Quad-core). FM1, FM2 AM3, AM3+. What are the differences between these? As fas I could tell the lowest seemed to be AM3, then FM1, then FM2, and AM3+ seemed to be the most advanced processors. I went with a slower AM3+ processor because it and its mother board were about the same price as some of the other stuff. Is there more things that I should be considering with this decisions?

FM1 and FM2 are for CPUs with IGPs in them. AM3+ is a continuation of the old AM2 socket.
AM2 begat AM2+ begat AM3 begat AM3+, in which each CPU could often go in a board with an older socket version, or an older CPU in a newer socket (FI, an AM3 CPU in an AM3+ socket, or an AM3 CPU in an AM2+ socket). FM1 and FM2 are each unqiue, like Intel sockets, and don't have any of that mess going on.

I liked the mother board I chose because its expandable to 32gb of RAM, however, it seems that since it only had 4 slots, the RAM chips would have to be 16GB each. Single 16GB RAM modules do not seem to be readily available, is this correct?

With 4 slots, each DIMM can only be 8GB (32=4*8). 16GB unbuffered DIMMs are not commonly supported.

Is this enough to complete the build? Am I missing something stupid like a case fan, CPU heat sink, thermal compound, wirres, etc..
Does the mother board match the processor?
Does the case match the mother board?
Does the RAM match the mother board?
Is the power supply enough to power this unit?

Not quite. You'll need something to give you a display. You might want to consider moving to an FM2 system, for the IGP, or using an older-chipset motherboard (here's a GB AM3+ 760G).

And, if you're going to do a lot of writing to the SSD, you might want to choose another one, known for good idle-time GC. With a Windows desktop, TRIM-dependent can be OK, but it's often iffy otherwise, as they get written over a lot (Linux works on many small files regularly, which ends up being harder on them than typical Windows, TRIM or not). In fact, that particular SSD had broken TRIM, so many users were screwed anyway. The Sandisk Ultra Plus is fairly new, but does well in steady-state testing, and is cheap (Sandisk appears to have learned their lesson about not being able to control firmware). It might not last 20+ years of hard use, and won't offer Corsair Neutron or Seagate 600 performance when hammered, but it's cheap, and looks pretty good so far (with that SSD and the AMD build, I'm assuming you're after cheap-but-good).

Was hoping to screw attach everything together, plug it in and be able to install Linux Centos. Is this a reasonable expectation or might I have to screw around with the BIOS or something?

Maybe, but if so, not much. Some motherboards come with odd settings for compatibility reasons, like SATA set to IDE mode, virtualization disabled (I simply cannot understand this default), or other oddities. But, just set up the first one, and change anything that needs changing. If anything does need changing, it won't take long on the other ones. Also, since they're the same hardware, you could simply clone the 1st one's SSD, if it's going to be locally installed.

 

[mfenn]

You have a pretty weird config here:

1. Dual-core CPU (effectively)
2. 16 GB of RAM
3. SSD

Either your workload is very unusual for cluster compute or you're going about this architecture in the wrong way. Can you tell us a bit more about the codes you intend to run?

As far as the 2TB 5086B-TRF is concerned, the cluster is meant to be horizontally scalable, so the idea is to build it from multiple smaller nodes.

Desktop chassis are about the worst way to scale up the number of nodes. Even 7 desktops you're planning to build take up a lot of space. More than a dozen or so is untenable in any kind of normal space. You really want get rackmount gear, even if you don't have a rack just yet.