building 32GB ram machine - feasible?

Toggle sidebar Toggle sidebar
H

hypothalamuse

Junior Member
Nov 13, 2007
21
0
0
I'm looking into a machine with 32GB of ram for some microarray analysis I'm working on. I've built desktop pc's before and I'm wondering how much more is involved in setting up a machine with a ton of memory beyond picking out a mobo that will support it.

Just looking at Dell's prices it seems like I could save a few thousand dollars doing it myself. Thanks!
 
A

AmberClad

Diamond Member
Jul 23, 2005
4,914
0
0
So, this sort of a setup rules out a typical desktop board. The most those will support is 8GB, I believe.

Tyan and SuperMicro both have server motherboards that will take in 16 DIMMs and support up to 64GB. Quite expensive: ~$500 for the board alone, plus 16 x 2GB = 16 x ~$50 each = $800.

After that, you're on your own. I don't know too many people besides sys admins that would have experience working with that elaborate of a setup :p. Although, one would assume that a 64 bit OS would be required. What do the Dells use? Windows Server 2003?
 
O

OdiN

Banned
Mar 1, 2000
16,431
3
0
Nothing really complicated.

You will need:

Motherboard that supports it.

Probably something like this:

http://www.tyan.com/product_board_detail.aspx?pid=562

The RAM - here's the thing...you will probably have to get Registered ECC RAM for boards that support 32GB. They won't work with regular for the most part. But if you are doing large scale data analysis, then you would want the Registered ECC anyway. Also have to get 4GB modules at least for most boards, unless you got a board with 16 RAM slots.

Then the OS. You will need a 64-bit OS of course, otherwise all of that RAM is useless. Depending on what you want to do, you will need something compatible with your program you are running. Vista may be out for server boards too. So if XP x64 would work, that would be one option, otherwise you will need Server 2003.
 
H

hypothalamuse

Junior Member
Nov 13, 2007
21
0
0
thanks for the input guys - dell is asking ~6K or 10K for dual quad xeon setups with 16 or 32GB ram respectively

I can't find any motherboad with 16 dimm slots - if you know of a specific one shoot me a link. Certainly cheaper than 8x4GB.
 
bamacre

bamacre

Lifer
Jul 1, 2004
21,030
2
61
Originally posted by: hypothalamuse
thanks for the input guys - dell is asking ~6K or 10K for dual quad xeon setups with 16 or 32GB ram respectively

I can't find any motherboad with 16 dimm slots - if you know of a specific one shoot me a link. Certainly cheaper than 8x4GB.
Click to expand...

Check my link above.

Consider the fact that if you build this yourself, you're gonna be supporting it yourself as well. And if something goes bad, are you going to have spare parts around to test and diagnose what is wrong? In these kinds of cases, it is nice to have direct system support, and using Dell Outlet, sometimes it is even cheaper as well.
 
A

AmberClad

Diamond Member
Jul 23, 2005
4,914
0
0
I think these were two I found. I believe both of those are dual Xeon boards. The SuperMicro is out of stock at the moment.

Amusingly enough, if you check the review for the Tyan, some idiot actually built a gaming setup with it...
 
T

thorin

Diamond Member
Oct 9, 1999
7,573
0
0
Originally posted by: hypothalamuse
I'm looking into a machine with 32GB of ram for some microarray analysis I'm working on. I've built desktop pc's before and I'm wondering how much more is involved in setting up a machine with a ton of memory beyond picking out a mobo that will support it.

Just looking at Dell's prices it seems like I could save a few thousand dollars doing it myself. Thanks!
Click to expand...

This could be true however how important will the machine be once it's in production and being used for these analyses? If you put it together yourself you may save a few grand but if it ends up down for a week or more while you try to fix/replace something down the road is that going to cost more than the initial savings? It might be worth spending a bit more on the Dell w/ same day on-site support.
 
JackMDS

JackMDS

Elite Member
Super Moderator
Oct 25, 1999
29,471
387
126
I actually saw the Tyan few days ago, it is actually a big Mobo with 4 CPU sockets each has it own 4 Dimm sockets.

The thingy runs 4 Quads. I.e it can be configured as a 16 cores animal.

The size is designed for a Blade Rack mounting.
 
Knavish

Knavish

Senior member
May 17, 2002
910
3
81
This is very doable. I've got a Dell Precision 690 at work. It has 2x CPU sockets and 16x FBDIMM slots. The FBDIMMS slots are in riser boards that stand up off the motherboard. It's pretty similar to this Tyan board:
TYAN S5382WAG2NRF

I'm sure you could build one yourself, but I'd be a bit concerned...
  • How do you support the riser cards in the case? They stand many inches off the motherboard & probably need some kind of a rack or support
  • FBDIMMs use a lot of power. The Dell riser boards have their own power connection from the CPU. How do you make sure that you have enough air flow to cool fully populated riser boards in a home built case?

I think the easiest thing to do would be to buy a stripped down Dell Precision 690 (make sure you get it with riser cards!) and then add the ram later. The 690 starts at about $1900. Get the base model with the 1000W power supply and a low end cpu, video card, & HD. That'll be about $2000. Maybe you should just go ahead and get the CPUs at Dell if it's not a huge ripoff. Then add your own ram:
32 GB = 16x 2GB FBDIMM = 16 * about $100 = $1600 for 32 GB!!!

If you want to add your own 2nd CPU, you'll have to buy a spare heatsink from dell. I'm sure they'll sell you one -- you probably just got to call them. You have to use their special passive cooling tower since it's designed to work in the airflow of their case.

Go for it! I bet you can get a 8 core, 32 GB ram system for about $5k in a warranted & supported dell box.

-Knavish

BTW, what OS are you going to run??
 
H

hurtstotalktoyou

Platinum Member
Mar 24, 2005
2,055
9
81
A quick look at Newegg suggests a dual Opteron 2210 custom build with 32GB RAM would cost $3,000-$3,500, depending on what other components you need.
 
M

MarcVenice

Moderator Emeritus <br>
Apr 2, 2007
5,664
0
0
How long will you need it ? Why not rent something ? Dunno if it's a viable option, but figured I might as well throw in my 2 cents.
 
H

hypothalamuse

Junior Member
Nov 13, 2007
21
0
0
I agree with the $3500 number - somewhat more for the fastest processors, but really what I need is the ram.

I'll be analyzing similar networks for the next two years, so investing in the machine seems like a decent idea. Really appreciate the input.
 
Knavish

Knavish

Senior member
May 17, 2002
910
3
81
On second thought, the AMD box would make more sense. That's the only field where they are beating Intel in some respect. It won't have the same raw horsepower, but you save a lot of money.

AMD 2x socket, 16x DIMM motherboard ~ $450
AMD 2x Barcelona 1.9 Ghz ~ $800
2GB DDR2 667 ECC unbuffered x 16 ~ $1600
Case + PSU ~ $200

+ video card, drives, accessories

...not too bad

-Knavish
 
H

heyheybooboo

Diamond Member
Jun 29, 2007
6,278
0
0
If you are going to try this I think you best be up-to-date on concepts of virtualization, parallelism, the apps being used for your microarray analysis and how they are compiled.

First off, can your microarray analysis software/processes be run by 'distributed computing' on a gigaLAN ?? As exciting as a 2p/4p server with 32Gb of RAM sounds - if your software permits - a 'desktop farm' may be faster, cheaper and less of a hassle.

I think your best success would be with an enterprise 'nix OS and Optys but as OdiN noted you may not have this option depending upon your apps and expertise. I think Red Hat RHEL has been talking about "Para-virtualization" (did I just make up a word? :D ) but I've never seen it in action.

To make all this work at the level you want you will need some serious disk I/O !

And though I think the Opty's may reign supreme, config setup is a more serious issue to gain performance advantages. You may get near linear scaling with AMD on certain processes but NUMA systems will kick yer arse with page faults if you don't ?localize" processes to specific cpu/memory banks.

This is not an issue with the 'northbridge' Intel/Xeon` arch.
 
T

thorin

Diamond Member
Oct 9, 1999
7,573
0
0
I agree with heyheybooboo, if it's possible to setup a small farm for the analysis you might be further ahead in the long run. Especially if this analysis system is only partially product (ie: no on-site same day support). If a single node in a small farm goes down at least you can still work/get results whereas if you have a single honking huge server with a junky support package you're dead in the water for days to as much as a week or two.

Tyan's new flexBlade system(s) might be an option, or I suppose you could build something like this (http://www.mini-itx.com/projects/cluster/ ... maybe you'd want beefier processing like Intel's new D201GLY series instead of Via's m-itx but u get the idea) as well. Even a few of those Shuttle bare bones box systems with a decent switch might fit the bill.
 
H

hypothalamuse

Junior Member
Nov 13, 2007
21
0
0
Hi Guys,

I've considered the suggestions here - really helpful, and based on this and work from the Uni PTB's, I'm going with dual quad core xeons with 32GB of ram.

Question: Is there a particular distro of linux that is easy to setup in this situation? I've been a user... just never installed before. Thanks!
 
You must log in or register to reply here.

TRENDING THREADS

COMPANY
RESOURCES
FOLLOW
Top Bottom