144GB RAM + MOBO Compatibility Question?

[MattM76]

I am building a large workstation for scientific computing, and need to double check with the Anandtech experts about hardware compatibility. The system will be a dual socket Xeon 5520 machine with a whole lot of RAM.. Mostly, I am concerned about motherboard-memory issues.

***Mobo: ASUS Z8PE-D18. Dual Intel Xeon 5500 series Server Motherboard. 18 DDR3 DIMMS: http://www.newegg.com/Product/Produc...82E16813131379

***The RAM: Patriot Signature 12GB (3 x 4GB) 240-Pin DDR3 SDRAM ECC Registered DDR3 1333: http://www.newegg.com/Product/Produc...82E16820220423

Other parts, just FYI

***PS: ENERMAX REVOLUTION85+ ERV1050EWT 1050W ATX12V / EPS12V SLI Ready CrossFire Certified: http://www.newegg.com/Product/Produc...-039-_-Product

***HD: Western Digital VelociRaptor WD1500HLFS 150GB 10000 RPM 16MB Cache SATA 3.0Gb

***HD#2: Seagate Barracuda 7200.11 ST31500341AS 1.5TB 7200 RPM 32MB Cache SATA 3.0Gb

Would anybody be able to take a look at the RAM-MOBO combination and help me decide if they willl work together?

 

[superunknown98]

A starting point is to look for a compatability list on Asus' website.

 

[vailr]

18x 4 Gb = 72 Gb, not 144 Gb.
Checking Newegg, they don't seem to offer any 8 Gb DDR3 sticks.

 

[theevilsharpie]

A starting point is to look for a compatability list on Asus' website.

Another option is to go through the RAM configurator on the RAM manufacturer's web site.

Patriot's configurator seems to be broken, but Crucial's is working, and gave me this recommendation. You'll need six of them to max out the PC's memory. I hope you've got your loan papers in order

 

[Gamingphreek]

I am building a large workstation for scientific computing, and need to double check with the Anandtech experts about hardware compatibility. The system will be a dual socket Xeon 5520 machine with a whole lot of RAM.. Mostly, I am concerned about motherboard-memory issues.

***Mobo: ASUS Z8PE-D18. Dual Intel Xeon 5500 series Server Motherboard. 18 DDR3 DIMMS: http://www.newegg.com/Product/Produc...82E16813131379

***The RAM: Patriot Signature 12GB (3 x 4GB) 240-Pin DDR3 SDRAM ECC Registered DDR3 1333: http://www.newegg.com/Product/Produc...82E16820220423

Other parts, just FYI

***PS: ENERMAX REVOLUTION85+ ERV1050EWT 1050W ATX12V / EPS12V SLI Ready CrossFire Certified: http://www.newegg.com/Product/Produc...-039-_-Product

***HD: Western Digital VelociRaptor WD1500HLFS 150GB 10000 RPM 16MB Cache SATA 3.0Gb

***HD#2: Seagate Barracuda 7200.11 ST31500341AS 1.5TB 7200 RPM 32MB Cache SATA 3.0Gb

Would anybody be able to take a look at the RAM-MOBO combination and help me decide if they willl work together?

1. 4 * 18 = 72 GB not 144GB
2. Why, with all of that RAM are you only using Dual Xeon's? Is there a point to you maxing out the RAM?
3. You are building a scientific grade machine and you are using a consumer grade power supply?
4. You are building a scientific grade machine and you chose to use a Raptor combined with a 1.5TB Seagate drive? With a machine where you have maxed out the RAM, why in the world are you not using SAS or, at the very least, SCSI?

You really need to tell us why you think you need all of that RAM with the rest of the components being so consumer grade.

-Kevin

 

[MattM76]

A starting point is to look for a compatability list on Asus' website.

Yeah, did that. They dont list it, but then again, they don't list hardly any..

 

[MattM76]

Patriot's configurator seems to be broken, but Crucial's is working, and gave me this recommendation. You'll need six of them to max out the PC's memory. I hope you've got your loan papers in order

Yeah, thats exactly why I have to stick with the 4GB sticks... Those 8GB are WAY too expensive!!

 

[MattM76]

1. 4 * 18 = 72 GB not 144GB
2. Why, with all of that RAM are you only using Dual Xeon's? Is there a point to you maxing out the RAM?
3. You are building a scientific grade machine and you are using a consumer grade power supply?
4. You are building a scientific grade machine and you chose to use a Raptor combined with a 1.5TB Seagate drive? With a machine where you have maxed out the RAM, why in the world are you not using SAS or, at the very least, SCSI?

You really need to tell us why you think you need all of that RAM with the rest of the components being so consumer grade.

-Kevin

Thanks Kevin.. Yes, there is a poing to maxing out the RAM.. I am using the computer for analysis of next generation sequence data.. In short, we are talking about the analysis of 50 million (give or take) 100 base pair reads.. Do the math, but 50,000,000 * 100 letters each is a sh!t load of data, which is represented in a single deBrujin graph. Here is the link, in case your not sure what that is (http://en.wikipedia.org/wiki/De_Bruijn_graph)..

Unfortunately, most of these tasks are non parallizable, and so the entire data set is loaded into RAM.. Even with the amount of RAM I am currently able to stick in this machine, I won't come close to being able to load the complete dataset at once, and combining multiple runs is, problematic at best..

The bottleneck with these analyses is RAM.. easily. CPU and hard drive read/write times, while important, are not limiting. Its like building a gaming rig (with which you are obviously familiar) with the best CPU possible, and crappy video card, then wondering why you can't play full resolution (or whatever the gaming metric of goodness is)..

Do you have a rec. for a better PS? Will the 1050W Enermax not do the job?

Thanks. Matt

 

[tcsenter]

***Mobo: ASUS Z8PE-D18. Dual Intel Xeon 5500 series Server Motherboard. 18 DDR3 DIMMS: http://www.newegg.com/Product/Produc...82E16813131379

Meh, should be going with a real enterprise/server board manufacturer such as Intel, Tyan, or SuperMicro. I would prefer Seasonic, PCP&C, or Zippy/Emacs for PSU.

 

[wwswimming]

in 5 years or whever you scrap it, can i have it ?

 

[Gamingphreek]

Meh, should be going with a real enterprise/server board manufacturer such as Intel, Tyan, or SuperMicro. I would prefer Seasonic, PCP&C, or Zippy/Emacs for PSU.

My thoughts exactly.

Thanks Kevin.. Yes, there is a poing to maxing out the RAM.. I am using the computer for analysis of next generation sequence data.. In short, we are talking about the analysis of 50 million (give or take) 100 base pair reads.. Do the math, but 50,000,000 * 100 letters each is a sh!t load of data, which is represented in a single deBrujin graph. Here is the link, in case your not sure what that is (http://en.wikipedia.org/wiki/De_Bruijn_graph)..

Unfortunately, most of these tasks are non parallizable, and so the entire data set is loaded into RAM.. Even with the amount of RAM I am currently able to stick in this machine, I won't come close to being able to load the complete dataset at once, and combining multiple runs is, problematic at best..

The bottleneck with these analyses is RAM.. easily. CPU and hard drive read/write times, while important, are not limiting. Its like building a gaming rig (with which you are obviously familiar) with the best CPU possible, and crappy video card, then wondering why you can't play full resolution (or whatever the gaming metric of goodness is)..

Do you have a rec. for a better PS? Will the 1050W Enermax not do the job?

Perhaps I am misunderstanding what you typed, but
50,000,000 * 100 = 5 Billion letters.

I am assuming that, by letter, you mean a char. A char is 1 byte. Thus 5 Billion bytes should only be 4.7 GB of memory.

Furthermore, saying that you are entirely dependent on RAM and not processor or hard disk is incorrect.

Consider the speed of the interconnect between the RAM and the processor. (ie: FSB). If this increases the data throughput increases.

As for the HDD, I suppose if you aren't polling it constantly, you don't need a SCSI or SAS. In that case, save yourself some money and ditch the overpriced Raptor. Take that money and ensure data integrity and grab 2 more cheaper HDD and set up a RAID 5 array across the 3 HDDs.

-Kevin

 

[schenley101]

if your running into ram limitations, you might want to look for a quad socket opty or wait until intel comes out with their new quad socket(the 7400 is not worth it!!)

 

[Cruisin1]

What's your budget? This sounds like an enterprise type of machine... shouldn't you be looking into a REAL manufacturer like IBM or HP?

 

[wwswimming]

when it's de-commissioned, would you accept a bottle of Foppoli or Chateau Montelena Chardonnay in trade ?

 

[hymy]

We needed a super cruncher for metabolic modeling, which is also very ram intensive. I did post here and was ridiculed as well. The problem being that IT people don't get what scientists are trying to do. The head IT guy was very critical of the amount of ram we wanted. He changed his tune after he saw the screen shot of the out of ram message, that is after he regained his color. Apparently it's necessary for them to actually see the 24 gb matlab matrix loaded into ram before they comprehend.

Using that same motherboard w/dual xeon 5530 we did:

We got 48 gigs of this:
http://www.newegg.com/Product/Produc...82E16820220421
It's important to get dual rank ram, what you chose is.

We considered velociraptors, but went with raid 1+0 with 4 of these:
http://www.newegg.com/Product/Produc...82E16822136284
This peaks at about 2 Gbits/s, but recently some numbskull initalized the raid array.

This turns out to be a decent machine. Buy a video card, the onboard video is abysmal. Its an ssi eeb spec motherboard, and technically you should get the appropriate case. we put ours in an atx tower and about 3 screw holes didn't get used.
http://www.newegg.com/Product/Produc...82E16811133056

we used this power suppy:
http://www.newegg.com/Product/Produc...82E16817153072

And these adaptors:
http://www.newegg.com/Product/Produc...82E16812198003
http://www.newegg.com/Product/Produc...82E16812198006

It runs great, except it's down for recovering the data after the array was initialized. Apparently the motherboard isn't idiot proof and one can initialize the raid in very few keystrokes. It has matrix raid and lsi mega raid, and it doesn't come with the pike card you'd need in order to use SAS drives. We installed winXP 64bit pro with no problems whatsoever.

Oh these processors run hot, so you want fans, fans, and fans. Ours run at about 79-80C under a full load, apparently they like it and the cpu fans are still very quiet even at that temp. Intel says these things will run at 75 at normal wattage. They clock up a little under load and pull more watts though.

 

[tcsenter]

We needed a super cruncher for metabolic modeling, which is also very ram intensive. I did post here and was ridiculed as well.

I was very helpful, for the record. :biggrin:

 

[KamiXkaze]

I could have sworn there are waterblocks for those cpu's

kXk

 

[faxon]

I am building a large workstation for scientific computing, and need to double check with the Anandtech experts about hardware compatibility. The system will be a dual socket Xeon 5520 machine with a whole lot of RAM.. Mostly, I am concerned about motherboard-memory issues.

***Mobo: ASUS Z8PE-D18. Dual Intel Xeon 5500 series Server Motherboard. 18 DDR3 DIMMS: http://www.newegg.com/Product/Produc...82E16813131379

***The RAM: Patriot Signature 12GB (3 x 4GB) 240-Pin DDR3 SDRAM ECC Registered DDR3 1333: http://www.newegg.com/Product/Produc...82E16820220423

Other parts, just FYI

***PS: ENERMAX REVOLUTION85+ ERV1050EWT 1050W ATX12V / EPS12V SLI Ready CrossFire Certified: http://www.newegg.com/Product/Produc...-039-_-Product

***HD: Western Digital VelociRaptor WD1500HLFS 150GB 10000 RPM 16MB Cache SATA 3.0Gb

***HD#2: Seagate Barracuda 7200.11 ST31500341AS 1.5TB 7200 RPM 32MB Cache SATA 3.0Gb

Would anybody be able to take a look at the RAM-MOBO combination and help me decide if they willl work together?

i currently own one of at least everything there in bold. the barracuda drives are the only thing i would be concerned with. all 10 of the 7200.11 drives that me and my step dad own (combined total) have failed or are developing bad sectors at an alarming rate. the Vraptor is a nice drive however, and the 85+ 1050 is one of the best, if not the single best PSU in its class. it should be an excellent pick for a scientific computing rig with a large RAID array, dual CPUs, and an HPC GPGPU set of cards. i picked it up for its outstanding power output quality and stability under load, but the only gripe i have is that some boards seem to simply not want to work with it, specifically certain varriants of the EVGA Classified, and the Gigabyte GA-EX58-UD5. if you wanna know about classified related issues, talk to aigomorla, im pretty sure he had this exact issue at one point

 

[ricoPan]

I am also building a big RAM machine. I looked at the Tyan, Asus, and Supermicro Xeon 5520 dual boards with 18 RAM slots, and eventually decided on the Supermicro X8DAH, as the Tyan boards had a fair number of complaints, and the Asus looked good but were supposedly a bit cramped. The problem with the Supermicro has been getting a case that will fit the board (13 x 13.7") and a power supply with an additional 2 12 volt cpu connectors. Eventually it seemed best to get the supermicro tower built for the board, which just arrived. Like you, I plan on maxxing this out with 4 gb rather than 8 sticks.

It is funny how resistant many admins or others are to the idea of needing more than 8 gb or so a process -- don't they understand the general space vs time tradeoffs? Isn't hard to need more RAM -- sometimes to make up for quick and dirty practices, other times for real. For me, it will be a matter of convenience (eg simple code to find transitive closures of big graphs that won't fit in normal RAM) and necessity -- linear algebra on big matrices of all vs all comparisons. But its really quite inexpensive now, and no, just because I need more RAM does not necessarily mean I need blazing cpu speeds.

I registered on this board just to reply -- I'll be interested how your machine(s) works out. As a side note, any interest in GPU for cpu bound scientific problems? I'm just starting -- hoping that the gods provide a nice numpy (python) interface soon.

 

[Rubycon]

the 85+ 1050 is one of the best, if not the single best PSU in its class. it should be an excellent pick for a scientific computing rig with a large RAID array, dual CPUs, and an HPC GPGPU set of cards. i picked it up for its outstanding power output quality and stability under load, but the only gripe i have is that some boards seem to simply not want to work with it, specifically certain varriants of the EVGA Classified, and the Gigabyte GA-EX58-UD5. if you wanna know about classified related issues, talk to aigomorla, im pretty sure he had this exact issue at one point

I was affected by this issue (Galaxy EVO 1250 + EVGA Classified 759). Enermax is aware of this and all supplies with SN beginning with 95 and higher now work with this board. If you have an older PSU that does not work with this board they will RMA it for one that does.

 

[hymy]

As a followup on this ASUS board. You can't lock the storage menus with a bios password so be aware that a few, a very few keystrokes you can wipe the drives. You need the pike card if you want SAS drives. If your going to fill to the max with dimms, make damn sure they are dual/2 rank modules.

Just and aside to those engaged in this. The excel 2010 64bit beta, changes the game for excel in science. Multi threaded, cluster aware and no limits to spreadsheets and graphs.

 

[jvroig]

It is funny how resistant many admins or others are to the idea of needing more than 8 gb or so a process -- don't they understand the general space vs time tradeoffs?

No, they don't. If my teaching experience has taught me one thing, it's that CS/IT people now are more concerned with databases and Java and hardware, and less on actual CS concepts regardless of how relevant it may be to their fields.

Whenever I talk about space-time tradeoff in a meeting with IT people, I get pretty much the same reaction I get when I introduce space-time tradeoff in class to my students: they all initially think I'm talking about Star Trek or time loops or something. It's crazy.

 

[HannibalX]

We needed a super cruncher for metabolic modeling, which is also very ram intensive. I did post here and was ridiculed as well. The problem being that IT people don't get what scientists are trying to do.

Not all IT people would have put up a roadblock for you. There are smart IT people and dumb IT people, just like there are smart scientists and dumb scientist. It's not an all or nothing proposition. Stereotypes are silly.

Had I been working for your organization and you came to me with your requirements after reviewing examples of your work environment I would have had zero problem building a machine to the specifications required by the task.

Please don't lump us all into one category.

 

[Mothergoose729]

I am not sure about all this server what not, but those seagate 1.5tb drives are really horrible. A similar capacity/price samsung drive can be had, that is much more reliable, and from a company that doesn't have a recent history of firmware debacles.

 

[tweakboy]

I think the max for Xeon is 128GB. gl