Multiphysics Simulation / Numerical Optimization Workstation

[thadman]

Hello Anandtech members:biggrin:

I am an engineering student and am currently working on a research project related to the fatigue behavior of loudspeakers. I am interested in building a linux workstation which is capable of solving a fully coupled system model (acoustic, mechanical, thermal, electromagnetic). Simulations will be contrasted with the collected experimental data.

In addition, these models will be integrated within a multi-objective genetic algorithm to solve for the global optimum of a particular system.

I have a lot of experience building PCs (had a little business going in junior high), but I've been out of the market for a while.

What would be considered optimal for such an application?

How soon will Intel be releasing its new server chips?

Best Regards,
Thadman

 

[wwswimming]

Hi !

how big is your budget ?

how much would your model & research be compromised by memory & CPU limitations ?

i think it's generally a good idea to buy the CPU that costs 20% & delivers 80% of the performance. i suppose in this case that would be the 750.

then bank the savings for your NEXT workstation, since there will be another one.

but if budget is no object ... there's a thread here where someone is building a dual 1366 server. probably a good guide.

http://forums.anandtech.com/showthread.php?t=2024033

 

[thadman]

Hi !

how big is your budget ?

Being a full time student, I do not have a large budget. We'll see how much money I'll be able to scrape together. I recently placed the vast majority of my loudspeaker drivers up for sale to finance this project. I should be able to manage $1500-2000, MAYBE $4000.

how much would your model & research be compromised by memory & CPU limitations ?

I am aware that FEA software (I'll be running CalculiX) is VERY memory intensive. If the RAM on the system can accommodate the entire sparse matrix, it can be solved in-core. However, if the amount of RAM is not sufficient, the sparse matrix is written to a file, passing through the disk I/O subsystem. I believe the system is bandwidth limited, rather than CPU limited.

I am not sure if 36gb would be sufficient or 72gb to be more appropriate for such an application. The workstation will be expected to integrate the coupled fluid-structure model within a genetic algorithm.

I've spoken with Dr. Crossley, a faculty member in Purdue's Aerospace Engineering department who teaches a graduate level course on Multidisciplinary Design Optimization. He expressed that the solution associated with very advanced models may require a significant amount of time (ie weeks) before it is realized.

If the system simply has to be solved once, processing time will be a minor inconvenience. However, if the system has to be solved 1,000,000 times (initial population = 1000, number of generations = 1000), advanced models may quickly approach impracticality. I would like to avoid this, if at all possible.

 

[zephyrprime]

Man, that's going to be tough to do on such a budget! Maybe cloud computing would be a viable solution for you since you only need this computing power for a limited amount of time.

You have to avoid using IO at all costs because that will slow you down by at least 1 order of magnitude if you need to write each generation to disk.

You're not going to be able to buy cutting edge stuff with that sort of budget. However, you could buy some last gen stuff like:
http://www.newegg.com/Product/Produc...82E16813151085
http://shop.ebay.com/i.html?_nkw=AM...dmpt=CPUs&_odkw=AMD+Opteron+8*+quad&_osacat=0

Ebay has a lot of high end last gen stuff that goes for pretty cheap.

Try to get a precise estimate of the amount of memory you need. And try to figure out what kind of intermediate results you want to write to disk.

Memory costs will be your biggest expense.

 

[thadman]

Man, that's going to be tough to do on such a budget! Maybe cloud computing would be a viable solution for you since you only need this computing power for a limited amount of time.

You have to avoid using IO at all costs because that will slow you down by at least 1 order of magnitude if you need to write each generation to disk.

You're not going to be able to buy cutting edge stuff with that sort of budget. However, you could buy some last gen stuff like:
http://www.newegg.com/Product/Produc...82E16813151085
http://shop.ebay.com/i.html?_nkw=AM...dmpt=CPUs&_odkw=AMD+Opteron+8*+quad&_osacat=0

Ebay has a lot of high end last gen stuff that goes for pretty cheap.

Try to get a precise estimate of the amount of memory you need. And try to figure out what kind of intermediate results you want to write to disk.

Memory costs will be your biggest expense.

I don't think I have the room for a cluster.

Assuming I should focus on memory, what processor(s) would you recommend (i7 or Xeon)?

 

[Idontcare]

thadman I've been in your shoes, grad student scrapping money together to buy rigs for HPC stuff (in my case ultimately going with a 12 node cluster, DIY beowulf style) and the best advice I can think to offer you is "measure twice, cut once" meaning you really need to make sure that the rig you are building is actually going to do the job and not just wing it and be hopeful.

Whoever makes the software, start with their sales/support guy via email and get your hands on some actual real-world scaling results. Find out from people who use the specific program, not just ones that are like it, and nail down just how critical that memory value needs to be.

Big difference between 36GB capable platforms versus 72GB, you are crossing a threshold in there and before you decide you need to do that you need to make darn sure you really actually do need to do that because that will be a big cost driver.

Your choice of CPU is going to be totally secondary at this stage if your memory needs are truly as critical as you have been led to believe them to be. Once you know your ram requirements then you go shopping for a platform (be it AMD or Intel) and once you have your mobo and chipset selected (plus ram) then you get to figure out just how little you will have left for buying your CPU.

This is assuming the premise that you led into is correct, that ram is critical and cpu is secondary because it will be completely hamstrung if there isn't enough ram.

 

[wwswimming]

my experience is that government agencies and companies like giving money to grad students for projects that benefit those agencies/and or companies.

i worked for an ANSYS VAR once and his main machine was a minicomputer with at most 144 MB (this was in 1988, RAM was $500 a MB). we used Compaq Deskpro's etc. (386-20 etc.) for front-end & back-end work, and gave the model to the mini for analysis.

my point being ... you can probably get some useful work done with an affordable amount of RAM, e.g. 8 GB to 12 GB.

one approach is to build an affordable solver, then ask some organization that might sponsor you for more $$. e.g. "if i can do this much work with 12 GB of RAM, just think how much work i can do with 36 GB".

 

[tcsenter]

Get a better handle on your budget and then we can help you get the most for whatever you have to spend. There isn't any other way to go about it if your budget is limited.

 

[Cogman]

I read up on CalculiX, its free, yes, but I could have sworn there was a CUDA alternative that does the same thing (This type of problem SCREAMS ridiculously parallel/use video cards). You might consider switching programs to one that utilizes the video card more as you should be able to get some pretty decent speed increases as a result.

As for the memory requirements, you're in a tough situation, You could almost spend $2000 on memory alone if your requirements are about right (72GB). I would almost plan on paging and get 2/3 hard drives in a raid 0 or raid 1 configuration. Unless you actually bring together 72gb of ram, I would plan on hitting the HD, which means your disks need to be as fast as possible.

 

[Fox5]

The only way you'll get the amount of ram you want is to go with a server system.

An Intel Xeon system would be preferable (the i7 based xeons), but you might be able to get an AMD Opteron system with the same amount of ram cheaper.

4GB sticks of ECC DDR3 run about $37 a GB. There are Intel LGA1366 dual socket motherboards with 18 slots, so those will old 72GB, though that'll cost you over $2664 just for memory. Still, for $4000, you should just be able to do it.

4GB sticks of ECC DDR2 run about $27.50 a GB, so an AMD system would come out cheaper. Still, dual socket AMD systems seem limited to 16 slots, so you'd max out at 64GB at a cost of $1760.
You might also be able to pull of a quad socket system, if you don't mind using some seriously outdated opterons. Then you could plop in ram up to 72GB, with room for expansion later. It'd be a challenge to pull that off within $4000 though, even using 2Ghz Athlon X2 based opterons.

 

[Idontcare]

Cogman is right, he may as well just assume his rig is going to be paging and build a raid-0 subsystem accordingly instead of blowing his entire wad trying to get enough ram to keep the simulation in memory.

 

[Fox5]

Cogman is right, he may as well just assume his rig is going to be paging and build a raid-0 subsystem accordingly instead of blowing his entire wad trying to get enough ram to keep the simulation in memory.

It's doable within the upper limits of his budget, what would the performance difference be? He could also get more cpu power with less memory, or a RAID array of SSDs to use for paging.