Beowulf Project

[Dark1]

Has any one experimented with it? If so can you offer any suggestions or comments on how to run this configuration.

thanx

 

[Armitage]

That's a very vague question.
What do you want to know?
Remember, google is your friend ... there is a ton of information out there on computational clusters.

 

[Dark1]

The question was has anyone in these forums experimented with it. Half of the questions asked in these forums can be found through google has that stopped any one form posting them. This was to get a somewhat live conversation about the subject.

 

[Armitage]

Sure, I work with a 16 CPU DEC Alpha cluster writing/running astrodynamics modeling & simulation code. I do some consulting on a 32 CPU AMD cluster down the hall.

 

[Dark1]

Estimate of how much that thing cost?!!!

 

[Armitage]

Originally posted by: Dark1
Estimate of how much that thing cost?!!!

The 32 CPU AMD is brand new ... delivered 2 weeks ago.
I think they paid on the order of $60K for it.

The Alpha was bought before my time, but it was considerably more.
I might take one of my models up to the AMD rack this afternoon and see how they compare.

 

[Dark1]

Oh yeah, what types of software are you running and what type of work are you doing that would warrant that type of computational power?

 

[Dark1]

what would some thing like that rate on specpref (if thats the right name for it).

 

[Armitage]

My work is primarily modeling & simulation for space systems. I have some genetic algorithm based applications for optimizing satellite constellations for various tasks. Even on the cluster those runs can take a few days depending on the parameters. Alot of our cluster use also tends toward Monte Carlo validation of statistical algorithms.

The SPEC benchmarks don't apply to clusters, just monolithic systems.

 

[Dark1]

That's some heavy stuff. With all of that processing power it still takes a few day's that's amazing. Have you looked into clusting high end systems like the Itanium 2, I heard that a dual proc set up was more powerful an IBM Power 4 4 proc setup and a above all cheaper.. I'm not sure if they will fit in a 1u though. That is what you use for that set up right or am I mistaken.

 

[Armitage]

Originally posted by: Dark1
That's some heavy stuff. With all of that processing power it still takes a few day's that's amazing. Have you looked into clusting high end systems like the Itanium 2, I heard that a dual proc set up was more powerful an IBM Power 4 4 proc setup and a above all cheaper.. I'm not sure if they will fit in a 1u though. That is what you use for that set up right or am I mistaken.

I benchmarked some of my code on an Itanium once. It sucked pretty bad actually. I think you need Itanium if you have very large datasets/adress spaces, etc. But for floating point, it sucked.

Actually, the dual Alphas are in 4U rackmounts, and the AMD cluster is in 2U cases. I think cooling is probably to marginal in a 1U case if your not in a dedicated hosting environment.

 

[kylef]

I worked closely on an older Beowulf cluster called ARGUS at Duke University for about a year. It executes conebeam tomography algorithms to backproject three dimensional computer models from an array of 64 two dimensional CCD cameras. At one point in time (for about a week in 1998), ARGUS was on the Top 500 Supercomputers list.

All of the software was custom written using C and MPI. For the backprojection stuff, we were able to achieve about 1 fps performance. That is, we could capture a 1fps "3d video" of whatever was moving around inside the ARGUS array.

I benchmarked some of my code on an Itanium once. It sucked pretty bad actually. I think you need Itanium if you have very large datasets/adress spaces, etc. But for floating point, it sucked.

This is a little OT, but the secret for getting fast running code on the Itanium is to optimize heavily. Were you using GNU compilers or the expensive Intel compilers? Or, did you hand-tweak your IA64 assembly code?

 

[Dark1]

Did either of you compile the code and how difficult would you say it is to do this? Oh yeah The NEw Itanium 2 I here Is much better. What do you guys think of Sun clusters?

oh yeah thanks for your input

 

[Armitage]

Originally posted by: kylef

This is a little OT, but the secret for getting fast running code on the Itanium is to optimize heavily. Were you using GNU compilers or the expensive Intel compilers? Or, did you hand-tweak your IA64 assembly code?

They were Intel compilers, but my access to the machine was very limited, so I didn't have much time to really wring it out. Wish I could find those results now... but the cost/performance ratio was absolutely terrible compared to what was available in the x86 world. In fact, the absolute performance was less then what I could get with x86.

I just assumed it was because my app is very floating point intensive, and doesn't give a rat's a$$ about bandwidth. I did some tests on my Athlon rig switching between 100 and 133 MHz FSB at the same chip clock. The difference was trivial, but the performance does scale almost linearly with cpu clock speed.

 

[kylef]

Did either of you compile the code and how difficult would you say it is to do this?

In the case of MPI, it isn't any more difficult than compiling with a library. MPI handles all of the tedious process distribution stuff.

 

[Dark1]

I sorry that I didn't respond to you guys earlier but I thought this thread had died so to speak (though I'm Sure you want it to). Do you guys think x86 is better in these clusters then RISC. How easy is (or was) it for you to maintain these type of clusters. How often would you say one machine totally crapping out on you? This doesn't bring down the whole cluster does it? How long would it take you to recover from that if it happened?


silly question here:
Do you think that it would be possible to make a version of Wine/VM that can fully utilize this technology with out the Windows program being written to do such a thing?

 

[Armitage]

IMHO, the cost/performance ratio at this time definitely favors x86, unless you need 64 bit CPU. And that will change when Hammer hits the streets.

As for maintenance ... Minor problems are compounded by the number of seperate machines, so you don't cut corners on component quality. We've had serious problems with CPU fans dying on our alpha cluster. If it was just a few workstations, it'd be no big deal. But when you have a few racks full it starts to become a real headache. For awhile we were replacing a few fans per month, but none for awhile since we switched to a better brand and changed the internal configuration of the cases somewhat.

What effect a single machine dying has depends alot on the applications. If the main server machine goes down, you're pretty much SOL. But, if you use PVM, you can design your distributed apps to tolerate failed compute nodes although it is a bit more work. MPI doesn't support dynamic process management currently, so is more fragile in this respect. FWIW, I've worked with both PVM and MPI, and I much prefer PVM.

Do you think that it would be possible to make a version of Wine/VM that can fully utilize this technology with out the Windows program being written to do such a thing?

I don't understand this question at all???
PVM & MPI both work on Wndows. So you could build a windows cluster. It's generally a Bad Idea though. A group here has a small NT cluster, and it's nothing but trouble.

 

[Dark1]

Yes, you?fre right. I did a little more research on windows clusters and It looks like a good idea if you really want to have the Widows Sever environment.