Mixing Athlons and Opterons

[wacki]

Is it possible to have a cluster that has nodes from Althon computers and Opteron based computers and access them as a whole, or access them seperately depending on if you need 64 bit computing or not?

Example:
You have a cluster with 20 dual athons and 20 dual Opterons.

You are running Blast (a 32 bit program) it runs on all 40 computers.

Then you run Bayes (needs 64 bit computing) it runs on the 20 Opterons only.

 

[drag]

depends on what types of clusters.

Beowolf style clusters are computational clusters. They are generally built up from nearly scratch. You could, but it would make your work 10x harder. Instead of having universally same hardware your going to end up having to construct metrics and algorythms based on hardware capabilities. That's why these types of clusters generally have the same hardware for all the nodes, it makes making it efficient easier.

OpenMosix type clusters are clusters that create a sort of SMP computer were threads are moved from computer and computer to load balance the thing. Right now, no, that wouldn't work. Unless you ran everything in 32bit mode, then it has a good chance. It automaticly balances out the computers, but with this model a single program will never run faster then the fastest computer, but multitasking is improved considurably. Were you run into problems is stuff like using SSE instructions in a program on one computer but then switching to a computer that can't handle those instructions because it's CPU doesn't support it. Then the program usually fails.

The other types of clusters are based on increasing networking capabilities or splitting the load from one server to several servers. Like used for big/fast Databases. So those wouldn't be bothered so much as long as the software runs on everything the same way.

Then the last type of cluster is were you have computers that remain in backup, unused, until a server fails and then they leap in to replace it transparently. You could mix and match hardware with that, I suppose.

 

[wacki]

You sound like you know what your talking about, good!

Here is some more information:

We have a Beowolf Cluster running NPACI Rocks release 3.1.0, a version of Linux. It consists of 37 Dual Athlon MP 1900+ servers with 512 MB of RAM each for a total of 74 available processors.

I'm in a debate on what type of cluster we should buy, or if we should add nodes to the current cluster. We are currently running programs that use Bayesian analysis (likes 64 bit computing but doesn't need it), MPI (message-Passing Interface) and non MPI programs, and numerous 32 bit linux programs i.e. BLAST. We are currently using PBS (Portable Batch Systems) to implement the data on numerous nodes. It would solve all of our problems if we could get the Opertons to work with the Athlons in a cluster format with the current Beowolf cluster. Creating the performance matrices and algorythms probably wouldn't be worth the effort. But then again I don't know, I'm not that intimate with clustering.


If we just started from scratch:
If there was a way to use to create a cost effective dual athlon node cluster, and tie with with a dual or quad Opteron Node cluster and be able to run them together efficiently, how would you do it?

Or is all of this simply not worth the effort?

 

[BlackMountainCow]

(( you see wacki, right place --> good answers ))

 

[wacki]

Originally posted by: BlackMountainCow
(( you see wacki, right place --> good answers ))

Yes, thank you very much

 

[wacki]

Also, why are we going to have to construct metrics and algorythms based on hardware capabilities? Isn't there a job queue and one job runs until it's done, then another job takes it's place?

 

[drag]

Originally posted by: wacki
Also, why are we going to have to construct metrics and algorythms based on hardware capabilities? Isn't there a job queue and one job runs until it's done, then another job takes it's place?

Actually your probably right.

My understanding of the actual cluster and it's functionality is realy limited. If your applications are able to adjust for the different speeds of the computer their shouldn't be any problem anyways.

What may happen is that you get the faster machines getting done with their calculations much faster then the slower machines, so that if your application(s) has to wait for all the threads to get finished you can end up with a situation were you have the fast machines sitting idle while the slower ones finished.

All I realy know is from things like from here

Q: I read your cluster series. Can I mix different CPU speeds and vendors (Intel and AMD) in a cluster? Is a dedicated subnet really necessary?

A: Although the ideal Beowulf cluster has identical hardware with identical configurations. Still, for most purposes, mixes hardware and configurations is fine. Dedicated subnets will make machine-to-machine communications more efficient, especially with high-power workstations, but it is not a necessity. I run a cluster from a network 10/100 switch with no problems.

Multithreaded applications are still completely over my head....

I think it would have mostly to do with the applications, and how well their multithreaded... If your doing multithreaded stuff, that is.

However one thing that I saw somewere is that it's normal to combine different types of clusters, so that if you end up having issues with machines remaining idle because of drastic speed differences you can use stuff like Mosix/OpenMosix to migrate threads around to make load balancing easier (although you then run into problems with different archatectures. I don't know if 32bit-mode Opteron is enough to cause issues with Althon. Since Mosix is running in kernel this can be more of a issue...).

I suppose one way to find out is to setup a test cluster and mix in slow computers with fast computers and see how well everything works.

 

[wacki]

cool, thx for the info

Does anyone know of a more specialized list serve or forum I might be able to get more detailed information at?

Or, since we are on the topic, does anyone have any recommendations on places to go to buy Opteron nodes?

 

[drag]

try here for beowolf

There are some links to mailing lists there.
I am sure that they would have a bit more accurate information then what I can provide.
(wish I could of been more helpful)

Also if your interested here is the home page for openmosix

 

[wacki]

thank you so much for your help

 

[Smilin]

Drag rocks!

 

[drag]

Originally posted by: Smilin
Drag rocks!

No I don't.

I was wrong about quite a few little details.

That's why whenever you get advice you always do your own research and find real experts.

 

[wacki]

Originally posted by: drag

Originally posted by: Smilin
Drag rocks!

No I don't.

I was wrong about quite a few little details.

That's why whenever you get advice you always do your own research and find real experts.

You were the only one I've talked to that could provide any answers at all. You pointed me in a new direction that I hadn't considered before, and you gave me many stepping stones I didn't have before. Not only did you help me, but in an indirect way you helped a project that will save millions of lives a year. I currently work for the Global Infectious Diseases program at the MBL. IMO, some will disagree, we are currently wasting millions of dollars on overpriced machinery. e.g. $250,000 on a single quad 64-bit processor server. Alpha's are the best computers out there speed wise, but not worth the $$$ IMO. If I can figure out how to mix Opterons and Athlons in a cluster format, and make it user friendly enough for a Biologist/Chemist to use, it will allow us to save boatloads of cash and allow use to divert it to wetlab research. Plus it will allow us to use a more homogeneous OS environment. Right now we have around 36 different O.S.'s, most of which are flavors of UNIX/LINUX/BSD. Scientific computing is a bitch! And all that will save time and even more $$$. And again that extra money will go to more research.

So IMHO you do rock.

Again, thanks for your help.