VT Supercomputer

[TheLonelyPhoenix]

I'm a student at Virginia Tech, and recently found out about a presentation (which is taking place right in my dorm) on the VT supercomputer. For those of you who don't know about it, the supercomputer here is a cluster of 1100 dual-proc G5s; third fastest computer in the world and by far the cheapest of those three. I'd like suggestions on some in-depth questions I can ask if I get the chance. I'll be sure to come back to post the answers to questions that I get answered. So far, here's what I (and a friend of mine) came up with:

What kind of infrastructure provides power to that many machines?
What software do you use to coordinate/control all the computers?
What are the intended uses of the supercomputer?
What are the advantages/disadvantages to using Mac hardware and architecture?
What would you do differently if you did this project over again?
Can I play UT2003 on it sometime? Please?

Not sure if the computer is open for students to see yet, will check on that. If it is, I'm borrowing one of my roommate's thinkgeek.com t-shirts... this is too good a photo-op to pass up The presentation is today around 8:30 PM, so make suggestions fast!

 

Tell them, "You know, I bet the only thing it's good at is Photoshop effects, since it's a Mac." See what they say.

 

[acemcmac]

Awesome... Good luck :beer:

Wish we had tech people like that even utter the name of our university, let alone attend it :beer:

 

[acemcmac]

Originally posted by: jumpr
Tell them, "You know, I bet the only thing it's good at is Photoshop effects, since it's a Mac." See what they say.

LOL Owned :beer:

Seriously, when you're done your line of questions, ask that!

 

[MichaelD]

1. What kind of backbone links all those computers? (similar to your first question, but I am talking "Gigagibt?" etc)
2. Are all these systems full-time dedicated to the share project, or does it run in the background "as needed" like SETI?
3. Run some common benchmarks on it, like SANDRA CPU test..."PR=98,343" :Q

 

[Goosemaster]

What kind of infrastructure provides power to that many machines?
Prolly its own 150,000 volt set of power lines..nothing special

What software do you use to coordinate/control all the computers?
Tons of stuff. I don't know if it is a beowolf or not....

What are the intended uses of the supercomputer?

Some use it for weather predictions..calculation of massive differential equations....

What are the advantages/disadvantages to using Mac hardware and architecture?

Uniformity.
What would you do differently if you did this project over again?

Prolly. AMD's offering are fantastic, but there isn't really anyone that can produce adaquate machine for such an application in such a massive quantity. IBM is usingAMD now, but my guess is they weren't fast enough and $$.


Can I play UT2003 on it sometime? Please?

Worthless. This type of setup is about compiling daily builds of UT2003, not playing it. Video cards do different things compared to main CPU's.

 

[EvilYoda]

A little out of the loop, eh? Or maybe they've just waited this long to make a presentation on it to the students...they've fully migrated from tower G5s to the XServe racks, right?

 

[Goosemaster]

Originally posted by: MichaelD
1. What kind of backbone links all those computers? (similar to your first question, but I am talking "Gigagibt?" etc)
Q

Depends. These are not web servers, application servers or plain database servers. Aplication servers can sometimes require Giabit and 10G cores. Webservers and databases don't use that much traffic compared to app servers.

Clusters like this prolly transfer intermittantly from what I have read. THey transfer amounts of data to process, and let the cluster do its thing and then request more data when it is done.

 

[Goosemaster]

Originally posted by: EvilYoda
A little out of the loop, eh? Or maybe they've just waited this long to make a presentation on it to the students...they've fully migrated from tower G5s to the XServe racks, right?

I REALLY want to find out more about that. What EXACTLY are they going to do with the 1100 towers? For campus wide distribution? or just a waste of money?

 

[Eug]

Originally posted by: TheLonelyPhoenix
I'm a student at Virginia Tech, and recently found out about a presentation (which is taking place right in my dorm) on the VT supercomputer. For those of you who don't know about it, the supercomputer here is a cluster of 1100 dual-proc G5s; third fastest computer in the world and by far the cheapest of those three. I'd like suggestions on some in-depth questions I can ask if I get the chance. I'll be sure to come back to post the answers to questions that I get answered. So far, here's what I (and a friend of mine) came up with:

What kind of infrastructure provides power to that many machines?
What software do you use to coordinate/control all the computers?
What are the intended uses of the supercomputer?
What are the advantages/disadvantages to using Mac hardware and architecture?
What would you do differently if you did this project over again?
Can I play UT2003 on it sometime? Please?

Not sure if the computer is open for students to see yet, will check on that. If it is, I'm borrowing one of my roommate's thinkgeek.com t-shirts... this is too good a photo-op to pass up The presentation is today around 8:30 PM, so make suggestions fast!

First check out the VT Terascale website, and the Apple video about it.

Ask them:

1) Have they completed their G5 Power Mac --> G5 Xserve migration yet?
2) Do they plan only to keep 1100 dual Xserves, or are they planning to add more now that they've freed up some space?
3) Was ECC support a key reason to migrate to the Xserve?
4) Did they do any real jobs on the Power Mac cluster, or was it only for test purposes?
5) How much is the G5 Power Mac --> G5 Xserve migration costing them?
6) What did they do with all the extra memory, since the Xserve will be running ECC, and the Power Macs did not?
7) Are they using the same Infiniband cards or are they getting new ones?
8) Have all the old Power Macs been sold off?
9) When did the Xserves start arriving?
10) Why are they using 2U spacing for 1U cluster nodes?
11) When is the system supposed to be available for real jobs?
12) The Power Mac cluster was 10.3 - 10.9 Tflops/s. What have they gotten the Xserve cluster up to?

jumpr:
3rd fastest supercomputer in the world my friend.

Michael:
1) Backbone is Infiniband (and Cisco gigabit Ethernet)
2) pass
3) Sandra runs on Windows. The VT computer runs Mac OS X. You can't run Sandra on a cluster anyway.

Goosemaster:
I've seen them advertised at Mac online stores now, as refurb VT supercomputer units. Not very cheap, but I guess they think think they can get a premium for computers part of that cluster.

 

[MichaelD]

Cool. That's some interesting stuff, Eug.

I didn't even know Apple HAD rackservers.

that sure is a lot of raw computing power! :Q 10.9 TFPS is nothing to sneeze at, that's for sure! I'd imagine the "extra memory" was/will be sold off w/the "old equipment." Helluva deal if you can get in on it, I'd wager.

 

[ThaGrandCow]

Hey fellow VT'er! I've been meaning to see if I can get near that computer to take a look since I first heard about it, I just have never looked into if I needed a special pass or if they have a tour or anything.

One thing I'd love to ask them: What was the reason they decided to use 1100 G5's and then barely 2 months later transition everything to the rackmounts (why not just wait for the rackmounts)? How much money did we lose to depreciation when trading the G5's in for the rackmounts?

 

[Eug]

Originally posted by: MichaelD
I didn't even know Apple HAD rackservers.

G5 Xserve

By the way, the G5 Xserve is the first chip to use the 90 nm IBM PowerPC 970FX chip, which won the Microprocessor Report's Analysts' Choice Award for Best Desktop Processor of 2003.

 

[rival]

ask em if you can use it cuz your college tuition helped pay for it

 

[EvilYoda]

yeah, they're selling off the "used" G5 towers for like $200 off of normal price, with some nice upgrades in them, I believe. Slashdot had a link a few days ago

 

[ThaGrandCow]

Originally posted by: rival
ask em if you can use it cuz your college tuition helped pay for it

Actually, if I had to guess I'd say Michael and Marcus paid for a LARGE portion of the computer. Same thing with the medical school we're going to be getting soon.

 

[TheLonelyPhoenix]

No, we're not out of the loop; news of the completion of this project showed up in the college paper, Wired News, and a few other tech sites at about the same time. However, there hasn't been any official presentation about the VT Supercomputer before now. At least, not that I've been aware of.... VT is a big place, its entirely possible I didn't hear about previous ones.

The backbone is Gigabit Ethernet with Cisco switches. Not sure of the architecture details past that.

Goosemaster, its pretty clear you're guessing; I'm looking for questions to get hard answers from the people that worked on this thing. I really doubt they just plugged a 150,000 V power cable into 1100 surge protectors... does it run off the power plant on the north side of campus? what protects it from surges? etc, etc. I know the reason they chose Mac was simple: Intel server chips was too expensive and AMD was relatively slow. However, I'm more curious about the specifics of Mac hardware which affected this project, either positively or negatively, in comparison to Intel/AMD hardware, without regard to price or pure horsepower. And the UT2003 thing was a joke Take it easy.

I've seen both the Terascale website and the Apple movie already, they've been linked to by the Virginia Tech main site before. However, they're a little too-much sales pitch and not enough technical info.

Hey TheGrandCow, nice to meet a fellow computer geek VT'er at 1 in the morning Do ya' live on campus? The presentation is in Pritchard Hall, ya could always stop by.

 

[TheLonelyPhoenix]

Nah, I'd be suprised if a dime of this came from college tuition. Virginia Tech is a huge research school; on par with any of the Ivy Leagues in engineering. Between alumni donations and companies funding research within the school, I'd say this project most likely never touched a tuition dollar. Its no secret some of our grad students are basically paid to come here, between the scholarships and the research projects they are salaried to work on.

 

[Goosemaster]

<---aspiring Vtech or UVA student ..in communicty college right now

 

[TheLonelyPhoenix]

If you're in for engineering, VT is definitely one of the best. Provided you survive Engineering Fundamentals.... *shudder*

 

[Goosemaster]

Originally posted by: TheLonelyPhoenix
If you're in for engineering, VT is definitely one of the best. Provided you survive Engineering Fundamentals.... *shudder*

i know what you mean:|

calc is a bitch:|...and physics for engineers is its mean big brother :|:|

 

[toant103]

Originally posted by: TheLonelyPhoenix
If you're in for engineering, VT is definitely one of the best. Provided you survive Engineering Fundamentals.... *shudder*

like 1015 and 1016?



i think they changed the course # now.

<---- alumni

 

[TheLonelyPhoenix]

Now its EF2984 and EF1016

Last I heard, EF2984 had a 74% 'success rate' - success rate indicating people who didn't drop the course and finished with a C+ or better.

 

[TheLonelyPhoenix]

Oops, I meant C- or better.

 

[Goosemaster]

Originally posted by: TheLonelyPhoenix
Now its EF2984 and EF1016

Last I heard, EF2984 had a 74% 'success rate' - success rate indicating people who didn't drop the course and finished with a C+ or better.

what kind of course is that? I think I misinterpreted what you meant. Are you talking about the FUNDEMENTALS: CHEM, PHYSICS, CALC or a class title "Engineering Fundamentals" ?

If it is a class, please ellaborate on what it involves.