Win a 48-core Magny Cours system

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

aigomorla

CPU, Cases&Cooling Mod PC Gaming Mod Elite Member
Super Moderator
Sep 28, 2005
21,131
3,667
126
LOL..

if i say i would drop the biggest OMGWTFBBQ custom water setup in which the world has never seen..

would that give me a +1? :p

Watercooling a 4P system using high performance waterparts, i think would be the epitome of watercooling.

I think i could even get some pretty nice sponsorships on a system like that...
Starting from Koolance... and finishing on some online vendor sponsorships even.

Ive always wanted to water a system that massive, its just the pricetag on the core hardware which i couldnt coop with.
 
Last edited:

cbn

Lifer
Mar 27, 2009
12,968
221
106
LOL..

if i say i would drop the biggest OMGWTFBBQ custom water setup in which the world has never seen..

would that give me a +1? :p

Watercooling a 4P system using high performance waterparts, i think would be the epitome of watercooling.

I think i could even get some pretty nice sponsorships on a system like that...
Starting from Koolance... and finishing on some online vendor sponsorships even.

Ive always wanted to water a system that massive, its just the pricetag on the core hardware which i couldnt coop with.

Seeing the power consumption differences between air and water would be worth it in my opinion.
 

aigomorla

CPU, Cases&Cooling Mod PC Gaming Mod Elite Member
Super Moderator
Sep 28, 2005
21,131
3,667
126
Seeing the power consumption differences between air and water would be worth it in my opinion.

lol... to be honest the machine belongs in a folders hands.

And not a one that struggling.. but a well established folder/cruncher that would never let this machine have a down time.

In my hands it would be a Toy... and not a machine... well i could have it replace my current gainestown, but lind is doing a good job at what she is doing.

In all likelyhood i probably wont enter on that reason... being it deserves to be in a folder/crunchers hands...

:p

But i would love to watercool one from head to toe, and polish it up very pretty.

I am pretty confident, by the time i was done with my custom builders, and meeting with vendors... i would have one hell of a package built.
Being serious about this statement... It would most definitely be something that the world has never seen...

:D
 
Last edited:

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
27,406
16,255
136
lol... to be honest the machine belongs in a folders hands.

And not a one that struggling.. but a well established folder/cruncher that would never let this machine have a down time.

In my hands it would be a Toy... and not a machine... well i could have it replace my current gainestown, but lind is doing a good job at what she is doing.

In all likelyhood i probably wont enter on that reason... being it deserves to be in a folder/crunchers hands...

:p

But i would love to watercool one from head to toe, and polish it up very pretty.

I am pretty confident, by the time i was done with my custom builders, and meeting with vendors... i would have one hell of a package built.
Being serious about this statement... It would most definitely be something that the world has never seen...

:D

Well, I hope that means that I have a good shot. If I get it, maybe you can help me cool it !
 

JFAMD

Senior member
May 16, 2009
565
0
0
They actually run pretty cool. We did a power demo on analyst's day last week (I did a blog on it, link in my sig) and we had an istanbul next to a magny cours. Generally the same power at each utilization level (idle, 20% for regular load, 60% for virtualization and 100% for HPC). Actually at idle it was lower than istanbul, but nobody is going to want that many cores to be sitting idle.
 

aigomorla

CPU, Cases&Cooling Mod PC Gaming Mod Elite Member
Super Moderator
Sep 28, 2005
21,131
3,667
126
LOL JF we watercool not only for performance but for aesthetic reasons as well.

To some of us, its no longer just a machine.. but a work of art. :)
 

aigomorla

CPU, Cases&Cooling Mod PC Gaming Mod Elite Member
Super Moderator
Sep 28, 2005
21,131
3,667
126
you contradict yourself.

no... submersing something in mineral oil so it gets everywhere, and later becomes a UBER PITFA to clean is totally different from removing a few blocks.
 

cbn

Lifer
Mar 27, 2009
12,968
221
106
LOL JF we watercool not only for performance but for aesthetic reasons as well.

To some of us, its no longer just a machine.. but a work of art. :)

http://www.pcworld.com/businesscent..._highend_servers_returns_to_watercooling.html

According to this watercooling is also beneficial for data centers when the actual energy supply to the building becomes a limiting factor.

Aigomorla, I think you need to team up Markfw900 and build something sleek and exotic. Then measure the before and after power savings. It would be an awesome project.
 
Last edited:

JFAMD

Senior member
May 16, 2009
565
0
0
Water cooling could be huge for data centers, but it needs to be standardized. Until that happens people are not going to jump in. When the back of a server has a water input and output that are standardized like a USB or ethernet port, it will have a chance.
 

cbn

Lifer
Mar 27, 2009
12,968
221
106
Water cooling could be huge for data centers, but it needs to be standardized. Until that happens people are not going to jump in. When the back of a server has a water input and output that are standardized like a USB or ethernet port, it will have a chance.

We also have to consider what type of Carbon Emissions laws are being passed in California (and other places)

It might be finding the most efficient use of the "heat waste" helps reduce some of these costs you are talking about.

So maybe Markfw900 and Aigomorla can team up and use both the crunching power (for folding) and the "heat waste" for something else (or more realistically think about what to do with the heat). Then you would have two productive uses for the electricity entering the system.

The IBM article mentions using the hot water for building heating systems. But what to do with those heating systems? How much total heat is produced from cooling efforts in the winter vs the summer? Is there a way of using that heat to drive another process that actually reduces CO2 emissions to a point that is below zero. (ie, carbon consuming rather than carbon producing). Or is there an even more productive use for the heat that makes more sense than reducing CO2? I think evolving carbon credit laws will be the driving force in this type of scenario.
 
Last edited:

Rubycon

Madame President
Aug 10, 2005
17,768
485
126
No overclocking?
Server fans. That's what is used. No boy racer stuff in those boxes. What will be expensive is feeding them with data. Lots of it and with minimal latency. A duplexed pair of SAS hosts with no less than four to six SAS 6Gbps SLC drives in RAID0 will work nice. 4.8GB/S with <200µS response time is pure bliss! :awe:
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
27,406
16,255
136
No overclocking?
Server fans. That's what is used. No boy racer stuff in those boxes. What will be expensive is feeding them with data. Lots of it and with minimal latency. A duplexed pair of SAS hosts with no less than four to six SAS 6Gbps SLC drives in RAID0 will work nice. 4.8GB/S with <200µS response time is pure bliss! :awe:
I already feed my 12 boxes with 48 cores just fine (F@H). It really depends on what you want to feed it, pure crunching stuff, or database stuff ( an Oracle database, I support a 30 tb farm powered by a IBM P590 with 16 processors and one tb memory, different class of performance).
 

Dadofamunky

Platinum Member
Jan 4, 2005
2,184
0
0
LOL JF we watercool not only for performance but for aesthetic reasons as well.

To some of us, its no longer just a machine.. but a work of art. :)

I know, it's gonna be the Watts Towers of watercooling.

John Brunner, in his book Stand On Zanzibar, envisioned a supercomputer that was entirely watercooled. Funny how that's never occurred to my knowledge but serious hobbyists routinely do so.

Water cooling could be huge for data centers, but it needs to be standardized. Until that happens people are not going to jump in. When the back of a server has a water input and output that are standardized like a USB or ethernet port, it will have a chance.

LOL. I can't imagine any corporate datacenter that would even contemplate such a move. But if companies like IBM and HP standardized it, as you say, it could have a chance. But who would want to watercool a 48-core box like this when there's no chance of OCing it? I wouldn't see any reason to do it, anyway. Does that Tyan board support it? I would doubt it. I'd compare it to the Supermicro boards, which of course don't allow tweaking.
 
Last edited:

Rubycon

Madame President
Aug 10, 2005
17,768
485
126
I already feed my 12 boxes with 48 cores just fine (F@H). It really depends on what you want to feed it, pure crunching stuff, or database stuff ( an Oracle database, I support a 30 tb farm powered by a IBM P590 with 16 processors and one tb memory, different class of performance).

F@H isn't i/o bound by any means. It's like running P95/SETI/RC5 (remember that?) ;)

I'm talking doing real work here like RTE of streams for real time motion capture.
 

cbn

Lifer
Mar 27, 2009
12,968
221
106
LOL. I can't imagine any corporate datacenter that would even contemplate such a move. But if companies like IBM and HP standardized it, as you say, it could have a chance. But who would want to watercool a 48-core box like this when there's no chance of OCing it?

According to the article I linked, water cooling is allowing an increased density of CPUs relative to rack space. Apparently this is reducing the air conditioning requirements.

Here is an IBM video with more details .

Notice the water cooling channels very close to the chip. (No doubt they already have a good amount of patents filed for this). Still we have to hope their is more room for true and legitimate innovation.
 
Last edited:

JFAMD

Senior member
May 16, 2009
565
0
0
LOL. I can't imagine any corporate datacenter that would even contemplate such a move. But if companies like IBM and HP standardized it, as you say, it could have a chance. But who would want to watercool a 48-core box like this when there's no chance of OCing it? I wouldn't see any reason to do it, anyway. Does that Tyan board support it? I would doubt it. I'd compare it to the Supermicro boards, which of course don't allow tweaking.

Within the datacenter, you could have 2 benefits of water cooling: cost and noise. Water cooling can be more efficient than air cooling for systems, but the challenge lies in the standardization of it. Unfortunately today there is a catch-22. In order to make enough money to make water cooling efficient, someone needs to come up with a new, proprietary method. However, in order for water cooling to take off it needs to be cheap and standardized. And that is why air still cools the data center.

As to overclocking, that is not supported on servers. Overclocking is a consumer activity and enterprise customers do not see the value in it. Performance is much lower down the list behind stability and power consumption. Overclocking will increase something of lower value and negatively impact two things of higher value.