[Feasibility Question] Turning some/most/all of works data center from aircooled computers to water cooled?

[menorton]

Fellow forums members,

I have been recently hired at a good IT/mgmt firm straight out of college. I went to the data center and it is quite big. It is cooled by ubiqitius vents in the ground, but the big guns are the 3 HVACs we have. We have similiar data centers located in various parts of the country, since work has a global network. These HVACs are expensive, both in cost to purchase, and in electicity to run. Their sole purpose being to cool the computers.

But if they computers don't need cooling?

Being a computer enthusiast, i know about the virtues of watercooling in high performance rigs. Does it not make the computers silent and virtually worry free about heat? The computers we have in the data centers are servers from dell, hp, and the like. don't think towers and desktop computers, think storage racks like the ones pictured. I assume we can't do much to the ones we buy straight from dell, sun, hp, but what of the ones we make ourselves? Is there anyway to find out which machines make the most heat?

Basically, i am trying to weigh cost of implmentation against cost savings. I can forsee cost of implementation including: downtime of servers, cost of installation, cost of de-polarized water that wont short out systems in case of a leak, and potetional clean up in case of a spill. Cost savings i see as no longer having to run the 3 huge HVACS, salvage from their selloffs, savings on electricity to run, etc..


I am not having AT forums do my work here, but i would appreciate places to read about water cooling servers. And also just to throw this idea out and see what people think, or if people have come before me and tried this before.

thanks!

Sample pics of some of the servers:

Sun 1
Sun 2
Sun 2
Storage Tek

 

[dBTelos]

Something you might want to try out in a small solution first, like a server or two. Then see if you like the results, if so, then expand to the other servers.

 

[menorton]

Oh obviously. This has got to start small and scale up. But at this stage, i am thinking of numbers. how much does it cost to change a single server (assuming single cpu) to being watercooled? I am thinking of total implementation: cost of materials, maintenance projected. And savings: how much extra heat is dissipated and much power is now needed to run computer?

 

[pkme2]

I have a question about the ambient temperature of the room the servers are in.

AFAIK, I've always thought that servers were usually housed in a air-conditioned room that was temperature-controlled. Anyhow, that how our servers at school are that way.
The servers at our R&T Park are also temperature controlled too.

So adding water cooling to your servers should be interesting endeavor. Hope you achieve your goals. Good working......

 

[menorton]

i want it to be watercooled so that we can make it less of a temperature controlled environment. Remember, more temp. control = more costs.

 

[Seekermeister]

I don't know the first thing about water cooling, so I guess it doesn't really matter...but I would hesitate to water cool my own computer, much less the computers owned by the company for which I work. I'm not saying that the idea is bad, but a leak could do alot of damage to the computer, and even more to your job security. Now that I've said that, forget it and go on with the discussion.

 

[Howard]

If the radiators are in the same room, you'll still spend the same amount of temperature control.

 

[BoberFett]

Water cooling would be the perfect solution for voiding the warranty on all of their mission critical servers.

 

[alpha88]

Both traditionally heatsinks and watercooling both just transfer the heat to the air.

Since watercooling is more effective (since it allows one to use a larger radiator), one could argue that it will heat the room faster. But in any case, you won't be changing the amount of heat produced by the computers that must be pumped out of the room via HVAC.

What you could do is:

a) Consolidate the radiators, such that only they need to be actively cooled by AC
b) Allow the room to be warmer (say 80 instead of 64). The AC will still have to do the same amount of cooling, however, it can cool more efficiently at 80 degrees.

I don't think that the energy savings will be that great.

It will cost a lot to implement (I'd say 200 bucks a computer), and dramatically increases the maintainence and level of risk.

 

[CurseTheSky]

In theory, you could have all of the water-cooled servers in one room, and have flexible, easily detachable pipes (tubes) running from the back of the computer out into another very small, auxillary room. The room could solely contain the radiators for the water cooling systems, and have passive ventillation allowing the heat to escape through many vents in the ceiling / roof. If you want to get even more exotic, the external radiators could be replaced with one central radiator, containing hundreds of different water paths for different computers, which could be set up outside the building itself.

However, it would be very costly and difficult to implement... not to mention the fact that it would add more maintainence and worries to an already problematic situation. It's probably worth mentioning at a meeting as an idea for the future, but I highly doubt any company would be willing to do this on a large scale. It never hurts to experiment, though.

 

[menorton]

oh well. good thing i go to smart tech forums before bringing this up with the upper mgmt.

 

[Rubycon]

You would forget using a radiator on the server altogether. Each server would have its own supply and return line fitting that would connect to a chilled water loop for the building. A mixing valve would maintain the desired temperature set by a qualified engineer.

 

[DaveSimmons]

Air cooling is probably lower maintenance and cheaper per server.

Another approach to cutting power use and heat generation would be to upgrade servers more than a year old to this year's much higher CPU-power-per-watt designs from intel, AMD, Sun, etc..

If you have PC level servers in the room, you might find that you can replace a power-sucking slow 4-processor server with a new single socket dual-core that's just as fast, or use a dual-socket dual-core to replace a small cluster. Blade servers might also make sense.

Even with larger servers they probably have newer, more power efficient designs.

Reduce power enough and you might reduce heat enough to keep one of the HVACs turned off as a spare unit, reducing power use even more.

 

[tcsenter]

The heat from a data center is attributable to combined dissipation of many dozens of computers, not from inordinate dissipation of any individual computer or components. The heat from a high performance desktop is attributable to the later; unusual or inordinate heat dissipation from high-output and high-watt components. Servers generally are already well designed and built to run relatively cool with more efficient components that individually don't require special cooling measures.

Water cooling may go a long way towards countering the heat dissipation of high-output and high-watt components, but its efficacy at cooling more efficient computers that individually don't have a problem with heat dissipation is questionable. Cooling a whole room full of such computers would be even more questionable.

Liquids and a data center just don't mix.

 

[Rubycon]

Originally posted by: tcsenter

Liquids and a data center just don't mix.

Properly engineered systems are no trouble. Most super computers are liquid cooled. Of course we're discussing systems that bear no resemblance to the watercooling kits that are popular with the enthusiast community. Those also have no place in a server room.

 

[tcsenter]

Properly engineered systems are no trouble. Most super computers are liquid cooled.

Engineered from the get-go with a cooling system designed specifically for the hardware and chassis (and vice versa), not retrofitted with a commercially produced 'universal fit' product.

Typically, 'super computers' only crunch numbers, they don't store sensitive or valuable data nor are they up-time/mission critical. A super computer goes down for a couple days, everyone just has to reschedule their projects. A data center goes down for a couple days, people lose big money.