I run 4 nodes with each being a Q6600 / 8GB RAM over GbEthernet, and have a few dual core and couple of single core nodes additional to those. They're not dedicated to cluster use since sometimes they're used for sandboxes, development, individual applications, et. al.
Yes, it is only as power intensive as 4 computers, but most computers aren't quad core, and they mostly run at around 10% or less computational load. Since you're talking about power being dominated by the actual computations, the energy consumed rises pretty significantly as you start to increase the computational work load to consume well over 70% of the total CPU power on the units. I believe 200W-350W is pretty average for a Q6600 system with a typical motherboard, hard disk, efficient power supply, a few fans, and some modest video card. I can dig up specific benchmarks of actual numbers measured from the UPS or from the wall socket, but they're usually consistently in that range unless very idle or equipped with very high power GPU card(s).
Still 250W x 4 = 1kW = 30 days / month * 24 hours / day * 1kW = 720kWh / month just for the PCs running 24/7. That's probably at least a little more than the cost of your entire household's current usage for an average US household.
There are some good cost estimators of the cost of running computers 24/7 under heavy load done by some of the distributed computing participants like people running Folding@Home.
Here's an actual AC power draw benchmark; Q6600 whole system = 177W IDLE, 230W LOAD; keeping in mind that your 'LOAD' may be more intensive than theirs if your software makes more efficient use of your hardware and you may be overclocking which will add power consumption proportionally:
http://techgage.com/article/in...2_quad_q9450_266ghz/12
Anyway figure your specific cost given your utility rates per kWh in the 1MWh/month to 2MWh/month range and you'll know the direct electricity cost. Figure added costs for cooling / ventillation and you'll have more realistic numbers for operational costs.
1kW of heating is around 3400 BTU / hour added heat, so figure that amount on top of whatever AC size would normally be needed to keep that room size / type below 80F on a hot day to get your minimum AC size; usually that ends up being a minimum 5000BTU unit for a small closed room + 3400 = 8400 ... round up and that's 10k BTU, more like 12k or 15k if the duty factor will be large and you've got hot environments / rooms even before the PCs are added and you're contemplating possible equipment expansions.
Heat is basically watts for watts so if you assume that 100% of your power draw from the wall will be dissipated as heat (a good assumption), then you're looking at around 1kW or so of heat load in the room for 4x250W units, or something approximating a modest space heater or hair dryer. Not HUGE, but hardly insignificant. It is usually 10F hotter in the computer room here than in the adjacent room and that's usually with the doors / windows open and some fans going etc. AC or strong ventillation is just a necessity on hot days one for staying comfortable and two for keeping the systems cool enough that the heat doesn't give them problems (typically you really don't want to exceed a CPU LOAD temperature of 65C or so, and often you choose your overcllock to keep it near that level on "normal" temperature days).
Your cluster could use less power than 150-250W / node if your nodes aren't very busy computationally or if they're using extremely efficient CPUs; the Yorkfield is more power efficient at low loads / idle than the Kentsfield by far, for instance, then again why build a cluster if you're going to keep it at low loads a lot of the time?
A single high end GPU would use something like 150W all by itself, with some being well higher than that, so basically add another 300W or so to the total power budget for a pair of those in each system case if you're going GPGPU.
Iterative calculations may be able to be made more tolerant of SEU (single event upset .. random glitches changing a bit value) errors since they can be programmed to include some kinds of sanity checking on each iteration as to the reasonableness or consistency of the calculations. Of course it helps if the problem is well conditioned such that small errors don't magnify over time to yield large catastrophic ones. You'd have to check with your code providers to see what their experiences have been with error rates and handling on individual workstations and clusters, keeping in mind that most academic / commercial clusters will already be using ECC corrected RAM & hardware from the start, so they'll see dramatically less errors than you might with consumer commodity hardware. Maybe the Pande Group (Folding@Home) can suggest helpful resources for estimating your error rates given commodity PC clustering.
Wow! Thank you for such a detailed response! The thing I don't get though is that this should only be as hot and power intensive as 4 computers, right? Or are those quad cores much more power hungry than most other computers? I guess that I never even realized how much power computers use compared to other things in the house. This means that a typical computer is consuming 25% of the electricity in the house or is it different because it is running all the time? I have never even realized how much a single desktop heats up the room. Does this really becomes essential to consider at 4 processors or a huge cluster? How large was the one that you made? Now that I look into it, the plans that I read about the 24 core cluster only used 400 W when running. Why wouldn't my potential cluster use around ~260 W then? If that's the case, it should only run as hot as 2.5 light bulbs, right?
I definitely need to do some research about where exactly the bottleneck is though. The programs I'd want to run do density functional theory calculations, namely VASP and SIESTA. I've noticed significant speed ups when using more nodes on the cluster I use now, but I only have a qualitative sense of it. I can't even thank you for this response though, there are so many things, like error correction, that I take for granted in normal computer operations, that I didn't think about how your setup would vary if you can't accept any. I would say that my calculations would take a few days to a week max. Its largely would do iterations trying to find the lowest energy state of a system.
Facebook
Twitter