Patience (Oct 16 2007)
Turns out the air conditioner coolant was actually down to near 50% full. After the tech filled it to normal levels this morning the temperatures immediately dropped about 5 degrees Celsius all over the closet. Sweet. They'll check again for leaks in the coming days.
The Tuesday outage for database backup/compression went just fine, except we wanted to take this opportunity to get a couple more Sun 220s shut down and removed from the closet, as well as get Eric's hydrogren database server ewen railed up and moved elsewhere in the racks (to improve its air flow). Well, none of that happened - once again despite having actual rails made for ewen they wouldn't fit in any of our non-standard racks in any configuration. Lots of heavy lifting, bolting/unbolting, cabling/decabling, and nothing to show for it. Very frustrating. And due to routing/apache configuration issues galore we ultimately couldn't shut down our old public web servers. In fact, we had to move klaatu out of the way for what we thought was going to be a successful ewen relocation, which meant turning penguin back on and making *that* a public web server. And then I realized there were libs that only existed on klaatu's disks, so I had to recompile php/apache on kosh/penguin to remove that dependency. All these efforts, and we're basically where we were yesterday afternoon. Except the air conditioner is working for realsies.
Maybe sometime this week I'll get back to what I was working on before all this nonsense. Hmm... What was I working on?
#____Total Work Done____Todays WD_______AWD________overtake________Team-nameHot Heat (Oct 15 2007)
So the past two days we were fighting with what to do about sudden rising temperatures in our server closet. This sort of thing happens every year around this time, as the regular lab air conditioner which "assists" our closet by keeping things extra cool in the sunny summer obviously doesn't do the same as we enter foggy fall. We also have some nagging tiny imperceptible coolant leak so we need to recharge that every so often. In any case, the systems were getting hotter, so we ultimately had to shut everything down (the idle disks and CPUs generate far less heat).
This morning the right people were called to inspect the situation. Turns out our air conditioner was more or less okay (we'll add more coolant soon) but the lab air conditioning system did konk out over the weekend. Apparently the lack of assist - even the slight amount during this wet weekend - pushed us over the edge.
Before we figured this all out we had a meeting and planned on several courses of action to remove as many aging, less efficient systems from the closet. I planned to get three systems out by the end of the day (download server, and the two public web servers) but due to annoying little nested problems I've been only able to get the download functionality out of the closet so far. Downloads are currently being served from host vader. I'll shut off penguin shortly - it's not so much a crisis now but we've been meaning to get off those Sun 220s for years.
04______111.572.078_______33.855_______56.162______impossible______BroadbandReports.com Team Starfire
06_______93.550.903_______15.808_______66.162______impossible______Czech National Team
08_______70.872.999_______41.810______127.763______impossible______The Knights Who Say Ni!
09_______43.622.822______-11.481______-18.694______2.334 days______OcUK - Overclockers UK
10_______38.472.912_______19.208_______39.727______impossible______Team Art Bell
14_______28.931.557________6.417_______34.813______impossible______The Planetary Society
16_______17.048.830______-13.930______-16.568______1.029 days______Ars Technica
17________4.079.514______-11.345______-26.507________154 days______Universe Examiners
21_______-3.553.488_______12.599_______46.484_________76 days______Team China
22_______-5.186.242_______-4.591_______-1.719______impossible______Team Starfire World BOINC
25______-11.316.924_______-5.249______-17.760______impossible______Dutch Power Cows
26______-11.366.662______-12.808______-23.198______impossible______Amateur Radio Operators
27______-12.209.322_______-1.032______-40.748______impossible______PC Perspective Killer Frogs
32______-19.134.373______-10.977______-10.692______impossible______UK BOINC Team
33______-19.575.928______-14.975______-17.454______impossible______BOINC SETI@home RUSSIA
40______-24.957.320_______28.137_______54.230________460 days______AUSTRIA - NATIONAL - TEAM
46______-29.434.017______-24.376______-55.122______impossible______HispaSeti & BOINC
48______-31.259.968______-24.636______-51.770______impossible______SETI Sverige [Sweden]
Appart for Anandtech's stats, it shows how much more/less than Anandtech.
Also shows based on Average Work Done how many days for Anandtech to overtake the team, or be overtaken by a team behind