• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Damn!! Damn!! Damn!! Damn!! I was trying to leave work early and KABOOM!!!

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.
&)(*^#$(&^ Compaq parts depot!!! We've had a guy there waiting on the part and they won't release it because the paperwork hasn't come through yet. Nevermind that I called the problem in on our Compaq Platinum Agreement at 3pm and they have all the info and were supposed to send through the paperwork. Nevermind that the depot was called to confirm they have the part. Nevermind the fact that we pay them tens of thousands of $ a year for the Platinum Agreement. Oh hell no....they can't release it until the paperwork comes through. So....our tech is sitting at the depot twiddling his thumbs and the server is still only marginally operational.
 
Originally posted by: vi_edit
(poor)visual aide

LOL!

I had that happen to my little PC once. I accidentally closed the case after working on it onto a floppy disk power cable, severing it. The second I turned the PC on, boom! Smoke everywhere. I turn it off quickly and open the PC up. I'd say about 1 ft of cable just disintegrated and there was some fire damage inside. Luckily, my PC still worked.
 
Originally posted by: FallenHero
Originally posted by: shinerburke
Originally posted by: vi_edit
(poor)visual aide
You know we actually had some servers smoke once. Christmas Eve last year an asshat security guard went into our Comm Room for some reason. Why? Nobody knows. Well anyway he couldn't figure out how to get back out of the room, doorknobs must confuse him, so he saw a big read button. The big red button. Now this big red button is sealed in a plastic see through box on the wall next to the door. The box has a sign on the front of it that was "Warning: Power Off Entire Room!" The box is also locked. The key is taped on the side of the box, you know the box with the warning sign. Anyway his ass takes the key off, unlocks the box, and then hits the button. Guess what. The entire server farm and all the networking equipment went down. I was at home, relaxing when my pager and cell phone both started having seizures. I answer the phone and it's my boss, who was in the position I have now at the time, in a panic saying "All the servers are down. How fast can you be at the hospital?" I told him it would take me 9 minutes to get there. I get there and the power is still off, luckily. We, another server guy had made it in, are in the process of going around and unplugging all the equipment so that there would not be a surge when the button was pulled back out. We had unplugged about 12-14 servers when the security supervisor shows up and starts asking the guard what happened and why he pushed the button. Well in the process of his explanation the dumbass pulled the button back out which turns the power back on. Oh it was interesting.....bright lights come back on and I think "oh sh1t!" but it was too late....the resulting surge in power smoked several of the power supplies in older servers, a few hard drives, and a motherboard in one of our mail servers. It also took out some network gear.... Needless to say that dumbass guard lost his job and the server team got to spend Christmas Eve in the Comm Room. Our SQL guy, same poor guy that is dealing with it today, had to spend all night and most of Christmas day getting his SQL server back.

Note to self: Stay out of server room

Note to self: Don't push big red buttons
 
Things got a little interesting right at the end of the shift yesterday for me also, but it was a little less high tech.

Me, four supervisors, from area supervisor on down, a welder, a sledge hammer, a 75mm locknut, and a ten foot pipe. All this while I was standing in an area covered in grease.

And all this after being called in from my forced vacation.

 
Originally posted by: KingNothing
For everyone in this thread

New episodes
rolleye.gif
I read through the first few chapters and I didn't laugh once, nor did I even crack a smile. That was the stupidest crap I've ever had the displeasure of reading. Not only was that jackass not funny, but his (obviously) made up stories made him seem like a childish brat.
 
Originally posted by: KnightBreed
Originally posted by: KingNothing
For everyone in this thread

New episodes
rolleye.gif
I read through the first few chapters and I didn't laugh once, nor did I even crack a smile. That was the stupidest crap I've ever had the displeasure of reading. Not only was that jackass not funny, but his (obviously) made up stories made him seem like a childish brat.

BAD GEEK!! BAAAAD GEEK!!! You are no longer worthy to come to Anandtech. BOFH is frickin hilarious!!!
 
Originally posted by: AgaBooga
Wow... sounds like a great way to start a weekend, hehe 😉

Maybe the boss won't be demanding on Monday...
I am the boss. Well....except for upper upper management.....most of the time they aren't seen though.

 
Originally posted by: shinerburke
Originally posted by: RayH
If I recall correctly, in our server room hitting the shiny red button means you have a seconds before you suffocate to death.
Our mainframe room is like that. They haven't switched out the Halon yet.
Yeah, our server room is like that too. Fortunately, as a developer, all that happens when a server blows up is I go home early 😉.
 
Originally posted by: AmigaMan
Originally posted by: KnightBreed
Originally posted by: KingNothing
For everyone in this thread

New episodes
rolleye.gif
I read through the first few chapters and I didn't laugh once, nor did I even crack a smile. That was the stupidest crap I've ever had the displeasure of reading. Not only was that jackass not funny, but his (obviously) made up stories made him seem like a childish brat.

BAD GEEK!! BAAAAD GEEK!!! You are no longer worthy to come to Anandtech. BOFH is frickin hilarious!!!

i have to agree with KnightBreed; those stories are not funny one bit.
 
ok ... i'm gonna be entering into the job market in a few months and stories like this one always freak me out. a few questions:

1. can u screen ur calls to avoid ones like this ... and then say something like "i was out of town"?
2. if so, what are the consequences?
3. do u get paid extra for the crap tht u got go thru ... overtime, 150% ?
4. do u get calls from work while ur on vacation?
5. if u HAVE to cancel ur vacation becuz of a really serious problem, do u get reimbursed?
6. think i should become a professional student?
 
Originally posted by: Vonkhan
ok ... i'm gonna be entering into the job market in a few months and stories like this one always freak me out. a few questions:

1. can u screen ur calls to avoid ones like this ... and then say something like "i was out of town"?
2. if so, what are the consequences?
3. do u get paid extra for the crap tht u got go thru ... overtime, 150% ?
4. do u get calls from work while ur on vacation?
5. if u HAVE to cancel ur vacation becuz of a really serious problem, do u get reimbursed?
6. think i should become a professional student?

I don't work in IT so I can't really help you with any of these questions 🙂
But what I can tell you is that things like this are in no way limited to IT.

I actualy got a (sad) chuckle from reading this thread because I didn't get out of work until a bit past 9pm yesterday (friday) and there was no mishap involved.
 
If you have a PHB who keeps bugging you every ten minutes when something like this happens, you can usually solve it. First thing is to call the PHB, give a complete update, and tell PHB that as soon as you learn anything else you will let PHB know. If an hour goes by, call PHB just to say "nothing new, I'll keep you posted." Do that every hour or so. PHB will quickly tire of this, and will tell you, "No need to call every hour, just keep me updated if something changes."

You have to be the one to initiate the calls to get PHB to quit.


Originally posted by: Vonkhan
ok ... i'm gonna be entering into the job market in a few months and stories like this one always freak me out. a few questions:

1. can u screen ur calls to avoid ones like this ... and then say something like "i was out of town"?
2. if so, what are the consequences?
3. do u get paid extra for the crap tht u got go thru ... overtime, 150% ?
4. do u get calls from work while ur on vacation?
5. if u HAVE to cancel ur vacation becuz of a really serious problem, do u get reimbursed?
6. think i should become a professional student?

Vonkhan, just from reading your questions I suggest you find another career. You are paid to keep the systems running. That means occasional nights/weekends/long hours. It's part of the job, you don't look to avoid the phone calls. You may not get paid extra. You may have to take calls from work while on vacation.

In other words, you could screen your calls, and say "I was out of town". But your boss is going to conclude that your job is to keep stuff running, and you failed to do so. You avoided the phone call, but you are still screwed.
 
We had a large UPS battery fail for one of our phone switches. The engineer went in and flipped the switch to start charging the backup UPS instead of the main one. Later a seperate tech came in to fix it, took the battry model down, and flipped the switch back to the main UPS and went to order the battery replacement, thinking nobody had been there. Left it for the weekend. sunday afternoon the main UPS battery exploaded releasing acid into the entire phone room. It was horrible, and the engineer had to go back in and switch it back off the main system. He was out for a few days with acid burns in his throat.
 
We had a massive failure on a raid array that the mail server was running on at like 7pm the day before I had a final exam at 8am. The rebuild was finished at like 3:30, that kind of sucked but wasn't horrible.

A UPS overloaded one day and started smoking considerably instead of switching off too. APC still has yet to impress me, powerware please. Mmmmmmm ozone....
 
some good stories here 🙂

the worst thing that happened to me was those damn IBM 75GXP drives, I had 3 out of 4 fail on a friday. thank God for raid1 and backups.

and was does this crap always happen on the weekend?
 
Originally posted by: shinerburke
Hardware....one of the array controllers failed. Luckily it's an old Proliant 7000 with redundant array controllers so we have it back up....just slow....I have one of my guys working it, but since I'm the Server Engineering Manager I have to stick around until the part is replaced.....why? So I can rport to upper management when they call every 10 minutes wanting an update. Here's an update for ya....quit calling me and making me call my people....when that happens it just slows them down.

Well, then take your balls into your own hands and tell them that! Every 10 minutes is out of line.
 
Originally posted by: shinerburke
&)(*^#$(&^ Compaq parts depot!!! We've had a guy there waiting on the part and they won't release it because the paperwork hasn't come through yet. Nevermind that I called the problem in on our Compaq Platinum Agreement at 3pm and they have all the info and were supposed to send through the paperwork. Nevermind that the depot was called to confirm they have the part. Nevermind the fact that we pay them tens of thousands of $ a year for the Platinum Agreement. Oh hell no....they can't release it until the paperwork comes through. So....our tech is sitting at the depot twiddling his thumbs and the server is still only marginally operational.

Someone that we contract with to provide a service to us does something like that we charge them by the 10th of an hour for any lost time we have as a result. Last i knew the charge was something in the neghborhood of 5 grand every 6 minutes. Once charged a freight company $26,000 because their idiot driver failed to set his brakes, chock his wheels and level his trailer in the dock. The dock has an automatic loader that loads seat sets for buicks in sequence. The system stages up a full load of seats in sequence then loads them in the trailer. When the loader tried to run the load into the trailer the first row of seats going in got caught on the row dividers because the trailer wasn't level. Normally they just wouldn't go in and the operator would stop the loader until the truck driver corrected the problem but this particular driver had the tractor in neutral no brake set and no wheels chocked so before the operator could stop the loader it pushed the truck right off the dock and proceeded to spill the seat sets into the lot outside. Lost 7 complete Buick Park Avenue seat sets into the lot. We charged the freight company the cost of every set damaged + the cost again to build replacements for them.
 
Back
Top