Explain to me the reasoning behind not letting low level I.T. guys reset a server

brainhulk

Diamond Member
Sep 14, 2007
9,376
454
126
Access was down to one of our hospital's web pages. So I naturally called I.T. to stop facebooking and fix it. The weekend guys said they are not allowed to reset those servers. They told me to wake up my department I.T. specialist and see if he can fix it remotely. If he cant fix it, they have to call their supervisor to reset it.

So i wake up my department's I.T. specialist and he says the server is located in the I.T. department and they have to reset it...

Wwwwwhhhhhhhyyyy do you need a supervisor to reset a fucking server. It does not make any fucking sense

Jesus H. Christ

23ea3146ea334d5741c306054889811d.jpg


tumblr_m86bgej5B01qksombo1_400.gif


The-Monkey-2.jpg
 
Last edited:

xSauronx

Lifer
Jul 14, 2000
19,582
4
81
I wouldn't give a low level anyone permission to do that.

really.

honestly, they are probably operations people who are allowed to reset passwords, log tickets, and maybe help you with email. if the server reboot [or just service reset] had any issues i bet they couldnt fix it.

the technical people with knowledge about those systems are more likely on call, and have a range of things that are critical they would *have* to fix on the weekend or at nights, and things they could wait and handle during the week


basically, it just doesnt make sense if you havent worked in a large IT operation with lots of people, equipment, software and services
 

Lean L

Diamond Member
Apr 30, 2009
3,685
0
0
Why are you assuming the answer is to 'reset' a server?

If something goes wrong, you look at the logs and possibly strace the process to figure out what happened. How do we know it's even the server and not a network issue? This isn't a windows desktop. RCA not required for healthcare?
 
Last edited:

brainhulk

Diamond Member
Sep 14, 2007
9,376
454
126
Why are you assuming the answer is to 'reset' a server?

If something goes wrong, you look at the logs and possibly strace the process to figure out what happened. How do we know it's even the server and not a network issue? This isn't a windows desktop. RCA not required for healthcare?

My department's I.T. specialist said on the phone (i called him at his house) it just probably needs a reset
 

Red Squirrel

No Lifer
May 24, 2003
70,282
13,640
126
www.anyf.ca
There is a ridiculous amount of politics especially in hospitals. Its why I left the server scene for NOC. Best move ever even though my original dream was server tech.
 

rudeguy

Lifer
Dec 27, 2001
47,351
14
61
LOL - "just" a server reset. What could possibly go wrong?

:whiste:

:D

Our IT Director took one day off to move. It was the only day where he wasn't available 24/7. We never have major issues and there were 2 of us who are more than capable of fixing almost anything.

So the morning he is moving, the server crashes. No internet, no CRM, no accounting, no network drives, no anything. Me and the other guy tried for 3 hours just to figure out wtf was broken, not even how to fix it.

We never did get it. The IT Director had to come in. I felt totally useless in my big chance to shine.
 

saratoga172

Golden Member
Nov 10, 2009
1,564
1
81
:D



Our IT Director took one day off to move. It was the only day where he wasn't available 24/7. We never have major issues and there were 2 of us who are more than capable of fixing almost anything.



So the morning he is moving, the server crashes. No internet, no CRM, no accounting, no network drives, no anything. Me and the other guy tried for 3 hours just to figure out wtf was broken, not even how to fix it.



We never did get it. The IT Director had to come in. I felt totally useless in my big chance to shine.


Domain controller issue and DNS?

It feels like that where I work now in that when the director takes time off or is out of town something always happens. Power outage, line down, something crazy happens and people are unable to work. Luckily I've been able to solve everything and get things rolling just not quite to his level....yet.
 

rudeguy

Lifer
Dec 27, 2001
47,351
14
61
Domain controller issue and DNS?

It feels like that where I work now in that when the director takes time off or is out of town something always happens. Power outage, line down, something crazy happens and people are unable to work. Luckily I've been able to solve everything and get things rolling just not quite to his level....yet.

DNS was my guess but the error logs didn't show shit.

Turns out it was a bad switch that set off a chain reaction. I didn't feel so bad once I found out what had happened. It was well above my pay grade. It also convinced the IT Director to upgrade everything in the server room. We had been piecing together things for too long and there was no documentation on anything. So if he had died, our whole company would have too. Now everything is shiny, new and up to date. We also have a consultant we can use if SHTF.
 

Leros

Lifer
Jul 11, 2004
21,867
7
81
Because resetting the server during business hours could cause havoc and damage the business.

An IT guy at my office added a misconfigured switch to our network during business hours and took down the office network for a few hours (removing the switch didn't fix the problem it created). I can only imagine the dollar amount of damage that caused. IT doesn't touch the network during business hours anymore.
 
Last edited:

nageov3t

Lifer
Feb 18, 2004
42,808
83
91
I remember back when I worked at a NOC, some guy hard booted a major switch because some random client on the phone told him to.

in his small, small defense, it was his 2nd day on the job and he was fresh out of school... he didn't get in nearly as much trouble as the guy who was supposed to be supervising and training him.
 

rudeguy

Lifer
Dec 27, 2001
47,351
14
61
I remember back when I worked at a NOC, some guy hard booted a major switch because some random client on the phone told him to.

in his small, small defense, it was his 2nd day on the job and he was fresh out of school... he didn't get in nearly as much trouble as the guy who was supposed to be supervising and training him.

We call that "on the job training". Fuck up, learn from it, don't do it again.

Stuff happens and luckily our leadership doesn't freak out. We were trying to integrate Fedex into our inventory and accounting system. It sounded like a great idea and Fedex even came out to install it. Well they managed to take down our Fedex server, so we couldn't ship any product that day. This was on a Wednesday, we ship around $1M on Wednesdays :D
 

Brovane

Diamond Member
Dec 18, 2001
6,252
2,484
136
Access was down to one of our hospital's web pages. So I naturally called I.T. to stop facebooking and fix it. The weekend guys said they are not allowed to reset those servers. They told me to wake up my department I.T. specialist and see if he can fix it remotely. If he cant fix it, they have to call their supervisor to reset it.

So i wake up my department's I.T. specialist and he says the server is located in the I.T. department and they have to reset it...

Wwwwwhhhhhhhyyyy do you need a supervisor to reset a fucking server. It does not make any fucking sense

Normally the guys you call on the phone for T1 support (Help desk) are not going to have the security access to reset servers. It sounds like the Help Desk supervisor is the one with the required access to reset the server. They probably took one of the helpdesk guys that was more technical and gave him server admin access and then slapped a supervisor title on him so they can make him salary and not pay him for OT. If it is like where I work you have IT operations is only responsible for the OS down and above the OS is the responsibility for team that manages the App, in this case a web server. So the help-desk guys are asking you to contact the app-owner and make sure it isn't a issue above the OS before they call the supervisor which is probably just going to restart it. If it is like my work you would have also had to open up a Emergency Change request for the server reboot. :cool:
 

Possessed Freak

Diamond Member
Nov 4, 1999
6,045
1
0
I remember back when I worked at a NOC, some guy hard booted a major switch because some random client on the phone told him to.

in his small, small defense, it was his 2nd day on the job and he was fresh out of school... he didn't get in nearly as much trouble as the guy who was supposed to be supervising and training him.
The *day* I was hired as a glorified A/V guy, the sys admin went on vacation for a week. And it was a 2 man IT dept, which was now just me... talk about trial by fire.
 

GagHalfrunt

Lifer
Apr 19, 2001
25,284
1,997
126
Let the words "low level" be your guide. If they know what they're doing they wouldn't be low level and if they don't know what they're doing they shouldn't be allowed to touch the servers.
 
Feb 6, 2007
16,432
1
81
Responsibility. Maybe it's just a quick reset, in which case a monkey could do it as well as anyone, only faster. But what if something goes wrong? It's the server admin's ass on the line, even if he's not the one flipping the switch. You don't let low-level people have that responsibility until you know you can trust them since their mistake will ultimately come back on you.