• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Hospital had generator and UPS failure

Page 5 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.
Ns1 said:
Out of curiosity, how expensive is a project like this? 7 figures, 8?
Click to expand...

I'm led to believe it's in the low-mid 7 figures range.

2 New transformers, new generator, construction work for new substation/generator, architects fees, electrical design and consultancy fees
New switchgear, redesign and rewiring of an large building
Dual redundant UPSs - these alone are nearly $1million.
Excavation of roads for new HV cable installation.
Contribution to the construction of the new power company substation (split between a bunch of other customers who also need improved redundancy or more capacity).
 
The worst I have personally seen is a failed UPS causing a momentary loss of power when the generator is powered on, then again to cut back over to utility power.

The worst I have heard second hand (from one of the causes) is for the oxygen and backup oxygen to run either side by side, or through the same line from the sub building, and it (or both if they were side by side) being drilled through due to bad records.
 
Mark R said:
I'm led to believe it's in the low-mid 7 figures range.

2 New transformers, new generator, construction work for new substation/generator, architects fees, electrical design and consultancy fees
New switchgear, redesign and rewiring of an large building
Dual redundant UPSs - these alone are nearly $1million.
Excavation of roads for new HV cable installation.
Contribution to the construction of the new power company substation (split between a bunch of other customers who also need improved redundancy or more capacity).
Click to expand...

Yes, a megawatt (ok, a megavoltamp) is a ton of power. The UPSs are going to be nutty expensive.
 
Wow a megawatt is incredible. To put it into perspective I work at a telco CO and there is tons of equipment in here and it would probably compare to a mid size data centre. The building pulls about 400 amps per phase. I'm not sure if that's at 208v or at 120 though... not familiar with 3 phase systems. At the rectifiers we're pulling 1600 amps or so at 54v. This is probably half the load of the building considering the AC units, lights, and other stuff of that nature are not running off DC or inverter power. Basically, we're pulling maybe a couple 100 kw at most. I think our two diesels are 160kw if I recall. The two are in a small building similar to a shipping container.

Speaking of AC, they're putting in two large 30 ton units they ran a new electrical panel that is taller than I am... scratch that... TWO new panels. Though the reasoning is redundancy. We basically have two switch gears so one from each splitter.

The project is in the millions I believe, it's been going for a bit over a month now. It's actually quite incredible to see big projects like this go into place. For some time it felt like we had an Ebola outbreak or something because of all the plastic and zippers we had to go through. But that was required to keep the construction dust from getting to the equipment.
 
I would believe 8 figures. If I had to guess, without seeing the full scope of the project, that it probably comes up to around 10-30 million. Because it's a hospital, take what you would normally guess such project to cost and times it by about 5.
 
The one hospital I primarily work in has redundant generators where both are needed to fully power everything that is wanted up in the case of a power outage, but if one fails the other can be used for essential equipment. We also have the ability to call in portable generators that can be up and running in hours so I'm confused at what happened here. Granted we're in a very large city and there are no doubt more services available to us, still I assume there must be some sort of built-in infrastructure maintenance has to allow getting the big portable units that arrive on a trailer up and running so quickly. I've been on shift when this happened before and the portable unit was called in and up and running before I left.

Someone(s) have some accountability in this situation and/or this hospital has some deficient protocols.
 
The problem was that the hospital management had changed several times.

The building had been built in the late 80s, and generally hadn't been upgraded - the electrical system was the same as it was when it was built, and the various codes allow things to be "grandfathered in".

The other issue was that the construction contractors look like they took a lot of shortcuts, and underspecified a lot of stuff. One was the generator and power infrastructure.

It's only really been the new management who have decided to perform a complete overhaul of all protocols and infrastructure.
 
Thought I'd post another update.

The contractors had brought a massive load bank on site by the electrical building, and it was running most of the day. Not quite sure what they were testing, because the generators weren't running.

A dude was also going round with a camera, possibly a FLIR camera.

I'm not sure what was happening. But I'm guessing that the new switchboard and/or new transformer has been installed, and is being tested before the main building is transferred over to the new switchboard.

Also, I walked the other way off site, taking the scenic route, and a massive (about 10 foot tall with massive fans) new transformer had appeared behind a small warehouse on the corner of the campus. I'm guessing that this is part of the power company's infrastructure upgrade to support the increased service to the hospital.
 
Last edited:
whats the costs for the upgrades/overhall?

and where did the $ come from?
I thought hospitals are normally cash strapped?
 
Last edited:
Mark R said:
Thought I'd post another update.

The contractors had brought a massive load bank on site by the electrical building, and it was running most of the day. Not quite sure what they were testing, because the generators weren't running.

A dude was also going round with a camera, possibly a FLIR camera.

I'm not sure what was happening. But I'm guessing that the new switchboard and/or new transformer has been installed, and is being tested before the main building is transferred over to the new switchboard.

Also, I walked the other way off site, taking the scenic route, and a massive (about 10 foot tall with massive fans) new transformer had appeared behind a small warehouse on the corner of the campus. I'm guessing that this is part of the power company's infrastructure upgrade to support the increased service to the hospital.
Click to expand...

They put a massive load on the panels and use the FLIR cameras to verify joint and electrical connections. While this is happening they will have all the panels open to make sure that someone didn't forget to tighten a lug to spec and cause melted / flaming cables.
 
Thought I'd post an update. The work is now complete.

Everything has been fully tested, dual-fed, both generators and UPSs tested, including a full "blackout test", which included a UPS shutdown, so that the new SOPs for total electrical failure could be tested.

No load banks were installed, but grid paralleling switch gear was installed on the generators, so that for testing purposes, they can synchronise with the grid, and be tested at whatever load is desired without the need for either a load bank, or for a power interruption.

New fuel tanks were installed which hold 200 hours of fuel.

Summary:
Service upgraded from 1 MVA to dual-redundant 2 MVA supplies.
Second supply comes from a new power company substation which is itself supplied from a different bulk supply substation.
Single 500 kVA low power generator supplemented by a new 1.2 MVA generator.
Generators now have paralleling switchgear to allow planned switching between mains and generator without power interruption.
Dual redundant UPSs for most circuits (with the exception of stuff like non-critical HVAC, etc.)
Main switchboards, sub-boards and some final panels upgraded to dual feed to mitigate against cable failure, or failure of a main switchboard.
ICU and ORs upgraded to IPS (isolated power supplies) supplies to mitigate against short-circuit faults to ground (as the supply is isolated, a hot-to-ground fault is not dangerous on its own and will not blow fuses/trip GFCIs/etc. Instead of disconnecting the power a fault will trigger an alarm, allowing the faulty equipment to be taken out of service in a controlled manner).
 
Long story short...

Looks like it cost them a bunch to get up to spec. But, it does appear that the new management is doing a better job than the old.


Brian
 
Mark R said:
Thought I'd post an update. The work is now complete.

Everything has been fully tested, dual-fed, both generators and UPSs tested, including a full "blackout test", which included a UPS shutdown, so that the new SOPs for total electrical failure could be tested.

No load banks were installed, but grid paralleling switch gear was installed on the generators, so that for testing purposes, they can synchronise with the grid, and be tested at whatever load is desired without the need for either a load bank, or for a power interruption.

New fuel tanks were installed which hold 200 hours of fuel.

Summary:
Service upgraded from 1 MVA to dual-redundant 2 MVA supplies.
Second supply comes from a new power company substation which is itself supplied from a different bulk supply substation.
Single 500 kVA low power generator supplemented by a new 1.2 MVA generator.
Generators now have paralleling switchgear to allow planned switching between mains and generator without power interruption.
Dual redundant UPSs for most circuits (with the exception of stuff like non-critical HVAC, etc.)
Main switchboards, sub-boards and some final panels upgraded to dual feed to mitigate against cable failure, or failure of a main switchboard.
ICU and ORs upgraded to IPS (isolated power supplies) supplies to mitigate against short-circuit faults to ground (as the supply is isolated, a hot-to-ground fault is not dangerous on its own and will not blow fuses/trip GFCIs/etc. Instead of disconnecting the power a fault will trigger an alarm, allowing the faulty equipment to be taken out of service in a controlled manner).
Click to expand...

um.. why wasn't this done in the first place for the Hospital?
and how could they afford this upgrade?!

it would have been cheaper to build a new hospital with these specs instead of upgrading the old one.
 
Update: The power comapny's new substation caught fire and shorted out both 33kV main feeders to the campus.

The generators started as expected and there were no major problems, except that despite the power upgrades management insisted on shutting down all medical imaging and other big ticket equipment, as the equipment vendors would not guarantee correct operation on generator supply.
 
You must log in or register to reply here.

TRENDING THREADS

Back
Top