Electrical problem: something is frying my components!

[OmNominous]

Normally I'd supply a TL;DR for a post this long but I think most of the details are important...

I thought about putting this in Computer Help, but since this is definitely a hardware problem I thought this forum would be the best place for it (I see a few other help oriented requests).

I ordered the Oculus Rift DK2 in May and bought a used video card (MSI R9 290 with a water block already installed) and everything else I needed to build a complete water cooling loop. (This wouldn't be my first water cooling build, but it had been a long time.) I also bought a beefy new power supply (EVGA 1200 P2) because my previous one wouldn't be able to support an R9 290 at full load plus anything else. (I went with a 1200 watt supply in case SLI performance with the Rift was improved in the coming months. And it has a 10 year warranty!)

I set up the cooling loop and let it run for a number of hours to ensure there weren't any leaks, then fired up the system. Everything was fine for two weeks. I played with the Rift, I did some gaming on my tv, I ran the system at high GPU load for hours at a time with no problems whatsoever.

Then last Saturday morning I wake up, have a 20 minute Skype call, then fire up Planetary Annihilation and after a half hour or so the system spontaneously powers off (as opposed to shutting down). I go look at the case and the power LED is blinking and the system doesn't respond to the reset or power switches (even holding it down).

So I shut off the switch on the power supply, wait a bit, power it back on then turn on the computer. It runs for about 3 seconds then turns off with the same blinking power LED.

I shut off the power supply and disconnect everything except the cooling pump and I plug the main power connector into the testing harness that came with my power supply (it just connects one of the black wires to the green wire), turn the power supply on and the pump starts working (and there are no apparent leaks).

I gradually add more devices until I have everything except the motherboard and the video card and everything is still working. I remove the testing harness and hook the main power connector to the motherboard (and the extra 4 pin cable) and connect a monitor to the motherboard's integrated video. This also works. I connect a low-end video card (which requires no additional PCI-E power connections) and this also works.

I connect the R9 290, power the system on, and a few seconds later there's a pop, a FLAME and of course some magic smoke. I replace it with my previous video card (an 8800 GT) and that works fine. I conclude that the used R9 290 must have been on its last legs.

For the last week my system has been exactly as it was for the last two years parts-wise except with a new power supply and a water cooling loop chilling the CPU, and it's been running fine (though now my Rift is pretty useless).

So a couple of hours ago I step away from my computer but I know I'll be back in a little while so instead of shutting it down I fire up the distributed.net client on all 4 cores of my Phenom and walk away. While I'm working on the task away from the computer I notice it turn off... and the power LED is blinking. Oh boy.

Here's where things really start getting interesting.

So the first thing I do is power cycle the power supply - but as soon as I flip the switch on the power supply back on my UPS starts squealing (as it does when the battery is being severly taxed) and everything connected to the UPS loses power. My UPS has not caused me any problems up to this point. I replaced the battery last winter and it's been working fine.

I hit the power button on the UPS (the brand is APC) which stopped the squealing then turned it back on after a few minutes (with the computer's power supply OFF) and all was normal.

First thing I did was pull my old video card, since I didn't want to fry it... (At this point I'm starting to wonder if that R9 290 was actually perfectly healthy before I installed it.) I disconnect just the motherboard power cables and hook up the testing harness, activate the power supply and the pump, fans, drives turn on and the UPS is silent. So far, so good.

So I deactivate the power supply, hook up the motherboard's power connectors, activate the power supply (no squealing from the UPS), hit the power button on the front of the case, and the fans come on for a fraction of a second and the system abruptly powers off.

Being resourceful, I grab my previous power supply (which was fine other than being rated for just 430 watts), and use its 28 + 4 pin motherboard cables to power the motherboard while the new power supply powers everything else (with the testing harness connected).

I power up everything except the motherboard, then I press the power button on the front of the case and I don't remember if I heard a POST beep or not, but what I do remember is several seconds later noise and SMOKE coming out of my trusty old 430 watt power supply.

ARG! I mashed the UPS power button which cut power to everything, opened the room's window and set up some fans to clear the presumably toxic smoke out of the house.

Thankfully I have more than one computer (though THIS machine isn't going to be doing any 3d gaming anytime soon).

Thoughts?

I feel confident that there isn't a fault in the new power supply, since *IT* wasn't smoking or causing other components to smoke in the events of this evening.

Since my old power supply had been working flawlessly for the past two years the fact that it suddenly gave up the ghost after being hooked up to a motherboard that the new power supply was unhappy about would seem to indicate there's either something about the motherboard causing an inappropriate power draw... or perhaps there's something wrong with my UPS and it's delivering more power than it should?

I guess the most troublesome aspect of this scenario is that the system ran fine for two weeks with the R9 290, and then ran fine for another week with the 8800 GT. Is a coolant leak the most likely explanation? How would a coolant leak cause the power supply (in this case, a power supply which wasn't even inside the case) to smoke? I don't know much about failure modes when it comes to electrical shorts...

Or is possible the UPS (another component in both events) "surged"?

For all I know my motherboard and 8800 GT are still OK, but I'm reluctant to try booting them without having some clue as to what is causing the problem.

I do have a "kill-a-watt" so I can verify the voltage the UPS is putting out...

What can I do to diagnose the issue? What are the likely culprits?

 

[Burpo]

Likely culprits may be a bit of coolant in a PCI slot or worse, processor socket. Seems you have a short somewhere.

 

[mfenn]

I agree with Burpo, you have a short somewhere. My bet is that your R9 290 flame out caused some collateral damage to the motherboard that wasn't immediate obvious (either mechanical like a somewhat conductive piece of debris or purely electrical). Your old PSU might not have had as good of an OCP circuit as ther new one, and so also flamed out when trying to power a shorted connection (evidenced by the UPS complaining about high draw).

Given that your core system seems like it's fairly old (a Phenom), I wouldn't spend too much money trying to 100% confirm a mobo issue, when it seems quite likely that there is one.

 

[OmNominous]

Thanks for the insightful replies - I was definitely planning on getting a shiny new DDR4 motherboard (probably paired with an Intel CPU) but I was hoping to wait until the consumer version of the Rift was available (so that my dollars would go farther, since that's probably a year away)...

In any event, I'll definitely be draining my loop and setting it up outside of the case so that I can run it for longer under closer inspection for any minuscule leaks. If I can convince myself that it's not leaking then maybe I'll install it on an older system I can afford to lose and run it there for a month or so (and live with the noise of OEM coolers on any new components I buy in the meantime).

But it could well be that it was just some fault in the R9 290 and it's flame out deposited something or fried something on the motherboard that wasn't immediately obvious.

Thanks for the help!

 

[PliotronX]

IMO nothin like fried hardware to hasten the upgrade timetable. You know you want 64Gigs of DDR4 :sneaky:

 

[inachu]

Need to know a few things.

Does the Oculus Rift DK have its own power supply or does it leach off of the PC power supply? Would be nice to see a picture of your pc innards.

I still suspect that power supply. I would get a voltage meter for it.

Put your nose close to the power supply when it is in operation.
Does the passing air smell like an electrical burn?
At this point I would open it up and take a visual to look for any signs of burns.
That is if you do not mind voiding its warranty otherwise just RMA it back to the company who makes it.

If your situation happened to me I'd be putting that mother board under bright lights and looking for other stuff like bad resistors and blown caps.

 

[inachu]

Likely culprits may be a bit of coolant in a PCI slot or worse, processor socket. Seems you have a short somewhere.

As far as I know the majority of all collant for computers is totally neutral and should not be shorting anything out even when the system is on.

Those coolants are made so you can even bathe the motherboard 100% in it and nothing would happen.

But yes I would investigate also what brand coolant was used and so forth.

 

[Gtokie]

This post is what made me Leary of going liquid cooked for this very reason of having a leak. Being somewhat of a noob my education on the matter is limited. I'm in the process of doing my first build and paranoid about everything. After reading this I went and make extra sure my case had stand offs pre installed because I never got any. I have a vengeance c70 so I think I'm good or so I hope. Don't want to see any sparks. Eek

 

[mfenn]

As far as I know the majority of all collant for computers is totally neutral and should not be shorting anything out even when the system is on.

Those coolants are made so you can even bathe the motherboard 100% in it and nothing would happen.

But yes I would investigate also what brand coolant was used and so forth.

As soon as you dissolve any sort of salt into the coolant, it will become conductive. There's plenty of residues in a typical not-100% clean computer case which will fit the bill.