Solved! SOLVED: Diagnosing Hard Shutdowns: Help Me Narrow the Root Cause?

[jkteddy77]

Issue: For ~2 months I've been experiencing Hard Shutdowns. PC shuts down, and oddly I have to actually unplug and re-plug the power cord before pressing the power button to start up again.

It seems to occur the longer it's running.
First Crash happens in 2-4 hours. Then if I keep pushing it another could happen within 10 minutes.

Machine Information:
Build Parts List: https://pcpartpicker.com/user/jkteddy77/saved/NxGxbv
Latest Bios Installed
Latest CPU Chipset and GPU Drivers
Tried Factory Default Bios Settings

Symptom:
At one point, One of my 4 fans died (replaced), and it was shutting down the MOMENT I opened a game and hardware usage increased.
This leads to me believe Temperature Overheating could be the root cause.

However, I used HWInfo64 to record the entire system's sensors the moment it crashed while gaming, a perfect snapshot to debug with.
BUT no part seems to be thermal throttling. Shown are the highest temperature sensors on each part.



Clues:

1. The 750w Platinum PSU is 6 years old, purchased in Late 2019. I don't have any sensor readout for it, but it was never hot to the touch.
   • Could it be suffering Voltage Droop on a rail?
   • I have to actually unplug and re-plug the power cord before pressing the power button to start up again.
2. The Crucial T705 PCIe 5.0 drive runs notoriously hot, and is unstable when it is. Its thermal limit is apparently 80C however
   • But never exceeded 55C
3. Motherboard is very saturated with heat. I'm intaking air from the top to give the CPU coolest air first, but its heat then flows directly across the MOBO and NVME Drives.
   • No sensors went over 65C, even Chipset the hottest.

My Next Steps:

1. I am replacing the PSU with a SFX 1000W Platinum
   • I'm upgrading to the 5090 at the most unstable time, but this necessary upgrade to accommodate could rule out the PSU.
2. Reversing the Fan Airflow. Bringing Fresh air from the bottom and exhausting through the CPU Radiator could prevent the Passive heatsinks in the system from being heat-saturated, and I can manage a few more degrees on the CPU.

SOLUTION:
Issue solved following Next Steps 1 & 2. It was either the CPU's heat being pumped into the PSU intake overheating or failing it, or CPU heat over the motherboard sinks.

Any suggestions are greatly appreciated​

Thanks for your interest and time.​

 

[mikeymikec]

1) According to the official specs, that PSU isn't enough for that graphics card (PowerColor suggests a 900W PSU minimum). That aside, a 750W PSU with enough miles on the clock to constitute say half its life will have dropped in capability (which is normal for a PSU) so it probably can't serve up 750W any more anyway. Another thing to bear in mind is that modern GPUs recently (around 2020-2023?) went through a phase of drawing a tonne of extra power for an instant which was causing seemingly sufficient (according to the figures) PSUs to fall over and crash the system. While I think that problem has been brought to heel since (because it was getting out of hand), it wouldn't surprise me if these micro spikes still occur even on a lesser scale. Furthermore, I bought a 750W PSU for my 2023 X3D system because I changed to a PSU for a GPU that had a minimum recommendation of a 650W PSU (which I already had), yet 'only' had 1x8pin and 1x6pin power needs, compared to your card's 3x8pin.

PCIe slot provides a max of 75W power to a graphics card
6-pin PCIe connector provides max 75W
8-pin PCIe connector provides max 150W

Logically your graphics card needs a max of 525W, your CPU's max is something in the region of 150W, your board, RAM and other components are going to use something. You always want to plan power needs with some spare capacity in mind.

2) You're using a top vent as an air intake source? Heat rises. The top vent should always be to exhaust air. You need to encourage an air flow through the case, e.g. from front out the back, the hot air that escapes via the top vent can handle itself. Front fans should pull air in, and rear fans (if any) should be sucking air out.

3) Try a CPU-only stress test (e.g. Prime95) for half an hour and see how it fares, at least to narrow the list of suspects.

My bet is that the PSU isn't beefy enough any more.

 

[jkteddy77]

1. The end of the graph is the wattage at time of crash, but could be it didn't catch a microspike. It only occurring over time and occurring frequently more doesn't line up with randomness however. 750w plat was pushing it.
2. It's a Small Form Factor case called the Sliger SM580. It only has a Top and Bottom pairs of 140's. Reviewers at the time oddly benchmarked that a top down flow kept both the CPU and GPU cooler in such a tight space, but those tests weren't over hours where I am sure saturation is occurring. I will flip the flow but the CPU could gain 4-5 degrees from the GPU is why it was this way in the first place
3. I will do a longer prime95 to rule out the OC. Oddly I would reset BIOS bone-stock and it would crash again in 5 minutes when run too long, that is why I suppose it's heat rather than a voltage spike or unstable OC.

To add fuel to the fire... I got a 5090 this week. So it's being replaced with a 1000w platinum regardless to rule the PSU.
I appreciate your interest and input

 

[mikeymikec]

I'm probably not going to be much more help here because I've never tried to pack what I regard to be high-end hardware into such a small container, and I would only attempt to do so purely out of morbid curiosity and an entirely disposable hardware budget.

An image from the case maker's site, I assume this is how things are approximately looking for you:

IMO (and I have never tried this before), I would try getting the air to flow from bottom to top, so intake at the bottom and exhaust at the top. It seems ludicrous to fight basic physics and make hot air go downwards.

One simple suggestion is to see if it handles the situation better with the side panel(s) off.

 

[In2Photos]

My guess is that some component in the PSU is getting heat soaked and causing the issue. It takes a while to get there, but once it does it keeps causing the shutdown.

IMO I would replace the PSU first and wait to put in the 5090. Make sure you have the problem solved before putting another new device into the mix.

I think I would also try using all 4 fans as intake. Fresh air for both the CPU and GPU. There's enough perforations at the back top of the case for exhausting the incoming air.

EDIT: Just saw the pics of the build and it looks like the GPU side of the case is vented, not glass. Not sure about the best airflow knowing that information now. Seems pretty open, almost like having no side panel.

 

[Hans Gruber]

I would suggest going into your bios and restoring default settings on your motherboard. Let it run for 2 or 3 days and see if that solves your crashing problem.

 

[jkteddy77]

never tried to pack what I regard to be high-end hardware into such a small container, and I would only attempt to do so purely out of morbid curiosity and an entirely disposable hardware budget.

You perfectly sum up what drives the Small Form builder haha. 280mm rad and 4x 140's are a large precaution, but in this case the air volume itself may be limiting me, just don't understand why it didn't the first 8 months of this upgrade.

I have removed the sidepanels after the first crash, but it doesn't help at that point. I'll try removing them at startup before heat build-up and see if it's a determining factor.

 

[JackMDS]

i am not sure that a 355Watts Graphic card feels ""'comfortable"" in an ITX case.

 

[jkteddy77]

I purchased the 5090 Founders from another ITX PC owner 😅
Nvidia put a lot of engineering into making a card that would fit, never asked enough if they should. But the puzzle of the strongest possible PC in 14.8 liters is too enticing to solve.
The CPU and GPU temps are very happy with their heatsinks, it's the PSU MOBO SSD that are suffering right now

 

[JackMDS]

It is the General Heat in such small environment that can Not be tolerated.

 

[C1]

It is possible that due to heat (effectively cooking over time) a component such as a capacitor somewhere (eg, on the MB) has been damaged. (This idea arises because of the statement about having to unplug the PSU before being able to actually restart the system,)

Whenever something weird like this happens & heat is suspected, I go to the local grocery & get a block of dry ice & break it into pieces to be used for diagnosis. For example, in your case, tilt the system at an angle & put chuncks of dry ice on the memory cards and on the MB components. Use large chunks & position to make as much physical contact as possible to components plus take advantage of the cold evaporative air flow off the dry ice to keep select components really cold then try the system to see if anything changes.

If anything does, then you can mess with experimenting with the dry ice in attempt to isolate more specifically which section of the system is involved.

Ya, I know it's weird, but you could be screwing with this system for many hours/days trying to determine the cause plus spend a bunch of money investing swapping of parts and dry ice is inexpensive (at least realtively).

Anyways just my two cents.


Good luck

 

[jkteddy77]

UPDATE:
Swapping the PSU to a Lian-Li SFX 1000w Platinum and reversing the fan airflow to intake from the bottom, exhaust from the top has solved the crash.

My machine shouldn't be possible with a 5090 in a 14 Liter case. All of the parts are cool and happy and I couldn't be more proud of it.