Question The Ultimate Windows Cluster-**** -- A Year's Worth Of Trouble

[BonzaiDuck]

I've had a profile on the forums lately, as if the Duck-Meister had become Chicken Little. No -- not "The Sky is falling!" "Something's wrong with my stellar starship-flagship computer, and I can't figure out what it is!" Everyone has seen my recent threads. They're on PSUs, Motherboards, Memory and STorage. And as I say in all of those threads, "It all began with an accident of stupidity, charging a vaping-pen in my computer's USB port."

Now it turns out that the accident and necessary hardware replacement -- the USB controller on the motherboard had died from the accident -- there may have been a confusion of chronological coincidence. I think there was a feature upgrade for Windows 10 last year, and some may correct me. It just seemed to me once the initial necessary hardware swap was completed two or three months after the accident, I recall there may have been a feature update.

The random shut-downs and restarts were noticeable as a first occurrence around June, maybe a few weeks after the motherboard swap. Over the last year, there have been some 16 of them, with an average interval of 14.5 days. Hard to diagnose something like that! I swapped a spare UPS, thinking that it had been failing, but the old unit was still good. I swapped in a new PSU, but that didn't resolve the problem. I had known-good RAM of the same spec -- again -- NO CIGAR! Finally, I replaced the (new) open-box motherboard with the ASUS RMA replacement. No change; intervals of four to five days. So I have two, good spare motherboards, because last year, I'd purchased another ASUS board with the same chipset -- a more feature-rich workstation model. Finally, we swapped in a twin, known-good graphics card, and replaced a twisted, tortured PSU-to-PCIE cable that connected to the graphics card. No prize for that, either.

What was left? Two simple NVME PCIE interfaces with drives that hadn't missed a lick. One auxiliary SATA-III Marvell controller in an X2 slot -- to which my backup disk for Macrium is connected. Drivers replaced. Everything that could be done to save what is now considered an "old" computer, was done.

So I stumbled across quite a collection of forum posts -- at sites other than Anandtech's -- all dealing with "Windows 10: Random shutdowns and restarts". My system would either shut down, requiring removal of the AC power cable from the UPS so that I could boot it, or it would restart and return to my Windows log-in screen.

Take a look at this, from Microsoft Community.

There are many other forums and threads turning up the same problem around the internet, several with parallel prescriptions for solving the same problem.

There is not one, single piece of hardware in my system excluding the CPU which hasn't been replaced to locate a hardware problem -- Oh, the two NVME cards and the Marvell controller -- both working just fine. The CPU will go day and night through iterative cycles of Intel CPU Diagnostic Tool. "Pass, Pass, Pass . . . Pass Pass Pass, (etc.) [next iteration] Pass Pass Pass Pass . . . . " The CPU is a year old. The way Silicon Lottery took pains to de-lid and re-lid -- no -- nothing wrong with it. Otherwise? A CPU that throws a random error every two weeks? That's hard to believe.

MORAL OF THE STORY.
Some symptoms seem catastrophic. Maybe they seem that way after an accident that actually had a catastrophic effect on a motherboard quickly gone South. So we naturally assume "Hardware! Hardware! Hardware!"

I've been fiddling around with PCs since 1983. I haven't bought an OEM computer since 1994. As much extra money, time, trouble, sweat and tears as it takes to execute a deliberate plan to build a perfect system, it seems worth it to me.

So again -- the MORAL OF THE STORY. IT may SEEM like hardware failure, but do two things right away: Run an elevated command prompt to execute "sfc /scannow". OK! I had some minor HDD corruption, which Windows fixed. I ran this procedure two more times, with nothing to report in the CBS.log file. A-number one. Stellar.

In the 1990s, the Sleep and Hibernate features were already reliable if you took the time to make them work. What we have now is a set of power or sleep states that are moving toward a full dozen. Should I have problems with a seven-year-old processor and a system built five years ago with Windows 10? I think not. But -- likely -- I do. I'm about ready to prepare for an "Installation Upgrade", with good backups of the configuration causing the trouble, stellar Macrium and Windows server backups waiting in the wings in case it flounders. With a creation of my latest Macrium rescue disk (USB flash), I'm going to do this with great deliberation and care. But I've been exercising deliberation and care all along, and I treated this system with deliberation and care, but for the vaping pen thing. But that problem was solved in May, 2021. This? Likely something else, and something else that seems to be too common.

Windows . . .

The hits just keep on comin'.

 

[VirtualLarry]

I tried telling you, plug the PC into mains power and not that nasty non-sinewave UPS.

 

[BonzaiDuck]

I tried telling you, plug the PC into mains power and not that nasty non-sinewave UPS.

But that nasty non-sinewave UPS is working fine on two machines -- until March, three. All with the "voltage protection" turned on.

But really. Tell me you're kidding about this. One PC every two years or so for decades -- now every five years. Do I have knowledge? Only some. Expertise? Not enough. Experience? Relative to some -- occasional.

But this problem began with one APC UPS, to which the system had been connected without fail for four years. Four . . . Years! Actually -- Four Plus. I replaced it only this year, after some 13 of the subject power events. The batteries were still good, although I'd initially thought they were failing. I've seen the system freeze because the UPS with old batteries wasn't doing its job. That was one case of "experience".

But -- no -- you don't have to tell me your kidding, but I think you must be, old friend.

There's nothing of hardware left to suspect. It's something related to sleep-states or power-states and Windows. But we'll see! The fallback -- the last resort -- is a Win 10 "Installation-Upgrade". After that?

An Alder Lake with a Z690 board? Win 10 was working with this gen6/7 system all along, since early 2017. It should be working now without fail.

Here's a question, though. The 7th gen processor list at Intel shows "Launched, Launched Launched" for all models giving the year. The 7700K is shown as "Discontinued". What was that all about?

 

[VirtualLarry]

But really. Tell me you're kidding about this. One PC every two years or so for decades -- now every five years. Do I have knowledge? Only some. Expertise? Not enough. Experience? Relative to some -- occasional.

I'm telling you that I've had similar problems, random restarts caused by "power gliches", that went away, once I bypassed the APC UPS's battery backup plugs.

I've had outright bad / flaky APC UPSes as well.

I've been a "UPS Believer" for literally going on 20+ years now. Probably longer. But I'm mostly now soured on the idea, as long as the power isn't bad in your area, given the amount of problems that buggy, flaky, and non-sine UPSes have given me. IMHO, the actual quality of "consumer" (under $500) UPS units, has gone into the toilet the last few years. (Yes, even APC sine units, even.)

Much like some of my experiences with anti-virus software; you might just be better off without your "security blanket".

At least, test your PC plugged into the "surge only" outlets for a month.

 

[BonzaiDuck]

I can do that, but I'm going to wait and see about either the power-state tweaks in Windows or an installation-upgrade. Given the circumstances I've tracked with first one UPS and then another, unless it's "something about Kaby Lake" as opposed to the SKylake that was in the socket for four years, I'm more inclined to at least focus on the software/OS issues at this moment versus going back to hardware.

It seems counterintuitive to go directly to the wall socket! I remember in 1982, I was fiddling with a cheap Sinclair Z80 system, connected to the wall socket. Every now and then, the refrigerator would cut on. I don't know why I got started with . . . that toy, but things were much better with a barrel-plug surge protector and my first IBM-compatible PC. Then, there was that storm in Virginia with green-lightning I already mentioned. I lost hardware! I became convinced about something more than a surge protector . . .

We're going to have power outages here this summer . . . They'll probably have them all across the continent, so -- you also.

Anyway -- this is the "CPU" forum, and was wondering about the Kaby Lake I put in this system, so I did some quick searches. Nothing bad about it, but it just didn't seem to offer that much more than the previous. It was good for over-clocking! It ran hot! But I took care of that, and sent it to Katy, Texas.

I just don't want to be interrupted with my work here, because of another power-event. And a person gets attached to a system that was better than perfect for four years. As I said in another thread here, I've got my eyes on something better, but I have too many diapers to change now.

 

[VirtualLarry]

You're using a PSU that is at least "Haswell compatible" aren't you? Otherwise disable C6 State in BIOS.

 

[BonzaiDuck]

You're using a PSU that is at least "Haswell compatible" aren't you? Otherwise disable C6 State in BIOS.

I think that was it! Thank you! I HOPE that was it! But it would have to be "Haswell compatible" if it were a Seasonic purchased within the last four years, and this one is essentially new. I'll have to ask my friend in Albuquerque if he's upgraded from his Haswell, since you mentioned that processor..

The C-state disable was a recommendation in a Skylake OC'ing guide. It was a good guide, but I can't find it -- always possible that I saved it to disk somewhere, but I'd have to look -- haven't found the time. The other wisdoms, such as the ASUS ROG, haven't mentioned it.

Whole point here, though, is that -- sure -- I may dial up the voltage to 1.29 and the frequency to 4.8, but I don't think it makes a hilla beans difference -- the 300 Mhz. I have a lot less time to run stress tests -- even OCCT -- now.

I was running HW-Info64 -- the monitoring software after the Windows power-tweaks I've made. Processor seems to behave properly. settles down to a low idle frequency, and only bounces up once in a while unless it's being used. So the tweaks salvage the EIST performance. I thought it odd that someone would recommend setting the power switch to simply "turn off the monitor". Sounds flakey. But the across-the-board advice seems consistent.

I shouldn't feel too confident about this -- I need to wait patiently and see. And I'm still going to refresh the entire windows install. I've got good backups to restore if something isn't right about it.

I think I'll start looking around for last year's processor and Z690 board. I wonder how much the DDR5 is going to cost. Those are the only items I should have to pay for. It's going to take me a while. So hopefully, this "old system" can be returned to the perfection it had before January a year ago.

 

[VirtualLarry]

Whole point here, though, is that -- sure -- I may dial up the voltage to 1.29 and the frequency to 4.8, but I don't think it makes a hilla beans difference -- the 300 Mhz. I have a lot less time to run stress tests -- even OCCT -- now.

So, your:
1) complaining about crashes,
2) while overclocking,
3) on an older system,
4) and "don't have time" to run appropriate stress-tested?

You know, your balls don't shrink, just because you're running your PC at stock...

Edit: So, my "professional suggestion" is, if you're running an older rig overclocked, and have done so for some time, consider running it at "Intel stock". Which may still require setting manual things, and not leaving mobo on "AUTO".

I don't know how you feel about AMD, but I honestly think that you would get a kick out of AM4 / Zen3 (NOT "3D"), and some of the advanced stuff, like Ryzen Master, and Curve Optimizer, and 1mem.

 

[VirtualLarry]

I think I'll start looking around for last year's processor and Z690 board. I wonder how much the DDR5 is going to cost. Those are the only items I should have to pay for. It's going to take me a while. So hopefully, this "old system" can be returned to the perfection it had before January a year ago.

Before you do that, look up and watch JayzTwoCents video on YT where he talks about problems, BSODs, on 12th-gen. Which I myself have also experienced to a limited extent, on even a non-OCable CPU, a G7400 Pentium Gold, using a Z690 Asus board with DDR4-3200.

Edit: If I were to drop $330 on a 32GB kit of DDR5-6000, I'd wait for a 16C/32T Zen4 CPU to pair it with.

 

[Insert_Nickname]

Much like some of my experiences with anti-virus software; you might just be better off without your "security blanket".

...and may even add to your attack surface. Some attackers are specifically targeting badly written security software. Even well written security software. Also, some AV is literally adware/scareware/malware (or indeed all three) by itself these days.

 

[BonzaiDuck]

So, your:
1) complaining about crashes,
2) while overclocking,
3) on an older system,
4) and "don't have time" to run appropriate stress-tested?

You know, your balls don't shrink, just because you're running your PC at stock...

Edit: So, my "professional suggestion" is, if you're running an older rig overclocked, and have done so for some time, consider running it at "Intel stock". Which may still require setting manual things, and not leaving mobo on "AUTO".

I don't know how you feel about AMD, but I honestly think that you would get a kick out of AM4 / Zen3 (NOT "3D"), and some of the advanced stuff, like Ryzen Master, and Curve Optimizer, and 1mem.

Let me be blunt. The clock settings have been set to stock since last fall, when I began to see the power-event time-series. I already said that I'd concluded that the OC setting hardly seems to make a difference. As for balls, I'm 74. I'm ready to announce Willie's funeral. I'm hiring a midget carpenter to build a 6" coffin, so we can have the "viewing" before the funeral.

For your very last thread mentioning the YouTube video and an outlay that could be over $1,000, I'm not going there too soon if I can help it. My deferral of new technology is grounded in my caution about just the sort of thing you suggest. I have to do the reading, the lab test reviews, the "spec" inspection. Some people with day jobs and other distractions probably do their new-PC builds much faster than I. That's why I might get a processor released two years prior -- still hot for enthusiasts -- rather than simply go ape-shit over a processor on the market for only a couple months.

I DO have one more minor hardware swap -- the main cabling from the PSU to the 24-pin (?) motherboard socket -- near the RAM. I'm then going to go forward with the "installation-upgrade" approach to refreshing the Win 10 OS. As I do this, I have the old Skylake processor, the workstation board, plenty of RAM(!), the identical computer case (CM Stacker 832 (!), and my formula case-mods are essentially finished with hindsight refinement. The WS board, processor, RAM and TR heatsink are ready to just "drop in".

So eventually one of these boxes will get the recent-gen CPU/board/RAM upgrade. Isn't the Z690 primarily made for DDR4? See -- I haven't even had time to investigate those questions. Yeah -- just checked -- DDR4.

After I'm sure that this isn't a "flakey cable", I'm going to do the upgrade-refresh after making a new Macrium rescue disk.

The only other minor irritant, assuming I defer the $1,000+ in outlays for Z690, is the OS license I'll have to buy for the "twin" machine coming together on my patio table. Today, I'm going to take a little time to harvest the Swiss Chard in the patio garden. A little olive oil, some sweet peppers and onions -- better than well-seasoned spinach for Sunday dinner . . .

There's a combination for you. Vegetable gardening and computer-building.

Here's another question. I bought the Seasonic Titanium 650W PSU for this system in late 2016 -- ten year warranty. I still have it. Is there ANY reason why a Seasonic 750W Titanium purchased last year would not play well with this system? I couldn't imagine that. And like I said, my power events began before the PSU swap.

Just as afterthought, these power-event shutdowns still leave the LED lights on the mobo chipset aglow, as they would be just for the PSU connection and trouble-free operation with the system normally shut down. No flashing lights on the mobo to indicate anything, but I've already swapped in two boards in succession There shouldn't be a problem with any of that stuff -- like "Mem-OK" etc.

My "reputation" has suffered as a result of this ordeal. People who DON'T build their systems from scratch look at my 30-year-long strategy with a jaundiced eye.

 

[VirtualLarry]

Let me be blunt. The clock settings have been set to stock since last fall

My bad, I apologize for being super-critical.

 

[BonzaiDuck]

My bad, I apologize for being super-critical.

I always understood the drill. One doesn't want unstable overclock settings to lurk as a possible factor of causation or multiple causation. I think with the SKY and KABY CPUs, you could achieve highly stable OCs with voltages that are lower by some margin than those reported for "auto" settings. But I"m not playing around with this.

OC"d or not, it's a computer that I was regularly using for more than just gaming.

As you had urged, however, I'm going back into BIOS so I can check and remove some "Auto" settings that were never part of any mobo profiles I'd created at any time for this chipset. And I always create a profile for the stock settings, but items are changed from "Auto" in some cases. With two motherboard swaps in this situation, I think I let ASUS have its way too much in setting some items to "Auto".

I've run Belarc Advisor on this system last week. Every license key, every piece of software is listed in the report. And you know as well as I that over time, you get stuff installed on your system for which you're not aware, or never had the need. You have a stability-stellar system, comfortable in using it, and we may think nothing of installing trial versions, testing programs, and other schlock.

 

[BonzaiDuck]

Oh, wow. When I run a web-search, I often leave the listed results open as an EDGE tab. I happened to open a link to a Ten Forum thread -- again -- the same problem. And again -- the twisted cable issue that I suspected, but for the guy with the problem, it was a stinking SATA cable. Moreover, he'd used a cable that had previously been installed for a different PSU.

Now -- these Seasonic Titaniums all seem to have the same cables. But since this started after the first motherboard swap, I think I'm going to replace every cable with those that came with last fall's PSU swap.

So apparently, the symptoms intersect and overlap troubles people occasionally have with modular PSUs. Just flakey connections; too much resistance or something.

 

[BonzaiDuck]

So -- I summon the ghost of VirtualLarry! Lar-ry! Even if connecting the system to the wall-socket was the wrong direction to go with this, it should've sparked a light-bulb in my head.

This WAS about "plut-in" connections, alright. But the source of trouble was "plug-in connections" and wiring between the PSU and the components.

I should've checked those connections at the beginning, when I first noticed these troubles in December.

As I was explaining to Igor -- I really feel a bit stupid. Call it "decrepitude".

 

[VirtualLarry]

So -- I summon the ghost of VirtualLarry! Lar-ry! Even if connecting the system to the wall-socket was the wrong direction to go with this, it should've sparked a light-bulb in my head.

@VirtualLarry would have worked just as well...

Anyways, I hope that has tracked down your issues.

I wouldn't feel stupid in the least; that's about as obscure a reason that I can fathom for crashes, PSU modular connections.

 

[BonzaiDuck]

@VirtualLarry would have worked just as well...

Anyways, I hope that has tracked down your issues.

I wouldn't feel stupid in the least; that's about as obscure a reason that I can fathom for crashes, PSU modular connections.

More or less -- I agree -- obscure. But with modular PSUs, even if "top-end", you have double the number of contact points to potentially cause trouble.

Here are my recommendations to anyone building their own system with modular PSUs.

First -- your "build" should be planned well enough that you can add SATA and PCIE cables to accommodate only the most likely additions you would make later. But that's why they make modular PSUs -- helping you avoid extra clutter in the case.

Second -- if you can help it, avoid situations in which you remove the plug from the PSU and then replace it. And if you replace it, make sure its latch is locked in place.

Third -- avoid twisting and bending your cables into pretzels.

Fourth -- don't be lazy. If you replace the PSU -- replace all the cables. If you need to add or remove cables from the PSU (especially if you add them) -- remove the PSU from the case if the case and PSU location require you to use an LED, a mirror and baby hands.

Fifth -- If you see something that looks like "catastrophic hardware failure" and you know you have all fresh or unabused hardware, check the freaking cables! If you're going to swap out RAM, PSU (especially), motherboard (especially) -- check the freaking cables!

AN EPILOGUE
My friend down the hill -- former high-school classmate -- had been an electronics technician, but he's a Mainstreamer. Savvy Mainstreamers find corporate refurbs from reputable sales-points -- even Xeons -- fixed and prepared with Windows. Others go to COSTCO or some place where OEM systems are sold.

So my friend tells me "None of us Mainstreamers have the troubles you have!" [Referring, of course, to my long-standing strategy of building systems and avoiding OEMs.] I have to grab him by the shoulders and give him a shake!

This all started with the sort of thing that Mainstreamers could do to their OEM computers. Charging a dumb device in a USB port, like a vaping-pen. Or forgetting to remove a non-bootable software install disc from the optical drive. It intrinsically had nothing to do with a strategy of building my computers. I wouldn't have need to replace the motherboard. And then, for the disc in the optical drive that I'd forgotten, I wouldn't have chosen right away to swap in a new PSU.

For all of those things, I was getting careless about cables. Of course, the Mainstreamers would send their warrantied systems in for repair, and the repair facilities -- you would HOPE! -- would attend to the required care before sending it back.

I remember my sister-in-law's computer -- a Gateway OEM -- was acting flakey. I was the one who was troubleshooting, and I'd never opened the box before. What did I find? A loose screw rolling around on the motherboard! So there is risk -- even under warranty.

 

[VirtualLarry]

Fourth -- don't be lazy. If you replace the PSU -- replace all the cables. If you need to add or remove cables from the PSU (especially if you add them) -- remove the PSU from the case if the case and PSU location require you to use an LED, a mirror and baby hands.

Fifth -- If you see something that looks like "catastrophic hardware failure" and you know you have all fresh or unabused hardware, check the freaking cables! If you're going to swap out RAM, PSU (especially), motherboard (especially) -- check the freaking cables!

Unless you are swapping exactly like for like when replacing a modular PSU, you MUST replace all of your current cables with the new modular cables. At worst, if you do not, you risk a FIRE! Yep, seen it with my own eyes, on a friend's rig.

 

[BonzaiDuck]

Unless you are swapping exactly like for like when replacing a modular PSU, you MUST replace all of your current cables with the new modular cables. At worst, if you do not, you risk a FIRE! Yep, seen it with my own eyes, on a friend's rig.

That goes without saying -- certainly if old and new modular PSUs are of different manufacture or model. I can defend my laziness with the observation that all the recent model Seasonic Titaniums ship with an identical cable-set. But if you are going to replace the PSU, it is laziness if you fail to use the new cables anyway.

 

[JoeRambo]

Unless you are swapping exactly like for like when replacing a modular PSU, you MUST replace all of your current cables with the new modular cables. At worst, if you do not, you risk a FIRE! Yep, seen it with my own eyes, on a friend's rig.

+1. Just don't. I have seen results of such "swap". It was between Seasonic and i think CoolerMaster, but just don't even if it is the same vendor. Only exact same model is safe.

 

[thecoolnessrune]

That goes without saying -- certainly if old and new modular PSUs are of different manufacture or model. I can defend my laziness with the observation that all the recent model Seasonic Titaniums ship with an identical cable-set. But if you are going to replace the PSU, it is laziness if you fail to use the new cables anyway.

Yeah you would think it goes without saying, but add me to the list of people that have seen this happen first hand by acquaintances. In all 3 cases I know, it was fueled by an "eventual" upgrade from a 1xxx series card to a 3080 / 3090, and given the shortage, the people were pre-emptively replacing their power supply with a larger unit while they waited to be able to get a card. In all 3 of those cases, the folks installed a new power supply, but left the cabling in place because they didn't want to re-do their cable organization and the "plugs look identical". 3 people who lost their whole system in the hope of saving 15 minutes (granted, more for at least 1 of them who was meticulous with his cable management).

In a system full of standardized connectors, it sucks that modular power supplies hold out as the proverbial upside down floppy connector, waiting to catch someone who doesn't know any better.

 

[DaaQ]

Yeah you would think it goes without saying, but add me to the list of people that have seen this happen first hand by acquaintances. In all 3 cases I know, it was fueled by an "eventual" upgrade from a 1xxx series card to a 3080 / 3090, and given the shortage, the people were pre-emptively replacing their power supply with a larger unit while they waited to be able to get a card. In all 3 of those cases, the folks installed a new power supply, but left the cabling in place because they didn't want to re-do their cable organization and the "plugs look identical". 3 people who lost their whole system in the hope of saving 15 minutes (granted, more for at least 1 of them who was meticulous with his cable management).

In a system full of standardized connectors, it sucks that modular power supplies hold out as the proverbial upside down floppy connector, waiting to catch someone who doesn't know any better.

They are only standardized on one side of the modular connectors.

 

[BonzaiDuck]

They are only standardized on one side of the modular connectors.

I'll say it again -- I was buying Seasonic Titaniums, only differentiated by wattage (650 and 750). they ship with the same cable set. OK -- that may seem reassuring. But I mangled those cables trying to keep them "out of the way". New power supply? Take the g*****n time to replace old cables with new! Am I kicking other people? I'm kicking myself!

 

[Insert_Nickname]

They are only standardized on one side of the modular connectors.

Which is kind of odd isn't it. I mean everything else is standardised inside the case, except the modular connectors.

(Yes, yes. We all know why of course. Still doesn't make a whole lot of sense. I mean, if PSU manufacturers could skip providing included cabling, they'd probably do it in a heartbeat. Oh, </deity>. Better stop here before someone get good ideas)

 

[BonzaiDuck]

Well, to repeat myself, I've left a trail of panicky threads in a few forums for a problem I created -- likely for just swapping in a new PSU in November. Probably a total of 17 power-events over that time, but they were occurring almost once daily after I did the second motherboard swapout, also an indicator that some cables were failing.

So -- here it is! I now officially declare my old desktop is back and in good shape! I've put it through the sleep and hibernate ringer; it's been running non-stop now for a week, no strange behavior -- just a standup rig.

I am DEFINITELY catching up on piles of paperwork now. I just hate playing catch-up, with all the other ongoing burdens.