The WinXP Windows Update CPU saturation bug

[Mushkins]

Are you OK with Microsoft sabotaging a product they want you to move off of?

Those are some strong words. Feel free to prove that its not a bug but intentional sabotage. Until you can show us some proof of Microsoft executives twirling their handlebar moustaches, adjusting their monocles, and laughing maniacally about creating a situation where Microsoft Update, a product that has continued to evolve over 10 years intentionally does not support IE6 to push people to more modern operating systems, you're just making things up.

The reality of the situation *is* that Microsoft Update and .Net frameworks have continued to evolve over the past decade, and one of the more recent iterations does not play nice with a base install of Windows XP regardless of service pack. It's not a show-stopping issue for people still on XP, there are workarounds for the one-off time that it happens outside of a corporate setting, and support for this product ends in April. Microsoft support is certainly aware that there is an issue, but they're also aware that it's absolutely not worth their time or effort to dedicate support and development resources on this. It's like putting $3000 tricked out rims on a 1989 Chevy Malibu with a busted transmission that you're about to have towed to the junkyard.

In two weeks it will be 2014, Windows XP was released August 2001. It's over man, let her go.

 

[TerryMathews]

lUntil you can show us some proof of Microsoft executives twirling their handlebar moustaches, adjusting their monocles, and laughing maniacally

The antitrust trial proved that they would in fact stoop to sabotage.

 

[mikeymikec]

The antitrust trial proved that they would in fact stoop to sabotage.

It wasn't really the same, and furthermore (making it difficult for competitors to work with your products, as opposed to sabotaging your own products), I am pretty sure there are still a large number of systems belonging to large organisations (including those belonging to governments) , and if there was proof that MS created this problem intentionally, I think those organisations would consider MS responsible for sabotaging their systems. That's a heck of a lot of enemies to make for the sake of shifting some Windows licenses.

Windows is one of Microsoft's safer products (from the threat of competition), but Microsoft really is an organisation under attack from pretty much every front. Fixing the XP WU bug would be a good thing from a PR perspective, but I think they're arrogant enough to believe that they don't need to bother. Not attaching a high priority from an unintentional bug in an old product is not anywhere near as bad an idea as sabotaging your own product is.

 

[TerryMathews]

It wasn't really the same, and furthermore (making it difficult for competitors to work with your products, as opposed to sabotaging your own products), I am pretty sure there are still a large number of systems belonging to large organisations (including those belonging to governments) , and if there was proof that MS created this problem intentionally, I think those organisations would consider MS responsible for sabotaging their systems. That's a heck of a lot of enemies to make for the sake of shifting some Windows licenses.

Windows is one of Microsoft's safer products (from the threat of competition), but Microsoft really is an organisation under attack from pretty much every front. Fixing the XP WU bug would be a good thing from a PR perspective, but I think they're arrogant enough to believe that they don't need to bother. Not attaching a high priority from an unintentional bug in an old product is not anywhere near as bad an idea as sabotaging your own product is.

As has been pointed out in this thread, large organizations have support contracts and use WSUS.

To be perfectly clear (go back and review my posts if you think ive maintained differently):

-I don't believe they intentionally sabotaged XP
-I believe there is a bugged update out there
-I further believe they are still obligated to fix it, just as if your mechanic broke your water pump during an oil change. He doesn't get to say "Dude, this Shelby Cobra is old not my fault. Go buy a Honda."

This is not the clock race bug from Windows 98. This is not an unforseen bug brought upon by machines faster than were intended to run the OS. This bug appears to affect all types of systems.

An ethical business would want to protect their reputation. Microsoft has proven in the past that they dont care and will in fact work against you to make a buck.

I dont see how an intellectually honest person can disagree with any of these statements.

 

[Mushkins]

As has been pointed out in this thread, large organizations have support contracts and use WSUS.

To be perfectly clear (go back and review my posts if you think ive maintained differently):

-I don't believe they intentionally sabotaged XP
-I believe there is a bugged update out there
-I further believe they are still obligated to fix it, just as if your mechanic broke your water pump during an oil change. He doesn't get to say "Dude, this Shelby Cobra is old not my fault. Go buy a Honda."

This is not the clock race bug from Windows 98. This is not an unforseen bug brought upon by machines faster than were intended to run the OS. This bug appears to affect all types of systems.

An ethical business would want to protect their reputation. Microsoft has proven in the past that they dont care and will in fact work against you to make a buck.

I dont see how an intellectually honest person can disagree with any of these statements.

You're the one who brought up Microsoft sabotaging their own product, and your car analogy falls flat as your mechanic is not the manufacturer of the Shelby Cobra. Auto manufacturers, as well as all other manufacturers very frequently stop making and providing support for your old ass beater. You cant get that muffler fixed because they dont *make* that muffler anymore, it stopped being profitable for them to do so as they haven't sold that model of car for nearly 15 years. Your mechanic that broke it is most likely going to have to go dig through some junkyards to find a replacement in decent condition. Is your mechanic on the hook for fixing his work? Absolutely, just like if your IT vendor rolled out a bad patch in your environment. Is the manufacturer going to go back and fix a design flaw in that muffler they're about to discontinue? Not a chance, its a waste of everyones time and money, and someday you're going to have to buy a new car.

 

[ninaholic37]

Apparently Microsoft knew about these Windows Updates problems since at least 2010:
Updates are taking up a high percentage of the CPU and bogging the computer down

It seems quite possible that they are intentionally exploiting the bug to get people to switch to a newer OS (especially considering the "timing" and how it goes along with all their recent public statements pushing people to get off XP). Many people have no idea what the problem is and think their entire OS or computer is broken so they "upgrade", so their plan seems to be working. It's great that threads like this one exist so that more people (who are not technical wizards) are aware of what is really going on.

It also affects XP Pro. I wonder if using WSUS would be a sensible solution on home computers?

 

[TerryMathews]

and your car analogy falls flat as your mechanic is not the manufacturer of the Shelby Cobra. Auto manufacturers, as well as all other manufacturers very frequently stop making and providing support for your old ass beater. You cant get that muffler fixed because they dont *make* that muffler anymore, it stopped being profitable for them to do so as they haven't sold that model of car for nearly 15 years. Your mechanic that broke it is most likely going to have to go dig through some junkyards to find a replacement in decent condition. Is your mechanic on the hook for fixing his work? Absolutely, just like if your IT vendor rolled out a bad patch in your environment.

No, your analogy falls flat. I will highlight the flaw in your argument for you.

In this example, Microsoft is both the manufacturer and the mechanic in that the faulty patch in Windows Update is akain to the mechanic breaking the water pump. Both are unintended consequences of regular maintenance.

Again, this is not a design flaw. WXP runs perfectly if it's not patched or if it's manually patched. This is not the runaway clock bug from Windows 98. This is a faulty patch.

 

[mikeymikec]

I started this thread as a focal point for people experiencing the bug and for suggestion-making for its resolution (rather than a hundred threads about it), not for conspiracy theory discussion. While it's a valid point to wonder why MS haven't fixed this yet, wondering about that does not help fix the problem.

Apparently Microsoft knew about these Windows Updates problems since at least 2010:
Updates are taking up a high percentage of the CPU and bogging the computer down

It doesn't read like the same problem to me*. I've seen WU hog a lot of memory and CPU plenty of times before and in my experience it is usually resolvable. What happens with this bug isn't excessive memory usage at all but absolute CPU saturation, grinding the machine to a standstill. I see a lot of XP machines in my line of work, and according to my browser history I first searched for this problem in October this year.

* - the problem here is of course that one is relying on another to identify whether it is the same bug. You know know the competence of the other person. Also, they're running PCs with 512MB RAM. Microsoft Update and that aren't really a good mix most of the time.

 

[nemesismk2]

Was your install a fresh one?

The bug happens on new installs.

Yes it was a new install because i bought a new copy of windows xp sp3.

 

[ninaholic37]

It doesn't read like the same problem to me*. I've seen WU hog a lot of memory and CPU plenty of times before and in my experience it is usually resolvable. What happens with this bug isn't excessive memory usage at all but absolute CPU saturation, grinding the machine to a standstill. I see a lot of XP machines in my line of work, and according to my browser history I first searched for this problem in October this year.

* - the problem here is of course that one is relying on another to identify whether it is the same bug. You know know the competence of the other person. Also, they're running PCs with 512MB RAM. Microsoft Update and that aren't really a good mix most of the time.

You're probably right. That was about the 4th thread from 2010 I found before finding any from 2013, I'll have to check if they're all about RAM/Microsoft Update and not the 100% CPU thing. As you said, it's hard to tell if the people agreeing and pressing the "Me too" button are all experiencing the same thing, but it sounds likely (based on your experience) that this particular problem did start in October. If anyone is bored, there's over 1000 threads of Windows Updates problems on their site to sift through, might make some good weekend reading material :awe:

http://answers.microsoft.com/en-us/...pdate?tab=Threads&dir=Desc&sort=LastReplyDate

 

[mikeymikec]

Some news:

http://arstechnica.com/information-...m-making-windows-xp-miserable-could-be-fixed/

 

[mikeymikec]

It doesn't seem quite right to me that calculating supersedence lists would suddenly result in a CPU saturation scenario like this, surely there would have been a steadily increasing period of CPU saturation over the years, not a sudden one like this. The only way I can think that their logic applies would be if there's a bug in calculating the supersedence list once it gets beyond a certain length, but I would have thought that would result in a non-completion scenario (let's say the list gets thrown into memory, but the memory allocation size is too small, possibly resulting in a malformed list, causing supersedence calculation to hit some sort of WTF loop). Unless a list that's too long gets recalculated in error repeatedly because it is too long or something... dunno.

One thing I've noticed recently is that on a machine that doesn't experience the bug, it has the msxml4 DLL files in system32, whereas one that does experience the bug doesn't have those two files.

One that experiences the bug:

14/04/2008 12:00 506,368 msxml.dll
14/04/2008 12:00 701,440 msxml2.dll
14/04/2008 12:00 37,916 msxml2r.dll
05/06/2012 15:50 1,172,480 msxml3.dll
14/04/2008 12:00 44,032 msxml3r.dll
06/11/2012 02:01 1,371,648 msxml6.dll
14/04/2008 12:00 79,872 msxml6r.dll
14/04/2008 12:00 26,624 msxmlr.dll

One that doesn't has msxml4.dll and msxml4r.dll, is what I mean. MSXML6 is supposed to supersede 4, but perhaps not quite?

I might try removing IE8 from my XP virtual machine, put it back on but without updates, then trigger WU to see whether it is anything to do with IE cumulative patches.

- edit - I booted XP, ran WU, no bug. I then removed IE8, ran WU, experienced bug, then stopped WU, installed IE cumulative patch manually (KB2898785), then ran WU, no bug. Perhaps MS is right?

 

[ninaholic37]

The only way I can think that their logic applies would be if there's a bug in calculating the supersedence list once it gets beyond a certain length, but I would have thought that would result in a non-completion scenario (let's say the list gets thrown into memory, but the memory allocation size is too small, possibly resulting in a malformed list, causing supersedence calculation to hit some sort of WTF loop).

That's what I was thinking too. The explanation by Doug Neal sounds interesting:

Due to security requirements, the Internet Explorer product was required to continue building a chain longer than what is normally permitted in Windows Update. Over time, this exception exacerbated the previously unknown inefficiency in the Windows Update Agent. Upon review, MSRC agreed to do away with the (now outdated) requirement to maintain long supersedence lists and permit a shortening of the supersedence to only reasonable numbers like most Microsoft products on Windows and Microsoft Update.

That might be why always doing the latest IE8 cumulative patch manually gets around the problem (at least for me).

If only the source code was open, I'm sure someone would have debugged it by now. :awe:


edit: This comment doesn't sound very promising (in general):

Microsoft thought that it had this problem fixed in November's Patch Tuesday update after it culled the supersedence lists. That update didn't appear to fix the problem. The company thought that its December update would also provide a solution, with even more aggressive culling. That didn't seem to help either. Although the company said that it did test these updates, for one reason or another, its test scenarios didn't reflect the experience of real Windows XP machines.

It really makes you wonder, how many "patches" Microsoft released that don't actually work, or make things worse.

 

[ViRGE]

It doesn't seem quite right to me that calculating supersedence lists would suddenly result in a CPU saturation scenario like this, surely there would have been a steadily increasing period of CPU saturation over the years, not a sudden one like this.

Keep in mind it's an exponential algorithm, O(2^n). So the length of the check doubled with every patch. An operation that took a few minutes 2 patches prior would take 15 minutes afterwards, which is why it crept up so quickly. No one really noticed the problem until it went from minutes to large fractions of hours.

 

[Matt1970]

I just got one on Vista. Unreal.

 

[nemesismk2]

I just got one on Vista. Unreal.

I have a few family friends with vista so I had better get myself a copy because I seem to get lumbered with tech support

 

[PliotronX]

Keep in mind it's an exponential algorithm, O(2^n). So the length of the check doubled with every patch. An operation that took a few minutes 2 patches prior would take 15 minutes afterwards, which is why it crept up so quickly. No one really noticed the problem until it went from minutes to large fractions of hours.

I'm sorry I'm having trouble finding which process stage exactly is causing this as I haven't seen it running WU on any XP boxes...yet. Is it the checking updates, autoupdate, applying updates?

 

[ViRGE]

I'm sorry I'm having trouble finding which process stage exactly is causing this as I haven't seen it running WU on any XP boxes...yet. Is it the checking updates, autoupdate, applying updates?

It's checking updates. The algorithm is comparing the available updates to see whether an update has been superseded by another update or not. It sounds like they're using a naive implementation, so every update is compared to every other update to see if it's been superseded.

http://arstechnica.com/information-...m-making-windows-xp-miserable-could-be-fixed/

 

[VirtualLarry]

It sounds like they're using a naive implementation, so every update is compared to every other update to see if it's been superseded.

Could it be that they made a size / speed tradeoff during design, in favor of using less RAM, because of the lower system requirements of XP?

i'm not sure I buy the theory that this so-called "algorithm choice" is the reason behind the issue.

When I first encountered the issue, I tried a bunch of things, and the thing that seemed to fix it for me, was a MS "Fix-it" that claimed that my Windows Update binaries were corrupt (corrupt from a fresh install???), and claimed to fix them. After that, all the patches came down smooth as silk. All 122 or so of them.

If this algorithm was indeed the problem, then why didn't I continue to have the problem as it was determining which updates were needed, out of the total pool of updates?

Something about that explanation doesn't quite ring true for me.

I had to do another XP install a month later. I tried to repeat what I had done the first time, but I couldn't get the fix-it to tell me my binaries were corrupt and then fix them. In the end I had to simply suffer through several hours of waiting.

 

[ViRGE]

Could it be that they made a size / speed tradeoff during design, in favor of using less RAM, because of the lower system requirements of XP?

i'm not sure I buy the theory that this so-called "algorithm choice" is the reason behind the issue.

When I first encountered the issue, I tried a bunch of things, and the thing that seemed to fix it for me, was a MS "Fix-it" that claimed that my Windows Update binaries were corrupt (corrupt from a fresh install???), and claimed to fix them. After that, all the patches came down smooth as silk. All 122 or so of them.

If this algorithm was indeed the problem, then why didn't I continue to have the problem as it was determining which updates were needed, out of the total pool of updates?

Something about that explanation doesn't quite ring true for me.

I had to do another XP install a month later. I tried to repeat what I had done the first time, but I couldn't get the fix-it to tell me my binaries were corrupt and then fix them. In the end I had to simply suffer through several hours of waiting.

I don't see any reason to doubt Microsoft's explanation. It makes perfect sense given the facts presented - the problem ramped up very quickly - and Microsoft doesn't have any reason to lie. Now using that algorithm may be a size-speed tradeoff in and of itself, but it's not as if there's something else going on.

 

[Cerb]

I think the recent removal of old updates did some good. I just had to reload an ancient P4 2GHz box. It took about a day before the 100% CPU usage subsided, and has then been fine, to what degree such a PC can be considered fine (I hate the idea of fixing a PC running an OS with a kill switch set for next May!). Compare that with several days for XP Mode on IB and Haswell i3s and i5s, previously.

 

[VirtualLarry]

I think the recent removal of old updates did some good. I just had to reload an ancient P4 2GHz box. It took about a day before the 100% CPU usage subsided

I'm not doubting you in the least, but conceptually, that is so very wrong. The fact that any of us put up with that #%% from MS, is proof of how well we've been trained to accept MS's mistakes.

 

[Cerb]

I'm not doubting you in the least, but conceptually, that is so very wrong. The fact that any of us put up with that #%% from MS, is proof of how well we've been trained to accept MS's mistakes.

As opposed to...what? There's no other option. Other people already made decisions that got us stuck with MS software only. When your choice is to do business and accept their mistakes, or not do business, what are you going to do?

 

[monkeydelmagico]

I think the recent removal of old updates did some good. I just had to reload an ancient P4 2GHz box. It took about a day before the 100% CPU usage subsided, and has then been fine, to what degree such a PC can be considered fine (I hate the idea of fixing a PC running an OS with a kill switch set for next May!). Compare that with several days for XP Mode on IB and Haswell i3s and i5s, previously.

Did an install this weekend and the same thing happened. Couldn't figure out why the heck svhost needed 99% cpu utilization for hours on end. Decided to just wait it out and it finally moved on the updates.

If I had it to do over again I would install before going to bed then let it run overnight.

 

[Cerb]

Did an install this weekend and the same thing happened. Couldn't figure out why the heck svhost needed 99% cpu utilization for hours on end. Decided to just wait it out and it finally moved on the updates.

If I had it to do over again I would install before going to bed then let it run overnight.

Since I did know about it, that's exactly what I did. Except, it was before leaving the office, instead of going to bed . Updating XP Mode instances, however, had previously taken 3+ whole days, with mostly 3rd-gen i3 and i5 CPUs. Virtualization will account for some added overhead, but that was just crazy, and it made some of the software that XP [mode] was needed for nearly unusable until it finished. Some people with older hardware, like slow P4s and Athlons, were complaining of it taking weeks, and that was just unacceptable.