For win32 users - a script that allows actual usage of 4GB+ of RAM....

[hamunaptra]

For years Ive wanted to find a way to use PAE to allow windows 32bit to actually use 4GB+ RAM...since thats basically what PAE is really for. On modern machines windows nearly always has PAE turned on because DEP(nx bit) requires PAE be used. So, by default most windows 32bit installs actually use the PAE enabled kernel.
But, even when PAE is forced on or is on because of DEP... RAM above 4GB is still locked away and unusable.
After coming across this article: http://www.geoffchappell.com/notes/windows/license/memory.htm

Ive been inspired to get this working for myself. So, Ive successfully been able to patch vista_32 & win7_32 kernels, resign them and boot from them. This has been tested on a couple different machines and works fantastic.
After confirming both kernel types are patchable, I set out to write an autoit script that automates the process on both vista & 7.
So, Ive made my first release of the patch. It basically patches the kernel, creates a cert, signs the patched kernel w/ the cert, modifies BCD to add entry of picking between original kernel & patched copy of the kernel.
NOTE: this is ONLY for vista or 7. Win8 32 I have not tested. XP this patch is not possible.
TO INSTALL: Simply run the exe

TO UNINSTALL: by creating a shortcut to my script, and putting 'uninstall' (without quotes) as an argument, it will remove the BCD entries and delete the modified kernel.

I am posting links to my script, one is the precompiled version(the executable)
The other is the source so you can see for urself what it does.
http://temp-share.com/show/HKdPx6LpA
http://temp-share.com/show/KdPf25qAh
Screenshot:
http://temp-share.com/show/dPf3apu7W

-
NOTE: after applying the patch, your system may show "test" mode water mark on the desktop in the corner(s). This is normal. There are guides to get rid of this watermark(by editing user32.dll.mui file). I may make it as part of my script in the future if there is enough demand for it.

UPDATE: noticed the link to geoff article broken, now fixed. Also the test mode watermark no longer appears ive implimented it into my script in an improved version(same download links)

UPDATE: the script now patches the mentioned mui files as well to remove the testing watermarks

 

[seepy83]

And you're going through all of this instead of just installing a 64-bit OS because....???

 

[hamunaptra]

And you're going through all of this instead of just installing a 64-bit OS because....???

Because, Im tryin to make my 6yr old 32bit OS install work long as possible =), months of setting up 6years worth of programs / settings / data is just no worth it till Im forced to go to 64bit.
I would go 64bit if there was an in place upgrade option, but alas, there isnt =P

 

[TigCobra]

Because, Im tryin to make my 6yr old 32bit OS install work long as possible =), months of setting up 6years worth of programs / settings / data is just no worth it till Im forced to go to 64bit.
I would go 64bit if there was an in place upgrade option, but alas, there isnt =P

I think this is the worst reason I have ever heard. I am sorry.

 

[dave_the_nerd]

Because, Im tryin to make my 6yr old 32bit OS install work long as possible =), months of setting up 6years worth of programs / settings / data is just no worth it till Im forced to go to 64bit.
I would go 64bit if there was an in place upgrade option, but alas, there isnt =P

Windows 7 has an in-place upgrade option.

 

[JackMDS]

Windows 7 has an in-place upgrade option.

Not from x32 version to a x64 version.

 

[Towermax]

Thanks, OP--I'll give this a try on a friend's 32-bit Vista. Like you, she has several years of detailed records, tweaked programs, etc, and has no desire to spend many hours reconfiguring her system after a new OS install. Plus, the system does all she needs, so she doesn't see any reason for an upgrade at this time.

(If it doesn't work, no problem restoring since her system is fully backed-up on WHS.)

 

[piasabird]

I have an older computer With Vista 32, but I dont plan on purchasing any DDR2 800 RAM.

 

[juvat]

A very interesting thread. Though I have Win8 64bit, I want to thank you for your diligence and hard work on this. I imagine there are many-many others out there who would benefit.

Towermax, could you please post the results of implementing his script?

jimb

 

[Mushkins]

Not sure I would willingly use this on something with all those important tweaks/programs/etc in a production environment. Sure, it might address that extra memory, but who knows what other issues it could bring to light or updates it could bork? Still kind of cool it can be done though.

 

[Towermax]

Towermax, could you please post the results of implementing his script?

Yes, but it won't be for a couple of months. I'm away from home and won't be able to do it till sometime in late April. Hopefully, someone else will give it a try before then . . .

 

[masteryoda34]

Very interesting. Would you care to elaborate on what parts of the kernel you needed to patch and how you figured out what you needed to do?

 

[hamunaptra]

Guys, if you read geoffs page, bleh Ill just link it... OH WAIT, just noticed the link is broken in my OP.. I just updated it, heres the link anyways: http://www.geoffchappell.com/notes/windows/license/memory.htm

So, if you go down to patch details, it shows the table showing the original opcodes and offsets then the resulting opcodes and offsets. Thats about it. My script utilizes DUP2 which is an image patcher (searches for pattern matches of those locations in the kernel image and replaces the code w/ the patched code as shown in the article)
To verify this patch works, you guys could make a 32bit guess OS instance in a VM and apply the patch(remember only vista32 or 7_32) ... set the OS RAM to over 4GB and on reboot you will see the rest of the RAM =).

This patch is easily undoable as it doesnt do anything PERMANENT to the kernel itself, it simply copies ur current kernel and patches it, then resigns it using a self made cert, then it basically adds the appropriate BCD entries to boot that kernel by default. (meaning it offers 2 boot options: the original kernel and patched one). To uninstall simply create a shortcut to my script w/ the argument 'uninstall' (without quotes)... it will remove the BCD entry and delete the patched kernel.

You can look at my source code ( the au3 script) to see exactly what it does.
Im happy to report after 2 weeks on my main gaming machine, its been running AMAZING. No stability issues and Im constantly saturating over 4GB of RAM(running more than 1 heavy 3d game and browsers w/ many tabs).

The only time this patch will incur stability issues if you are using extremely old drivers that dont have a 64bit provided variant from the manufacturer, there is a possibility of those drivers not supporting address ranges above 4GB. (note you dont need the 64bit drivers, it just means generally when manufacturer codes for 64bit drivers, their 32bit drivers will have all the needed code to work w/ address ranges above 4GB .. unless they put the extra effort into stripping this ability in their 32bit drivers upon compiling)

 

[masteryoda34]

Guys, if you read geoffs page, bleh Ill just link it... OH WAIT, just noticed the link is broken in my OP.. I just updated it, heres the link anyways: http://www.geoffchappell.com/notes/windows/license/memory.htm

Wow! Very detailed. This answers all my questions.

 

[xylem]

I find this to be interesting.. Thanks for the work and for posting. I may give it a try if I find myself with a surplus of time, and will post results if that happens.

 

[Anteaus]

Since virtually all software that operates in Win32 is still limited to 2GB of memory space regardless of total ram, does this really get any more traction? In other words, you might pull around half a GB back into action, but if you're only one or two heavy applications at a time or playing games I can't see that you'd see any tangible difference. I could be wrong though.

If you don't mind me asking, what are you doing with this machine that this fix would actually boost performance. I only ask because in your second post you specifically mention that you're trying to keep your 32-bit OS running as long as possible, but from my vantage point you're going to get the same life out of the machine whether you do this fix or not. At some point or another there is going to either be a software or hardware failure that is going to trigger an upgrade, and that is going to happen regardless of whether this PAE stuff works or not.

That said, if you can manage to get all this working then you would have succeeded in doing what even MS hasn't been able to do widescale without breaking something along the way. Third party software can sometimes be unpredictable.

 

[hamunaptra]

Since virtually all software that operates in Win32 is still limited to 2GB of memory space regardless of total ram, does this really get any more traction? In other words, you might pull around half a GB back into action, but if you're only one or two heavy applications at a time or playing games I can't see that you'd see any tangible difference. I could be wrong though.

If you don't mind me asking, what are you doing with this machine that this fix would actually boost performance. I only ask because in your second post you specifically mention that you're trying to keep your 32-bit OS running as long as possible, but from my vantage point you're going to get the same life out of the machine whether you do this fix or not. At some point or another there is going to either be a software or hardware failure that is going to trigger an upgrade, and that is going to happen regardless of whether this PAE stuff works or not.

That said, if you can manage to get all this working then you would have succeeded in doing what even MS hasn't been able to do widescale without breaking something along the way. Third party software can sometimes be unpredictable.

Well, firstly, often times most processes dont need the full 2GB address space, and when they do they generally can take advantage of the large address awareness flag(which Ive also increased userva to 2400MB in my script, user can always change as needed). Some games touch the 2GB address limit and other rare applications do as well. BF3 often times, 64p multiplayer server only uses around 1700M address space. Also, thats not entirely true for 32bit programs as a limitation. If coded properly(db engines n so forth) can utilize AWE pages which can see well beyond the 2GB(default user) address space.
Also, keep in mind even a 32bit process on a 64bit machine still only has a 4GB address space, in fact its even less, coming around 3.2-3.5GB due to wow64 overhead.

2ndly, the added performance of having more RAM comes from the fact I now can play 1-3 RAM intensive games at the same time & have a browser w/ 30+ tabs open(which I do nearly all the time). I experience no slowdowns w/ the added RAM.

3rdly, a software or hardware issue that would trigger an upgrade -well.. sure, if you dont back shit up this will happen sooner than later. In the 6+ years Ive had this same OS(several OS upgrades & several diff motherboards both AMD/intel) going .. smooth as a baby's but .. Ive not had it outright 'break' that would need an upgrade. The only thing really that would trigger an upgrade is when a majority of the programs I use in the future start hitting that 32bit address space limitation... more often then not(which hasnt happened).

4thly, as geoffs article points out. MS decided to keep addressable physical RAM limited to the same addressable virtual address space due to issues w/ drivers at the time. Remember when 64bit was new?(XP 64) and XP64 sucked for stability reasons...well thats because driver manufacturers were just then starting to code for 64bit. All those issues in 64bit would have been there in PAE 32bit drivers utilizing greater than 32bit PTE's. MS obviously didnt want to have 2 product lines on there hands w/ stability issues due to driver issues(32 & 64) so thats why they limited desktop OS's to 4GB max addressable physical memory. Also, as geoffs article points out, general software itself (non driver) is not concerned at all about virtual address -> physical address space mapping. This is solely handled by the memory manager, which is part of the kernel and completely transparent to programs.

 

[Anteaus]

Well, firstly, often times most processes dont need the full 2GB address space, and when they do they generally can take advantage of the large address awareness flag(which Ive also increased userva to 2400MB in my script, user can always change as needed). Some games touch the 2GB address limit and other rare applications do as well. BF3 often times, 64p multiplayer server only uses around 1700M address space. Also, thats not entirely true for 32bit programs as a limitation. If coded properly(db engines n so forth) can utilize AWE pages which can see well beyond the 2GB(default user) address space.
Also, keep in mind even a 32bit process on a 64bit machine still only has a 4GB address space, in fact its even less, coming around 3.2-3.5GB due to wow64 overhead.

2ndly, the added performance of having more RAM comes from the fact I now can play 1-3 RAM intensive games at the same time & have a browser w/ 30+ tabs open(which I do nearly all the time). I experience no slowdowns w/ the added RAM.

3rdly, a software or hardware issue that would trigger an upgrade -well.. sure, if you dont back shit up this will happen sooner than later. In the 6+ years Ive had this same OS(several OS upgrades & several diff motherboards both AMD/intel) going .. smooth as a baby's but .. Ive not had it outright 'break' that would need an upgrade. The only thing really that would trigger an upgrade is when a majority of the programs I use in the future start hitting that 32bit address space limitation... more often then not(which hasnt happened).

4thly, as geoffs article points out. MS decided to keep addressable physical RAM limited to the same addressable virtual address space due to issues w/ drivers at the time. Remember when 64bit was new?(XP 64) and XP64 sucked for stability reasons...well thats because driver manufacturers were just then starting to code for 64bit. All those issues in 64bit would have been there in PAE 32bit drivers utilizing greater than 32bit PTE's. MS obviously didnt want to have 2 product lines on there hands w/ stability issues due to driver issues(32 & 64) so thats why they limited desktop OS's to 4GB max addressable physical memory. Also, as geoffs article points out, general software itself (non driver) is not concerned at all about virtual address -> physical address space mapping. This is solely handled by the memory manager, which is part of the kernel and completely transparent to programs.

Thanks for the info!

 

[imagoon]

Lots of good information buried in conspiracy drivel. I find it neat he fails to mention that MS made the license codes 32/64 for most of them anyway. I figure the MS had some sort of good reason to do it the way they did. It might not be relevant now but they are not in business of making changes like that to the older OS's.

 

[hamunaptra]

Lots of good information buried in conspiracy drivel. I find it neat he fails to mention that MS made the license codes 32/64 for most of them anyway. I figure the MS had some sort of good reason to do it the way they did. It might not be relevant now but they are not in business of making changes like that to the older OS's.

I dont know if you are referring to me or to Geoffs article, but Geoff states several times its simply a license of RAM usable which is the limitation. Not the spoon fed "its impossible" lie weve been told over the years. The patch simply tells the kernel to bypass this check and allow all RAM usable. He also mentions this license of RAM isnt directly even mentioned in MS's EULAs, much less anywhere else.

 

[imagoon]

I dont know if you are referring to me or to Geoffs article, but Geoff states several times its simply a license of RAM usable which is the limitation. Not the spoon fed "its impossible" lie weve been told over the years. The patch simply tells the kernel to bypass this check and allow all RAM usable. He also mentions this license of RAM isnt directly even mentioned in MS's EULAs, much less anywhere else.

The "it's impossible" part has come from the fact that unless you get MS to either a) change the value (or license whatever you want to call it,) sign that modified kernel, or you are willing to run in test mode it is effectively impossible. It is fairly commonly known that PAE was used in the 32-bit OS's (DEP like he mentioned requires it). For the longest time I have had 32-bit 2003 Servers running 8 -> 16GB of RAM. To further that, installing SQL server on a 32bit OS, it will use the RAM even if the OS doesn't manage it. You can also use the PAE switches at boot to change memory handling.

The original reason they blocked it was driver issues while trying to maintain compatibility between the PAE / None PAE kernels back in the 2k days. XP needed to keep support for 'ancient' non PAE 2k stuff.

I would guess the real reason MS never changed the 32 bit code restrictions was because at the time 4GB of RAM was getting common on the desktop, 64bit versions were already available and registration key compatible so why waste the cash certifying the desktops.

This same key is used in the 64bit versions to limit RAM usage in Windows 7 Home/ Pro / Ultimate etc so I suspect the reason it was key was named "license" had more to do with future plans to limit the OS for app tiers.

It comes off conspiracy style because he feels the need to bring up all the arguments against it rather than simply saying: "If you change these values it works!" and then saying "I am aware of these arguments. Otherwise he has some really good information in there and I think it would be better delete several chunks of the text that talk about the restrictions.

 

[hamunaptra]

Yes, but what you dont realize is the 'word on the street' for the last several years ... is 'RAM above 4GB is unusable in desktop windows 32bit'(like as if it were to blame on the hardware or just 32bit itself) .. nearly every uninformed article written on this says it. This is the impression many communities have about it and this is also the knowledge of the avg joe schmoe that walks in the shop off the street.
I think thats what geoffs article is talking about. Basically the bad impression 32bit has gotten because of the 'max 4GB RAM limitation it has' ... based on plenty of wrong press and marketing.

 

[imagoon]

Yes, but what you dont realize is the 'word on the street' for the last several years ... is 'RAM above 4GB is unusable in desktop windows 32bit'(like as if it were to blame on the hardware or just 32bit itself) .. nearly every uninformed article written on this says it. This is the impression many communities have about it and this is also the knowledge of the avg joe schmoe that walks in the shop off the street.
I think thats what geoffs article is talking about. Basically the bad impression 32bit has gotten because of the 'max 4GB RAM limitation it has' ... based on plenty of wrong press and marketing.

I realize it fine. Provide me with a Microsoft documented and supported method to do what you are doing.

32bit Linux has used PAE for years. So did all of the 32bit server releases. You can't make this modification and actually boot Windows except in the test modes that allow unsigned kernels so it has been effectively true. That is the same as 'being true' for most intents and purposes.

Average Joe Schmoe isn't going to care and will just install Windows 64bit anyway.

 

[Anteaus]

Yes, but what you dont realize is the 'word on the street' for the last several years ... is 'RAM above 4GB is unusable in desktop windows 32bit'(like as if it were to blame on the hardware or just 32bit itself) .. nearly every uninformed article written on this says it. This is the impression many communities have about it and this is also the knowledge of the avg joe schmoe that walks in the shop off the street.
I think thats what geoffs article is talking about. Basically the bad impression 32bit has gotten because of the 'max 4GB RAM limitation it has' ... based on plenty of wrong press and marketing.

I see where you're going with that, but the "word on the street" isn't wrong, even if technically inaccurate.

Windows 32-bit has never supported +4GB of memory in any official capacity. It isn't a lie to say that no 32-bit Windows Desktop release can do 4GB+ because it is defacto truth that no "out of the box" release can do it.

I also don't believe its much of a conspiracy. I won't speak about software specifics that I don't know much about, but I'm reasonably confident that if MS had found a way to do it without causing problems for people they would have done it. Maybe it is bowing to the least common denominator, but at the end of the day they need to move units and avoiding unnecessary pain is part of the deal both from a PR and a financial standpoint.

Look what happened with XP SP2 when the PAE switch went mainstream. By SP3 MS had to close the door again because all sorts of issues came up. Obviously there are always cases where experienced programmers/admins can have success where others can't, but when you consider the total number of licenses in circulation compared to the number of people who can intelligently deal with these things it's easy to see why MS did what they did.

There is a guy on these forums who keeps telling everyone he is going to figure out how to unlock the power of the AMD Bulldozer CPU, in spite of the fact that the CPU has been put through every test imaginable and has simply falling short of what it was meant to be. I think this is one of those situations where you might prove to a few people that 4GB+ 32-bit desktop is not only possible and has real application, but in the end it what does it matter. 32-bit is dead to all but those who cling to it out of necessity.

Anyways, I think it's cool what you're doing even if it doesn't really have practical application with the average user. Good luck!

 

[hamunaptra]

I see where you're going with that, but the "word on the street" isn't wrong, even if technically inaccurate.

Windows 32-bit has never supported +4GB of memory in any official capacity. It isn't a lie to say that no 32-bit Windows Desktop release can do 4GB+ because it is defacto truth that no "out of the box" release can do it.

I also don't believe its much of a conspiracy. I won't speak about software specifics that I don't know much about, but I'm reasonably confident that if MS had found a way to do it without causing problems for people they would have done it. Maybe it is bowing to the least common denominator, but at the end of the day they need to move units and avoiding unnecessary pain is part of the deal both from a PR and a financial standpoint.

Look what happened with XP SP2 when the PAE switch went mainstream. By SP3 MS had to close the door again because all sorts of issues came up. Obviously there are always cases where experienced programmers/admins can have success where others can't, but when you consider the total number of licenses in circulation compared to the number of people who can intelligently deal with these things it's easy to see why MS did what they did.

There is a guy on these forums who keeps telling everyone he is going to figure out how to unlock the power of the AMD Bulldozer CPU, in spite of the fact that the CPU has been put through every test imaginable and has simply falling short of what it was meant to be. I think this is one of those situations where you might prove to a few people that 4GB+ 32-bit desktop is not only possible and has real application, but in the end it what does it matter. 32-bit is dead to all but those who cling to it out of necessity.

Anyways, I think it's cool what you're doing even if it doesn't really have practical application with the average user. Good luck!

I disagree, 'the word on the street' is wrong. True, MS simply says they dont want one using above 4GB of RAM on 32bit. People need to be made aware its a limitation MS put in place, not a 32bit limitation in of itself(hence giving 32bit a bad rep).
Also, of course MS found issues w/ it. Like I said before it was mainly because at the time 64bit was a new idea in MS world & drivers werent written properly for years when 64bit was new. Those problems would translate into the 32bit compiled drivers when dealing w/ addresses larger than 32bit. As time went on, MS ultimately didnt care because as 64bit drivers were maturing so was the rate of 64bit OS adoption .. so why unlock 32bit potential when they could keep it all locked up and use it as a method to leverage the advantages of 64bit.
I also disagree it doesnt have a practical application w/ the user .. sure it does .. getting more life outta 32bit then ever thought imagineable in a desktop OS.

Also, that guy about BD (PD) isnt entirely wrong. There are a few areas where that uarch is superior. I can name one specialized setting where written in proper assembly language can obliterate intel counterparts. That would be scrypt mining, utilizing AMD's XOP instruction set.
---
BTW, not sure about the whole XP SP2 thing you mentioned, but, generally ALL 32bit windows OS's have PAE turned on(using PAE kernel) out of the box. Its just the RAM limit is still in place as per the kernel setting. PAE kernel is required in order to use hardware DEP(which every processor since late P4's supports).

Also, thanks much for the compliment =) ... it just excites me that I (with the help of geoffs article) have come up w/ a way to get around something thats bugged me for years(mainly cuz Im still a 32bit user).