xp's ram limit

[rubix]

why is there about a ~3.2gb ram limit in windows xp? is this limit artificially imposed by the way microsoft designed xp?

if so, how difficult would it be for microsoft to change some of their xp code to make it support say 8gb of ram? would it be a complete overhaul of xp or just changing various dlls or something?

note: i didn't ask *if* they would change the code and release a new xp, just if they could do it and how hard it would be.

 

[bsobel]

Originally posted by: rubix
why is there about a ~3.2gb ram limit in windows xp? is this limit artificially imposed by the way microsoft designed xp?

if so, how difficult would it be for microsoft to change some of their xp code to make it support say 8gb of ram? would it be a complete overhaul of xp or just changing various dlls or something?

note: i didn't ask *if* they would change the code and release a new xp, just if they could do it and how hard it would be.

XP and 2003 server share the same code base. 2003 server can (in PAE mode) support >4gig of memory. When MS tried to do this with the XP code base, too many drivers crashed never expecting such configurations. Basically since the server uses a subset (generally speaking) of drivers that a consumer OS needed, it was supported in 2003 but no longer supported in XP.

 

[TheStu]

Originally posted by: bsobel

Originally posted by: rubix
why is there about a ~3.2gb ram limit in windows xp? is this limit artificially imposed by the way microsoft designed xp?

if so, how difficult would it be for microsoft to change some of their xp code to make it support say 8gb of ram? would it be a complete overhaul of xp or just changing various dlls or something?

note: i didn't ask *if* they would change the code and release a new xp, just if they could do it and how hard it would be.

XP and 2003 server share the same code base. 2003 server can (in PAE mode) support >4gig of memory. When MS tried to do this with the XP code base, too many drivers crashed never expecting such configurations. Basically since the server uses a subset (generally speaking) of drivers that a consumer OS needed, it was supported in 2003 but no longer supported in XP.

Wouldn't XP64 propperly support greater than 4GB RAM or is it also limited?

 

[bsobel]

Originally posted by: TheStu

Originally posted by: bsobel

Originally posted by: rubix
why is there about a ~3.2gb ram limit in windows xp? is this limit artificially imposed by the way microsoft designed xp?

if so, how difficult would it be for microsoft to change some of their xp code to make it support say 8gb of ram? would it be a complete overhaul of xp or just changing various dlls or something?

note: i didn't ask *if* they would change the code and release a new xp, just if they could do it and how hard it would be.

XP and 2003 server share the same code base. 2003 server can (in PAE mode) support >4gig of memory. When MS tried to do this with the XP code base, too many drivers crashed never expecting such configurations. Basically since the server uses a subset (generally speaking) of drivers that a consumer OS needed, it was supported in 2003 but no longer supported in XP.

Wouldn't XP64 propperly support greater than 4GB RAM or is it also limited?

It is not limited to 4gig. Since it was a new OS and 64 bit drivers (e.g. new drivers) had to be written for it, they didnt have the legacy problem that xp32 had.

 

[rubix]

i forgot there is a xp64. is xp64 literally the same as regular xp but can use more memory and all that... or is it a whole new os with "xp" slapped onto it's name?

does xp64 install basically the same files and take up the same space as xp? can it be hacked up like xp can with tools like nlite and xplite to be made "light weight".

other than needing special drivers, can xp64 run old emulators (snes9x, meka, kega etc) and stuff that work on regular xp? how about older games, like thief or alice (the american mcgee game)?

 

[Lepard]

The 4GB limit is a 32-bit limitation, not a Microsoft/XP imposed limitation.

e.g. Ubuntu 32-bit has the same 4GB cap.

 

[The Keeper]

64-bit Windows XP kernel is based on 64-bit Windows 2003 kernel. Basically 64-bit XP is just desktop version of Windows 2003 with XP as name. If you need to pick 64-bit Windows for desktop or notebook usage, Vista would be better option than XP. Otherwise stick with 32-bit XP.

 

[Raduque]

I think a better question is why do you want to put 4gb+ ram in a machine running emulators?

 

[Nothinman]

The 4GB limit is a 32-bit limitation, not a Microsoft/XP imposed limitation.

e.g. Ubuntu 32-bit has the same 4GB cap.

Not if you use a kernel with PAE enabled it doesn't. 32-bit Linux can use up to 64G of memory with PAE although it's much better to just use a 64-bit kernel anyway.

 

[Lepard]

Originally posted by: Nothinman

The 4GB limit is a 32-bit limitation, not a Microsoft/XP imposed limitation.

e.g. Ubuntu 32-bit has the same 4GB cap.

Not if you use a kernel with PAE enabled it doesn't. 32-bit Linux can use up to 64G of memory with PAE although it's much better to just use a 64-bit kernel anyway.

Isn't PAE technically 36-bit?

 

[nerp]

Originally posted by: Lepard

Originally posted by: Nothinman

The 4GB limit is a 32-bit limitation, not a Microsoft/XP imposed limitation.

e.g. Ubuntu 32-bit has the same 4GB cap.

Not if you use a kernel with PAE enabled it doesn't. 32-bit Linux can use up to 64G of memory with PAE although it's much better to just use a 64-bit kernel anyway.

Isn't PAE technically 36-bit?

I'm only qualified to say that the XP version is. That's all I know though. Maybe not for linux?

 

[tcsenter]

Originally posted by: Lepard
Isn't PAE technically 36-bit?

Yes, but there is more to PAE than just the end-result. The OS can cap the MMU to anything it wants.

if so, how difficult would it be for microsoft to change some of their xp code to make it support say 8gb of ram? would it be a complete overhaul of xp or just changing various dlls or something?

It would be relatively trivial. The problem would be the deluge of problem reports and complaints Microsoft would face after assuring developers for the past four years that they don't need to worry about their drivers and applications being able to cope with this because physical addresses beyond 4GB will never exist in 32-bit XP and Vista.

 

[bsobel]

Isn't PAE technically 36-bit?

Yes, PAE mode uses 36+ bit addressing, but the OS is still a 32bit OS and drivers must be specifically aware of the large addresses (hence the compatibility problems)

 

[degibson]

I feel like this problem comes up a lot. Perhaps we should sticky the answer.

Anyway, here's whats going on.

In a computer, all bytes in the memory system need a unique name, called an address. If you have, say, 2 GB of main memory, then there are 2147483648 bytes of RAM in your machine. To name them all, in binary, you need 31 bits. With 32 bits, you can name 4 GB. Here comes the problem: A lot of x86 hardware out there only has a so-called 'word length' of 32 bits -- it doesn't understand memory addresses longer than 32 bits.

So how do we get around this? Its something called Physical Address Extension (PAE). Think of this as adding a "street name" to your "address"**. 123 Elm St. is not the same as 123 Oak St. If your mailman only looks at the '123', then he could deliver a letter to the wrong place. The same thing is true of PAE -- individual programs haven't been compiled expecting memory locations to have 'street names' in additional to numerical addresses, so individual programs under PAE can still only use up to 4 GB. Kernels and drivers can be made aware of PAE, but they can still only use 4 GB ranges at a time.

In short: PAE is not that great. If you want to use more than 4 GB of RAM (3 GB in the case of Windows), buy 64-bit hardware and use 64-bit OSes.

** PAE is segmentation all over again. Clearly, it wasn't bad enough the first time around. ;-)

Edit: Nothinman and hanspeter corrected me on this. PAE != segmentation.

 

[Nothinman]

** PAE is segmentation all over again. Clearly, it wasn't bad enough the first time around. ;-)

No, PAE just extends the physical address to 36-bits, there's no segmentation going on.

 

[degibson]

Originally posted by: Nothinman

** PAE is segmentation all over again. Clearly, it wasn't bad enough the first time around. ;-)

No, PAE just extends the physical address to 36-bits, there's no segmentation going on.

The physical addresses are larger than the virtual addresses. Virtual addresses are multiplexed over PA ranges by a windowing function. Even if the don't call it segmentation, PAE is segmentation.

Edit: Let me be clearer on this. Even if contiguous VAs can be mapped to non-contiguous extended portions of the PA space, and there is no single 'segment' register, I still call it segmentation because a single VA space is still limited to a subset of the physical address space -- the subset may not be contiguous as in traditional segmentation.

Edit Again: Nothinman and hanspeter corrected me on this. PAE != segmentation.

 

[Nothinman]

The physical addresses are larger than the virtual addresses. Virtual addresses are multiplexed over PA ranges by a windowing function. Even if the don't call it segmentation, PAE is segmentation.

No, with segmentation you've got multiple idenfiers for a block of memory. The segment that it's in, the offset from the beginning of that segment and the length of the block. With virtual memory one virtual address points to one physical address, just because there's more physical address than there are virtual doesn't mean any segmentation is being done.

The kernel accesses the physical and virtual memory as two contiguous spaces and each process accesses it's own virtual space as one contiguous space. No segments are involved at all.

 

[hanspeter]

Nothinman is right. You have the exact same memory layout in 64bit mode. You have different virtual address spaces pointing to different areas in the physical address space. The only difference is the size of the virtual area that can be mapped for each process.

In the current x64 implementation, the physical address space is bigger than the virtual address space.

Edit: You might call it segmentation, if you code your application to use the AWE API. Here you get a window that you can move around in physical memory, so your application can address whatever amount of physical ram you have installed.

 

[degibson]

Originally posted by: hanspeter
Nothinman is right. You have the exact same memory layout in 64bit mode. You have different virtual address spaces pointing to different areas in the physical address space. The only difference is the size of the virtual area that can be mapped for each process.

In the current x64 implementation, the physical address space is bigger than the virtual address space.

Edit: You might call it segmentation, if you code your application to use the AWE API. Here you get a window that you can move around in physical memory, so your application can address whatever amount of physical ram you have installed.

Here's a ray of light for me. Thanks hanspeter. I thought that AWE was the only mechanism to use PAE. I'm going to note in my above responses that I was incorrect in my assertion.

I'm confused, however, about the line about x64 -- The x64 VA is 64 bits (well, not quite that big actually, because of hierarchical page tables, but its big). How is the PA space larger than the VA space? Are we including I/O spaces in this discussion?

 

[Nothinman]

Here's a ray of light for me. Thanks hanspeter. I thought that AWE was the only mechanism to use PAE. I'm going to note in my above responses that I was incorrect in my assertion.

AFAIK AWE is the only userland mechanism to work with PAE. And even it doesn't do anything special besides give you an API to manage sets of memory maps.

His point was that on a 64-bit system that physical and virtual address limits are different just like on a 32-bit system with PAE enabled. On AMD64 in long mode the system only uses 40 bits physical, 48 bits virtual. Neither are fully 64-bit yet.

 

[degibson]

Originally posted by: Nothinman

His point was that on a 64-bit system that physical and virtual address limits are different just like on a 32-bit system with PAE enabled. On AMD64 in long mode the system only uses 40 bits physical, 48 bits virtual. Neither are fully 64-bit yet.

I knew this. This makes perfect sense. The following is the claim I am confused about:

Originally posted by: hanspeter
In the current x64 implementation, the physical address space is bigger than the virtual address space.

 

[hanspeter]

It was about address translation. The virtual address (which is currently less than 64 bit) translates to a 52bit physical address (on page table level). The actual physical bus is less in current cpus.

 

[Nothinman]

I knew this. This makes perfect sense. The following is the claim I am confused about:

Well besides the fact that he mistakenly swapped virtual and physical, which should be obvious, we said the same thing.

 

[degibson]

Originally posted by: Nothinman

I knew this. This makes perfect sense. The following is the claim I am confused about:

Well besides the fact that he mistakenly swapped virtual and physical, which should be obvious, we said the same thing.

Ok, I'm now GTG. Next topic