Extra RAM and RAMDisk

[SunnyD]

Originally posted by: Nothinman

I am quite familiar with this, thank you.

You may be familiar with it but you still don't seem to understand some of the basic differences in terms. Read the memory management chapters in Inside Windows or Understanding the Linux kernel and you'll see what I mean.

By the way, you can easily turn off paging in Windows. Try it some time, it's under System Properties -> Advanced -> "Performance Settings". I wouldn't suggest doing so...

That's not paging, that's the pagefile.

I believe the discussion in this thread revolves around speeding up disk-based IO with respect to the system, where the whole crux revolves around paging memory to the pagefile does it not? If there's no pagefile, you're not swapping pages out of memory!

I think you've read way to far past the intent of this thread. Why would a ramdrive or i-RAM have anything to do with paging memory in and out of a CPU cache? The OP is taking about relieving disk IO bottlenecks.

 

[Nothinman]

I believe the discussion in this thread revolves around speeding up disk-based IO with respect to the system, where the whole crux revolves around paging memory to the pagefile does it not? If there's no pagefile, you're not swapping pages out of memory!

The pagefile is only part of the "problem", when memory is low mmap()'d files, executables, libraries, etc will also be evicted from memory and when you access them again (i.e. alt tab to an app or it needs to run for some other reason) they'll need to be paged back in causing I/O. You can't just focus on one small portion of the subject.

 

[SunnyD]

Originally posted by: Nothinman

I believe the discussion in this thread revolves around speeding up disk-based IO with respect to the system, where the whole crux revolves around paging memory to the pagefile does it not? If there's no pagefile, you're not swapping pages out of memory!

The pagefile is only part of the "problem", when memory is low mmap()'d files, executables, libraries, etc will also be evicted from memory and when you access them again (i.e. alt tab to an app or it needs to run for some other reason) they'll need to be paged back in causing I/O. You can't just focus on one small portion of the subject.

I'll give you that. I was focusing on more "common" complaints about system performance, which was seemingly what the OP was looking for. Again, in an optimal world the OP should just go with a SSD and use that as his secondary storage device, rather than a ram drive or whatnot.

 

[Nothinman]

I'll give you that. I was focusing on more "common" complaints about system performance, which was seemingly what the OP was looking for. Again, in an optimal world the OP should just go with a SSD and use that as his secondary storage device, rather than a ram drive or whatnot.

Common complaints are usually totally off base, most people see hard disk activity and assume pagefile activity even when it's not the case. But then again if the average use knew anything about memory management and how to use perfmon then those common complaints would probably dry up.

 

[WaiWai]

Originally posted by: Nothinman

Tell me if I'm wrong. But OS will still use paging (not only when RAM is insufficient) unless you switch it off completely. I am also wondering if it is a good idea to switch off paging completely (don't use virtual RAM at all) since I have plenty of RAM (3GB*).

You can't switch off paging, paging isn't a bad thing, it's how the system gets data to/from memory. The same goes for Virtual Memory. Get a copy of Inside Windows or Understanding the Linux kernel and read the chapter on memory management.

What about pagefile?
Should I switch it off completely if I have plenty of RAM?

The reason why I want to use RAMDisk is I want to keep accessing to/from hard drive at the minimal. As long as I can tell the OS and all apps to use RAM (instead of hard drive) when they are running, I'm fine with it.

The OS is already caching all reads and writes into main memory, sure they hit disk eventually and that may slow them down a bit but if your performance requirements are that high you probably need to rethink your whole setup.

No, I don't think so.
If you say all reads/writes have been cached into main memory, why is my hard drive still flashing even though my RAM hasn't been used up. (eg my PF usage is only 1.41GB, but I have 2GB real RAM).

*: I realise 32-bit OS can't make use of all 4GB, and I'm not ready to use 64-bit OS (application and drive compatibility issues). So I think I will only get 3*1GB DDR2 RAM at max.

You realize wrong, a 32-bit OS that fully supports PAE can access up to 64G of physical memory.

Yes it is supposed to solve the address space problem but it causes conflicts and incompatibilities issues. Microsoft has pretty much abandoned PAE in the 32-bit versions of Windows XP SP2 and Windows Vista. This is to avoid compatibility problems with the wide variety of third-party device drivers. For example, some incorrectly written drivers may cause problems when attempting DMA operations. Thus, setting the PAE option to "ForceEnable" using the BCDEdit program likely has little effect in 32-bit Vista.

PS: PAE theoretically support up to 64GB. However it can't in reality. The max is still 4GB.

 

[WaiWai]

Originally posted by: SunnyD

I believe the discussion in this thread revolves around speeding up disk-based IO with respect to the system, where the whole crux revolves around paging memory to the pagefile does it not? If there's no pagefile, you're not swapping pages out of memory!

I think you've read way to far past the intent of this thread. Why would a ramdrive or i-RAM have anything to do with paging memory in and out of a CPU cache? The OP is taking about relieving disk IO bottlenecks.

Yes your are right. I want to avoid the disk I/O bottlenecks as many as possible. If there is any way to move the disk reads/writes to RAM reads/writes, I would like to know.


HyperDrive (another type of hardware RAMDisk):
http://www.hyperossystems.co.uk/

 

[EagleKeeper]

Originally posted by: WaiWai

Originally posted by: SunnyD

I believe the discussion in this thread revolves around speeding up disk-based IO with respect to the system, where the whole crux revolves around paging memory to the pagefile does it not? If there's no pagefile, you're not swapping pages out of memory!

I think you've read way to far past the intent of this thread. Why would a ramdrive or i-RAM have anything to do with paging memory in and out of a CPU cache? The OP is taking about relieving disk IO bottlenecks.

Yes your are right. I want to avoid the disk I/O bottlenecks as many as possible. If there is any way to move the disk reads/writes to RAM reads/writes, I would like to know.


HyperDrive (another type of hardware RAMDisk):
http://www.hyperossystems.co.uk/

Your target files for the I/O wouild have to be on the RAM drive.
For programs that are already built, unless there is some config file/utility available to you, are hard coded to some directory, usually where the program is being run from or in relation to it.

To get around that, you would have to load the program itself from RAM disk and ensure that all associated directory structures and support files are also in the RAM drive.

 

[Nothinman]

What about pagefile?
Should I switch it off completely if I have plenty of RAM?

No, let the OS do it's thing. Contrary to popular belief the programmers at MS do know more about memory management than you and most people.

No, I don't think so.
If you say all reads/writes have been cached into main memory, why is my hard drive still flashing even though my RAM hasn't been used up. (eg my PF usage is only 1.41GB, but I have 2GB real RAM).

To save you from losing data in the event of a crash. Writes are delayed a bit so that if a page is updated frequently it won't hit the disk for every little change and even if the write does get pushed out to disk the modified pages will stay in memory for a while so that any subsequent reads don't need to hit the disk. It's likely just that the write delay isn't as long as you would like and no, I don't know if a way to tune that in Windows.

Yes it is supposed to solve the address space problem but it causes conflicts and incompatibilities issues. Microsoft has pretty much abandoned PAE in the 32-bit versions of Windows XP SP2 and Windows Vista. This is to avoid compatibility problems with the wide variety of third-party device drivers. For example, some incorrectly written drivers may cause problems when attempting DMA operations. Thus, setting the PAE option to "ForceEnable" using the BCDEdit program likely has little effect in 32-bit Vista.

PS: PAE theoretically support up to 64GB. However it can't in reality. The max is still 4GB.

I know the reasoning behind MS' decision to cripple their 32-bit client's PAE support and I think it's a piss-poor excuse. The only driver that I've specifically heard about problems with is the one from nVidia and I'm sure they'd fix it if pressured even a little bit.

So PAE does in reality support up to 64G of physical memory, just because Windows is crippled doesn't change the fact that PAE works.

 

[SunnyD]

Originally posted by: Nothinman

What about pagefile?
Should I switch it off completely if I have plenty of RAM?

No, let the OS do it's thing. Contrary to popular belief the programmers at MS do know more about memory management than you and most people.

WaiWai - please note that even though I said the pagefile can be turned off... Windows will actually run LESS efficiently without it (unless you have ample amounts of RAM : read "MUCH more than 4GB").

Have a blast shopping...

 

[WaiWai]

After reading many replies, what I can do is to have 3GB RAM [*], and let Windows do the rest. I don't need to adjust anything.
Hardware RAMDisk seems to be better option than software RAMDisk.

*: I'm still uncertain about the /PAE switch (so as to get 1GB RAM more). /PAE support is essentially dropped by MS.

 

[WaiWai]

So PAE does in reality support up to 64G of physical memory, just because Windows is crippled doesn't change the fact that PAE works.

Let's face the reality. It is just a theory only. No one has tested the theoretical limit of 64GB RAM.
No one is running 64GB with PAE on. No one can ensure no problems or bugs will arise when we are pushing towards the limit.

PAE is just a technology feature. It can't support 64GB on its own.
Yes PAE doesn't need to work with windows but it still need to work with an operating system to support 64GB.
PAE has to work with an operating system, driver, and other components to support 64GB in reality.
So I don't think it is correct to say "PAE does in reality support up to 64G of physical memory".

PAE itself does in theory support up to 64GB of physical memory.

 

[Nothinman]

Let's face the reality. It is just a theory only. No one has tested the theoretical limit of 64GB RAM.

It's not just a theory, all it does is add another level of page tables and tell the CPU to use 36-bit addresses instead of 32-bits and I know of people that have tested it with Linux at least.

So I don't think it is correct to say "PAE does in reality support up to 64G of physical memory".

Sure it does since just about that's all PAE does. Enabling PAE tells the CPU to use 36-bits for addresses, i.e. up to 64G of memory. Whether you can find hardware that allows you to install 64G of memory and an OS that'll let you use it is orthogonal.