Optimal Windows 7 "virtual memory" or pagefile size with new technology

[BonzaiDuck]

Now that we've got our feet firmly planted in the "Age of SSD -- freedom from storage bottlenecks," -- here's a question based on my own, real configuration of disks:

60 GB Patriot Pyro SSD as caching for accelerated HDD under ISRT

600 GB WD VelociRaptor -- the accelerated drive

500 GB WD Black HDD -- File backups, photo library, Windows Media Center buffer for Live TV and DVR recordings

Until an hour ago, I didn't include the WD Black in my virtual memory settings.

And I'd simply set the VM/pagefile size at ~ 16GB -- the amount of RAM in my system.

But cleaning up some issues surrounding the Windows VSS Volume Shadow-Copy Service, it dawned on me: "IF my boot disk (VelociRaptor) is cached under ISRT, how much of a page file does it REALLY need? Should it even benefit from or need a pagefile?"

I finally chose to just "let Windows decide" about the pagefile.

But really -- first -- under this specific hardware configuration, how much of a pagefile is necessary? And second -- how does this apply generally to a larger variety of configurations?

I think there is already a consensus about an SSD boot disk: You don't NEED a page file! Isn't that true?

Caller! You're on the air! . . . .

 

[Essence_of_War]

I wouldn't spend anytime worrying about this. Just use a pagefile. Are you short on space or something? Drives are so cheap that rather than muck around with the pagefile, the correct solution to the problem is to fix the actual problem. Get more capacity.

Why would caching your main drive make you think you don't need a pagefile?

 

[KingFatty]

You can actually test this, and it can be pretty fun.

Your particular scenario works with the ability to test it out, because you are already set to let windows decide.

I would suggest you set the pagefile to a rediculously small value, like 1 megabyte. There is no danger here, because if windows needs more, it will decide to enlarge that file when the time comes.

But the nice thing is you get to witness how long that will be, until the time comes. You have 16 GB of RAM, so I'm quite confident that windows will take a very very long time before it tries to enlarge that pagefile. Feel free to pick a bigger one or whatever, but I still think your initial choice of 16 GB is wasting space.

That's the nice thing about this test, you can now free up almost 16 GB of storage space off of your SSD or velociraptor or wherever it is, and I'm guessing for you that the freed storage space is very valuable, and it's a fun way to test this out.

 

[MongGrel]

Personally, I just run a small page file on the 4 WD RE3's in here I've had for many years on an Areca 1210 RAID card and turn the page file off on my SSD's.

 

[BonzaiDuck]

That's the nice thing about this test, you can now free up almost 16 GB of storage space off of your SSD or velociraptor or wherever it is, and I'm guessing for you that the freed storage space is very valuable, and it's a fun way to test this out.

Well . . . 16GB out of 600 with >50% of the HDD free . . . I hadn't thought of it. I recently came within a 100GB of filling up the other drive with DVR captures. I used to "trim" these more mercilessly, then finding myself re-record some movie I'd purged because of the memorable script-lines. (Coen Bros "No Country . . " was a long wait to get back).

But it DOES beg the question. I have 60GB of "disk cache" with ISRT. I have 16GB of RAM which seldom -- ever -- exceeds 50% usage (although I doubled it because the 8GB was always pushing 75% after a week's 24/7, DVRs and Media Center running 24/7.)

I was planning to build a Haswell-E system next year. It gives me pause: This almost-3-year-old SB-K system has a lot more than I'd need for the more practical business tasks. After a three month obsession with it, I think I just eliminated every red-bang node in my Event Viewer logs; system access the internet with less delay; backs up without a hitch every morning.

I think, very soon, I may just swap out the Pyro SSD and VelociRaptor after cloning over to a Samsung 840 Pro. But with so few "hourglass" experiences, I'm just not in a big hurry. I was saving that Sammy drive for next year.

 

[Essence_of_War]

But it DOES beg the question.

I'm not sure there was a question

 

[KingFatty]

Begging the question means you are simply "answering" the question by rephrasing it, thereby dodging the actual need to answer. Or something like that, like circular logic where nothing is gained. But more commonly, people use that phrase to really mean "raises the question" so the phrase will actually evolve to have a different definition over time.

 

[VirtualLarry]

I think putting a 16GB pagefile on your iSRT-accellerated HDD is fine.

 

[BonzaiDuck]

Well, it almost seems like KingFatty is peeved at me. Larry might think I'm worried about the issue, but his is a fair answer.

KingFatty DOES have a useful suggestion, even so -- "test it."

I just don't think anyone ever fully explored the implications of the ISRT caching. Suppose your SSD specs or benchmarks at 500 MB/s sequential read. Suppose the ISRT caching gives you an "effective" HDD access of ~ 350. And since there's so much RAM, it doesn't seem like the usual rule of thumb for pagefile size would apply.

But like Larry says, you don't need to worry about it. And if you were short on disk space, you could reduce the pagefile a tad, I'd think.

 

[KingFatty]

Sorry I didn't mean to suggest peevishness. But it would be good to know if you have a usage scenario where you'll run out of physical ram. I think virtual machines are the easiest way to use up lots of ram, but I'm not sure of how else you might use up all 16 GB of ram.

I'd even go so far as to just go all the way to the dark side and simply delete the page file and run your computer to see if you encounter an error. I'm too lazy to try this, but I don't have a luxurious 16 GB of ram either.

 

[BonzaiDuck]

Sorry I didn't mean to suggest peevishness. But it would be good to know if you have a usage scenario where you'll run out of physical ram. I think virtual machines are the easiest way to use up lots of ram, but I'm not sure of how else you might use up all 16 GB of ram.

I'd even go so far as to just go all the way to the dark side and simply delete the page file and run your computer to see if you encounter an error. I'm too lazy to try this, but I don't have a luxurious 16 GB of ram either.

Well, sah, I've grown more cautious in my old age, perhaps too much so since I come here often with questions I might answer myself. But this is indeed a question -- the "optimal" page file for an ISRT setup..

To be more specific, I have some prospective hardware upgrades to make on this machine and my WHS server-box -- some of them "dead certain" and others "optional."

One of the things I'm planning to do (eventually) is to dump the ISRT and replace it with a Sammy 840-Pro I have sitting here in the box. Even before I make the final decision to do that (because I could also "save" the Sammy for a build I plan for next year), I wanted to assess and then clean up my Win 7 installation on this here workstation. I had been concerned that there may have been disk corruption over 30 months time, but -- no -- it's pristine. I then wanted to clean up the causes of event-log "red-bangs." It's all in the "blue" now.

Well -- caveats -- I'm down to about two: one is a benign (and quite common among Win7 users) Event ID 2 "kernel event tracing" error which likely derives from some startup program. Another is a "side-by-side" error attributable to one software package that can be cleaned out and reinstalled. I had an "Schannel" error last night which likely results from my browser configuration and visiting to some particular site. That's about it . . .

So the only errors I could tolerate are those which don't set me back on all this progress. Sometime later this week, I may try scaling back the page file to half as much -- to see what happens.

 

[Essence_of_War]

Duck,

Maybe it would help if you explained exactly what you're trying to optimize by eliminating the page file.

Are you worried that your SSD cache is caching your page file and wasting "good caching space" on that? Have you tried to check how much paging you're actually doing? I have two machines I work on with 16 GB of RAM, and on my gaming machine I don't think I've ever seen the page file used, on my work machine I use it only when I'm manipulating really big matrices.

On top of that, I feel like you'd actually WANT the SSD to cache the page file anyway. If your OS is writing pages out, it is high probability stuff that will need to be in RAM again, so caching it to the SSD is probably better than writing it to a spinner.

 

[BonzaiDuck]

Duck,

Maybe it would help if you explained exactly what you're trying to optimize by eliminating the page file.

Are you worried that your SSD cache is caching your page file and wasting "good caching space" on that? Have you tried to check how much paging you're actually doing? I have two machines I work on with 16 GB of RAM, and on my gaming machine I don't think I've ever seen the page file used, on my work machine I use it only when I'm manipulating really big matrices.

On top of that, I feel like you'd actually WANT the SSD to cache the page file anyway. If your OS is writing pages out, it is high probability stuff that will need to be in RAM again, so caching it to the SSD is probably better than writing it to a spinner.

And . . . I'm pretty sure the SSD does cache the page file. How else would it work? I'm also wondering, though, absorbing what you have also said, if the optimal page file size might be smaller. For instance, suppose you just had a decently-performing HDD. The optimal cache size would seem to equal the size of RAM. But -- you're never or seldom using RAM to full capacity -- there's nothing to be swapped to disk. Yet the disk is "accelerated" by the caching, so you'd still think faster disk speed would point toward smaller cache size. Or would you?

Probably good reason to experiment with it for a while. All I can say is that any "bugs" I had in January are gone -- this system is "tip-top." If it continues another few weeks with this level of perfection, I'll switch the ISRT mode to "Maximized."

On the other hand, between event-logs and smooth-running stability, there was never a better time to clone the system to my new Sammy 840 Pro and save a drive image for good measure. In other words, I don't see an "upgrade-repair" or OS reinstall in the picture here. Dorothy has a worm-hole to Kansas, and definitely "out of the woods in OZ."

Ah! Decisions! Decisions!

 

[Essence_of_War]

And . . . I'm pretty sure the SSD does cache the page file. How else would it work? I'm also wondering, though, absorbing what you have also said, if the optimal page file size might be smaller. For instance, suppose you just had a decently-performing HDD. The optimal cache size would seem to equal the size of RAM. But -- you're never or seldom using RAM to full capacity -- there's nothing to be swapped to disk. Yet the disk is "accelerated" by the caching, so you'd still think faster disk speed would point toward smaller cache size. Or would you?

I think there is still some mis-communication somewhere between us.

The SSD cache does more than just cache the page file. It caches frequently accessed blocks from the spinner, so if everything is working well, your "usual" workloads should be much faster. It is not at all obvious to me why the optimal cache size would be equal to the size of RAM.

 

[KingFatty]

Also, maybe there is some misunderstanding, or at least approaching the issue backward.

You seem to describe the optimal pagefile for a machine as a function of the particular drive.

But I think instead, the optimal pagefile is a function of the amount of RAM and the type of programs you run.

So you think about how you use your computer, then you look at how much RAM it has, then you predict whether your amount of RAM will handle it. Then you choose the smallest sized pagefile that will keep these factors satisfied.

As far as the drive you will put the pagefile on, I don't think it's much of a decision. It should be on the fastest drive you have, assuming you have room to put it there, assuming you have made it as small as practical based on amount of RAM and intended computer usage. So thinking about the hard drive should be the last step in deciding how to set up the pagefile. I'd prefer to just delete it or set it to a very small number, but let windows adjust as necessary so you still have the flexibility of using one if you ever change your usage habits or remove RAM.

 

[sm625]

It is hard to test the utility of a pagefile when you have 16GB of RAM. A machine with 2GB of RAM will page much more often. It is still annoying, even when the page file is on an SSD.

 

[BonzaiDuck]

Also, maybe there is some misunderstanding, or at least approaching the issue backward.

You seem to describe the optimal pagefile for a machine as a function of the particular drive.

But I think instead, the optimal pagefile is a function of the amount of RAM and the type of programs you run.

So you think about how you use your computer, then you look at how much RAM it has, then you predict whether your amount of RAM will handle it. Then you choose the smallest sized pagefile that will keep these factors satisfied.

As far as the drive you will put the pagefile on, I don't think it's much of a decision. It should be on the fastest drive you have, assuming you have room to put it there, assuming you have made it as small as practical based on amount of RAM and intended computer usage. So thinking about the hard drive should be the last step in deciding how to set up the pagefile. I'd prefer to just delete it or set it to a very small number, but let windows adjust as necessary so you still have the flexibility of using one if you ever change your usage habits or remove RAM.

No, just to confirm my "credentials," that was always understood. With only traditional HDD technology and your run-of-the-mill SATA controller, the RAM drives the page file because you're swapping out program and data space to disk so that something else can have priority in queues to the processor via its three-level cache.

It always seemed that if Windows was allowed to pick the cache size, you would get it approximately to equal total RAM or most of it.

But we're "on the same page." However, ISRT would seem to change the rule-of-thumb expectations, since it's caching a queue of programs and data while also caching the page-file. But the ISRT configuration means a sustained transfer-rate of approximately 350 to 400 MB/s, as opposed to the best hard drive's ~140 MB/s (probably velocirapter), or hard drive RAID arrays -- which add wattage as well as speed. With four drives, I could never get my SATA-II 3Ware RAID5 to do more than come close to 300 MB/s read rate. And even with that, you're already spending $300+ extra on a controller alone.

So you might have 80% of effective SSD-standalone speeds. Even your writes would accelerate if you set the ISRT manager to "Maximum" or "Maximized" mode.

Since the system is writing changes to the cache SSD simultaneous to writing them to HDD, there has to be some nexus of logic that points to a smaller page file. First, in a spectrum of users and their patterns, many are not likely to see much RAM to swap to disk -- [remember it used to be called a "swap file"]. Effectively for purposes of another read taking place eventually, it's putting the page file into the SSD cache, so it can read it much faster.

Now just assume a hypothetical, that your system is rock stable, issue-free, backed up with an industrial UPS. You're not taxing your boot disk, since so many more reads come from the SSD cache. You could use "Maximum" mode so that the HDD is refreshed in bursts, so you don't have to wait much for HDD writes.

I think that you might be better for a smaller page file for these, and possibly other reasons.

It is hard to test the utility of a pagefile when you have 16GB of RAM. A machine with 2GB of RAM will page much more often. It is still annoying, even when the page file is on an SSD.

Yup. That's why they call it "Virtual Memory," and it's only "virtual" at hard disk speeds except in the circumstance you cite.

I think there is still some mis-communication somewhere between us.
The SSD cache does more than just cache the page file. It caches frequently accessed blocks from the spinner, so if everything is working well, your "usual" workloads should be much faster. It is not at all obvious to me why the optimal cache size would be equal to the size of RAM.

Well, SSDs were scarce and ISRT was an Intel project when Win 7 was released. I probably wouldn't have to strain my brain to start thinking of certain aspects of a page file as vestigial -- a sort of withered and less-necessary feature.

 

[Cerb]

In Windows, RAM + PF <= peak commit (which is harder to measure, these days). Forcing a smaller page file than default (or none) just reduces your possible commit. That may be good or bad, depending on how you use your PC, and what you expect of it. SSDs do not change that inequality one little bit, whether as dedicated drives, or as caches.

Windows has long used the PF to keep RAM that's little-touched freeable (backing cache), so a big/default PF may be higher-performing in many cases, and definitely will be for a RAM-starved PC. The size of it you want or need is about capacity, not speed.

The optimal size of PF is like the optimal size of drive: it's going to vary by the user. The default size and behavior is made to be good enough that 95% of people don't care, and is not necessarily an ideal, perfect-for-everybody, Microsoft-knows-what-you-need configuration, just a good enough one.

If you have 8GB RAM, and your commit never exceeds 5GB, FI, your page file settings won't even matter. If your commit can get right near 8GB, then you (a) need more RAM and (b) the page file matters, because it needs to be big enough to store everything that has to spill out of RAM.