What is the final word on the Win XP page file(s)?

[bsobel]

't believe that you're unaware of the performance advantages of a paging file.

Whatever Larry. First you say "No matter how much ram you have, paging increases performance" then you say " I'm not suggesting that phenomenon is in any way performance-enhancing, and in fact it signals that you don't have enough physical RAM period" Which, if you were paying any attention at all, was all my original point was.

I'll leave it to the others to discuss with you. I'm out, your one of those guys who just wants to argue about everything no matter how stupid or simple it is (any by god, if we can make an anti-MS conspiracy out of it all the better).

 

[Nothinman]

It is a waste of money to have more ram then your ever going to use. Especially true if you are a heavy multitasker. Why pay for enough memory to store all 19 or so programs when I can only use one at a time.

It's not a waste of money, memory will always be used for system cache to avoid disk access which is always a performance boost. And I may not use all 19 programs simultaneously, but I do multitask and operate more than one thing at a time, don't you?

 

[VirtualLarry]

Originally posted by: bsobel

't believe that you're unaware of the performance advantages of a paging file.

Whatever Larry. First you say "No matter how much ram you have, paging increases performance" then you say " I'm not suggesting that phenomenon is in any way performance-enhancing, and in fact it signals that you don't have enough physical RAM period" Which, if you were paying any attention at all, was all my original point was.

I was talking about "thrashing" there, not normal paging. Which I clearly stated in my original post. Paging is performance-enhancing, overall. "Thrashing" is not. Is my statement now clear enough?

Originally posted by: bsobel
I'll leave it to the others to discuss with you. I'm out, your one of those guys who just wants to argue about everything no matter how stupid or simple it is (any by god, if we can make an anti-MS conspiracy out of it all the better).

That's too bad, since this is a purely technical computer-science issue, and has nothing at all to do with MS, although that link I posted shows that they agree with my statement as well. It's pretty simple to see, if you look at it deeply enough. It effectively allows multiplexing your physical memory, whereas if that wasn't possible, it would require nearly 3x as much to achieve the same performance level for computing tasks.

In fact, paging is really a normal part of the memory hierarchy, just extended downward, since if you look at a CPU's L1/L2 cache-line replacement algorithm, it closely resembles "paging" as well. Yet, it is performance-enhancing. Why/how could that be?

Isn't it clear that the CPU cache functions more effectively, when the cache lines are used to their fullest, for the task at hand, rather than some of them being occupied by "stale" data, and not available to be used by the current tasks? How are memory pages in physical RAM any different? (This is likely why the NT VM model treats RAM as a cache of the pagefile, rather than the pagefile being a spill-over from the RAM, and why the pagefile is used, even when free physical RAM is available.)

The downside, of course, is the latency that occurs, shuffling the pages between RAM and the pagefile, and if the disk has a fragmented pagefile, or is otherwise experiencing heavy disk I/O, the paging I/O gets further delayed, causing more severe worst-case latency. So while paging can improve overall performance in the majority of cases, it can also suffer from even worse worst-case performance in the corner cases. That's why putting a (not necessarily the only) pagefile on a little-used spindle can be useful, and reduce paging latency/load on the other drives.

Edit: I will add that because of some other aspects of MS's actual implementation, the actual day-to-day performance will be somewhat less than the theoretical, but it should still offer some advantages. The biggest problem is lack of any sort of disk I/O prioritization scheme in NT, AFAIK, so paging activity can interrupt a stream of disk block I/O requests, and cause additional seek latency issues, unless your pagefile is on a less-used spindle.

I'm not entirely sure how much NT's "pre-emptive paging" of dirty memory pages actually helps though, since it may cause a larger number of pagefile-related disk I/Os than a purely demand-paged system. Additionally, the utility of paging out stale application memory pages, in order to use for disk cache, to do read-ahead pre-emptive caching operations, is dependent on the sequential access of data. In terms of worst-case random-access to a data file, depending on the level of read-ahead, far more application memory pages may be forced to the pagefile by the OS, than the number of pages used for disk caching that are actual used, and not simply disposed of. There are limits on how effective this is, just like there are effective upper-bound limits on branch prediction. In fact, it's interesting, because the CPU guys have taken this OS software feature, and implimented it in hardware (hardware cache line pre-fetch).

PS. If you thought this to be exhaustive, you should see the two threads on StorageReview that I was involved with last year or so. Oh, and for the record, I still respect you guys totally. My anti-MS rants are more of a hardcoded reflex action, I try not to make them too personal, sorry if anyone felt that way.

 

[stash]

I'll leave it to the others to discuss with you. I'm out, your one of those guys who just wants to argue about everything no matter how stupid or simple it is (any by god, if we can make an anti-MS conspiracy out of it all the better).

Glad to see I'm not the only one who finds Larry exhausting

All in good fun though Larry

 

[Smilin]

Originally posted by: STaSh

I'll leave it to the others to discuss with you. I'm out, your one of those guys who just wants to argue about everything no matter how stupid or simple it is (any by god, if we can make an anti-MS conspiracy out of it all the better).

Glad to see I'm not the only one who finds Larry exhausting

All in good fun though Larry

Yes, good old Larry. He'll start an MS rant here soon if he hasn't already. As usual I've skipped over his droning.

His original post stating that having a paging file even when you have more than enough physical memory is still valid though. I didn't read any further so he may have started babbling some nonsense. The details are way outside of the scope of this thread (for my lazy ass at least).

 

[bsobel]

His original post stating that having a paging file even when you have more than enough physical memory is still valid though.

And I didn't suggest otherwise. My only comment is that it was best to have more physical memory than your normal usage so you didn't need to page. VL comes in and claims that you WANT TO PAGE as it INCREASES PERFORMANCE in all cases :roll:

 

[Smilin]

Originally posted by: bsobel

His original post stating that having a paging file even when you have more than enough physical memory is still valid though.

And I didn't suggest otherwise. My only comment is that it was best to have more physical memory than your normal usage so you didn't need to page. VL comes in and claims that you WANT TO PAGE as it INCREASES PERFORMANCE in all cases :roll:

Don't argue with me Mister.

 

[tiap]

I recently installed a learning game for my 5 year old daughter. Her computer has 1 gig of ram. The program wouldn't install unless the pagefile was set at a minumum of 3 gigs. So now she has a static page file of 4 gigs and everything runs smoothly. I probably should check the processes for that program to see how much real ram is used or if it is just sloppy programming on the part of the game.

 

[VirtualLarry]

Originally posted by: bsobel

His original post stating that having a paging file even when you have more than enough physical memory is still valid though.

And I didn't suggest otherwise. My only comment is that it was best to have more physical memory than your normal usage so you didn't need to page. VL comes in and claims that you WANT TO PAGE as it INCREASES PERFORMANCE in all cases :roll:

Oh, I don't disagree that adding more RAM isn't better - it clearly is. If you hit the "pagefile thrashing" threshold, you clearly need more physical RAM to contain the working-set of your active processes. But given a fixed limit to the amount of RAM (any size, really), then paging can increase performance even further. Again, the pagefile is a valuable tool, that allows the OS to convert "stale" physical RAM memory pages into disk cache, and disk cache can be used to increase performance. Thus, not taking advantage of such mechanisms, would result in lower performance.

I've attempted to clearly explain it via analogy to a CPU's L2 cache-replacement algorithm. If you can't see the similarities, and thus the advantages, then I'm sorry for you. Perhaps studying computer science would prove to be fruitful.

Btw, please don't confuse my statements, with the implementation of such in W2K/XP, especially the non-demand/pre-emptive pagefile behavior for dirty pages, and the lack of disk IO priority, in terms of being seperate from normal process CPU priority.

In fact, the former is the root of the "problem", of no matter how much physical RAM you have, some pagefile activity always seems to take place no matter what. Meaning that this statement isn't exactly correct either - more physical RAM can never eliminate the amount of pagefile activity under W2K/XP, due to design.

Originally posted by: bsobel
Best advice I can give you is to have more physical ram than your actually using so your not actually paging (just creating reservations).

I'm curious what kind of performance advantages one might be able to obtain under typical load when that was feature disabled, and pagefile activity only occurred when free RAM pages were actually low, rather than constantly and pre-emptively in the background. I would also like to see the "minimum RAM page free list size" low-water mark adjusted upwards, to reduce the effective latency of page commits. Anyone know how?

Edit: Useful VM-related link (PDF), check page 47, what they refer to as "pre-cleaning" of dirty pages, is what I understand that NT-based OSes do, and why they generate pagefile activity even when free/available physical RAM pages are not low. Page 57 is also a good read, re thrashing.

Ok, found the link that I was originally searching for, for this thread here. Andrew Morton talking about the "swappiness" setting for the Linux kernel, and its effects on VM paging performance vs. interactivity.

I just find it so funny, that one of the top Linux kernel hackers, and MS themselves, document what I've mentioned (to be fair, I was finally enlightened by that above link some time ago, I didn't come to that conclusion without some outside input and persuasion), and yet some people still think that I'm somehow just making this all up, just because it goes against the grain of "conventional wisdom".