Understanding memory usage (and Vista)

Aug 23, 2004
84
0
0
I work at a place that sells computers, and I was looking over our various Vista systems and was rather surprised about something. No matter how much RAM they had (512MB, 1GB, or 2GB), the percentage of their physical memory used was pretty similar. I would assume that, at least, the 2GB systems should have twice as much free physical memory as the 1GB ones do. But that definately does not appear to be the case.

My understanding of memory is rather... limited. I think of RAM as the "working space" of the computer?the place things are when they're running. I know that part of the hard drive is reserved as pseudo-RAM in the form of the "page file" or "virtual memory," but I admit I don't know much about this.

So I decided to provide some specifics. Below are the make, model, basic specs, and Task Manager Peformance information for three Vista laptops. The first two are very similar aside from their RAM (and screen size), the latter not as much, but I'm trying to understand this and I would really appreciate any insights anyone could provide.

Toshiba P105-S6197:
Intel® Core? 2 Duo Processor T5200, 2048MB PC4200 DDR2 SDRAM, 160GB (5400 RPM) SATA harddrive, 17? screen running at 1440x900, Windows Vista Home Premium
Physical Memory: Total: 2037 MB, Cached: 1369, Free: 27. Memory usage reads: 1.07GB, 54%.
Kernel Memory: Total: 136 MB, Paged: 84, Nonpaged: 52
System: Handles: 22552, Threads: 821, Processes: 78, Up Time: 63:43:33, Page File: 1050M / 4296M

Toshiba A135-S4467:
Intel® Core? 2 Duo Processor T5200, 1024MB PC4200 DDR2, 160GB (5400 RPM) SATA harddrive, 15.4" screen running at 1280x800, Windows Vista Home Premium
Physical Memory: Total: 1013 MB, Cached: 475, Free: 9. Memory usage reads: 623 MB, 61%
Kernel Memory: Total: 104, Paged: 60, Nonpaged: 44
System: Handles: 20711, Threads: 784, Processes: 73, Up Time: 201:34:35, Page File: 977M / 2280M

HP Pavilion DV6205us:
Intel Pentium Dual Core (Core Duo, basically) processor T2060, 512MB DDR2 SDRAM, 80GB (5400RPM) SATA Hard Drive, 15.4" screen running 1280x800, Windows Vista Home Basic
Physical Memory: Total: 501 MB, Cached: 148, Free: 3, Memory usage reads: 414 MB, 82%
Kernel Memory: Total: 92, Paged: 63, Nonpaged: 28
System: Handles 17508, Threads: 776, Processes: 57, Up Time: 200:14:47, Page File: 739M / 1496M

I must admit, I don't know what the extra five processes were on the A135-S4467, but all three had nothing open when I did this except for IE7, the Task Manager, and whatever they boot up with on the Guest (limited) user.

I've heard Vista can "scale" itself somehow based upon the available system resources--is that what's happening here? If so, how? And if you had a system with 4GB of RAM, would it still find a way to use up 50% of the physical memory doing nothing? :)

Thanks!
 

ViRGE

Elite Member, Moderator Emeritus
Oct 9, 1999
31,516
167
106
Look up SuperFetch. Vista is using all the spare RAM as a cache(and yes, it will use every last byte it can get until it runs out of things to cache).
 
Aug 23, 2004
84
0
0
Originally posted by: ViRGE
Look up SuperFetch. Vista is using all the spare RAM as a cache(and yes, it will use every last byte it can get until it runs out of things to cache).

Yes, I assumed SuperFetch was why the "free" memory was so low (and what the "cached" memory value was). But I also listed the "used" memory and the reading from the little performance gadget thing, which shows (as a percentage) CPU and memory usage. In both cases, this seems to be exclusing the SuperFetch used memory to make the value useful.

Problem is, the numbers don't add up: on the Toshiba P105, if you add the "cached" value to the "free" value, you get 1396 MB, which is 68% of the 2048 total memory ("32% used," IOW). The 1.07GB of used memory does add up to the 54% value on the memory usage meter, but it would mean 952.32 MB of RAM would be available, and that value isn't on the performance values.
 

hardcandy2

Senior member
Feb 13, 2006
333
0
0
Any programs residing in memory? Turned off any scanners, Adobe, etc to make sure they are not sitting in memory waitng to run?
 

Lemon law

Lifer
Nov 6, 2005
20,984
3
0
If these are computers sitting in some store---turned on for eye candy but not really used as work
horses, you won't get an accurate picture of how vista superfetch uses free ram.---like the man says---it will uses every spare byte of ram until it runs out of things to cache.
 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
My understanding of memory is rather... limited. I think of RAM as the "working space" of the computer?the place things are when they're running. I know that part of the hard drive is reserved as pseudo-RAM in the form of the "page file" or "virtual memory," but I admit I don't know much about this.

Memory statistics are always misleading, even if you understand how it all works. Lots of things are miscounted because there are tons of pages shared between multiple processes with no easy way to go back and see just which processes a specific page is mapped into. It would possible to keep track of that but the amount of memory required to do that would remove any benefits from the more accurate statistics. And the pagefile isn't pseusdo-RAM, it's more like a storage locker because it's extremely slow and the data stored in it isn't directly accessible.
 

Smilin

Diamond Member
Mar 4, 2002
7,357
0
0
Dig around the web for "memory manager" or grab a copy of Windows Internals to read.

A few things to consider:
1. Unused ram is wasted ram. Show me a computer that's only using a tiny percent of it's ram and I'll show you a computer that is sluggish when you ask it to do something new.
2. Much allocated ram is "disposable". You can try to predict what a user will do and pre-load into ram. If the user does something different the ram can be discarded in just a few CPU cycles.
3. Many apps allocate memory that they "may" use but aren't doing so yet. The OS will do certain things to account for this like putting such pages immediately in the pagefile. You can also have read-only pages of memory shared by multiple instances of an app. If one need to write a copy of the page is made before the change.

If Vista's memory management befuddles don't worry too much. Take a close look at disk activity instead. If Vista starts hammering the disk to grab pages when you go to do something then you should start worrying. I'm betting what is happening instead is Vista is hitting your disk when you are doing *nothing* but when you find you want to do something the disk barely moves (because it successfully predicted).

Another thing you might want to do to check out the behavior: load up a bunch of stuff that consumes gobs of memory and see what happens. On your Pavilion for instance, load say 8 different 100meg files into memory. Once they are loaded, close them all quickly and see what happens to your memory. Although you only had ~750 megs of free memory all 800megs of files somehow fit. You will also see some disk activity as pages are being preemptively copied (not moved...yet) out to disk.
 
Aug 23, 2004
84
0
0
Originally posted by: Smilin
2. Much allocated ram is "disposable". You can try to predict what a user will do and pre-load into ram. If the user does something different the ram can be discarded in just a few CPU cycles.
But would this not be considered "free" RAM by Vista?

Originally posted by: Smilin
If Vista's memory management befuddles don't worry too much.
Actually, all memory management kinda confuses me. :) I don't know much about the "nuts and bolts" of how computers work beyond how it tends to affect things. (Games need video cards so the processor doesn't have to do that work. Computers need RAM to load things so they can work with them faster than working off the hard drive. And so forth.) So I'm trying to get a grasp of this particular nuts and bolts issue since this is something Vista is especially criticized for: its RAM consumption. And this RAM consumption seems totally counter-intuitive.

Two on-the-sales-floor, never-owned demo computers of the same brand running in Vista's guest user should use the same amount of RAM "idle." If computer A has twice as much RAM as computer B, it would seem logical that its "real RAM consumption" (not preempitive caching) should be half, as a percentage, computer B's. And yet I'm not seeing that. I'm trying to understand why. Is it really just SuperFetch? Shouldn't Vista's RAM-usage meter ignore that since it's misleading?
 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
So I'm trying to get a grasp of this particular nuts and bolts issue since this is something Vista is especially criticized for: its RAM consumption.

Well congratulations on choosing one of the most conceptually complicated parts of the OS to start with. =) Smilin is right, you'll never really understand what's going on unless you do something like read the memory management chapter(s) in Inside Windows.

If computer A has twice as much RAM as computer B, it would seem logical that its "real RAM consumption" (not preempitive caching) should be half, as a percentage, computer B's.

But memory used for caching is still technically used.

Is it really just SuperFetch? Shouldn't Vista's RAM-usage meter ignore that since it's misleading?

Most likely yes and probaly not, no, respectively. There has to be a way to tell how much memory is being used for filesystem caching, if taskmgr ignored it then you'd have to use something like perfmon for that.
 

BD2003

Lifer
Oct 9, 1999
16,815
1
81
Originally posted by: Nothinman
So I'm trying to get a grasp of this particular nuts and bolts issue since this is something Vista is especially criticized for: its RAM consumption.

Well congratulations on choosing one of the most conceptually complicated parts of the OS to start with. =) Smilin is right, you'll never really understand what's going on unless you do something like read the memory management chapter(s) in Inside Windows.

If computer A has twice as much RAM as computer B, it would seem logical that its "real RAM consumption" (not preempitive caching) should be half, as a percentage, computer B's.

But memory used for caching is still technically used.

Is it really just SuperFetch? Shouldn't Vista's RAM-usage meter ignore that since it's misleading?

Most likely yes and probaly not, no, respectively. There has to be a way to tell how much memory is being used for filesystem caching, if taskmgr ignored it then you'd have to use something like perfmon for that.

The "cache" entry in the system monitor is *NOT* the disk cache (superfetch). I forget exactly what it is, but that's definitely what it is not. The amount of disk cache = total memory - free memory - used memory.

So in the case of your first rig its 2048 - 27 - 1095 (1.07gb) = 926mb. You can verify this by adding the amounts of the three entries for disk cache in the perfmon, which I again, can't remember the names of off the top of my head (one begins with standby, I believe another is priority something).

As far as to the reasoning behind the differing amounts of RAM consumption, part of it is due to the screen resolution, drivers, background programs etc. It will even be affected by whether or not a program is minimized or not.

Like XP, Vista will also preemptively page out of memory things that may never be used at take up space that could be better used for disk cache, but won't be as proactive about this if there's memory to spare. There are also some processes such as explorer, etc, that will vary in RAM usage depending on memory conditions.

This is counter intuitive, but it's theoretically a good thing. It's called "memory management" for a reason. It's trying to utilize the limited resource as best it can. If the resource isn't quite as limited, it will be safer with it's management and not try to do you too many favors by assuming which pages are and aren't important. It would be downright silly for all three computers to use the same amount of memory - that'd be simple, but it wouldn't be efficient.

Rather than looking at total amounts, it'd be more telling to look at the amounts of memory used by the individual processes that are common to all three, such as explorer, lsass, crss, dwm etc.

People get WAY too caught up in how much memory so and so program uses, including the OS. The entire point of memory management, virtual memory and all that good stuff is to allow programs to take memory when they need it, and take away memory from programs that don't need it. Saying a program uses xMB of memory is a *VAST* oversimplification - a lot of memory is only used temporarily, shared between programs, reserved for future use, etc.

So don't sweat the numbers. Just know two things. Free memory = wasted memory, and more memory is ALWAYS better.
 

Lemon law

Lifer
Nov 6, 2005
20,984
3
0
The thing to understand is that what is currently in ram can be accessed around a thousand times faster than what is on the hard drive. Compared to hard disk access, memory is really fast.
Especially when you start talking about moving the read heads on a hard drive---where you talking many milliseconds---and memory access where you are talking nano seconds times a number of clock cycles. And there are a million nano seconds in a milli second.

Your danger in ram caching involves loading the wrong things into ram---and then having to load something else---and then going back and loading what you had before---I believe the term is called cache thrashing---but the more surplus ram you have---the less danger that poses.----but to work efficiently and predict user behavior, vista has to build up a data base of user behavior. Which is why computers sitting in stores being vista eye candy will not be accurate predictors.
 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
The "cache" entry in the system monitor is *NOT* the disk cache. I forget exactly what it is, but that's definitely what it is not. The amount of disk cache = total memory - free memory - used memory.

If that's true, and it's possible as there's a lot more than just filesystem caching going on, that only means that taskmgr is even more worthless.

(one begins with standby, I believe another is priority something).

The standby list is where memory pages go when the kernel notices you haven't used them in a while, it's the last step for them before being reused.
 

BD2003

Lifer
Oct 9, 1999
16,815
1
81
Originally posted by: Nothinman
The "cache" entry in the system monitor is *NOT* the disk cache. I forget exactly what it is, but that's definitely what it is not. The amount of disk cache = total memory - free memory - used memory.

If that's true, and it's possible as there's a lot more than just filesystem caching going on, that only means that taskmgr is even more worthless.

Taskmgr IS very misleading when it comes to giving an accurate picture of the memory situation. Not on purpose, but because it's using terminology that is easily confused and is misinterpreted by those with an oversimplified view of memory.

According to perfmon:
Cache Bytes is the sum of the Memory\\System Cache Resident Bytes, Memory\\System Driver Resident Bytes, Memory\\System Code Resident Bytes, and Memory\\Pool Paged Resident Bytes counters.

System Driver Resident Bytes is the size, in bytes, of the pageable physical memory being used by device drivers.

System Code Resident Bytes is the size, in bytes of the operating system code currently in physical memory that can be written to disk when not in use.

Pool Paged Resident Bytes is the current size, in bytes, of the paged pool. The paged pool is an area of system memory (physical memory used by the operating system) for objects that can be written to disk when they are not being used.

It has *NOTHING* to do with the disk cache. It's more a subset of used memory than anything else.

(one begins with standby, I believe another is priority something).

The standby list is where memory pages go when the kernel notices you haven't used them in a while, it's the last step for them before being reused.[/quote]

I have to look up the exact entires a bit later when I'm at home on my vista system. I'm pretty sure the standby (IIRC it's standby to begin with) I'm thinking of is not the one you're referring to. I went through every entry to try to figure out what was going on, and after a while I managed to sort it all out. A lot of the perfmon entries overlap, so it can be very confusing.

The one thing I did manage to figure out 100% for sure is that total - used - free = disk cache. I set it all up in samurize and along with perfmon I can literally watch upon boot what happens.

First it loads the OS and startup programs, and the amount of "free" memory = total - used.

Then it starts loading from all over the disk into the cache. The used memory stays the same, but the free memory begins to drop like a rock until it reaches near zero. If I monitor what it's actually loading off the disk, I'll notice all the programs I typically use, hell it even preloads a decent amount of world of warcraft I've been playing it so much. :p

After that's done, it will start loading up the readyboost cache. Which I believe (not 100% sure) is also managed similar to superfetch. Which is why they say it's best to have 1-2.5x the amount of flash as you have ram. If you have less flash than you have unused memory, a lot of it will be redundant.

 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
Taskmgr IS very misleading when it comes to giving an accurate picture of the memory situation. Not on purpose, but because it's using terminology that is easily confused and is misinterpreted by those with an oversimplified view of memory.

And because in places the terminology is just plain wrong, like the PF Usage graph in XP, I'm not sure if it's still named the same in Vista.

According to perfmon:
Cache Bytes is the sum of the Memory\\System Cache Resident Bytes, Memory\\System Driver Resident Bytes, Memory\\System Code Resident Bytes, and Memory\\Pool Paged Resident Bytes counters.

And System Cache Resident Bytes is defined as "Shows the size, in bytes, of pageable operating system code in the file system cache."
 

BD2003

Lifer
Oct 9, 1999
16,815
1
81
Originally posted by: Nothinman
Taskmgr IS very misleading when it comes to giving an accurate picture of the memory situation. Not on purpose, but because it's using terminology that is easily confused and is misinterpreted by those with an oversimplified view of memory.

And because in places the terminology is just plain wrong, like the PF Usage graph in XP, I'm not sure if it's still named the same in Vista.

I think taskman is still a bit off, but Vista does include the much better perfmon, which gives a much clearer picture of whats going on in your system.

According to perfmon:
Cache Bytes is the sum of the Memory\\System Cache Resident Bytes, Memory\\System Driver Resident Bytes, Memory\\System Code Resident Bytes, and Memory\\Pool Paged Resident Bytes counters.

And System Cache Resident Bytes is defined as "Shows the size, in bytes, of pageable operating system code in the file system cache."[/quote]

Yep, I forgot that. But that's only one part of the disk cache. I honestly don't understand what use to anyone that Cache bytes number is.
 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
Yep, I forgot that. But that's only one part of the disk cache. I honestly don't understand what use to anyone that Cache bytes number is.

Because people want to know where their memory went and if you can say "Look here, it's the filesystem cache." they'll usually go "Oh, ok." and forget about it. Of course there are the people that are so thick they'll continue asking how to limit the filesystem cache and things to keep more memory free, but you can't protect them from themselves forever. =)
 

BD2003

Lifer
Oct 9, 1999
16,815
1
81
Originally posted by: Nothinman
Yep, I forgot that. But that's only one part of the disk cache. I honestly don't understand what use to anyone that Cache bytes number is.

Because people want to know where their memory went and if you can say "Look here, it's the filesystem cache." they'll usually go "Oh, ok." and forget about it. Of course there are the people that are so thick they'll continue asking how to limit the filesystem cache and things to keep more memory free, but you can't protect them from themselves forever. =)

That would be a pretty silly reason, now wouldnt it? :p

My personal favorite is those silly little programs that "free memory" for you. :p

The more I think about that entry, it doesnt make sense to me. "Shows the size, in bytes, of pageable operating system code in the file system cache." Why would it leave something pageable in the file system cache in RAM, why not just keep it in resident "used" memory instead?
 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
That would be a pretty silly reason, now wouldnt it?

No, the whole point of presenting those statistics is so that you can tell what's using what.

Why would it leave something pageable in the file system cache in RAM, why not just keep it in resident "used" memory instead?

How about you buy a copy of Inside Windows and let us all know?
 

Matthias99

Diamond Member
Oct 7, 2003
8,808
0
0
Originally posted by: BD2003
The more I think about that entry, it doesnt make sense to me. "Shows the size, in bytes, of pageable operating system code in the file system cache." Why would it leave something pageable in the file system cache in RAM, why not just keep it in resident "used" memory instead?

Random guess (having not used Vista myself): it sounds like they are tracking blocks requested by (or prefetched for) the OS separately from blocks requested by (or prefetched for) user processes.

If that is the case -- no, it isn't particularly useful to most people, other than to get a breakdown of what the filesystem cache is actually being used for. The OS itself may use it to try to make better decisions on what to page out or replace.
 

BD2003

Lifer
Oct 9, 1999
16,815
1
81
Originally posted by: Matthias99
Originally posted by: BD2003
The more I think about that entry, it doesnt make sense to me. "Shows the size, in bytes, of pageable operating system code in the file system cache." Why would it leave something pageable in the file system cache in RAM, why not just keep it in resident "used" memory instead?

Random guess (having not used Vista myself): it sounds like they are tracking blocks requested by (or prefetched for) the OS separately from blocks requested by (or prefetched for) user processes.

If that is the case -- no, it isn't particularly useful to most people, other than to get a breakdown of what the filesystem cache is actually being used for. The OS itself may use it to try to make better decisions on what to page out or replace.

Now that I'm home on vista I can make a little more sense of it.

The three entires I was referring to that correspond to the superfetch cache are:
Standby Cache Core Bytes
Standby Cache Normal Priority Bytes
Standby Cache Reserve Bytes

In order of priority. Killing the superfetch process will stop it from reloading and clear those to near zero (but not entirely).

"Standby Cache Core Bytes" - seems to contain files that it really, really wants to keep in cache for you. No amount of pressure on memory seemed to lower this number. I can't tell whats in it, because it doesn't reload it so I can monitor it.
"Standby Cache Normal Priority Bytes" - seems to contain things you use often. It loaded a ton off my readyboost stick, and a good amount of microsoft office and WoW.
"Standby Cache Reserve Bytes" - Appears to contain data that isn't as crucial. It loaded a massive chunk of WoW, about 400mb of a movie I watched this morning. If I had to guess, I'd say this part of superfetch was most similar to XP's cache - it keeps things you recently use, whether you do it often or not.

The "System Cache Resident Bytes" isnt part of superfetch - it exists in XP as well. If I understand it correctly, it's dlls and other good stuff. It's definitely the last in line to go from my testing, but put enough pressure on it, and it'll start to shrink. As far as I can tell, its considered both used memory and cache at the same time, which sorta kinda makes sense when you think about what it is.

The taskman cached entry in *Vista* displays the sum of these 4. But taskman is still misleading..it's double dipping somewhere.

Right now, I have according to taskman.
710mb used
1456mb cached
53mb free
=
2219mb.

Out of 2045 mb.

The System Cache Resident Bytes at 86mb doesnt make up this difference, so it's more than just that where it's double dipping, but I've no idea where.
 
Aug 23, 2004
84
0
0
Originally posted by: Nothinman
Taskmgr IS very misleading when it comes to giving an accurate picture of the memory situation. Not on purpose, but because it's using terminology that is easily confused and is misinterpreted by those with an oversimplified view of memory.

And because in places the terminology is just plain wrong, like the PF Usage graph in XP, I'm not sure if it's still named the same in Vista.

In Vista the "PF usage" graph has been replaced by a memory usage one.
 
Aug 23, 2004
84
0
0
Originally posted by: BD2003
The taskman cached entry in *Vista* displays the sum of these 4. But taskman is still misleading..it's double dipping somewhere.

Right now, I have according to taskman.
710mb used
1456mb cached
53mb free
=
2219mb.

Out of 2045 mb.

The System Cache Resident Bytes at 86mb doesnt make up this difference, so it's more than just that where it's double dipping, but I've no idea where.

So I'm not completely crazy. :)
 

BD2003

Lifer
Oct 9, 1999
16,815
1
81
Originally posted by: RobertMcDonald
Originally posted by: BD2003
The taskman cached entry in *Vista* displays the sum of these 4. But taskman is still misleading..it's double dipping somewhere.

Right now, I have according to taskman.
710mb used
1456mb cached
53mb free
=
2219mb.

Out of 2045 mb.

The System Cache Resident Bytes at 86mb doesnt make up this difference, so it's more than just that where it's double dipping, but I've no idea where.

So I'm not completely crazy. :)

No, you're not...it's double dipping, yet its somewhat accurate. System Cache Resident Bytes can be considered active code and cache at the same time, depending on how you look at it.

I've also noticed that after a night of browing photos, general system usage etc, the active memory usage will rise, but overnight it will fall. The only explanation is that vista is doing it's thing, and using memory when useful, and curbing it when not.
 

Nothinman

Elite Member
Sep 14, 2001
30,672
0
0
In Vista the "PF usage" graph has been replaced by a memory usage one.

True, although neither the numbers in that graph or the one in the Page File section under System match up with the numbers provided by perfmon.