Surprising caching effect in Windows. How

[wsaenotsock]

I just observed an odd scenario where Windows had unexpectedly cached some data, I was very surprised and don't know exactly how it worked. Here's what happened:

-I viewed (in VLC, a media player) a 3.5GB video file from a USB device, which only has a transfer rate of 25MB/s.

-I then closed this file and application and performed a copy operation of this file to my local disk. It copied basically instantly, which would normally take a few minutes at the given transfer rate.

-To verify that this was a result of caching, I tried copying a different large file which I did not view beforehand. It took the normal amount of time to copy.


What surprises me about this, is that Windows must either have:

a) Not unloaded this file from RAM, after the application closed, and then knew to reference the RAM to copy it.

or

b) When the file was viewed from the USB device, it was actually copied to a temp file and cached from there. This file was being buffered in the background the whole time the viewer was open. Windows knew to reference this temp file when the copy operation occured.


This is some sophisticated write caching at work. It should mean then, that any data accessed off a USB device is cached for the duration it is connected.

 

[SOFTengCOMPelec]

That sounds quite normal to me. When things are automatically cached to memory, windows (or any other relatively low security) operating system, might as well keep cached stuff already in ram, until either the ram is needed for something else, or so much time has passed there is too much danger of the data being invalid.

 

[Tuna-Fish]

a) Not unloaded this file from RAM, after the application closed, and then knew to reference the RAM to copy it.

That's exactly what happened. When something is read from any device, a cache page is allocated, the contents are DMA'd into it, and this page is given (with read-only access) to the program. There is no reason to release this page until the ram is actually needed again, so any further accesses are satisfied from the cache. When ram is needed, clean pages like this can be dropped and repurposed without having to do anything to them, because the data is still available on the device.

This is some sophisticated write caching at work. It should mean then, that any data accessed off a USB device is cached for the duration it is connected.

If by sophisticated you mean: "Everyone but Windows has been doing it like this since the 70's." Not kidding.

Windows finally stopped dropping pages unnecessarily in Vista. This is one of the baby steps they did to drag the NT kernel towards modernity. It's the reason why all the internet flame wars about Vista taking so much memory started.

 

[Tuna-Fish]

That sounds quite normal to me. When things are automatically cached to memory, windows (or any other relatively low security) operating system, might as well keep cached stuff already in ram, until either the ram is needed for something else, or so much time has passed there is too much danger of the data being invalid.

There is no danger of the data being invalid. This is perfectly safe, as the operating system manages writes to the device and can verify that all caches remain correct.

 

[SOFTengCOMPelec]

There is no danger of the data being invalid. This is perfectly safe, as the operating system manages writes to the device and can verify that all caches remain correct.

There is a thread about it on these forums, I can dig it out if you want.

Windows periodically invalidates/erases/stops using old ram cached stuff. There are details in the book (mentioned in the other thread), which explain how its timed mechanism works. If I remember correctly, the main reason is for security.

EDIT: http://forums.anandtech.com/showthread.php?t=2316212

But anyway, I could easily be wrong about it. On re-reading the thread, it looks like I may have been mixed up between different types of windows ram usages.
I don't have the book, and have never read it, so I will happily stand corrected.

 

[Tuna-Fish]

Windows periodically invalidates/erases/stops using old ram cached stuff. There are details in the book (mentioned in the other thread), which explain how its timed mechanism works. If I remember correctly, the main reason is for security.

EDIT: http://forums.anandtech.com/showthread.php?t=2316212

But anyway, I could easily be wrong about it. On re-reading the thread, it looks like I may have been mixed up between different types of windows ram usages.
I don't have the book, and have never read it, so I will happily stand corrected.

That's not what it's doing. That's just clearing pages in the free set. It doesn't move pages from cache set to the free set.

 

[SOFTengCOMPelec]

That's not what it's doing. That's just clearing pages in the free set. It doesn't move pages from cache set to the free set.

I agree, (as I tried to explain in my previous post), I had misinterpreted the information in the thread I linked to above.

 

[tweakboy]

The more RAM you have the more you can cache and speedier system. gl