How to flush a Windows disk cash?

[Carson Dyle]

I considered asking this in several other forums, but this is a programming and software testing question in most respects.

I want to test the performance of a software package that reads and indexes tens of thousands of files (music files of various types). It's been shown that disk caching of the files has a huge impact on the timings that are found during testing.

How can I flush the Windows (different flavors) OS cache of the files read from disk, as well as any hardware cashes? I need to do this from the command-line, so that it can be dropped into a testing script.

 

[Cogman]

http://msdn.microsoft.com/en-us/library/windows/desktop/aa364218(v=vs.85).aspx

In short, the application doing the reading/writing needs to interact with the Win32 api and needs to explicitly disable disk read caching.

 

[Carson Dyle]

Hmmm. I'm wondering if the best and simplest approach might be to just reboot the machine between each run of the test.

 

[Cogman]

That might not be best. Windows is smart. On startup it will cache most frequently used files. Rebooting the system may eventually get you to the point where windows caches the files for you.

If you want good results on performance of the application, your best bet is to run the application multiple times until the results become stable. (caching be damned). If you want to benchmark performance, that is the best way to do it.

If you want to see how fast your HD is, use actual HD benchmarking tools. Using something not designed for benchmarking an HD to benchmark an HD will only end poorly for you.

 

[Carson Dyle]

The times do seem to stabilize after the first run, which takes longest. But I'm wondering - the files in question for the recent tests are 40,000 in number and take about 1.25 TB of disk space. How much of that could Windows actually cache? And given that each is hit exactly once per test, there aren't any of these files that should register as "frequently" used.

 

[Cogman]

The times do seem to stabilize after the first run, which takes longest. But I'm wondering - the files in question for the recent tests are 40,000 in number and take about 1.25 TB of disk space. How much of that could Windows actually cache? And given that each is hit exactly once per test, there aren't any of these files that should register as "frequently" used.

So my question, what are you trying to get out of this? If you get a good, solid, stable number after the first run, why do you care if windows caching, or potentially some other mechanism, is coming into play. What is this test trying to accomplish and measure?

The worst kind of test I can think of is one that requires a reboot of the system. You're times will be unstable, and you will have to time the test to run when the system has stabilized (isn't loading anything new up).

 

[DaveSimmons]

Rebooting will affect more than just the cache. It might clear some caching but any "check for update"services (OS, AV, Adobe programs, etc.) and starting up services will take a variable amount of time. For 8.x Windows might also start downloading updates in the background.

 

[imagoon]

The times do seem to stabilize after the first run, which takes longest. But I'm wondering - the files in question for the recent tests are 40,000 in number and take about 1.25 TB of disk space. How much of that could Windows actually cache? And given that each is hit exactly once per test, there aren't any of these files that should register as "frequently" used.

Considering Windows caches based on blocks and not the files themselves, if the app it just reading the meta data say something like ID3 or what ever that is about 256bytes of data, potentially everything that the app needs to touch. A 64kb block size with 40,000 files is only 2.5gb so assuming the RAM is there it would use it.

I am also a bit curious what your goal is here as it doesn't make a whole lot of sense to disable caching.

 

[Leros]

It seems to me that you might want to disable disk cache for certain applications because one application is going to read a ton of stuff from disk once (so it won't benefit from the caching) and enabling disk cache would result in a bunch of stuff getting evicted from the disk cache decreasing performance for other applications.