NTFS Compression

[WobbleWobble]

When Windows reads an NTFS compressed file, does it need to uncompress it onto disk before reading it?

 

[Smilin]

I was under the impression it was all done in memory. That could equate to disk usage for large files since the pagefile may get used.

You'll often have plenty of freespace anyway. When compressed files are written, they are written first then compressed.

 

[WobbleWobble]

I thought about it and I think I'm asking the wrong question.

Scenario: Reading large NTFS compressed text logs versus uncompressed to be processed for web analytics

Assuming NTFS compressed files are compressed down to 10%, disk activity (assuming enough memory) should be 10% of what it would be versus reading an uncompressed file. Disk is the slowest part of the equation right? Yes, it takes some CPU time to uncompress the file but CPUs are fast enough so impact on CPU should be minimal. But the file still needs to go to the CPU so in effect, we're going back to a PIO-like mode rather than a DMA-like mode. Is there a performance impact there in getting the file read? Like I said, assume the CPU isn't being a bottleneck but it's part of the pathway in uncompressing the file.

 

[bsobel]

Originally posted by: WobbleWobble
When Windows reads an NTFS compressed file, does it need to uncompress it onto disk before reading it?

No, it is an in memory operation. Your third post is correct, you trade disk IO for cpu IO. In many cases this is a good trade.

 

[WobbleWobble]

But does needing to access the CPU slow down the operation?

I haven't experimented with this in a while, but going from PIO mode to DMA mode made disk transfer speed significantly faster. Were CPUs bottlenecks back then or does the path it takes just that much slower? Or was it just interface transfer speeds? I think PIO was capped at 16MB/s

 

[bsobel]

Originally posted by: WobbleWobble
But does needing to access the CPU slow down the operation?

Generally speaking, if your cpu has headroom (e.g. it isn't already maxed out) then NTFS compression will be faster than the equivalant disk IO. With PIO mode, you had a alot of interupt overhead to factor in.