Did some benchmarking of an SSD caching solution

[CatMerc]

The Ryzen FuzeDrive got me interested, and so I went ahead and downloaded PrimoCache to check out what sort of boosts I can get from a combined SSD+HDD setup. I don't care much about synthetic tests, I'm more about real world tests that match my workloads, so don't expect to see Crystal Disk Mark here.

Anyhow, here are my system specs:
Ryzen 7 1700 SMT Off @ 3.9GHz
2x8GB G.Skill TridentZ at 3200MHz CL14
Gigabyte AX370 Gaming K7
EVGA GTX 1080 Ti SC2
OS Drive: 500GB 960 EVO
Other drives:
120GB Sandisk Plus SSD <-- Used as cache
240GB Transcend SSD
2TB WD Blue
1TB WD Black <-- Used as main storage in tests

The RAM Cache test uses 2GB of RAM dedicated to speeding the HDD up.
The All Cache test uses 2GB of RAM + 112GB of SSD cache for speeding up the HDD.
In both cases the block size was set to 32KB, which induces a 500MB of overhead on RAM.

Testing methodology:
I picked 3 games, being The Witcher 3, Rainbow Six Siege, and Battlefield 4. For the two former I picked three saved games/situations, and on the latter a multiplayer map and singleplayer level, to simulate loading different levels of the same game, since in theory a lot of data should be shared between them, leading to caching algorithms to do a speedup even on a first load of a different map after you loaded a different game level.

For these tests, I ran three different setups. One being just pure HDD loading and letting the OS do its own caching and prefetching. After that I cleared the RAM of standby memory to reset any caching the OS did, and used just the RAM Cache to check for speedups there. Then finally, I did the all cache test, using the RAM as L1 cache and SSD as L2 cache. Finally after all of that I moved the games to the SSD and checked the performance loading purely from the SSD. Battlefield 4 wasn't included in the pure SSD tests both due to time reasons, and for reasons that will be clear in the data.

Now the point of this entire thing is to save on precious SSD storage, so I recorded how much data was being cached in the SSD at any given test, to compare with just installing the game on the SSD.

Results:

Quick Analysis:
Looking at the pure HDD results, we can see the OS caching in action. Windows 10 makes use of unused memory, and caches things in standby memory to speedup future accesses. It is obviously quite effective here, as can be seen in the Witcher 3 and Battlefield 4 test results after the first run, being within margin of error with the full cache result (and SSD for Witcher 3).
It isn't nearly as effective however with Rainbow Six Siege. At best load times are reduced by 20%, which is a far cry from the 2.8x and 5.2x(!) speedup on Witcher 3 and Battlefield 4, respectively.
Important to note that these speedups only last until system restart or standby memory is evicted in favor of something else.

Moving on to the RAM Cache, there were no observable speed ups with The Witcher 3 and Battlefield 4, however Rainbow Six Siege did get boosted slightly. About a 16% speedup for the first run, and up to 35% for subsequent runs.

And finally, using the entire caching scheme, here things get interesting.
There was a small speedup in the initial Witcher 3 first run, but it's really nothing worth discussing.
On subsequent runs, we can see the full cache scheme being within margin of error with the rest of the results. Looking at the cached data usage after all 9 runs, it was just 2.23GB! And we were getting the full speedup of an SSD here. The full game install is 51GB for perspective.

Moving on to Rainbow Six Siege, the first load isn't any faster than the RAM cache result, however subsequent loads of the same level show the full speedup of an SSD. After that, my theory of game asset sharing is in effect, and we can see loading different levels for the first time sits between SSD and RAM Cache in performance, and significantly below the base HDD result, showing a 1.733x speedup before the system even got the chance to fully cache everything. Subsequent loads of the same level show a 2.25x speedup over a mechanical drive. After all 9 runs, the total storage use (Which includes the Witcher 3 runs), went up to 6.21GB. The two games together take up 90GB to fully install.

Finally, with Battlefield 4, while I didn't test the SSD due to time constraints, it should be fairly obvious that the caching scheme is highly effective and after the first run, saves significant time equaling an SSD for subsequent loads, without costing much storage. Total usage went up to 8.23GB. Having all three games installed would cost 158GB of storage. More than this SSD can hold, and about 19x more than the cached result. To be fair, I didn't load ALL levels, however from testing it can be seen that the main bump in storage requirements comes from the very first map, and subsequent loads don't add much. I expect after everything is cached that the result would be about a 10x storage efficiency increase, down from the 19x.

Now at first glance it might seem like aside from Rainbow Six, the HDD and SSD provide similar performance after first load. However what I mentioned previously is that the OS cache is far less "sticky". It can be lost from a restart, or even just using the computer for long enough that the cache is evicted from memory. Same can be said for the SSD cache, however the huge difference in capacity means it can hold far more before having to evict anything. And of course nothing is lost after a restart.

To make sure, I evicted everything from memory and tested the effectiveness of the cache. As can be seen, the performance matches the pure SSD runs from the very first run (or arguably slightly faster, depending on the margin of error).

Summary and tl;dr
From my testing I conclude that an SSD caching scheme is far more efficient use of your SSD storage space. Aside from not having to manage your storage tiers manually, it provides most of the SSD benefit after the first level of a game loads, even when loading different levels. When you load the same level again, you get pure SSD performance, with significantly less storage space used up.

+1 to PrimoCache, I'll keep using, and probably buy it past the 60 day trial.
Alternative software would be VeloSSD, however I didn't try it myself and therefore don't know anything about it.
Intel users can look here: https://www.intel.com/content/www/us/en/software/intel-cache-acceleration-software-performance.html

 

[BonzaiDuck]

The Ryzen FuzeDrive got me interested, and so I went ahead and downloaded PrimoCache to check out what sort of boosts I can get from a combined SSD+HDD setup. I don't care much about synthetic tests, I'm more about real world tests that match my workloads, so don't expect to see Crystal Disk Mark here.

Anyhow, here are my system specs:
Ryzen 7 1700 SMT Off @ 3.9GHz
2x8GB G.Skill TridentZ at 3200MHz CL14
Gigabyte AX370 Gaming K7
EVGA GTX 1080 Ti SC2
OS Drive: 500GB 960 EVO
Other drives:
120GB Sandisk Plus SSD <-- Used as cache
240GB Transcend SSD
2TB WD Blue
1TB WD Black <-- Used as main storage in tests

The RAM Cache test uses 2GB of RAM dedicated to speeding the HDD up.
The All Cache test uses 2GB of RAM + 112GB of SSD cache for speeding up the HDD.
In both cases the block size was set to 32KB, which induces a 500MB of overhead on RAM.

Testing methodology:
I picked 3 games, being The Witcher 3, Rainbow Six Siege, and Battlefield 4. For the two former I picked three saved games/situations, and on the latter a multiplayer map and singleplayer level, to simulate loading different levels of the same game, since in theory a lot of data should be shared between them, leading to caching algorithms to do a speedup even on a first load of a different map after you loaded a different game level.

For these tests, I ran three different setups. One being just pure HDD loading and letting the OS do its own caching and prefetching. After that I cleared the RAM of standby memory to reset any caching the OS did, and used just the RAM Cache to check for speedups there. Then finally, I did the all cache test, using the RAM as L1 cache and SSD as L2 cache. Finally after all of that I moved the games to the SSD and checked the performance loading purely from the SSD. Battlefield 4 wasn't included in the pure SSD tests both due to time reasons, and for reasons that will be clear in the data.

Now the point of this entire thing is to save on precious SSD storage, so I recorded how much data was being cached in the SSD at any given test, to compare with just installing the game on the SSD.

Results:

Quick Analysis:
Looking at the pure HDD results, we can see the OS caching in action. Windows 10 makes use of unused memory, and caches things in standby memory to speedup future accesses. It is obviously quite effective here, as can be seen in the Witcher 3 and Battlefield 4 test results after the first run, being within margin of error with the full cache result (and SSD for Witcher 3).
It isn't nearly as effective however with Rainbow Six Siege. At best load times are reduced by 20%, which is a far cry from the 2.8x and 5.2x(!) speedup on Witcher 3 and Battlefield 4, respectively.
Important to note that these speedups only last until system restart or standby memory is evicted in favor of something else.

Moving on to the RAM Cache, there were no observable speed ups with The Witcher 3 and Battlefield 4, however Rainbow Six Siege did get boosted slightly. About a 16% speedup for the first run, and up to 35% for subsequent runs.

And finally, using the entire caching scheme, here things get interesting.
There was a small speedup in the initial Witcher 3 first run, but it's really nothing worth discussing.
On subsequent runs, we can see the full cache scheme being within margin of error with the rest of the results. Looking at the cached data usage after all 9 runs, it was just 2.23GB! And we were getting the full speedup of an SSD here. The full game install is 51GB for perspective.

Moving on to Rainbow Six Siege, the first load isn't any faster than the RAM cache result, however subsequent loads of the same level show the full speedup of an SSD. After that, my theory of game asset sharing is in effect, and we can see loading different levels for the first time sits between SSD and RAM Cache in performance, and significantly below the base HDD result, showing a 1.733x speedup before the system even got the chance to fully cache everything. Subsequent loads of the same level show a 2.25x speedup over a mechanical drive. After all 9 runs, the total storage use (Which includes the Witcher 3 runs), went up to 6.21GB. The two games together take up 90GB to fully install.

Finally, with Battlefield 4, while I didn't test the SSD due to time constraints, it should be fairly obvious that the caching scheme is highly effective and after the first run, saves significant time equaling an SSD for subsequent loads, without costing much storage. Total usage went up to 8.23GB. Having all three games installed would cost 158GB of storage. More than this SSD can hold, and about 19x more than the cached result. To be fair, I didn't load ALL levels, however from testing it can be seen that the main bump in storage requirements comes from the very first map, and subsequent loads don't add much. I expect after everything is cached that the result would be about a 10x storage efficiency increase, down from the 19x.

Now at first glance it might seem like aside from Rainbow Six, the HDD and SSD provide similar performance after first load. However what I mentioned previously is that the OS cache is far less "sticky". It can be lost from a restart, or even just using the computer for long enough that the cache is evicted from memory. Same can be said for the SSD cache, however the huge difference in capacity means it can hold far more before having to evict anything. And of course nothing is lost after a restart.

To make sure, I evicted everything from memory and tested the effectiveness of the cache. As can be seen, the performance matches the pure SSD runs from the very first run (or arguably slightly faster, depending on the margin of error).

Summary and tl;dr
From my testing I conclude that an SSD caching scheme is far more efficient use of your SSD storage space. Aside from not having to manage your storage tiers manually, it provides most of the SSD benefit after the first level of a game loads, even when loading different levels. When you load the same level again, you get pure SSD performance, with significantly less storage space used up.

+1 to PrimoCache, I'll keep using, and probably buy it past the 60 day trial.
Alternative software would be VeloSSD, however I didn't try it myself and therefore don't know anything about it.
Intel users can look here: https://www.intel.com/content/www/us/en/software/intel-cache-acceleration-software-performance.html

Interesting. I'm lazy about defining and pursuing meaningful tests beyond what the benchmark scores show. So this just offers more confidence for me about the decision I made to purchase a 3-pc and 1-pc license for the systems we have here. I could even investigate the server version, even for the extra expense. My new server will have 16GB of DDR3. I'm quite sure I can use half of that for a RAM cache. With a drive pool, I should have to create a caching task to cover four storage drives, and not the virtual drive defined by the pool.

I had looked into VeloSSD. I can't remember precisely what ended my attention to it. I can't remember if VeloSSD provided both SSD-caching and RAM-caching. I know for sure that SuperCache only offers the RAM option.

 

[XavierMace]

First off, it's nice to see somebody finally post useful numbers rather than it feels faster. Not naming any names, but.....

However, I'm going to have to disagree with your conclusion. First off, your first run scores put both caching solutions closer to the spindle base numbers than the SSD base numbers. That's to be expected but that doesn't mean that number can just be dismissed as that's always going to be your first impression. Secondly, you're dedicating a 120Gb SSD for caching. How is that more efficient than just installing two of those games on the SSD? Thirdly, caching will wear out your SSD faster as it's constantly doing writes to it at a far faster rate than it would be doing if it was being used as a normal drive. Lastly, at least in your case, you've got almost 1TB of flash storage in your system anyways. I'd argue the more efficient solution would be to just install the games you're actively playing on the SSD.

If you tested further to simulate more real world tests, such as jumping between applications, the numbers are just going to be worse. The price of PrimoCache is the cost difference between a 128Gb and 256Gb SSD. If you're the type of person who basically only plays one or two games at a time, a 256Gb SSD is enough to allow you do that and get the fast speeds ALL the time. If you're the type of person who likes to jump around between a bunch of games at once, then that's going to decrease the effectiveness of your caching.

Currently I'm running a single 512Gb SSD in my system. I've got 9 games installed with 200Gb space free. Obviously some games are far larger than others. But I can uninstall and reinstall games in 20 minutes. I'd far rather do that when I get bored of a game.

 

[BonzaiDuck]

Well, I may have already said it. It all depends on what extra resources you want to buy and use. I'll bet a lot of people settle for 8GB RAM in their systems; others buy the 2x8 kits. I spent maybe $180 on my second 2x8 kit, when the first had cost me about $140. I think the 256GB 960 EVO I've deployed for caching was about $130. I have way more storage space than I need on my 1TB NVMe, and way more than I need on a Crucial MX300 SATA.

But I can look at the hit rates reported in real time by Primo. Your usage profile may have a lot to do with performance. But I don't have to run a 2x or 4x NVMe storage system in RAID. I don't need any striped RAID configurations. My hit rates for RAM-cached NVMe hovers around 70%, and my hit rate for the SSD-cached SATA drive is around 70%, but often pushing above 90%.

I think the RAM-caching adds something more, but unless I want or need another PCIE slot for something different, I'm not missing anything.

Looking at Magician's reports, the EVO has been in the system for about seven months, and it's racked up 2.9 TBW. That's not a lot of wear and tear. The other drive -- a 960 Pro cached to RAM -- has passed 11 TBW, But part of that arose from rewriting the entire NVMe drive two or three times when I was struggling with the dual-boot troubleshooting. It was probably around 10 TBW in the middle of last year.

 

[CatMerc]

First off, it's nice to see somebody finally post useful numbers rather than it feels faster. Not naming any names, but.....

However, I'm going to have to disagree with your conclusion. First off, your first run scores put both caching solutions closer to the spindle base numbers than the SSD base numbers. That's to be expected but that doesn't mean that number can just be dismissed as that's always going to be your first impression. Secondly, you're dedicating a 120Gb SSD for caching. How is that more efficient than just installing two of those games on the SSD? Thirdly, caching will wear out your SSD faster as it's constantly doing writes to it at a far faster rate than it would be doing if it was being used as a normal drive. Lastly, at least in your case, you've got almost 1TB of flash storage in your system anyways. I'd argue the more efficient solution would be to just install the games you're actively playing on the SSD.

If you tested further to simulate more real world tests, such as jumping between applications, the numbers are just going to be worse. The price of PrimoCache is the cost difference between a 128Gb and 256Gb SSD. If you're the type of person who basically only plays one or two games at a time, a 256Gb SSD is enough to allow you do that and get the fast speeds ALL the time. If you're the type of person who likes to jump around between a bunch of games at once, then that's going to decrease the effectiveness of your caching.

Currently I'm running a single 512Gb SSD in my system. I've got 9 games installed with 200Gb space free. Obviously some games are far larger than others. But I can uninstall and reinstall games in 20 minutes. I'd far rather do that when I get bored of a game.

First run scores are obviously as expected, but I'm not at all worried about it. As I said, since games share data, loading a different level of the same game would still be quite a bit snappier than an HDD if the memory got cleared for whatever reason (Which happens all the time). A restart and or simply cache rolling will remove the speedup.

As for efficiency, I did mention how much data is used by the caching compared to how much it would cost to just install these games on the SSD. Since not all data is hot enough for the caching algorithm to move it to the SSD, it's more selective in that regard. Meaning the SSD space is used more efficiency. As you can see, even after a restart clearing the standby memory, performance was still on par with an SSD despite using a fraction of the storage.

As for wearing out the drive, I'd argue the low volume of writes compared to just installing the games, let alone when you hop through a lot of games requiring you to constantly juggle them around, is actually easier on the drive. Besides, it's just not a factor for desktop users IMO. SSD's die in data centers and workstations, not from average desktop workloads. This SSD was my main OS drive for two years, was benchmarked to hell and back, and had games constantly moved around it. S.M.A.R.T reports 19TB of writes. At this rate it will last me a decade with the conservative estimate of modern SSD lifetime.

This is actually better for game hoppers, since less data is required per game to get the SSD speedup. Yes the initial first load will be slower, but after that it will be a looooong time until that data gets replaced with the 120GB of cache I dedicated.

 

[BonzaiDuck]

With me, having tried RAID0 on three or four different controller models, I wanted to try something different that would still widen the bottleneck of mass storage at the bottom of the pyramid. When I picked my Skylake parts, I was a bit slow in coming to terms with the NVME option, but by the time I was ready to put it all together, I couldn't resist buying the 960 EVO just to experiment with an NVME cache. I set it aside when I totally lost my fiscal discipline and counted out six Franklins for the 960 Pro 1TB. For a while, I was caching to a volume at the end of the boot system disk, and that worked. But I was then a bit anxious for the effect XavierMace postulates, and decided to put the EVO back in exclusively for caching.

It was just a matter of my curiosity about how NVME M.2 might be used in addition to just availing of its native speed. I remember several people thinking quite rightly that you wouldn't miss the difference between SATA speed at 500 MB/s and NVME speed at ~3,000 MB/s.

It always seemed to me that storage slowed things down. That's why RAID0 became "fashionable" among a wider group of enthusiasts for a while.

Incidentally. The RAM caching can be made to be semi-persistent. You can arrange to have the contents saved at Shutdown or Restart and reloaded at boot time. For some reason, that's really not accelerating TBW as much as I thought it would.