SSD RAID and PERC Cache: Oddity with Read Ahead

[SketchMaster]

I have a PERC h700 with four (cheap) 128GB SSDs in RAID 0 (because it seemed like fun to try). Today I decided to play around with cache settings to see how much the SSDs benefit from it.

This is what I found:

Write Through, Read Ahead:
-----------------------------------------------------------------------
CrystalDiskMark 5.1.2 x64 (C) 2007-2016 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 1216.747 MB/s
Sequential Write (Q= 32,T= 1) : 1521.770 MB/s
Random Read 4KiB (Q= 32,T= 1) : 357.083 MB/s [ 87178.5 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 220.209 MB/s [ 53762.0 IOPS]
Sequential Read (T= 1) : 1088.565 MB/s
Sequential Write (T= 1) : 1026.841 MB/s
Random Read 4KiB (Q= 1,T= 1) : 42.154 MB/s [ 10291.5 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 46.398 MB/s [ 11327.6 IOPS]

Test : 1024 MiB [E: 83.2% (370.3/445.0 GiB)] (x5) [Interval=5 sec]
Date : 2016/08/28 12:57:10
OS : Windows 10 Professional [10.0 Build 14393] (x64)

Write Through, Adaptive Read:
-----------------------------------------------------------------------
CrystalDiskMark 5.1.2 x64 (C) 2007-2016 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 1301.205 MB/s
Sequential Write (Q= 32,T= 1) : 1518.119 MB/s
Random Read 4KiB (Q= 32,T= 1) : 355.963 MB/s [ 86905.0 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 218.520 MB/s [ 53349.6 IOPS]
Sequential Read (T= 1) : 884.402 MB/s
Sequential Write (T= 1) : 1019.686 MB/s
Random Read 4KiB (Q= 1,T= 1) : 42.460 MB/s [ 10366.2 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 43.953 MB/s [ 10730.7 IOPS]

Test : 1024 MiB [E: 83.2% (370.3/445.0 GiB)] (x5) [Interval=5 sec]
Date : 2016/08/28 13:12:03
OS : Windows 10 Professional [10.0 Build 14393] (x64)

Write Through, No Read ahead:
-----------------------------------------------------------------------
CrystalDiskMark 5.1.2 x64 (C) 2007-2016 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 948.613 MB/s
Sequential Write (Q= 32,T= 1) : 1495.216 MB/s
Random Read 4KiB (Q= 32,T= 1) : 492.761 MB/s [120303.0 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 522.755 MB/s [127625.7 IOPS]
Sequential Read (T= 1) : 822.362 MB/s
Sequential Write (T= 1) : 1261.691 MB/s
Random Read 4KiB (Q= 1,T= 1) : 30.069 MB/s [ 7341.1 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 106.234 MB/s [ 25936.0 IOPS]

Test : 1024 MiB [E: 83.2% (370.3/445.0 GiB)] (x5) [Interval=5 sec]
Date : 2016/08/28 17:22:35
OS : Windows 10 Professional [10.0 Build 14393] (x64)

*****************************

Suddenly the write speeds are through the roof with read caching disabled. I ran the test multiple times to verify it wasn't a fluke. I'm stumped as to why this could happen. Write through is enabled, so writes shouldn't be getting held up by read data left in the cache. I'm hoping someone with a better understanding of storage can shine some light on this. The SSD RAID is a gaming/scratch disk, so if we want to play around with it I can blow it away and rebuild it with different settings. This may be a fun experiment...

Mods, please feel free to move this into storage if you feel it belongs there.

 

[XavierMace]

Off the top of my head, I don't remember the reason why, but I do know that write through/no read ahead is the recommended settings when using SSD's on those cards.