SSD RAID0 TRIM is causing major problems!

[tlzmazaite]

Hi all,

I've been running an SSD RAID0 configuration for some time now (standard Z77 + RST). One of the things I've noticed since switching to RAID0 is the 'aggressiveness' of TRIM. I never noticed this while running the single SSD, it seems to be more obvious with the two in RAID.

If I delete a large file from the SSD the TRIM command is executed as a single uninterruptable operation from start to finish. It totally blocks up the ATA command queue. Any application requiring disk access effectively freezes up while waiting for this TRIM operation to finish.

I deleted a 6GB file and this froze up most of the system for over a minute while it was trimmed off.

Why does all of the data have to be TRIM'd off as a single operation? Can't it do small chunks of blocks at a time, when idle? Why does it have to be done immediately and all in one go?

There's no point me having TRIM enabled if I can't solve this, I may as well switch to RST 12.5 - performance was actually better overall, despite obviously taking a write performance hit over unconditioned blocks.

Any ideas folks?

 

[Cerb]

Why does all of the data have to be TRIM'd off as a single operation? Can't it do small chunks of blocks at a time, when idle? Why does it have to be done immediately and all in one go?

Because the drives must be kept in sync, and TRIM's implementation was half-assed. They're now pushing through a deterministic TRIM variant, to take care of just these kinds of issues, but how many years will that take to get into drives, and why wasn't it thought of initially?

TRIM, as it exists, makes no guarantee of the state of the blocks. So, in a RAID, it is advantageous (and, in the case of RAID 1, 10, 5, etc., necessary for sanity) to, if any TRIM be performed at all, execute TRIM for all outstanding blocks, before allowing anything any further commands (so, whether there is support for, or use, of queued TRIM or not, Intel probably wants to err on the paranoid side).

Now, taking over a minute was not simply TRIM. But, performing TRIM over many blocks quickly could have triggered some panic mode in your SSDs, much like some reviewers' crazy stress-tests have been able to, that might have resulted in many small GC cycle pauses.

Any ideas folks?

Don't use Sandforce drives for anything but single-disk desktops, pay attention to steady-state performance when researching your next SSD purchase, and over-provision a bit, for a RAID 0 volume.

 

[tlzmazaite]

Thanks for the info.

I'm only really running them in RAID because I needed extra SSD space, it was cheaper to buy a 2nd matching SSD and simple to put them in RAID rather than buy a whole new larger drive, but next time I'll be sure to look into more detailed options if I intended to continue down the RAID path. The extra performance of course is nice too!

I've figured as a solution for now, set DisableDeleteNotify to 1 for normal usage (fsutil behavior set disabledeletenotify 1). Once a week or two I can temporarily set DisableDeleteNotify to 0 and run the Windows 8 drive optimiser which TRIMs out the SSD partitions in one big pass. That should keep the performance relatively good and means I don't suffer from freezing when deleting large files on a daily basis.

e.g.

@echo off
fsutil behavior set disabledeletenotify 0
defrag /O /V C: F:
fsutil behavior set disabledeletenotify 1

 

[KentState]

Thanks for the post. I've ran into this before where everything seems to look up for an extended period of time.

 

[johny12]

I found this presentation from the Flash Memory Summit on why Sandforce SSDs are good in RAID environments. http://www.lsi.com/downloads/Public/Flash Storage Processors/LSI_PRS_FMS2012_TE21_Smith.pdf

 

[Mark R]

Thanks for the post. I've ran into this before where everything seems to look up for an extended period of time.

TRIM by its nature must lock the drive while it is in operation.

Because the drive is allowed to accept multiple commands, and then re-order reads and write operations, there are data corruption issues, or security issues (and ineffective TRIM issues) if writes/reads are re-ordered around a TRIM. This re-ordering/multi-tasking facility is called NCQ or TCQ.

For this reason, TRIM commands require that the drive disable NCQ/TCQ, before the the TRIM command. Then while the TRIM runs, no other commands can be accepted. Once the TRIM is complete, NCQ can be re-enabled and the drive can return to normal multi-tasking operation.

 

[Mark Rejhon]

Theoretically, TRIM could be designed to be done in segments at a time -- rather than on the whole file at once. Basically, block the NCQ, trim the corresponding sectors on both drives, unblock the NCQ, let things run, rinse and repeat. That would allow RAID 0 TRIM without the freezes. I wonder if any drives do things this way.

I have a pair of Intel 520's -- 240GB SSD's -- running in RAID 0. Supposedly supports TRIM but I've never gotten freezes. Right now, I'm not 100% sure if TRIM is operating.

 

[VirtualLarry]

I deleted a 6GB file and this froze up most of the system for over a minute while it was trimmed off.

That's not normal, and I doubt the reason for that is TRIM. When I do a fresh format of my SSD with the Win7 bootable DVD, when it goes to format (does a full-disk TRIM pass), it takes less time than that.

My estimation, is that one/both of your SSDs is failing. Are they SandForce? It wouldn't surprise me.

Oh, and ignore the SandForce-pimping shills posting marketing materials from their parent company.