ZFS with advanced format drives

[taltamir]

How does ZFS handle advanced format drives? those drives use internally 4kb sectors, but fake 512b sectors to maintain backwards compatibility with winXP (eww!)
http://www.anandtech.com/show/2888

So how does that affect ZFS? does ZFS align properly on those drives? What if you are using variable block size?

 

[Emulex]

a 4K alignment is aligned to 512K too.

But what i do is experiment with stripe size and cache ratio for the intended application - you'd be surprised when you have a 1GB FBWC and varying levels of protection (raid levels) that it is really application specific so partitioning the raid can provide better performance for some apps.

seems like there is a flatline on all raid when the sets get too large - and you need to bust a stripe over a raid set (50,60,100, etc).

does ZFS level? can it use an SSD for ECC storage?

 

[taltamir]

a 4K alignment is aligned to 512K too.

to quote anandtech:

With the emulation of 512B sectors, there’s the risk that a partition could be misaligned compared to the 4K physical sectors - where it would be unwittingly started in the middle of such a sector. As a result, the clusters of a file system on that partition would end up straddling 4K sectors, which would cause performance problems.

But what i do is experiment with stripe size and cache ratio for the intended application - you'd be surprised when you have a 1GB FBWC and varying levels of protection (raid levels) that it is really application specific so partitioning the raid can provide better performance for some apps.

ZFS automates all lower level handling, as such I can not touch the stripe size.

I already found my answer elsewhere, ZFS cannot handle the problems caused by drives lying about their sector size, which causes it to have atrocious performance with those "advanced format" drives.

 

[sub.mesa]

If you have disks to test, now may be a good time to try out my newly released ZFSguru web-interface. I just uploaded a new version with Benchmarking support, particularly designed to test performance of 4K sector disks which use emulation.

You might find this thread interesting as well:
http://hardforum.com/showthread.php?t=1546137

Have you seen the preliminary results of benchmarks with WD EARS 4-platter yet? Those weren't too bad, except the RAIDZ2 performance under 6 disks.

 

[sub.mesa]

to quote anandtech:
I already found my answer elsewhere, ZFS cannot handle the problems caused by drives lying about their sector size, which causes it to have atrocious performance with those "advanced format" drives.

I think that's a too strong statement. I don't have real reliable test data to base an well-founded opinion on yet, but so far on the systems i've tested i've got quite decent performance out of these drives; all i need to do was some ZFS tuning. That may also have been necessary with normal 512-byte disks.

I also did tests with geom_nop providers on them, which transform the sector size from 512 bytes to a multiple of that; like 4096 bytes giving you both physical 4K and logical 4K sectors; just as if the drive wasn't actually lying about it.

By doing that, performance did indeed increase. But not any significant margin. We also have to recognize that even with native 4K sectors, if your filesystem needs to write only 2000 bytes, it would need to read the unchanged portions and write the modified 4K sector again, thus causing some overhead and performance loss as well. The same occurs on 512-byte HDDs when you need to write 40 bytes; your filesystem would read the 512-byte sector, apply the modifications (40 bytes) and write the sector again to HDD.

But that's the filesystems job; translating a block-device to a char-device filesystem. Now at least one piece of software is doing that job, unlike with 4K sector disks out-of-the-box where both HDD and Filesystem would be doing the same job; only the HDD can't do as good as job as the filesystem can.

But all in all, you can use 4K sector disks in ZFS, unless you absolutely want highest performance. Personally, i would invest in SLOG and L2ARC SSD to gain such performance, and use low-rpm disks for sequential performance. This is much more 'clever' since a small SSD can boost performance of alot of large capacity HDDs.

For ZIL/SLOG you have to wait for newer SSDs though; the Intel G3 would be suitable for this. This is because these have a supercapacitor to allow them to write their buffer on power loss; otherwise you may have a corrupt ZIL and lose your pool altogether. It may also be wise to wait with this feature until you run ZFS pool version 19, which can continue to function if your ZIL/SLOG device is lost.

L2ARC or Cache device would be safe and can be done even with cheap SSDs; even usb sticks and compactflash.

 

[taltamir]

thank you, I was basing my statements on some forum threads (which I found via google) which claimed atrocious performance (less than 1/10th) when using 4k sector drives.
But it seems there is more to it than that, and that there are things you can do.
I am looking at that other thread linked here for more info right now.

For ZIL/SLOG you have to wait for newer SSDs though; the Intel G3 would be suitable for this. This is because these have a supercapacitor to allow them to write their buffer on power loss; otherwise you may have a corrupt ZIL and lose your pool altogether. It may also be wise to wait with this feature until you run ZFS pool version 19, which can continue to function if your ZIL/SLOG device is lost.

Thanks for the suggestion. I would definitely wait for v19 then. I wouldn't see the point of running a redundant array of the loss of one specific drive causes you to lose the pool?

 

[sub.mesa]

No real point there, indeed! However, before pool version 19 it was recommended to mirror the SLOG; meaning you need two SSDs. But this is kind of a waste since all the SLOG does is sequential write; you gain nothing by mirroring it. And you wear both SSDs by writing the same data to it. Not really sexy, i think!

So until you get pool version 19, only use L2ARC to accelerate random reads.

The SLOG accelerates writes instead and can yield significant performance improvements. But normal SSDs are unsafe as SLOG and again can cause your pool to fail; you need an SSD with supercapacitor. The remaining risk would be bad firmware in the SSD causing it to corrupt itself. The chance of this is low for a major brand like Intel, but nonetheless, you should consider that losing a SLOG means losing all your data; assuming you run ZFS version lower than 19.

After ZFS v19, you can lose the SLOG and it will revert to some seconds before the last crash/power outage. Without dedicated SLOG device, your pool (including its redundancy) will hold the SLOG; alot safer but less performance.

 

[taltamir]

I just checked, and it seems the latest stable build of FreeBSD is using Zpool v14. I am currently using FSv4 & Zpool V22 (using latest osol). So I can't really switch to BSD or any derivative at the time.
I could, however, use an SLOG, but I cannot justify the price for what I am using it for.

Now that we got 3TB drives, I wonder how it handles those.

 

[tweakboy]

My HD specs are 512k per sector / cluster size. However dumb OS installs or should I say formats it to 4k cluster and its soo much slower drive and not working to potential. I changed it to 512k sector then I defragged which took 7 hours and baaam my Crystal Mark went from 124 read 124 write to 155 read 155 write and high scores down on the bottom too. Its the fastest mechanical drive,, one platter ,, F4 pownz me! next best thing to SSD and a passmark of 970 higher then some SSD's