My adventure with maczfs - ZFS for OSX

[silverpig]

Okay, so I finally got my array all built. I'd like to share my experience so far with ZFS on OSX as it may be useful to some of you.

Old hard drives: 2x1.5 TB WD Greens, OCZ Summit (boot disk), 500 GB, 320 GB, 300 GB, 250 GB.

New drives: 2x1.5 TB WD Greens.

Goal: RAID-5 the 4x1.5 TB Greens, boot off the Summit.

Problems:
1. The ~2 TB of data I wanted to keep
2. ICH10 southbridge has no RAID-5, didn't want to run linux mdraid and virtualize the hackintosh on top, didn't want to buy a rocketRAID card.

Preliminary Solution: ZFS raidz pool of disks.
Basically, this is like RAID5 (1 parity stripe and [n-1] non parity stripes) but slightly better as it avoids the RAID5 write hole. It's all software and ZFS has some cool features related to expanding pools etc.

Now, ZFS is ZFS. It's stable and great, but maczfs, while still ZFS, is community maintained off of a dead Apple project to bring ZFS to OSX. It works and works well, but it is a bit behind the Solaris version. I read through most of the documentation and planned it all out and figured I had my solution.

One thing about raidz is you can't ADD a new drive to the raidz array. If you make it with 4 disks, it has to have 4 disks. You can't grow it to 5 disks and let it rebuild. With my spare drives not able to hold all the data, and me wanting to use all 4 1.5 TB drives in the array, I had to get a little creative.

Basically, I partitioned one of the new 1.5 TB drives into 3x0.5 TB, and created a 4 disk raidz array using those three partitions and the 0.5 TB drive I had. I then had a 1.5 TB raidz array I could copy stuff to, and then swapped in the other new 1.5 TB drive for one of the 0.5 TB partitions.

Now, because the way this works is the size of the raidz array is 3/4 * (number of disks) * (size of smallest disk), my array would be stuck at 1.5 TB in size until I did the final swap and ended up with 4.5 TB.

I then copied as much data from the old 1.5 TB drives to the new array and the spare disks, and swapped those in for one of the 2 remaining partitions and the 0.5 TB drive.

So now I had in my array:

[1.5][1.5][1.5][1.5] drives

[0.5][1.5][1.5][1.5] partitions

So I had 4x1.5 TB drives in my raidz array, with three of them being full 1.5 TB partitions, and the fourth being a 0.5 TB partition that I hadn't yet swapped out.

The last step was to take the final 0.5 TB partition of the 1.5 TB drive I used for the initial raidz array offline, repartition and reformat the drive to the full 1.5 TB space, and then put it back into the array.

So I did that, but the array still came up as 1.5 TB. After much googling, the autoexpand flag (which was in the man pages for the version of maczfs I installed) isn't actually IN maczfs. They're a bit behind the Solaris implementation and this is one of the features not yet implemented.

Solution: Get a live DVD of OpenIndiana (basically OpenSolaris), import the array, have it re-set the size, then reboot to OSX. Win.

Oh, I ran some benchmarks with Xbench:

4x1.5 TB WD Greens in ZFS raidz1:

Results	1667.03	
	System Info		
		Xbench Version		1.3
		System Version		10.6.7 (10J869)
		Physical RAM		4096 MB
		Model		MacPro3,1
	Disk Test	1667.03	
		Sequential	978.08	
			Uncached Write	325.07	199.59 MB/sec [4K blocks]
			Uncached Write	2547.84	1441.57 MB/sec [256K blocks]
			Uncached Read	2023.82	592.28 MB/sec [4K blocks]
			Uncached Read	7888.36	3964.63 MB/sec [256K blocks]
		Random	5639.17	
			Uncached Write	2109.55	223.32 MB/sec [4K blocks]
			Uncached Write	5822.33	1863.94 MB/sec [256K blocks]
			Uncached Read	65706.06	465.61 MB/sec [4K blocks]
			Uncached Read	20696.61	3840.40 MB/sec [256K blocks]

OCZ Summit 60 GB SSD:

Results	204.01	
	System Info		
		Xbench Version		1.3
		System Version		10.6.7 (10J869)
		Physical RAM		4096 MB
		Model		MacPro3,1
		Drive Type		OCZ SUMMIT
	Disk Test	204.01	
		Sequential	167.21	
			Uncached Write	136.03	83.52 MB/sec [4K blocks]
			Uncached Write	138.74	78.50 MB/sec [256K blocks]
			Uncached Read	144.16	42.19 MB/sec [4K blocks]
			Uncached Read	412.26	207.20 MB/sec [256K blocks]
		Random	261.56	
			Uncached Write	129.08	13.67 MB/sec [4K blocks]
			Uncached Write	176.18	56.40 MB/sec [256K blocks]
			Uncached Read	2558.59	18.13 MB/sec [4K blocks]
			Uncached Read	675.91	125.42 MB/sec [256K blocks]

Previous OP:

Long story short, I was looking for the cheapest way to RAID5 4x1.5 TB WD greens on my hackintosh without having a southbridge that had RAID capabilities. There's no OSX software RAID, linux + OSX in virtualization means I'd have to buy more RAM, and I didn't want to spend money on new hardware. This lead me to ZFS for OSX via maczfs.

RAID-Z ftw baby!

The only issue I'm having is two of my 1.5 TB drives have data I want to keep on them, so I'm going through some complicated partitioning and drive replacing scheme to build the pool and copy data over in bits while I move drives into the array.

But still, why did Apple ever abandon this filesystem?

 

[ViRGE]

As far as anyone knows, it was a matter of licensing issues and development resources. Sun only got ZFS stable on Solaris after a great deal of work, never mind any other OS. Actually for that reason I would not suggest maczfs either.

 

[silverpig]

The page for it says maczfs is ZFS and is stable. I am building a raidz array right now and due to my hd situation, have had to repeatedly replace disks and transfer data back and forth as I set it up. All of my drives have been churning non stop for a few days now and things are going great so far.

I've learned a lot and will update the thread with what I did and how it went once the array is finished and when I am not on my phone.

 

[silverpig]

Updated OP.

 

[JSt0rm]

interesting. I researched doing a zfs backup system but just ended up going raid5 (on battery back up so probably wont corrupt). My data is pretty critical and I need to know its gonna be there (I have off site back up as well). I would love to have a system that I can add drives to expand the system though.

 

[silverpig]

interesting. I researched doing a zfs backup system but just ended up going raid5 (on battery back up so probably wont corrupt). My data is pretty critical and I need to know its gonna be there (I have off site back up as well). I would love to have a system that I can add drives to expand the system though.

The thing with raidz on ZFS is you can't add drives to the raidz device, but ZFS allows you to add drives to the pool for expansion. You can get what you want.

Let's say you create a pool called "data". You can then add a 4x2 TB raidz1 array under it to get 6 TB of storage on what appears to be a device called data.

data
..raidz
....device1
....device2
....device3
....device4

While you can't add a new device to the raidz, you CAN add ones to "data". Let's say you add 4x2 TB more drives... you can double the space:

data
..raidz
....device1
....device2
....device3
....device4
..raidz
....device5
....device6
....device7
....device8

Now your data drive will look like it is 12 TB. You still have redundancy and can grow your pool as you add more drives.

You can mirror as well (sort of like raid 1), and then add more mirrored sets to the pool.

data
..device1
..mirror1

then grow to

data
..device1
..mirror1
..device2
..mirror2

etc. It's pretty flexible. This is just stuff I've learned in the past 3 days of reading man pages, online documentations, and messing around myself. I'm no expert of course but I'm pretty happy with it right now.

 

[Emulex]

let us know when that explodes.

 

[Nothinman]

But still, why did Apple ever abandon this filesystem?

I'm sure it had to do with the CDDL licensing and possibly dealing with Sun. ZFS' data parity sounds good, but I don't like the bundling of all the different layers into one big ambiguous thing. I'd rather deal with Linux software RAID, LVM, iSCSI, Samba, etc individually.

 

[silverpig]

I'd have liked to have gone with linux software RAID but kind of wanted to run OSX for media center and wife reasons and didn't want to deal with virtualization. It's not like this is mission-critical stuff anyways. I'm doing it half for the learning process and experimentation.

*shrug*

 

[Swimgym]

I'm doing something very similar to you. Mine is a 2 partition mirror, copy data, reformat old data disk to zfs and zpool replace, all went well. I can't find any good info on removing one of the 2 partitions and then resizing the other to full disk. I wanted to do this without reformating the whole disk, because its a mirror and there will be no backup. Your msg wasn't clear on this point.

 

[Swimgym]

I got my answer, after reading the first post again, and seeing the "repartition, reformat" line. Looks like I'll have to take the reformat plunge, tho the data won't be any less protected that before I started down the ZFS path, It would have been nice to find a different answer.

 

[Tyranicus]

There's no OSX software RAID, linux + OSX in virtualization means I'd have to buy more RAM, and I didn't want to spend money on new hardware. This lead me to ZFS for OSX via maczfs.

Actually, there is OS X software RAID. I've use it in the past. If you open Disk Utility, and select a disk, you will have a RAID tab at the top. You can select, mirrored, striped, or concatenated.