Economical SFF ZFS box

[Pandamonium]

I want a relatively compact NAS appliance that also handles some stuff like torrents, streaming, etc. Probably no real-time transcoding, mostly because 1) I don't know what realtime transcoding is, 2) If I did need it, I'd know about it.

That said, this is what I want:

ZFS RAIDZ1 or Z2
ECC ram
Small footprint- I have a Synology DS209 hiding behind my TV with some room to spare. This machine will take its place.
Low power
Quiet
Cool
Room for at least 4x3.5" HDDs
+/- on deduplication- I kind of like having multiple copies of my wedding photos, etc.
Not too expensive.

Right now the only thing on the market that comes to mind is the HP Microserver. But the N40L is getting long in the tooth and I'm tired of waiting for the refresh. Plus I'm iffy on how well 8GB ECC ram will handle a 6TB RAIDZ2 or 9TB RAIDZ1.

Is there anything else on the market that would work for me?

 

[PCTC2]

Well, what's your budget? You can build your own, but it can get a little expensive. ECC will add a decent amount of cost (for SFF limited choices and also for ECC itself). Also, I don't get your comment on deduplication. You have multiple copies on the same box?

Here's a good starting point (with ECC) for a custom-built SFF box (it can be lowered by over $100 without ECC). The G630T is Intel's low power Pentium, but the G530 is full-power and will save you $25. Later on, for more power, you could upgrade to a Xeon E3-1220.

The Case is the Lian Li PC-Q25B. It has 5x hotswap bays and room for another 2x 3.5" hard-mounted drives or 3x 2.5" (or 1 2.5" and 1 3.5"). So you could have 5 3.5" storage drives in RAID Z1 (or Z2), and then the OS drive and SSD Cache drive hard mounted.

The power supply is an SFX-size (not ATX) for the sheer fact that ATX power supplies are tough to get into this case.

The SATA Controller Card is based on the older LSI 1068E controller, but I'm not too familiar with FreeBSD/FreeNAS/Solaris compatibility. Most systems now are compatible with this older controller (vs the newer LSI 2X08 controllers). You could always find one cheaper.

Motherboard: Intel DBS1200KP $160 from the Egg
Processor: Intel Pentium G630T (35W, 2.3GHz) $75 (You can get the G530 and save yourself $25)
RAM: Kingston 8GB (2x4GB) ECC Unbuffered Kit $70
Case:Lian Li PC-Q25B $120
Power Supply: Silverstone SFX 450W $78
SATA Controller Card: Intel SASUC8I 8-port (LSI 1068E) $150

Without drives: $653 before tax/shipping
Add 5x 2TB drives, an 80GB OS drive, and a 60-64GB SSD for cache, and you're golden.

 

[billyb0b]

+1 for the SFX class PSU. I'm running this PSU in my HTPC in a tiny Lian Li case and it's a great unit. It comes with an ATX adapter so it will fit almost anywhere.

 

[Pandamonium]

PCTC2: That Intel board looks interesting. The only gripe I've got with it is that it only has 4 SATA ports, so I'd have to get another SATA controller. I haven't run the numbers to figure out if it's cheaper to get 6 onboard ports vs a separate controller though.

Anyway, I was playing on google and stumbled on this case. There are other variants too, all made by U-NAS:

NSC-400, HT-400
NSC-600, HT-600
NSC-800, HT-800

I haven't figured out the naming convention, but they all appear to be the same kind of design. Anyway, these cases look almost ideal for me. The problem is that there is next to no information about them in english. I understand that there's a chinese ebay equivalent that has sellers listing these for sale individually, but I've never purchased overseas like this, so I'm a bit wary. Has anyone heard about this manufacturer? Or if there's a US Subsidiary?

edit: I saw the PC25B. It checks a lot of the boxes off, but the dimensions are tough for me to work with. The 14.5" length is going to be difficult to fit where it needs to fit. A CFI-A7879 is more along the dimensions I want.


edit2: The ideal case for my situation would have 4 hotswap 3.5" trays stacked on top of each other, a tall/narrow PSU next to the HDD trays, and motherboard on top. Maybe a flexible PCIe cable if someone really wanted an extension card. But mainly something that builds "up" rather than around.

 

[PCTC2]

Well, there most miniITX boards that exist only have 4 SATA ports. A few do have 5 or 6 SATA ports but do not support ECC. Do you really need ECC?

It is much cheaper to get a non-ECC ITX board with 8GB non-ECC RAM that has 6 SATA ports than it is to get a controller card.

As for a case, there are also Chenbro and CFI cases.
CFI A7879 (200W PSU ~ $169)
Chenbro ES34169 (discontinued)
Chenbro SR30169 (250W PSU - $110 on Provantage)

EDIT: Just saw you posted about the CFI case.

EDIT2: You could also find a non-functional, cheap Acer AH340 or AH342, or Sans Digital AccuNAS AN4L/+ and use them as a case. They should be miniITX capable and are around your correct dimensions as well.

 

[Pandamonium]

I'm pretty dead-set on 16 GB ECC ram for ZFS. I'd prefer a do it right/do it once type of approach.

Good call on looking for a dead Sans Digital AN4L chassis. I think I'll stay away from Acer EasyStores so I don't have to worry about case-modding for video-out.

 

[PCTC2]

I'm pretty dead-set on 16 GB ECC ram for ZFS. I'd prefer a do it right/do it once type of approach.

Good call on looking for a dead Sans Digital AN4L chassis. I think I'll stay away from Acer EasyStores so I don't have to worry about case-modding for video-out.

Well, the Acer EasyStores should also support ITX boards (they have a removable IO panel too, and a single PCI slot cover), so I meant to look for a dead EasyStore. Replacing it with another ITX board would solve the no-video issue.

As for ECC, the only ITX board I have found so far is the Intel board. Every other ITX board I have found, using the Q-series, B-series, Z-series, AND H-series chipsets do not use ECC and there are no other 'server' ITX board. No SuperMicro, no Tyan, no nothing. You're going to have to either go microATX, use the Intel board I linked, or ditch ECC.

In all honesty, you are not going to need ECC for a ZFS box. I am all about "Do it right or go home", but it's not needed. ZFS has ECC itself, and if data corruption was that much a worry for you, 1) I would not be building my own box and 2) I wouldn't go ITX. I would suck it up and get REAL SMB/Enterprise EQUIPMENT.

 

[Pandamonium]

I think I'll stick with the ZFS with ECC or bust camp. ZFS has end-to-end checksumming, so basically ECC for the filesystem. It doesn't have any sort of mechanism for detecting when/if memory is acting up. In general, if you move to ZFS, RAM becomes the weakest link. The cost delta (at present) isn't so exorbitant for non-ECC/ECC ram. Why upgrade the filesystem and leave your data exposed to ram errors that could be addressed with an extra ~$20?

Enterprise ZFS storage is sold by Oracle. AFAIK, ZenaVault/Zenabox is the only SMB/SOHO ZFS appliance. They are both "call us for a price" types of purchases. Cursory google queries haven't given me a pricepoint for the Zenavault, but CDW prices Oracle/Sun's 7140 (entry level model) on the order of $20k for 24TB.

But to be totally honest, I'm shelving my ZFS plans for the time being. I have a DS209 whose only faults are
1) running out of free space
2) not providing me with as much peace of mind as a ZFS box would

So my DS209 is doing 90% of what I need, the physical space where any new NAS would reside is relatively constrained, and HDD prices will remain inflated until ~2014. With that in mind, I decided to jump on the recent Seagate 2TB Newegg/Amazon deal and will be upgrading my DS209's storage.

I'll follow ZFS for the time being, and will reevaluate a ZFS server in late 2013. Maybe Synology/QNAP will support Btrfs and I won't bother with ZFS.

 

[sm625]

At a load of 90 watts, that power supply is burning up about 15. I dont see how it can command such a price premium when it is not any more efficient than a cheap Antec EA-380D. Indeed it may even be less efficient...

 

[jdcooper]

What do y'all think the scoop is with the N40L refresh? It sure seems like the Microserver line is superb value, but it also seems like it would be prudent not to buy right now.

 

[RaiderJ]

My ZFS file server build log is list in my signature. Doesn't use ECC, but I'm also not overly concerned with the risk of errors by using regular RAM. At the time of my build, early 2011, I spent ~$1,000 on my entire setup. I have something like 9TB of formatted storage.

 

[PCTC2]

Holy Thread Revival Batman (not really, but it didn't need to be revived about the N40L)...

The N40L is just a cheap small SOHO server. You can do better or worse. It's pretty underpowered, but if you're just using it for a straight storage server (not really ZFS), then you'd be fine. ZFS may be a little much for it.

Here's my ZFS server. Much like RaiderJ's, but in the PC-Q25b case.

 

[jdcooper]

That build looks nice. Do you have a thread describing it? Would love to know the details.

 

[Fox5]

Iirc, all AMD consumer cpus support ECC, although it's hit or miss finding a motherboard that does. I've heard Asus supports ECC across the entire line, but they don't have any AMD ITX boards. (other than a brazos based stuff)

 

[PCTC2]

That build looks nice. Do you have a thread describing it? Would love to know the details.

It's basically what I described in #2, but I didn't use high performance parts because I'm fine with 10-20 MB/s. It's a backup of a backup server. . I built it out of spare parts and there are much better choices.

It's a Intel Celeron E3300, 4GB DDR2 on a Zotac G43-ITX-A-E with an Apricorn Velocity Solo PCIe 2.5" Card with a MicroCenter G2 64GB SSD. The HSF is a Thermaltake SlimX3.
The drives are 5x Western Digital Green drives and the case and PSU are the Lian Li PC-Q25B with an FSP SFX PSU.
The OS sits on a 4GB USB Dongle.

If I wanted better performance, I would choose a Xeon E3-1220, Intel DBS1200KP Mini ITX board, 16GB DDR3 ECC RAM, and a better SSD (2x Intel 313 24GB comes to mind). I would skip the Apricorn card and instead put in a 8-port LSI card (9207-8i). The SSDs, instead of being mounted on the card, would be attached to the on-board 6Gbps Intel controller. The storage drives would be attached to the LSI controller. Possibly using a 16GB SSD instead of the 4GB USB Dongle for the OS. The bottom plate has space for 3x 2.5" drives. (The OS drive and 2x cache drives). I would still use the Thermaltake SlimX3 (1155 compatible now).

Also note that the 9207-8i is a rather expensive HBA. There are cheaper 8-port 6Gbps HBAs. I just said it because I use them at work.

The reason why I went for a SFX PSU is because of the space. Imagine all that free space you see taken up by the PSU. It would encroach on the HSF and block some airflow. ATX to SFX adapter plates are usually included with SFX PSUs anyways, and they're cheap if not included.

 

[jdcooper]

Thanks, that's very helpful.

I have managed to convince myself that I really want to do this build with ECC RAM and as far as I can tell the Intel S1200* motherboards are the only choice available that will let me do that. Unfortunately they appear to be quite fussy about, well, everything, including processor and RAM compatibility. (Newegg reviews are kind of a bummer, for example.)

I was thinking to use WD Red drives or possibly RE4 in RAID-Z2 configuration. Any thoughts on that?

I have not had great luck with single-drive redundancy in the past, and while I will have a backup plan in place, I'd rather not use it.

I've never worked with virtualization at home so am now thinking perhaps I can go for a more powerful build and set this up with ESXi, run FreeNAS in a VM, and another VM with Plex Media Server to handle transcoding duties to stream to my Xbox360. But there's a lot of learning about ESXi before I decide to do that -- I would probably set it up from the get-go as a straight FreeNAS box.

Cheers,
Jon

 

[PCTC2]

Well, if you want storage, ESXi is NOT the way to go. Any storage server will not have direct access to the drives. You generally want to separate your ESXi and storage servers (such as a compute node and an iSCSI node).

As for WD Red vs RE4, it depends on what its for. If it's for you, WD Reds are fine for low to mid performance storage servers with 5 or less drives. WD themselves do not recommend using more than 5 Reds. If you want more, they recommend RE4's.

TBH, I prefer Hitachi Ultrastars and Seagate Constellations over WD RE4s.

My Greens are in a RAID-Z configuration (don't really need dual-drive parity). Just make sure to align the drives to 4K sectors for WD Reds and Greens. There's a tutorial somewhere on the internet. I have my MC G2 as a cache drive (don't need logging). It's not very reliable. Dual Intel 313's will give good performance, have a decent size (48GB raw), and have great reliability (proven Intel controllers with Intel SLC NAND).

As for ECC, for ITX, the S1200KP is the only board, BUT it doesn't have AMT, which is a dealbreaker for me. I'd prefer the DQ77KP (which supports AMT 8.0). It doesn't have ECC but I don't need ECC.

 

[RaiderJ]

Don't know if it's still the case, but before FreeNAS's performance was awful compared to OpenIndiana (with respect to ZFS). I run OpenIndiana myself with napp-it as the front-end - very highly recommended. Also have a Ubuntu VM running on VirtualBox that has Plex on it. Works great, although I think my processor is too slow for the transcoding. Chokes on the really high bitrate MKV files.

 

[PCTC2]

FreeNAS had a lot of driver issues until the more recent releases. I still don't like OpenIndiana compared to FreeBSD, but I've never been much of a Solaris guy. OpenIndiana and Nexenta are based off the same kernel.

But IMO, the best is just running straight FreeBSD.

 

[Knavish]

Since the thread has been revived, here is a non-Intel microATX board with ECC. I'm pretty sure you *have* to run a Xeon (1-cpu model like the E3-1220) and not a Pentium, i5, i7, etc. to get ECC on any Intel system.
ASUS P8B-M
FYI the built-in RAID on this motherboard is total crap. I RAID-0'd a pair of Intel 510 SSDs and got something like 20-40MB/s on a couple benchmarks -- I forget the specifics, but plan on *not* using the onboard raid.

 

[PCTC2]

Since the thread has been revived, here is a non-Intel microATX board with ECC. I'm pretty sure you *have* to run a Xeon (1-cpu model like the E3-1220) and not a Pentium, i5, i7, etc. to get ECC on any Intel system.
ASUS P8B-M
FYI the built-in RAID on this motherboard is total crap. I RAID-0'd a pair of Intel 510 SSDs and got something like 20-40MB/s on a couple benchmarks -- I forget the specifics, but plan on *not* using the onboard raid.

Yes. You do have to run Xeon to get the full benefit of ECC. However, some boards still require ECC RAM even if you run Celeron/i3's... reference SuperMicro X9 1155 boards.

As for the onboard RAID... it's software/fakeRAID. Generally, software RAID0 gets better performance than you got, but it's still not hardware.

 

[jdcooper]

Cool, thanks for the ongoing feedback. It seems that finding a Mini-ITX motherboard that supports ECC is going to be a ginormous pain in the butt, plus, I'm not really space constrained, so I think I will end up doing a MicroATX or even full sized ATX. Cheers!

 

[stevech]

Cool, thanks for the ongoing feedback. It seems that finding a Mini-ITX motherboard that supports ECC is going to be a ginormous pain in the butt, plus, I'm not really space constrained, so I think I will end up doing a MicroATX or even full sized ATX. Cheers!

mATX, I'd say. Speed/heat and miniITX don't mix.

 

[PCTC2]

Cool, thanks for the ongoing feedback. It seems that finding a Mini-ITX motherboard that supports ECC is going to be a ginormous pain in the butt, plus, I'm not really space constrained, so I think I will end up doing a MicroATX or even full sized ATX. Cheers!

The SuperMicro X9SCM-F is a nice board.

 

[Insert_Nickname]

The SuperMicro X9SCM-F is a nice board.

I second that. I have one running as a general purpose server at work, and there has not been a single problem with it.