30TB FreeNAS & HTPC build questions

[imported_MrBungle]

I'm new to FreeNAS, ZFS and RAID-Z, but I've been doing some research on a first build and would greatly appreciate some feedback.

My ideal system would be a combined HTPC and NAS box with up to 30 TB raw disk space, running Windows 7 and virtualized FreeNAS. I'm not sure if this is all possible - just an ideal that I'm shooting for. My budget is flexible - trying to stay on the cheaper end, but I don't want to skimp. This will be using all-new parts.

Answers to the 10 questions:

1. Usage: FreeNAS storage (RAID-Z2) first, HTPC (Blu-ray, DVD, Netflix streaming) also if possible.
2. Budget: Flexible (ideally less than ~$1500 in total; already spent $880 on 8 drives (intend to buy 2 more), and $85 on case/PSU, so ~$400 left).
3. Country: USA; buying from NewEgg, Amazon, MonoPrice
4. No strong brand preferences, but I usually prefer larger, more established brands.
5. The box will contain all-new parts; I already have monitor and all peripherals.
6. I've been doing some research, but I'm at the phase where it seems like my options have only increased. Hopefully with some guidance, I can weed out the bad approaches and narrow down the field.
7. Probably no overclocking; just default speeds.
8. FreeNAS is headless (no monitor), but HTPC would be at 1080p.
9. Plan to build as soon as possible.
10. I'm thick-skinned. ;-)

Here is the list of essential hardware that I've put together:

- Asus E25M1-M PRO motherboard (AMD Zacate E-350 APU, 1.6GHz dual-core) - http://www.newegg.com/Product/Produc...82E16813131697
- (8x) Hitachi Deskstar 0S03230 3TB 5400 RPM SATA 6.0Gb/s HDD - http://www.newegg.com/Product/Produc...82E16822145493 (will probably add 2 more drives within a few months to end up with 30 TB raw, ~24 TB in RAID-Z2)
- G.Skill Ripjaws 8GB (2 x 4GB) DDR3 1333 (Model F3-10666CL7D-8GBRH) - http://www.newegg.com/Product/Produc...82E16820231402
- HighPoint RocketRAID 2720SGL SATA/SAS controller card - http://www.newegg.com/Product/Produc...82E16816115100

Some notes about intended usage and concerns:

- I'm confident that this hardware is more than capable of handling my HTPC needs (up to 1080p smooth Blu-ray playback). As I understand it, the GPU has hardware decoding that will keep the CPU mostly free (worst-case, ~25% load). However, I already own a sufficient HTPC, so using this build for double-duty (HTPC & NAS) is just gravy, and would serve to simplify my home setup. I want this to be a capable NAS first-and-foremost, so I can scrap the HTPC plans if needed.
- I'm overwhelmingly concerned about data integrity above all else. Performance, compression, and other features aren't very important to me aside from that. I'll be more than happy if I can just get half-decent file transfer speeds over the network. Downtime is completely unimportant. I just don't want to lose any data, so I hope to run RAID-Z2.
- I plan to use this mainly for music, movies, and photos, but also Time Machine backup for 2 Macbooks, a Mac Pro, and backup for random files. From the HTPC front - Netflix streaming and watching DVDs and Blu-rays.

Questions:

- I'm assuming that I can use any low-cost HBA/RAID Controller since I won't actually need it for its RAID capability (since I'll be using RAID-Z2). Is that correct? If so, is there a cheaper option even than the HighPoint 2720?
- The drives I'm using (I've already bought them) are SATA 6.0 Gbps, so I originally intended to buy other hardware with support for that. Will that even matter, though, assuming that all traffic in & out of the system will be via gigabit ethernet (i.e., will the latter be my bottleneck anyway even with SATA 3.0 Gbps)?
- Following up on the previous question, would even a couple of PCI 4-port SATA HBA cards (e.g., http://www.newegg.com/Product/Produc...82E16816124043) be sufficient to not cause a bottleneck with network transfers? Do I need to worry about FreeBSD compatibility if I'm just using them for extra SATA ports (no RAID other advanced features)?
- Will I have any issues creating a RAID-Z2 setup with 8 drives, then adding 2 more identical drives in a few months? Or am I better off just buying 2 more now to start with a 10-drive setup from the get-go?
- Will 8 GB RAM and the E-350 APU be enough horsepower to run ZFS/Z-RAID2? If so, is it close enough that I might want to avoid dual-tasking as an HTPC?
- If I ultimately go with the HighPoint 2720SGL and need/want the 6.0 Gbps capability, do I need special cables? Will some cheap cables from MonoPrice do the trick (e.g., http://www.monoprice.com/products/p...=10254&cs_id=1025406&p_id=8186&seq=1&format=2) or do I need to get ones built to a certain standard? Unlike with SATA cables, I don't see any SFF-8087 breakout cables specified as SATA II/3.0 or SATA III/6.0.
- Anything else I missed, or gotchas to be aware of? Would anyone recommend different hardware or a different approach?

Sorry for the barrage of questions... Hopefully based on all this info, you can get a feel for what I'm looking for and help to narrow the scope of my research a little further. Any help would be very much appreciated!

 

[ViRGE]

Wow, that's quite an ambitious build.

I can't even begin to answer all of your questions but I do have a couple of thoughts:

1) ZFS has a fairly high CPU overhead. You want a full-featured desktop processor; Zacate isn't going to cut it.

2) Your goals seem to be in conflict. Your NAS software is going to work best with direct hardware access. At the same time you want direct hardware/GPU access for Windows for GUI and decoding purposes. Having a single box doing both is probably not a good idea.

 

[imported_MrBungle]

Thanks, ViRGE! :thumbsup:

Those are both good points to consider. I sensed that I might need to drop the HTPC aspirations and focus just on the NAS - my main reason for considering the former was because as I narrowed down the hardware, I realized that it should be more than capable of replacing my current HTPC (at least if that was its only purpose). Good point about the virtualization, though; I'm not familiar enough with that approach to have considered that.

Probably a dumb question, but one I forgot to ask - if I use a more powerful CPU, is that likely going to make a big difference in idle power consumption? Obviously I'll use this system fairly frequently, but in all honestly it's likely to sit idle 90%+ of the time (I plan to run it at least 16 hours/day). I don't care much about power consumption while it's in use - especially if it's necessary for ZFS/RAID-Z2 resource demands - but I am concerned about keeping the idle power consumption as low as possible. I may not have looked hard enough, but I haven't found any discreet CPUs/GPUs that are nearly as low-power as Atom and Zacate.

Another quick question - will ZFS/FreeNAS take full advantage of multi-core processors?

Please let me know if anyone else has more feedback - thanks!

 

[ViRGE]

Probably a dumb question, but one I forgot to ask - if I use a more powerful CPU, is that likely going to make a big difference in idle power consumption? Obviously I'll use this system fairly frequently, but in all honestly it's likely to sit idle 90%+ of the time (I plan to run it at least 16 hours/day). I don't care much about power consumption while it's in use - especially if it's necessary for ZFS/RAID-Z2 resource demands - but I am concerned about keeping the idle power consumption as low as possible. I may not have looked hard enough, but I haven't found any discreet CPUs/GPUs that are nearly as low-power as Atom and Zacate.

Modern CPUs are pretty good about idle power. It won't be as low as Atom/Zacate (you're looking at 40W-50W for the CPU/RAM/mobo/GPU), but it will be low enough that all 10 disks spinning will be a bigger issue.

Another quick question - will ZFS/FreeNAS take full advantage of multi-core processors?

Individual file operations will generally be limited to a single core, but the moment that server is multitasking it will be using additional cores.

 

[SViscusi]

I agree with Virge, don't virtualize your server. If you afford to build two boxes, do that. You just adding an additional layer of potential problems, not to mention that many hard drives, even if they are low rpm drives, will produce heat, with the system tucked away in a closet you can get away with loud fans to cool it, in a living room or wherever you want your htpc, it'll be a distraction and end up adding to your cost.

You also don't need a huge investment in a cpu, whatever deal you can find for an athlon II or Pentium G will be fine, heck I think a Zacate would probably work although as Virge said your overall performance can be hindered by lack of cpu power. The more important aspect is going to be the motherboard. You need to make sure the nic is supported in freebsd, and if the mobo comes with any additional sata ports, that they're supported. Also don't skimp on the ram, 8gb's is cheap nowadays so that should be a good goal.

As far as controllers go http://blog.zorinaq.com/?e=10 this is a good place to look. The raid features on the Highpoint card are going to be useless as you've said, you can save a few bucks by going with something like this http://cgi.ebay.com/Dell-P-N-0HN359...285?pt=LH_DefaultDomain_0&hash=item335e64acfd, it's a one port barebones version of the Dell 5/i, and is supported in all os's.
http://cgi.ebay.com/Dell-P-N-0HN359...285?pt=LH_DefaultDomain_0&hash=item335e64acfd

 

[mfenn]

I agree completely with Virge. You don't want to be virtualizing file servers unless you're in an enterprise environment where you can take advantage of iSCSI or FibreChannel with NPIV. The added complexity is enormous.

As for your specific questions:
- Yes, you can use any HBA that is supported by FreeBSD
- Doesn't matter at all. Your drives cannot even approach SATA 3Gb/s speeds due to mechanical limitations
- Yes, you do need to worry that a driver exists for the chip that you want to use. In this particular case, the SiL3124 is supported as of FreeBSD 8
- Based on my understanding of ZFS (and I could be wrong), you cannot grow a RAID-Z (or Z2) vdev. You can of course add a new vdev to the overall pool, but only adding two disks will require you to do a mirror. What I would suggest that you do is to either start with 10 disks to begin with, or to create your initial pool with a single RAID-Z vdev of 5 disks and leave the other 3 disks idle until you purchase the final two disks. Then create another RAID-Z vdev of 5 disks and add it to the pool.
- You will probably by bottlenecked by Zacate when it comes to random operations, but I wouldn't worry about it since you only have a GigE pipe anyway.
- There is no physical difference between 3Gb/s cables and 6Gb/s cables, other than 6Gb/s cables have been certified to tighter tolerances. It doesn't really matter though (see above).

 

[imported_MrBungle]

@ViRGE, SViscusi, and mfenn:

Thanks so much for your feedback! I've taken it all to heart and revised my project goals and hardware list accordingly. Since it's changed pretty significantly, I posted a new thread here:

http://forums.anandtech.com/showthread.php?t=2190971

I'd really appreciate your feedback on that as well. Thanks again!

 

[KingGheedora]

30TB is a lot of storage for a home server? What do you do with all that storage?

 

[tracker1]

I know this is an older post. I'm currently running a FreeNAS box with the RocketRAID 2720SGL with 12x 3TB Western Digital Green drives (8 on sas controller, 4 onboard bus), using 10 in RaidZ2, and 2 in hot spare... It runs pretty good for the most part. I'm using an AMD FX-4100, and 8GB of RAM... the RAM seems to be the limiting factor under any amount of load, more so than the CPU... generally there's only a couple of things accessing the system at once (aside from when daily backups run from my servers (under ESXi on a different box).

As to the cable question, any internal mini-sas (SFF8087) to SATA cables should work, You may not want to get them too long though.

One word of caution, the SAS controller wouldn't "see" the drives until they had at least with a partition table (GPT) and one partition... I initialized the drives on another system first.

 

[mfenn]

Welcome to the forums tracker1! Thank you for posting additional information, but generally speaking, posting in threads that are several years old is frowned upon. I'm locking this thread, but feel free to create a new one to discuss FreeNAS, ZFS, or any other hardware related topic.

mfenn
General Hardware Moderator