Mini-ITX, ECC and VT-d, HDMI out .. does anything like this exist?

[repoman0]

I'm thinking about building a mini-ITX machine that can double as a NAS with ZFS and HTPC. I'm thinking I could run ESXi with either freenas or some other system providing the storage, and then a normal Windows or Linux OS to provide HTPC functions (mostly simple streaming and emulators). Unfortunately it seems like a pretty esoteric list of requirements, and I'm having a hard time finding a motherboard/platform that's small, supports ECC and bare metal virtualization, and a reasonable video output for a TV (not VGA). I guess I could use a really cheap low-end video card to provide the video if necessary.

First, does doing this even make sense? I have never used ESXi and only learned about it a few days ago. From the reading I've done so far, I could set up the NAS/ZFS OS, create the config and everything, then copy the config to a virtualized version, and have the original NAS OS on a USB stick in case anything goes wrong. I want to combine these two functions in the first place because I have a hard time justifying an extra computer just for providing storage to one other computer, but I do want the NAS functionality for reliability. And if I want it to be an HTPC too, having that separate from the ZFS functionality seems like a good idea.

Second, anyone know of a board that does all this if it does make the slightest bit of sense?

 

[Ken g6]

Why do you need ECC? Exclude that and the rest of your requirements seem pretty easy.

 

[repoman0]

Why do you need ECC? Exclude that and the rest of your requirements seem pretty easy.

I was under the impression that ECC is a good idea with the ZFS file system, and it seems like most of the boards that support the pci-passthrough type things that a NAS VM requires support ECC anyway. That said, I'm not really sure if the ECC with ZFS things is FUD, and if it is and there's a cheaper option for doing PCI passthrough and having it work right and reliably, I'm all for it ...

 

[frowertr]

I'm guessing because the FreeNAS community continually preaches this over and over again. Unfortunately, the majority of that community is made up of hobbyist that don't really have a clue how storage is handled.

I don't think ECC should be a deal breaker for a home NAS. Bit rot chances are so very tiny especially for home NAS size arrays.

 

[frowertr]

ECC is great, don't get me wrong. I just find it wasteful for a home NAS serving music, videos, and pictures, etc...

 

[Viper GTS]

Please don't try to make this machine do HTPC duty as well. I'm all for integrated ESXi/ZFS and have run such a setup successfully for 5 years now but for the love of god buy a Roku or something.

On the ECC or not to ECC debate:

If you care about your data enough to deploy ZFS why would you NOT also run ECC? The cost premium is not massive, and most people can get by with UDIMM instead of RDIMM. There are reasons workstations and servers run ECC, there's simply no reason not to.

ECC is great, don't get me wrong. I just find it wasteful for a home NAS serving music, videos, and pictures, etc...

Anyone who posts here is probably WAY over the wasteful line for home technology before getting to ECC. If you're going to be wasteful do it right.

Viper GTS

 

[repoman0]

I'm guessing because the FreeNAS community continually preaches this over and over again. Unfortunately, the majority of that community is made up of hobbyist that don't really have a clue how storage is handled.

I don't think ECC should be a deal breaker for a home NAS. Bit rot chances are so very tiny especially for home NAS size arrays.

Yeah I've done a bunch of searching and haven't seen anyone doing any actual analysis, beyond that one guy who pulls up all kinds of crazy hypotheticals trying to support his case but making no use of actual probability. I'm guessing it would be fine, and the only stuff I really actually care about are my pictures, which will be backed up to blu-ray and uploaded somewhere.

Please don't try to make this machine do HTPC duty as well. I'm all for integrated ESXi/ZFS and have run such a setup successfully for 5 years now but for the love of god buy a Roku or something.

Viper GTS

Any reasons for this, or is it just snobbery like half the freenas forums?

"I can make this work, in all my glory, but a peon such as yourself has no chance!"

 

[repoman0]

On the ECC or not to ECC debate:

If you care about your data enough to deploy ZFS why would you NOT also run ECC? The cost premium is not massive, and most people can get by with UDIMM instead of RDIMM. There are reasons workstations and servers run ECC, there's simply no reason not to.

Anyone who posts here is probably WAY over the wasteful line for home technology before getting to ECC. If you're going to be wasteful do it right.

Viper GTS

The ECC ram itself is not that expensive, but the motherboards and CPUs that support it seem to be. That said, I'd hope they're also correspondingly higher quality in other ways too.

 

[repoman0]

I should add that I'm really not actually that paranoid about data loss. The amount of data that I really care about, as mentioned above, is pretty small in size and can be easily backed up online. I care enough about the convenience of potentially not having to use those backups and to keep the other stuff that I sort of care about safe to buy two hard drives instead of one and to run a local mirroring/checksumming file system (currently using storage spaces and ReFS with mirroring). This works fine for me, except that there's no mirroring/checksumming file system that I can use from both Windows and Linux. So naturally, network storage is a reasonable step, and if it can double as something I can actually use and tinker with beyond initial setup, that's great. There doesn't appear to be anything that prevents a dual use machine from being feasible and stable, beyond initial setup time, but please let me know if I'm wrong.

I have no interest in a Roku because I like building my computers and care just as much about running goofy old SNES games as I do about watching TV (care level: not much in either case, but why not have the ability)

 

[frowertr]

Yeah, I'm not against ECC memory. But, as you have found out, your scope is so narrow you are having a hard time finding the parts you need.

There are plenty of NAS devices on the market without ECC. If you can find a board that supports ECC and your other requirements then great! But I wouldn't lose any sleep if you couldn't. ZFS is probably the best filesystem to actually guard against bit rot which is one of the key things that ECC protects against. And bit rot itself happening is such a small percentage. You'd more than likely have other issues before with your array well in advance of having rot.

 

[Viper GTS]

No snobbery at all, just functionally very different tasks and mixing infrastructure with end user devices is miserable. Plus then you're stuck putting your storage box next to your TV where it's going to be hot/noisy vs a dedicated box that is potentially completely silent. Also upgrades/changes to one don't have to impact the other. Want to install a new video card in your HTPC do use HDMI 2.0? Guess what you have to shut down your entire infrastructure. Have people currently watching Plex? Guess you'd better wait until they're done.

They just work better separated.

My setup for reference:

Dual Xeon x5660 w/48 GB ECC
ESXi 6.0
FreeNAS 9 stable path with 16x1 TB HDD + 2 ZIL devices (via PCI passthrough of 3 storage controllers)
Plex VM on Ubuntu
Various other infrastructure VMs (vcenter, ubiquiti unifi controller, crashplan uploader, etc)

I currently use a Roku 4 for Plex. Server gear all lives in the rack where it belongs.

Viper GTS

 

[frowertr]

I agree with Viper. I'm all for virtualized infrastructure, but I really like physical fileservers. They are just easier to maintain. It's understandable, though, for home use you need to conserve space so a single box is very attractive.

 

[repoman0]

Yeah, I'm not against ECC memory. But, as you have found out, your scope is so narrow you are having a hard time finding the parts you need.

There are plenty of NAS devices on the market without ECC. If you can find a board that supports ECC and your other requirements then great! But I wouldn't lose any sleep if you couldn't. ZFS is probably the best filesystem to actually guard against bit rot which is one of the key things that ECC protects against. And bit rot itself happening is such a small percentage. You'd more than likely have other issues before with your array well in advance of having rot.

Closest thing I've found so far is this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813157466

Around $100-120 more expensive than a normal comparable board, and would need a video card. I'd think "server" chipset hardware would be more stable, but there's plenty of DOA reviews there ...

No snobbery at all, just functionally very different tasks and mixing infrastructure with end user devices is miserable. Plus then you're stuck putting your storage box next to your TV where it's going to be hot/noisy vs a dedicated box that is potentially completely silent. Also upgrades/changes to one don't have to impact the other. Want to install a new video card in your HTPC do use HDMI 2.0? Guess what you have to shut down your entire infrastructure. Have people currently watching Plex? Guess you'd better wait until they're done.

They just work better separated.

My setup for reference:

Dual Xeon x5660 w/48 GB ECC
ESXi 6.0
FreeNAS 9 stable path with 16x1 TB HDD + 2 ZIL devices (via PCI passthrough of 3 storage controllers)
Plex VM on Ubuntu
Various other infrastructure VMs (vcenter, ubiquiti unifi controller, crashplan uploader, etc)

I currently use a Roku 4 for Plex. Server gear all lives in the rack where it belongs.

Viper GTS

I see, in that case thank you for the info, I will definitely keep it in mind. I think your server setup looks much more complicated than mine would ever be, and probably much hotter/louder than a little mini-ITX running only two simple things would be (I'd hope!)

I'll think more about it, but from your examples, I think I can live with the downsides, and if not I'll sell it all and do something else.

By the way I found an interesting analysis on the effects of bit flips in memory on the ZFS file system .. I've skimmed it and might read it in more detail if I get the time. But this is exactly what I was looking for rather than that one freenas guy's crazy hypotheticals.

http://research.cs.wisc.edu/adsl/Publications/zfs-corruption-fast10.pdf

 

[repoman0]

I agree with Viper. I'm all for virtualized infrastructure, but I really like physical fileservers. They are just easier to maintain. It's understandable, though, for home use you need to conserve space so a single box is very attractive.

That's another consideration ... I live in Boston and will be moving to a smaller apartment soon and possibly share it with my GF within a year. I have another computer, big desk, digital piano and guitars and amps, etc. A tiny computer box for the TV and for network storage is enticing for that reason.

 

[frowertr]

Closest thing I've found so far is this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813157466

Around $100-120 more expensive than a normal comparable board, and would need a video card. I'd think "server" chipset hardware would be more stable, but there's plenty of DOA reviews there ...

Doesn't support much RAM. I'm not a ZFS guru so I don't know how much you need for a given array size. I would think 16GB would be fine so long as dedupe is turned off but you need to research that.

 

[Viper GTS]

You can get by with a lot less than 16 GB for FreeNAS if you aren't using deduplication. I run with 12 GB pretty comfortably.

The problem here is that he is going to need 8-12+ GB for FreeNAS, 4-8+ GB for his HTPC VM, and ESXi itself has overhead. I would want 24-32 GB to do all that plus the other things I'd inevitably want in VMware once I had it available - 16 is not going to be enough. Also if you plan on ever doing anything really fun with vmware you will need vCenter - That's another 8+ GB.

Honestly it sounds like he really needs a Synology + an Intel NUC.

Viper GTS

 

[frowertr]

^

Forgot you were virtualizing.

 

[repoman0]

The other one I found is this:

http://www.asrockrack.com/general/productdetail.asp?Model=E3C224D4I-14S#Specifications

Four RAM slots, little bigger but should fit into the case I was looking at supposedly (PC-Q25B)

I can't decide if that short PCI-E x8 slot would accept a video card though, just not using some of the contacts. Other than that, I could step up to one of their newer DDR4 boards, which come in 16GB modules.

 

[Viper GTS]

Have you looked at Xeon D at all? If I were building a storage device/ESXi host today for home use I'd be looking at stuff like this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182973&cm_re=xeon_d-_-13-182-973-_-Product

I've actually been debating dropping my 5 year old setup and replacing it with a trio of Xeon D systems - One for dedicated storage, and two ESXi hosts for a two node cluster.

Viper GTS

 

[repoman0]

Have you looked at Xeon D at all? If I were building a storage device/ESXi host today for home use I'd be looking at stuff like this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182973&cm_re=xeon_d-_-13-182-973-_-Product

I've actually been debating dropping my 5 year old setup and replacing it with a trio of Xeon D systems - One for dedicated storage, and two ESXi hosts for a two node cluster.

Viper GTS

Wow, this looks great! I didn't know about Xeon D, something like this is perfect. Nice price point for CPU included too, especially a quad with hyperthreading. Thanks for the suggestion :thumbsup:

 

[XavierMace]

Yeah, if you're building server class hardware, go Supermicro or go home. Asrock makes decent consumer boards but they've got a lot to learn when it comes to server boards. Xeon-D is the way to go currently. Like Viper, I've been considering replacing my older setup with Xeon-D's.