Plex/Home server build

[RampantAndroid]

Hi all,

Looking for a critique of a possible build I've come up with. It's for a server that'd be used for running a Plex server (and I'd like to comfortably do 2 1080p streams + some room to run websites or game servers). I figured a Xeon E3 would probably be the best bet, but I'm unsure whether I should really care much about moving up to a E3-1246v3 for hyperthreading?

Any thoughts on the Toshiba drives I've selected? I'm planning to either use ZFS (less likely) or ReFS and storage pools (more likely) to do mirroring with some level of data integrity (I already have a server license.)

I've seen the Avoton C2750 recommended, but it sounds like it'd bog down with just 2 streams going, leaving no real room for a game server or website hosting (I have 3-4 websites that currently are on a free AWS VM that I'd move over to my server. Only one is really critical, and I'll leave it with AWS until it's no longer free...it won't be critical by then.)

I'm going with the 1226 (or 1246) so that I get the integrated graphics so I don't need to worry about finding a card for setup - it'll run headless otherwise.

The case I've chosen since it has a number of extra 3.5" bays I can expand into - I plan to just buy drive pairs and mirror them and skip striping with parity. 5TB is a good starting point I think.

http://pcpartpicker.com/p/sG8Zbv

Thanks!

 

[eton975]

Why not just get a cheap locked i5 instead of the E3 then? You're not using the Xeon's ECC or virtualisation capacbilities.

Would replace the PSU with a SeaSonic 300/350W, EVGA 430W/500B meself. All are cheaper.

 

[smitbret]

The Toshibas are fine for ZFS. I have used a couple of them in my FlexRAID setup and had no trouble. Backblaze seems to think they are pretty reliable in their storage pods so they should be fine for any software RAID.

I would agree that if you aren't going the ECC memory route, there is no point in paying for a Xeon.

Not sure why you picked the Arc Mini as the case but the Fractal Design Define Mini is quieter gonna be quieter.

 

[RampantAndroid]

I would not mind having the virtualization, as one option I've considered is running inside of ESXi since that would let me get freenas (or some other solution) in addition to getting Windows Server. I gather that Refs/storage spaces aren't as great as ZFS right now, but I'm unsure how Sever 2016 is changing that? I omitted ECC RAM because of cost, and because I'm unsure a small home server that serves as a place to dump some backups and movie content really was worth ECC RAM.

Does hyperthreading matter much for my workload?

 

[dave_the_nerd]

I'm unsure a small home server that serves as a place to dump some backups and movie content really was worth ECC RAM.

It's probably not. But there's some reason my current home server is a crapton more stable and reliable than the desktop-parts-based system it replaced, and ECC is one of the things it has that the old machine didn't.

Not trying to scare you, just saying - my anecdata says ECC is a possible contributing factor to my great degree of happiness.

Flipside is the cost, and finding a motherboard and CPU that support it. (Both have to be compatible.)

Does hyperthreading matter much for my workload?

If you're doing a lot of VMs, it's helpful. I don't think it's a big price difference.

Passing your disks through for a FreeNAS VM to manage is a pain in the arse, but it would also require an i5/i7/Xeon for hardware passthrough. If you wanted to do ECC RAM too, you'd need the Xeon. (It's not supported by i5/i7 CPUs.) It's doable, but for simplicity's sake, I'd usually recommend that your bare-metal OS be the one managing disks.

Ideally, Linux with ZFS-on-Linux to manage your disks / drive pool, and probably hosting your file shares. Then use Xen as a hypervisor to run whatever server OSes you want to, Plex server, etc. (That's basically what I'm doing - except I'm using FreeBSD and Virtualbox because reasons.)

My Plex VM (Ubuntu) mounts the Movies and Music folders from the NAS environment (bare-metal FreeBSD) via NFS, and runs fine out of there.

Also, many low and midrange server/workstation boards include onboard graphics (a crappy console GPU with 8MB VRAM, basically). Most commonly found on boards with IPMI. So if you're running a text-based OS like a Linux server distro, you can happily use a Xeon without onboard graphics. (See server in my sig for hardware that supports what I'm talking about.)

 

[dave_the_nerd]

Getting a similar build with ECC would be around $200 more, but the motherboard would support all the virtualization goodies, out of band management, etc.

RAM - http://www.crucial.com/usa/en/x10slm-f/CT4485065
Mobo - http://www.newegg.com/Product/Product.aspx?Item=9SIA24G3435376

 

[RampantAndroid]

Yeah - I understood that I needed ECC support from both MB and CPU - and that it cost a bit more. I had seen some boards with IPMI and I knew that it did something, but I don't know how to use it (it looks like it's an ethernet port. Do I just access the system through PUTTY?)

For running VMs - does it work well enough to run a Windows VM for example? The idea of hosting a game is one I don't think I'd want to drop...and I've never tried that within a VM.

How well does that Pentium work for you? It's about as powerful as the Avoton it looks like - I'm guessing it can only do one stream worth of transcode? How much does the system bog down when doing a transcode?

Finally, is ZFS really worth it? Will it be remarkably better for me over just giving up on Freenas and installing only Windows Server 2016 and using REFS/Storage spaces? I've read up some on them, and it sounds like REFS only has some integrity checksums on metadata, whereas ZFS has more? I don't think deduplication is anything I'll have a use for, since most of my space will be used by videos, music, photos and such...which don't lend themselves to dedup.

Thanks for the replies!!

 

[dave_the_nerd]

Yeah - I understood that I needed ECC support from both MB and CPU - and that it cost a bit more. I had seen some boards with IPMI and I knew that it did something, but I don't know how to use it (it looks like it's an ethernet port. Do I just access the system through PUTTY?)

Not quite. It's a web server, basically - you can log into it just like you do your router status page, and get information about the server, power cycle it, etc. It's also got a built-in KVM, so you can "remote control" the system from another computer on your network. (So I never need to have a monitor, keyboard, or mouse connected to mine.) You can also do things like attach ISOs or something over the network as virtual devices. (So, install Windows on a server from an ISO that's saved on your laptop.)

It's pretty cool. Definitely a nice-to-have and not a critical need, though.

For running VMs - does it work well enough to run a Windows VM for example? The idea of hosting a game is one I don't think I'd want to drop...and I've never tried that within a VM.

Yeah, I have a couple VMs on mine that are running desktop OSes. They're fine for what I use them for (stupid things that are limited by my ISP and can run overnight, but which I don't want to leave my main rig on for.) But that said, the hypervisor I use (Virtualbox) is a type 2 and not a type 1, so it's inherently slower, particularly for disk I/O.

How well does that Pentium work for you? It's about as powerful as the Avoton it looks like - I'm guessing it can only do one stream worth of transcode? How much does the system bog down when doing a transcode?

Well, I'm only dealing with standard def video, DVD rips mostly. 2-3 clients at a time uses maybe 50% of the CPU. But I'm not sure they're all actually transcoding either - if a video file can play on a client as-is, Plex won't bother transcoding it.

If you were really concerned about transcode performance for Blu-Ray rips or other HD content, you'd probably want to stick with the Xeon.

I can, however, play MineCraft (Another server VM) and watch a movie at the same time, so I'm happy.

The Pentium is probably slower than the Avoton (aren't they 8-core?) in overall compute, but the single-threaded performance is crazy-better, which is nice. It was also a $55 CPU that supported ECC. (!)

Finally, is ZFS really worth it? Will it be remarkably better for me over just giving up on Freenas and installing only Windows Server 2016 and using REFS/Storage spaces? I've read up some on them, and it sounds like REFS only has some integrity checksums on metadata, whereas ZFS has more? I don't think deduplication is anything I'll have a use for, since most of my space will be used by videos, music, photos and such...which don't lend themselves to dedup.

Thanks for the replies!!

If you're a Linux guy and want to do Linux/BSD, ZFS is the best file system available in terms of data integrity - it's also well documented and there's a fairly wide base of users and use cases, so it shouldn't be too hard to get your questions answered. (That can sometimes be a problem in the Linux world.)

It also does some cool stuff in terms of managing drive pools, taking snapshots, doing dedup, etc. And it implements software RAID, too. So it's a nice all-in-one solution.

If you're a Windows guy and want to do Windows, then ZFS isn't really an option and the choice is between Storage Spaces and something third-party like FlexRAID, both of which will manage your disks, then putting a file system on top of it.

I'm not sufficiently familiar with either to make a recommendation, but FlexRAID definitely has some fans. I also hear good things about Storage Spaces.

For a file system, ReFS (built into Windows 8 Server and later) is a journaling file system that also does copy-on-write, so it should be comparable to ZFS in terms of data integrity. (Although by saying so, I have probably caused several peoples' ears to start burning, and await the flame fest). I haven't messed with it myself, though.

 

[XavierMace]

Basically, ReFS and Storage Spaces is getting closer, but ZFS is really mature at this point. You're hard pressed to beat it. Especially on decent hardware (meaning plenty of RAM and an SSD for ZIL).

That's with 7 VM's running, including the File Server with an active stream. Pool is using standard 7200rpm 2TB Seagate SATA drives.

 

[dave_the_nerd]

Basically, ReFS and Storage Spaces is getting closer, but ZFS is really mature at this point. You're hard pressed to beat it.

Sure, but if you're a Windows guy...

Especially on decent hardware (meaning plenty of RAM and an SSD for ZIL).

That's with 7 VM's running, including the File Server with an active stream. Pool is using standard 7200rpm 2TB Seagate SATA drives.

...how many? How much RAM? Any L2ARC? If you duct tape enough model rockets together, you can fly a bus to the moon.

You have provided a single benchmark with no context, and I am obligated to give you crap. (For shame!) ()

Nice numbers though - mine isn't nearly that fast. Unless I tinker with the count so the size written/read fits neatly inside my cache, then it's as fast or faster. :awe:

 

[RampantAndroid]

In looking at what I want to do, I think I might just say Windows Server 2016 - at least to play with.

I've got a 128GB crucial SSD already, but I can get a board that supports M.2 and a new SSD. Is there a good reason to? It sounds like even if I go ZFS I don't need a ZIL for my workload (and my workload will mostly be sequential reads and writes.)

I looked into the avoton, but it was considerably more expensive than an i5, owing in part to ECC RAM and partially to the existence of <$100 h97/z97 boards. So I'll aim for a i5-4460 I think. It has the virtualization stuff, just not ECC or dual NICs.

 

[dave_the_nerd]

In looking at what I want to do, I think I might just say Windows Server 2016 - at least to play with.

I've got a 128GB crucial SSD already, but I can get a board that supports M.2 and a new SSD. Is there a good reason to? It sounds like even if I go ZFS I don't need a ZIL for my workload (and my workload will mostly be sequential reads and writes.)

For a PLEX/transcoding server, you're almost certainly going to be CPU limited almost no matter what you do (unless you're streaming uncompressing 4k video or something.) So I would tend to say it's not worth it.

You can always add a PCI-E SSD later if you need crazy-awesome cacheing.

I looked into the avoton, but it was considerably more expensive than an i5, owing in part to ECC RAM and partially to the existence of <$100 h97/z97 boards. So I'll aim for a i5-4460 I think. It has the virtualization stuff, just not ECC or dual NICs.

Okay. ()

 

[bbhaag]

I read threads like this and it makes me think I really under did my home server. It's literally an external 4tb drive and a couple of fire tv sticks running Plex on an old ass AMD 965.

 

[XavierMace]

Sure, but if you're a Windows guy...

...how many? How much RAM? Any L2ARC? If you duct tape enough model rockets together, you can fly a bus to the moon.

You have provided a single benchmark with no context, and I am obligated to give you crap. (For shame!) ()

Nice numbers though - mine isn't nearly that fast. Unless I tinker with the count so the size written/read fits neatly inside my cache, then it's as fast or faster. :awe:

I am a Windows guy. LOL. Check out the storage sub forum. I have to call out to Gea everytime I run into an issue. LOL. I am learning though, which was half the point. Solaris + napp-it gives me a free (for "development" use), robust, and fast storage system with an easy to use interface. Gea's guide on setting it all up is very detailed and easy to follow. The only thing that made my setup more difficult is I'm running Fiber Channel to the hosts rather than iSCSI.

Fair enough about the context. 12 drives in RaidZ2. 64Gb, no L2ARC. That wasn't meant to be a scientific test. In retrospect now that I'm not half asleep it's not even a terribly useful one, other than to show that it's still got plenty of oomph to spare even with medium disk access on it since the OP was asking about multiple streams. I've got my UTM, PDC, File/Media Server, 3 Utility servers (vCenter, vCops, and System Center), a Win7 sandbox VM, and a Game Server running at the same time. I think I had a network install of windows running when I ran that benchmark. The File/Media Server has 4 vCPU, sits at about tops out at 20% CPU usage when streaming a 1080p/5.1 stream. Disk Benchmark on the Media Server VM with the stream running:

-----------------------------------------------------------------------
CrystalDiskMark 3.0.3 x64 (C) 2007-2013 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

Sequential Read : 300.366 MB/s
Sequential Write : 28.690 MB/s
Random Read 512KB : 284.106 MB/s
Random Write 512KB : 23.875 MB/s
Random Read 4KB (QD=1) : 19.138 MB/s [ 4672.3 IOPS]
Random Write 4KB (QD=1) : 0.719 MB/s [ 175.5 IOPS]
Random Read 4KB (QD=32) : 243.971 MB/s [ 59563.3 IOPS]
Random Write 4KB (QD=32) : 6.295 MB/s [ 1537.0 IOPS]

Test : 2000 MB [C: 32.2% (64.3/199.7 GB)] (x1)

No impact on the stream while testing.

 

[dave_the_nerd]

I am a Windows guy. LOL. Check out the storage sub forum. I have to call out to Gea everytime I run into an issue. LOL. I am learning though, which was half the point. Solaris + napp-it gives me a free (for "development" use), robust, and fast storage system with an easy to use interface. Gea's guide on setting it all up is very detailed and easy to follow. The only thing that made my setup more difficult is I'm running Fiber Channel to the hosts rather than iSCSI.

Fair enough about the context. 12 drives in RaidZ2. 64Gb, no L2ARC. That wasn't meant to be a scientific test. In retrospect now that I'm not half asleep it's not even a terribly useful one, other than to show that it's still got plenty of oomph to spare even with medium disk access on it since the OP was asking about multiple streams. I've got my UTM, PDC, File/Media Server, 3 Utility servers (vCenter, vCops, and System Center), a Win7 sandbox VM, and a Game Server running at the same time. I think I had a network install of windows running when I ran that benchmark. The File/Media Server has 4 vCPU, sits at about tops out at 20% CPU usage when streaming a 1080p/5.1 stream. Disk Benchmark on the Media Server VM with the stream running:



No impact on the stream while testing.

Ah, I see. All your data set sizes (40GB) could fit inside the RAM on the NAS/SAN server with room to spare. It's probably using that as write cache.

Try doing the DD test from before, but triple the "count" parameter. The final result should be a better representation.

 

[smitbret]

I read threads like this and it makes me think I really under did my home server. It's literally an external 4tb drive and a couple of fire tv sticks running Plex on an old ass AMD 965.

Does it work for you?

If you answer "yes" then you are fine.