Cheap ATX-ish AES/ECC/IPMI home server - long term market watch

[_Rick_]

Hey all,

I'm currently running a multi purpose home server:

- Router (iptables)/dial-in
- SMB, NFS (main storage for my desktop)
- TV-streaming to clients via vlc
- audio output to my decoder via optical audio
- Teamspeak and possible game servers
- logitech media server
- FireWire card reader for my CF cards
- headless
- 12HDD + 2SSD + 1 ODD
- ATX original CM Stacker STC-01
- i5 650, 8GB DDR3, Gigabyte P55-UD5, Promise PCI ATA controller

The Realtek NICs appear to be crapping out though, with them dropping out and resynching to 100Mbit every now and then. Also throughput isn't great. Additionally, I've recently been getting weird issues, with programs crashing inexplicably. Could be bad RAM, but the one in there has done sterling services in my desktop for over a year.

What I'm looking for, is more of the same, but with two extra features ( as per the title):
ECC - I was hoping that the dual core i5 would have ECC, but this is not so. Therefore, to get ECC + AES, on the Intel side, we're looking at an E3 Xeon.
IPMI - Intel's AMT integrates IPMI onboard, if the OEM decides to support it. Not sure if AMD even supports something like that. This feature is probably worth around $50 to me - especially if there are onboard video outputs. Currently I have to install the graphics card every time I have to debug the boot, or SSH goes down. This is a bit of a hassle.

Low power consumption is also a must. Performance outside of AES is mostly irrelevant. I suppose a single core Atom would be pushing it a little. It's nice to have a reasonable level to make running Gentoo a continued possibility, but then I've been running that on a Celeron 333 until a few years ago. I might run ZFS on the machine, once there is open source encryption support for it.

Box has to run Linux (driver support for LAN/HBA/etc). Box has to support my two ATA SSDs as well as 12x HDDs and the ATA ODD. Ideally by supporting booting from the PCI-Promise. I would like to give

So, if I had to buy today it would be:

Xeon E3-1225V3 - 200 euro
2x8 GB Kingston Value RAM ECC 1333/CL9 - 130 euro
Marvell 88SX7042 4x SATA2 HBA - 65 euro
Supermicro X10SAE - 210 euro

I'd also like to (have to, due to lack of IDE) switch two 80GB HDDs that currently serve to host /var to SSDs. 160 euros for two Samsung 840 or Plextor M5S. Two mounting braces add another 10 euro or so.

Also would have to throw out one more old 40 GB P-ATA HDDs. Cheap 3 TB drives currently run around 95 euro. (Don't care for special RAID editions - I run RAID so my drives can be dispensable)

If I want to avoid using the IPMI-LAN as external interface, then I might need any old PCIe LAN adapter to hook up the cable/dsl/fiber/whatever. Another i210T1 would set me back around 60 euros.

Total sum at this point: 870 + 60. +10% shipping or limiting number of dealers, and we're at ~965 + 70.

The goal of this thread: Figure out whether there's a cheaper/better/more efficient way to achieve these goals, and keep ahead of the developments, as well as a log of my own progress towards an eventual buying decision.

So, feel free to comment/chime in.

 

[mfenn]

What country are you shopping in? What vendors do you use?

Your CPU and RAM look fine to me. I'd also look at the V2 Xeons because Ivy Bridge is certainly fast enough for your uses, and there will be more of a motherboard selection.

As for the storage, it looks like 14 + 1 for the ODD? If you get a board with the SuperMicro IPMI, then you don't need the ODD because you can use virtual media. That means 14 SATA ports.

As for the motherboard, you've picked out a workstation part, which means you're missing out on server goodies like built-in IPMI support. I would check out the SupeMicro X10SL7-F. It costs more up-front than the X10SAE, but you get 14 SATA ports and IPMI built-in so it will likely come out cheaper in the end. It has a dedicated IPMI LAN port, so you don't have to worry about sharing it with an external interface.

 

[Cerb]

Also, if you're spending so much money on CPU and mobo like that, get a new ODD. They're cheap, and SATA.

For optical out, you will probably need to buy a Linux-friendly sound card, that can function OotB as a transport. Turtle Beach makes a couple USB ones that might do the job, but I'd not sure of Linux compatibility.

Also, the Plextor M5S is probably a better bet than the Samsung 840, with Linux and an unknown disk/array configuration. The 840 has pretty fragile non-TRIM performance, and Linux servers are notorious for backing such drives into corners.

Oh, and I missed this on my first read-through:

Also would have to throw out one more old 40 GB P-ATA HDDs. Cheap 3 TB drives currently run around 95 euro. (Don't care for special RAID editions - I run RAID so my drives can be dispensable)

RAID edition drives are just as disposable as normal ones. You're being misled by marketing. RAID edition drives, and WD Reds, have a hard-coded 7-second error-correcting timeout. Normal desktop drives may try to correct for a couple minutes before giving up, which can cause them to drop out of an array. This is generally only a concern for hardware RAID. With software RAID, REs only provide a practical benefit of an added warranty, regardless of if the drives are binned for reliability or not.

 

[_Rick_]

Regarding Audio out: The ALC 1150 on the X10SAE should have Linux support. Normally those intel HDA / realtek solutions are well supported. The current 889A works flawlessly.

I'll be shopping probably in Germany, for simple availability reasons. Sona is one of the few shops where you can get Supermicro boards. Plus, the high number of e-tailers in Germany makes for lower prices. Otherwise I currently reside in France. UK shops are also an option. Can't spread myself too thinly though, because then shipping costs really are going to end up biting me 😀

As for the ODD - I'm using the machine also to rip my audio CDs. Getting a new ODD makes no sense, as I have the two IDE 8GB SLC-SSDs in there that run the OS. (For the next build I may RAID 1 the / partition) So I require a two port ATA controller, unless I want to throw those ~150 euro of investment that work mostly perfectly (one of them had hiccups, hence I got a second, different, one which has no issues). Technically I suppose I could do without them though, if I migrate to a pair of 128 GB SSDs. But then I'd still need 13 SATA, which is a bit of a worst case situation.


The micro ATX board is around 40 euros more expensive, saves me 60 + 65 for LAN and HBA, but I'd need a sound card. The Xonar DGX has very fresh ALSA support (3.5) and would add 30 euro, the DSX (50 euro)has no driver (but I could swap it with the DX in my desktop, which I only use for dolby encoding anyway- THX should be equivalent) and the DX is supported and costs 60 euro.
I would net around 20-55 euro then, with possibly 45 being a sweet spot, at the cost of losing expandability - both PCIe slots would be in use from day one with TV card and sound card. 14 SATA should be enough though, same for 3 LAN.
Of course, this only gets me ahead, if I don't want to use the second LAN port as the external interface. That depends on how AMT needs to be setup, and whether there's some way to put the port in IPMI mode, in case of a hangup, or something like that. Otherwise, it's 15 euro more, for 5 expansion ports less.

Edit: Actually, I could also step down to a 1220V3 with a board with dedicated IPMI. That would further reduce cost by around 20 euro. But i also forgot budgeting for the SATA ODD which eliminates that gain neatly.

 

[Cerb]

As for the ODD - I'm using the machine also to rip my audio CDs. Getting a new ODD makes no sense, as I have the two IDE 8GB SLC-SSDs in there that run the OS.

Ah, I was not aware of that (I was thinking the 2 new SSDs would be the 2 SSDs inside, once done). That said, the new SSDs could be made to run the OS, as well, if given small partitions for it, or just with the 2 SSDs as /.

If you have to get a new PATA card, make it a Silicon Image or JMicron (all the JMicron hate comes from customized motherboard implementations--reference mobo and card implementations have been rock solid, IME). If it's a SIL, and comes stock with a RAID BIOS (common), temporarily install it in a Windows computer, to easily flash it to the base firmware, which supports ODDs.

If/when you do get a SATA ODD, for CDs, the Asus DRW-24B1ST and Samsung SH-224BB are very good at C2 error reporting and handling. LiteOns are a risk, these days, as they'll have different controller chips for the same model drive, and extremely few retailers list the submodel (now a letter suffix). LGs vary by model, but the good ones are very good, too.

 

[mfenn]

As for the ODD - I'm using the machine also to rip my audio CDs. Getting a new ODD makes no sense, as I have the two IDE 8GB SLC-SSDs in there that run the OS. (For the next build I may RAID 1 the / partition) So I require a two port ATA controller, unless I want to throw those ~150 euro of investment that work mostly perfectly (one of them had hiccups, hence I got a second, different, one which has no issues). Technically I suppose I could do without them though, if I migrate to a pair of 128 GB SSDs. But then I'd still need 13 SATA, which is a bit of a worst case situation.

My advice, stop throwing good money after bad and sell off any IDE gear that you have. You're painting yourself into a corner of needed old tech and you're not getting anywhere near the performance you should be. Using a tiny SLC drives for your root filesystem makes next to no sense anyway, since you're rarely writing there.

The micro ATX board is around 40 euros more expensive, saves me 60 + 65 for LAN and HBA, but I'd need a sound card. The Xonar DGX has very fresh ALSA support (3.5) and would add 30 euro, the DSX (50 euro)has no driver (but I could swap it with the DX in my desktop, which I only use for dolby encoding anyway- THX should be equivalent) and the DX is supported and costs 60 euro.

I guess I don't understand how using a server to rip CDs is a win. You will have to get up to change them rather than just doing it at your desktop where you are already sitting.

Of course, this only gets me ahead, if I don't want to use the second LAN port as the external interface. That depends on how AMT needs to be setup, and whether there's some way to put the port in IPMI mode, in case of a hangup, or something like that. Otherwise, it's 15 euro more, for 5 expansion ports less.

The SuperMicro IPMI controller is really very flexible and well understood. IMHO, AMT is a complete pain in the ass if you're not doing a full Windows-centric enterprise deployment. I've never worked in a place that actually used it successfully on Linux workstations.

 

[_Rick_]

Must have missed this reply back then and just stumbled across the thread again.

My advice, stop throwing good money after bad and sell off any IDE gear that you have. You're painting yourself into a corner of needed old tech and you're not getting anywhere near the performance you should be. Using a tiny SLC drives for your root filesystem makes next to no sense anyway, since you're rarely writing there.

I definitely see the point of doing without the ATA SSDs by consolidating it with larger new SSDs.
As to why I chose SLC? Because they are so small, each write is much more "expensive". 80 TB writes is 10k writes to each cell in the best case, so going SLC with such a small device was a must to me. Plus, with the small drives, getting any kind of decent bandwidth also requires SLC, as parallelism isn't that great.

I guess I don't understand how using a server to rip CDs is a win. You will have to get up to change them rather than just doing it at your desktop where you are already sitting.

I usually just pop the disk in after buying it, and before putting it in storage. I have a reasonably automated setup for Linux, via ripit, which means I just have to keep feeding CDs every now and then. Plus, not having to shuffle data over the network kind of seems a nice idea. But yeah, it's a tiny bit idiosyncratic.

The SuperMicro IPMI controller is really very flexible and well understood. IMHO, AMT is a complete pain in the ass if you're not doing a full Windows-centric enterprise deployment. I've never worked in a place that actually used it successfully on Linux workstations.

Hmm, those news about AMT are a bit of a downer. I was hoping it'd be something that would "just work" once setup in the BIOS. After all, all it has to do is run the VNC server from the ROM and send out whatever the iGPU is supposed to render.

Of course, in the meantime the machine woes have calmed down a bit again. Possibly some of it was merely a bad kernel. Haswell board choices should in the meantime go up, and prices down, so I'll probably revisit this thread in January / February.