Building a PC-based NAS

[Balforth]

Are there any good guides out there? I'm not sure what to aim for with processing power, ram, power supply specs, using on-board controllers vs a separate raid controller, etc.

I need it to be windows 7 based, which isn't my first choice, but it will need to serve some additional duties that are dependent on windows 7. The main purpose of the NAS is simply streaming video to multiple clients.

It seems like software RAID might be the cheapest and easiest way to go... I've come across some software like FlexRaid, but I'm really not sure what other options may exist.

Can anybody offer any adivce?

 

[dave_the_nerd]

Windows 7 has built-in software RAID. You can use third-party software like FlexRAID too, but it's not necessary. If you're using software RAID, that nixes one of the advantages of a hardware RAID card. If you're only serving a few clients (home server) you can probably "get away" with using the motherboard's onboard ports. (The additional performance of a dedicated RAID controller will be bottlenecked by the network anyway.)

Most PC NAS boxes end up looking pretty similar - you'll probably end up with an ATX or mATX motherboard with lots of SATA ports and no SLI capabilities, a slower quad core CPU or a fast dualie, depending on threading and number of clients, and 8-16GBs of RAM which may or not be ECC, in a smaller case with easy-to-access HD bays. There are kits that convert multiple 5.25" bays to hot swappable 3.5" bays. Then throw in a few "Silent" fans, OEM HSF, and an 80+ PSU that isn't big enough to power dedicated GPUs.

What other stuff are you using the machine for? That will probably inform your buying decisions more.

 

[Balforth]

Ah, I thought there were some size limitation for a native windows 7 array. I'm currently only using 4x2TB drives right now, but I wanted the ability to expand in the future.

Outside of hosting the array, I'll need it to run a program called Media Center Master, utorrent, and sabnzbd. Obviously I'll also be using either Windows RDP or VNC for remote access. That will be it's dedicated functionality.

I'm not concerned with hot-swapability and have no problem just mounting the hard drives in the case. I'm a bit surprised at the psu -- only 80 watts? I guess I'm accustomed to building more powerful machines, so I'm not even aware of the existence of such low powered psu's

 

[aigomorla]

when setting up a nas...

1. File Formats... what do u want compatiable? are u using linux machines... will u be running linux? what OS is the NAS on?

2. Type of Storage... how are u setting it up... you seem to say you have 4x2TB.
How important is redundancy? How fast do you want the array to be if ur creating a Raid array.

3. Again... lots of file formats.. u can use ZFS, NTSF, or u can try on the failed new file system from w8 RTFS.


Typically the cheapest is to put sharing on a system thats typically on most of the time and map the share drive.

If you would like to build a Standalone Headless NAS... i suggest u look at FREENAS. Spend about 20 minutes on youtube and look at installation.

But u would typically want:
1. a Very low wattage cpu.... or the cheapest cpu u can get... a Pentium G2020 is fine here.
2. As much ram as u can possibly fit, large raid vol. benifit from large ram pool.
3. a Case which isnt too big that can handle all the drives u would like....
Fractal Node... CM101 <--- very cute case.. however very limited in HDD storage..
Ect...


Last advice..
1. NEVER and i say NEVER have your torrent machine be anything but a torrent machine.
2. Do not give it write access to your NAS on a torrent machine... ever... transfer the files it copies into a share dump drive, and then move it from dump to NAS on a different secured machine.
3. Your NAS is your vault... u dont want to put the vault in the front of the bank do you?

Those 3 rules will help u keep your NAS, clean.... dont attach a NAS to an outside gateway machine with unlocked access, that machine has the potential to nuke your NAS.
When setting up FTP rules... MAKE SURE u set proper root folders, and make sure u know what ur opening.
Your NAS u set to read only to all the public machines on your network... Write access u give a secure machine which u know is protected so it will NEVER nuke your NAS.
You can always RDP headless to that secured machine and transfer files though Remote Desktop Access.

 

[Balforth]

Ouch. Great advice, but it stings

1) All clients are Windows 7 or MacBooks. No linux File formats are nothing out of the ordinary... mkv, mpg, avi. Currently my NAS is a Netgear ReadyNAS NV+. The new NAS was going to run Windows 7 because I intended to double as my torrent/usenet/media center master machine.

2) I've been running RAID 5 because I like the idea of being able to recover from a single drive failure. I want the best performance possible, obviously, but I also don't want to lose my entire library if there is an issue.

My issue is that my NAS currently resides on my border router, with gig-e network connecting it to my living room switch where my HTPC lives. My HTPC is in constant use, streaming videos from the NAS. However, it serves as my torrent machine also, and I use Media Center Master to automatically process/move them to the NAS. When it does this, the HTPC starts stuttering and buffering the videos it's reading from the NAS. I started looking at options for a more powerful NAS and was considering this guy with pretty impressive performance:

http://www.smallnetbuilder.com/nas/nas-reviews/31909-synology-ds412-diskstation-reviewed

But then I though, why not just offload all the work to a PC and relieve my HTPC of all the downloading/moving/etc. That's when I got the idea to have a single machine to handle all of that + serve as the NAS. Media Center Master is windows based, so the NAS would have needed to be windows 7.

Definitely don't have the budget to build two new PCs, one for NAS and the other for torrents. Ugh... not sure what to do now.

 

[dave_the_nerd]

I'm a bit surprised at the psu -- only 80 watts? I guess I'm accustomed to building more powerful machines, so I'm not even aware of the existence of such low powered psu's

Not 80w, >80% efficiency.

Overview here:

http://www.hardwaresecrets.com/article/Understanding-the-80-Plus-Certification/742

Basically, if you're building a server, it's on 24/7. The more efficiently the PSU delivers its load, the less money you spend on electricity (and the less waste heat is generated, etc.)

So, ideally, you'd get a PSU that's "right-sized" such that your normal component power draw will be 50-70% of the PSU's rated capacity.

 

[dave_the_nerd]

Definitely don't have the budget to build two new PCs, one for NAS and the other for torrents. Ugh... not sure what to do now.

Torrent box = FreeNAS VM on main box, with separate storage?

Could work.

 

[Balforth]

Well, the problem is the torrent box will also run Media Center Master, which is the requirement for Windows 7. So the main box would have to be win7 with torrent/mcm, running a vm with freenas... Not sure how the separation of storage would work.. if i could leave the drives unmounted in windows but give freenas full direct control to the drives to build the array?

Thats beyond anything I've ever done, but if it would work, I would do it!

 

[dave_the_nerd]

Not quite.

Depending on the degree of separation you want between your torrent box and your storage, you build your everything-but-torrent box, then install a lightweight OS with a torrent client (I suggested freeness, but you could use nearly anything) on a VM. It might be separate "enough."

 

[Balforth]

Therein lies the problem My torrent box needs to also run Media Center Master, which is dependent on Windows. So the VM would have to be windows. Even on powerful machines, my experience running a Windows VM inside Windows is extremely poor.

Hmmm....

 

[code65536]

Note that Windows software RAID is only RAID-0 or RAID-1 for client editions (e.g., Windows 7). Windows software RAID-5 is and always has been limited to the server editions.

If you want parity (instead of mirroring) and Media Center, then you should be using 8.1 instead of 7. Storage Spaces with parity in 8.1 is just like RAID-5 (except with better flexibility).

My server:
* Ivy Bridge Celeron G1620 ($40 from Micro Center )
* 8GB of RAM (at the time, I thought I might also run some virtual machines; 4GB or even 2GB is fine; the advice to get more RAM makes sense if you're running a memory-heavy file system like ZFS, but for Windows and NTFS, that RAM would just go to waste)
* Windows 8.1 x64. I needed Windows for Media Center. The machine also has a torrent client.
* 4x4TB drives in parity in Storage Spaces; 12TB of data, 4TB of parity.
* 1xSSD for the OS/boot drive.
* 1x500GB 2.5" HDD as a "staging" drive. This is optional. I record shows and torrent into this staging drive and stuff I want to keep, I periodically move onto the 4x4TB. So if I torrent something overnight, downloading to the staging drive means that the 4x4 can idle and spin down instead of spinning for hours doing low-volume work, and this also means that Media Center doesn't have to wait for the 3.5" drives to spin up if they have been idle (2.5" drives spin up almost instantaneously, whereas 3.5" drives take many seconds). Anyway, my staging drive is useful to me mostly because my personal usage patterns are such that my 3.5" drives are idle and spun down most of the time; for others, such a drive may be pointless.

All this is in a nice Chenbro case (that was discontinued a couple of years ago) with hot-swap bays for the 3.5" drives. The PSU is only 120W, but that's more than enough because the system, at its peak, draws less than 70W (measured at the wall) (running Prime95 to load the CPU at the same time as copying data onto the array to load the disks). It idles at only 22W.

If you do go down the Storage Spaces route (which I've been very happy with), you should read this post about setting it up properly.

 

[dave_the_nerd]

Therein lies the problem My torrent box needs to also run Media Center Master, which is dependent on Windows. So the VM would have to be windows. Even on powerful machines, my experience running a Windows VM inside Windows is extremely poor.

Hmmm....

No. Your torrent box cannot run anything but the Torrent software. Not allowed.

Once you have verified that the downloads are virus/trojan free, you are allowed to copy them manually to your NAS.

See what we're getting at here? I mean, yeah, you can do it your way, it's just a really bad idea.

 

[Balforth]

Now that I think about it, I'm not maxing out my network, and I'm definitely now maxing out my HTPC (it's a pretty powerful gaming HTPC), I'm only maxing out the processing capabilities of my current NAS. So is my obsession with off-loading the work from my HTPC unfounded? As long as the NAS can keep up, I should still be able to stream video to my HTPC and have the HTPC copying files to the NAS over my gig-e network... And no matter where the file processing/moving is occurring (even from the OS/staging drive on the NAS it self), it's going to hit the same limitations if it's being copied to the array (from anywhere) while I'm trying to read/stream from the array.

Right?

So the only thing that's really going to help me here is a better NAS. The easy route would be going with the Synology DS412+ I was originally considering purchasing for $600. Can I get similar performance from a custom built FreeNAS system for less money? Smallnetbuilder benchamarked the Synology performing reads around 89.9MB/s and writes at 125.9MB/s.

 

[Balforth]

No. Your torrent box cannot run anything but the Torrent software. Not allowed.

Once you have verified that the downloads are virus/trojan free, you are allowed to copy them manually to your NAS.

See what we're getting at here? I mean, yeah, you can do it your way, it's just a really bad idea.

Ya, I get it But I just went from upgrading my NAS to building 3 machines... a torrent machine, a media center master machine, and a NAS. I'm not even separating that functionality now...

For several years now I've ran Media Center Master, which lets me add tv shows, automatically downloads new ones when they're available, downloads metadata, artwork, etc, renames them to a standard format, and moves them to the appropriate folder on my NAS. Whenever I sit down and open up XBMC, I have all the new shows sitting there waiting for me. That's half the joy of my entire home network/NAS/HTPC. I can't imagine manually moving each file. MCM does allow for custom post-processing execution, and I suppose I could kick off a virus scan, but I can't imagine running that many separate machines just for downloading.

I guess if I end up having negative consequences I'll just have to deal with it, but I imagine it'll take me less time to clean up any possible virus that might come through than the total combined time it would have taken me to manually rename and move every file I've ever pulled down over the past 3 years. I do keep daily backups of important documents/files on the NAS.

If I had more money, things would be different, but for now I guess I'll just keep living life on the edge!

 

[code65536]

No. Your torrent box cannot run anything but the Torrent software. Not allowed.

That's pretty extreme.

What security problems do torrents pose? There can be an attack against the torrent client, but the last time there was a security hole in uTorrent was years ago. For something as relatively simple and well-defined as a torrent protocol, the likelihood of an attack against the client is not that great and poses no more a risk than other Internet-facing software (e.g., a FTP server).

Once you have verified that the downloads are virus/trojan free, you are allowed to copy them manually to your NAS.

There is absolutely nothing dangerous about downloading malware. I do it on purpose now and then if there's an interesting piece of malware that I want to disassemble and analyze. Malicious code is nothing more than inert data until it gets executed. So if you do download malware, it poses a threat only to the system that tries to execute it, not to the system on which the file exists. And if you don't try to execute it, it poses no threat to any system.

The one and only security reason for keeping torrenting separate would be if you're worried about a breach in the torrent client itself, which, as I noted, is probably no more a security risk than the other Internet-facing services a NAS already has.

 

[dave_the_nerd]

That's pretty extreme.

What security problems do torrents pose? There can be an attack against the torrent client, but the last time there was a security hole in uTorrent was years ago. For something as relatively simple and well-defined as a torrent protocol, the likelihood of an attack against the client is not that great and poses no more a risk than other Internet-facing software (e.g., a FTP server).

There is absolutely nothing dangerous about downloading malware. I do it on purpose now and then if there's an interesting piece of malware that I want to disassemble and analyze. Malicious code is nothing more than inert data until it gets executed. So if you do download malware, it poses a threat only to the system that tries to execute it, not to the system on which the file exists. And if you don't try to execute it, it poses no threat to any system.

The one and only security reason for keeping torrenting separate would be if you're worried about a breach in the torrent client itself, which, as I noted, is probably no more a security risk than the other Internet-facing services a NAS already has.

If you're a coder downloading it on purpose to analyze, that's one thing. If you're just some dude (no offense, OP) who's trying to watch Newsroom without an HBO subscription, that's another.

The toolkit you have and the precautions you're taking are completely different. (Unintentionally executed code is, like, what ridiculously high % of malware problems?)

I'd also point out that if he's got his files being downloaded directly into his media folder for his media server software to access, then the potential vulnerabilities are not only the uTorrent client (which I wouldn't actually worry about) but the media software as well.

 

[code65536]

Now that I think about it, I'm not maxing out my network, and I'm definitely now maxing out my HTPC (it's a pretty powerful gaming HTPC), I'm only maxing out the processing capabilities of my current NAS. So is my obsession with off-loading the work from my HTPC unfounded? As long as the NAS can keep up, I should still be able to stream video to my HTPC and have the HTPC copying files to the NAS over my gig-e network... And no matter where the file processing/moving is occurring (even from the OS/staging drive on the NAS it self), it's going to hit the same limitations if it's being copied to the array (from anywhere) while I'm trying to read/stream from the array.

Right?

So the only thing that's really going to help me here is a better NAS. The easy route would be going with the Synology DS412+ I was originally considering purchasing for $600. Can I get similar performance from a custom built FreeNAS system for less money? Smallnetbuilder benchamarked the Synology performing reads around 89.9MB/s and writes at 125.9MB/s.

The first question you should ask is, "Why do I want a NAS instead of just shoving my drives in a regular machine?"

One reason is for very high resiliency; e.g., something like ZFS, which is a self-healing, integrity-checking filesystem with parity resiliency available only on Unix/Linux. This is what FreeNAS uses. Because it's Unix-only and because it takes up a LOT of resources (particularly RAM), ZFS should ideally be run as its own separate machine because it needs that dedicated hardware. If you want the absolute best reliability, this is it, and no Synology box is going to be able to come close to measuring up.

Another (different, conflicting) reason is power consumption. You usually want 24/7 availability of your data, but if you stick your drives in a high-end general-purpose machine, that means you have a high-end machine sucking power 24/7. And there are other things, like scheduled DVR or torrenting, that makes more sense for a 24/7 machine. So instead of keeping a high-end machine on overnight to torrent, you do it on the power-efficient system that you are keeping on 24/7 anyway. This is the route that I picked; the resiliency here isn't as good (I've got parity resiliency, but I don't have a self-healing, self-verifying file system), but I'm mostly concerned about power and heat.

So what do you want? If neither of these matter to you, then there's no reason to not just stick those drives in another PC (e.g., your HTPC) and use that instead. Unless you are using a monster file system like ZFS, storage is not going to burden any system, so there's limited reason to offload it.

As for NAS performance, the speed is going to be limited by the speed of the hard drives, not of the NAS box (unless you get some really crappy prefab); pretty much anything you self-build or any decent prefabbed thing like the Synology will perform similarly). The one exception to this is if you use parity for resiliency instead of mirroring. Parity is very taxing on write speeds, which is one reason why people who want to use parity instead of mirroring go for beefy FreeNAS/ZFS boxes.

 

[code65536]

If you're a coder downloading it on purpose to analyze, that's one thing. If you're just some dude (no offense, OP) who's trying to watch Newsroom without an HBO subscription, that's another.

The toolkit you have and the precautions you're taking are completely different. (Unintentionally executed code is, like, what ridiculously high % of malware problems?)

I'd also point out that if he's got his files being downloaded directly into his media folder for his media server software to access, then the potential vulnerabilities are not only the uTorrent client (which I wouldn't actually worry about) but the media software as well.

It makes no difference if he's a regular dude or a seasoned security researcher; the danger malware poses is to the computer executing it, not the computer physically hosting the file. If he torrents some malware and then goes to execute those files from his laptop, it makes no difference if that file is sitting on a dedicated torrent box or if it's sitting on the NAS.

Nobody says, "I wanna run this malware", so it's always unintentional--my point is that the danger is not in the downloading, but in the execution, and as a result, the danger is to the system executing, not to the system downloading.

 

[Balforth]

Dave: Haha no offense taken. I really don't want to get into this debate, but I do consider myself a bit more ethical than the average downloader -- I still spend tons of money every month on DirectTV, so I'm still paying premium prices for everything I end up watching. The HTPC is just so much more convenient.

Code: I definitely understand the benefits of NAS, and I will either buy a more powerful NAS or build my own. What I was bringing into question was my infatuation with off-loading the downloading/renaming/moving etc to another machine. The entire reason I'm looking at some sort of new solution is because of this:

I have fiber to my house, and an ethernet cable coming in from the demarcation point outside. That gets plugged into a cisco router/gig-e switch. Also connected to that is my Netgear ReadyNAS NV+ (piece of crap), and another cat6 cable running to my living room, where it connects to my ASUS Dark Knight. This serves as the WAP and connects my TV, DTV DVR, receiver, PS3, and HTPC. My HTPC is in constant use by my family and we all frequently watch shows from the NAS wirelessly in different parts of the house. The HTPC also runs Media Center Master. So while we're enjoying a show on the HTPC and MCM starts processing a file and moving it to the NAS, XBMC starts buffering and causing everybody to get frustrated. That is the only thing I want to solve.

They underlying problem is the processor in the ReadyNAS can't handle the simultaneous reading and writing of so much data. If I simply upgrade my NAS, I'll still have my HTPC reading data between the two routers and across that cat6 cable while it's writing data along the same path. If I built my own NAS and ran MCM, utorrent, and my usenet client all on one box, all of that bandwidth would be freed up. The NAS would be doing all that work, self contained.

My latest revelation is two fold... 1) It doesn't matter that the HTPC is both reading and writing to the NAS at the same time. It has plenty of power and I'm not saturating my network by any means.

2) Even if I had a windows 7 nas, with a staging/OS drive attached to my border router/swtich (internal facing), and my 4 2TB drives in an array, the array could still only read and write so fast... it wouldn't know the data was coming from a drive local to it's host vs over the network (I think). The same is true even if I had a separate torrent machine connected with the Synology NAS connected to the border router. No matter what, the real limitation is going to come from how well the NAS can handle the simultaneous reading and writing.

Am I making any sense? Just trying to figure out the real bottleneck and where I can throw the least amount of money and see the biggest results... ie, XMBC not stuttering when files are being copied to the NAS.

 

[code65536]

They underlying problem is the processor in the ReadyNAS can't handle the simultaneous reading and writing of so much data.

I doubt the problem is with the process or the NAS. You're probably seeing disk thrashing, which always happens with spinning drives, no matter the setup. User 1 reads some data from position A, then the drive needs to seek to position B to fulfill user 2's read, and then it seeks to position C to write data that's being fed to it, and then seeking back to position A, etc. Seeking is, by orders of magnitude, the slowest, most expensive operation on a spinning disk, so the drive is spending almost all of its time juggling between its tasks instead of actually performing its tasks. If this is the problem, and it likely is, it has nothing to do with the NAS.

 

[Balforth]

Well, this was my initial thread here about the issue:

http://forums.anandtech.com/showthread.php?t=2352439

The advice I got there was that the NAS itself has a very weak processor and can't match the speed of the hard drives:

That unit has a peak transfer of 36MB/s. That is basically 1/4 of gigabits max speed. Monkeying with the network won't fix your issue since it isn't the problem. You are simply running out of performance. You have a computer that can dump more data to it than the unit can handle. Since 4 WD black drives should easily do 80-100MB/s, The unit is likely maxing out the CPU. Droping you computer to 100mbps would likely fix the issue.

http://www.readynas.com/?p=331#Performance

I had though about applying QOS on the switch my HTPC is connected to (running Tomato firmware) to throttle the ports windows copies files on to something much smaller, but that just seemed a bit hack-ish.

 

[code65536]

Well, this was my initial thread here about the issue:

http://forums.anandtech.com/showthread.php?t=2352439

The advice I got there was that the NAS itself has a very weak processor and can't match the speed of the hard drives:



I had though about applying QOS on the switch my HTPC is connected to (running Tomato firmware) to throttle the ports windows copies files on to something much smaller, but that just seemed a bit hack-ish.

Well, it could be both, but I know for a fact that if I'm doing some heavy disk-intensive task while I'm streaming a high-bitrate video (like a recorded TV show), it will sometimes stutter, even if it's a local disk, so while upgrading your NAS may help, don't expect it to work miracles.

Also keep in mind that all those figures you see are for single-task sequential access.

 

[mfenn]

So the only thing that's really going to help me here is a better NAS. The easy route would be going with the Synology DS412+ I was originally considering purchasing for $600. Can I get similar performance from a custom built FreeNAS system for less money? Smallnetbuilder benchamarked the Synology performing reads around 89.9MB/s and writes at 125.9MB/s.

That's basically maxing out GigE, but I'm assuming that's simple sequential access. Most filesystems will slow down greatly under random I/O workloads.

I doubt the problem is with the process or the NAS. You're probably seeing disk thrashing, which always happens with spinning drives, no matter the setup. User 1 reads some data from position A, then the drive needs to seek to position B to fulfill user 2's read, and then it seeks to position C to write data that's being fed to it, and then seeking back to position A, etc. Seeking is, by orders of magnitude, the slowest, most expensive operation on a spinning disk, so the drive is spending almost all of its time juggling between its tasks instead of actually performing its tasks. If this is the problem, and it likely is, it has nothing to do with the NAS.

Your bolded is not true with ZFS. ZFS is a log-structured filesystem, which means that it always appends new data sequentially, no matter if the write I/O is completely random. That, combined with the ARC (read cache) and elevator algorithms really cuts down on thrashing under simultaneous concurrent access.

 

[aigomorla]

No. Your torrent box cannot run anything but the Torrent software. Not allowed.

Once you have verified that the downloads are virus/trojan free, you are allowed to copy them manually to your NAS.

See what we're getting at here? I mean, yeah, you can do it your way, it's just a really bad idea.

^ +1

he understands...


U never want unsecured data to enter a secured data pool.

Secured Data being your personal data... unsecured Data being everything from the internet until u checked and cleaned it to be secured.

You may think we are sounding very paranoid about security, but trust me, its called thinking safe, and experienced.
Guys who think like this have been nuked / saw a server get nuked...
Were trying to save u from a fallout b4 it can begin.

lemme ask you this... if you can say you downloading nothing but 100% legal virus free torrents, then id say go ahead.
If your sure no one in your family member will download anything bunk or click on a bad exe, then allow write access to all machines.

However if u arent confident about anything i have said, then, its best to lock up a NAS behind glass walls.


Also you can virtualize the torrent PC on your main PC, and only have it active when you are torrenting.
There are many things you can do, to keep the machines separate.

 

[Balforth]

So on any given night, my HTPC will start stuttering exactly when we are watching something, and while it is copying downloaded files to my NAS.

This afternoon, for the sake of troubleshooting, I started streaming a 4gb movie on my HTPC, a 3gb movie on my laptop, and a ~1gb tv show on my raspberry pi in my bedroom, whilst attempting to copy a 4gb file from my htpc to my nas.

I couldn't get so much as a stutter or a buffer out of any damn thing. It worked perfectly.

Now I'm starting to wonder if there isn't something going on with my HTPC, which shouldn't be (see the My Rig link in my sig). Or some phantom processes freaking out my HTPC. Or network. Or NAS. Now I'm so confused. I don't have the beastliest machine in the world, but my HTPC was built with pretty good specs a couple years ago and should still be able to easily handle any of this.