Networked Storage

LeatherNeck

Member
Jan 16, 2001
174
0
76
A pretty involved question here. I'm looking for detail and good justification for solutions chosen.

I have a good friend who needs networked storage for several hundred images that are about 400-500 MB each. If you had to build a file server, how would you build it - Case, RAID controller, number of drives, type of drives, RAID 0+1 or RAID 5, Operating System, ...

What do you think of Network Attached Storage devices like these:

http://www.newegg.com/Product/ProductLi...cription=network+storage&InnerCata=124

Assume a healthy budget and that reliability and speed are the driving factors.
 

Rainsford

Lifer
Apr 25, 2001
17,515
0
0
First of all, if you have a healthy budget, I'd go with a custom PC over a NAS specific device for 2 main reasons. The first, and perhaps most important, is that lower end NAS devices still have a lot of room for improvement. Good ones exist, but many are plagued by slow speeds, incompatable permissions setups, and reliability issues. Secondly, your flexibility with them is very limited. I don't know if this is important for you, but most of the NAS devices do not have features I consider vital to a good NAS device. They may support Samba, but not NFS, or FTP but not SFTP. That sort of thing may not be hugely important for your friend, but keep in mind that future expansion is also limited, and adding new features (say, snapshot capabilities) might prove to be impossible.

So, as long as budget is not a huge concern, I'd go with a custom PC running some form of Linux, assuming your friend is comfortable with it. The specific parts, other than the hard drive, are mostly your choice, except that you will do well with a high powered, well known name brand power supply (say, Antec of Sparkle). For the hard drives, I would suggest 4 or more drives, as long as the total number is even and the drives are the same capacity. Then, take half of the drives and put them in a RAID 0 array. Take the other half and create a second RAID 0 array. You can do this either with software RAID in Linux (not good for performance), or a hardware RAID controller that is well supported under Linux. Sorry, I don't have a recommendation on that, as I haven't done very much with hardware RAID under Linux.

Now, you have two RAID 0 arrays of the same size. Using some fairly simple snapshot creation scripts and rsync under Linux (Google for very helpful howtos on this topic), you can use the first array as the storage area, and the second array as a snapshot backup of the storeage area. Keep the snapshots however you like based on how often data is changed and how important it is, I like ever four hours for 24 hours and then daily for 7 days. The great thing about Linux snapshots is that, with a few simple scripts, the snapshots look like complete snapshots of the filesystem at the time of the snapshot, but they only take up the space of one copy of the file system, plus space for any changes between snapshots (plus a tiny bit of overhead). Obviously this could be a problem if the first array is almost filled to capacity, but creating the first array filesystem as a little bit smaller than the second will help solve this problem.

I prefer this over a complete RAID solution for a few reasons. First of all, mirrored RAID is not backup, if you accidentially delete a file or a virus or whatever gets it, it's instantly gone. Secondly, it creates some performance issues when it isn't really necessary. I assume making sure the data is safe is far more important than not having any downtime, so a solution that focuses on backup over redundency is best.

Hopefully this will help, let me know if you have any questions about this, as I just set up something pretty similar myself.
 

Goosemaster

Lifer
Apr 10, 2001
48,775
3
81
RAID 1 or 5, linux, 500GB SATA , case (antec p180) any mobo pc combo you like.....you don't neeed that much CPU power if the RAID controllers are hardware based, but having a new mobo with PCI express is nice:D
 

Atheus

Diamond Member
Jun 7, 2005
7,313
2
0
My file server is a dual PIII-S 1.1Ghz (the server version) on a Serverworks board with 512MB Crucial ECC memory and a mix of Seagate SCSI and IDE disks. It could take a processor failure, RAM stick failure or HDD failure and still work, and it cost very little to put together. It runs debian. I got a dual mostly for the redundancy - the CPU usage is usually 0-1% so don't bother with opterons or anything like that.

I'd recommend something like this over a NAS appliance for reasons Rainsford pretty much covered. If you want to spend a lot you could get a dell server or something, otherwise just order some nice parts and build it.
 

Goosemaster

Lifer
Apr 10, 2001
48,775
3
81
Originally posted by: Atheus
My file server is a dual PIII-S 1.1Ghz (the server version) on a Serverworks board with 512MB Crucial ECC memory and a mix of Seagate SCSI and IDE disks. It could take a processor failure, RAM stick failure or HDD failure and still work, and it cost very little to put together. It runs debian. I got a dual mostly for the redundancy - the CPU usage is usually 0-1% so don't bother with opterons or anything like that.

I'd recommend something like this over a NAS appliance for reasons Rainsford pretty much covered. If you want to spend a lot you could get a dell server or something, otherwise just order some nice parts and build it.

nice. disk access doesn't require high processing speeds if the controllers aren't software based so your setup is actually quite excellent

serverworks chipsets are NICE chipsets:D


:thumbsup:x10