Thinking about a fileserver setup + raid

[Saicho]

Hi all,

I have a perfectly decent box lying around wasting cycles that I'd like to turn into a server. It would be beneficial for me to have a web server running with things like SSH, SVN, with multiple user accounts, etc. I was thinking of running some sort of Linux distro (probably Ubuntu) to handle all of this when I realized....

A remote file server would be a good idea. I'd rather put everything I want in storage on that machine. Other members of the household would benefit from something like this as well since we all have multiple desktops/laptops, and mounting a network drive to access and store all of "storage type" files would be quite nice. Most of these machines are wired through gigabit and the rest are wireless N. Anyway, I think a fileserver is a good idea for my use case. Thoughts?

I haven't dove into the fileserver world much, but I hear Samba is a good solution? I would like multiple user accounts (on both Windows [Vista32/64bit & XP] and Linux platforms) with designated storage space to basically mount network drives and use them as they would local storage. Can Samba do all of this?

Also, I'd like all of this to have some sort of redundancy so if one drive fails the data can still be recovered. I know a bit about how RAID works, not much about RAID6, and even less about the features of RAID controller cards. Thus, I have a few questions.

I'm going to take what I think is a decent solution: an internal 16x PCI-E raid controller card for SATA drives running RAID5 (or 6) on that Linux box that does around 300MBps (since the gigabit ethernet will be the bottleneck anyway). Say I have the OS boot partition, / installed with all the goodies and blah blah on a 200 something GB drive not hooked up to the raid on ext3fs. Say I'd like to put in the raid controller and connect 3 1TB SATA drives to it, for 2TB worth of storage space (3TB minus 1TB in RAID5, I think it's the same for RAID6 no?).

1. Is there a way to use an OS independent fs on the card? What if I stop liking my linux server and want to put the raid controller and the 3 drives in a Vista machine. ext3 won't work, right? Is there a filesystem to solve this? I'd like the RAID OS independent (I guess if I'm using something like Samba to handle the actual file server stuff it won't really be OS independent but it would still be a pleasant fiction to at least be able to put the card in another linux system and have everything working just like it was before.)

2. Do these cards have good monitoring software? Lets say 1 of the drives fail. Will the card tell me immediately and which one? Will the RAID still operate (maybe not well performance wise) with one broken drive (that's the whole point, right?) How painful/less is it to put in a new 100% working drive and get the RAID to rebuild it? Does the raid software take care of this?

3. Lets say the card has room for 4 drives and I want to add another 1TB drive to the raid, for 4TB minus 1TB storage after I've ran the 3 drive RAID for a year or so. Will this be just plug-and-play? Are these cards smart enough to extend the RAID as I keep adding more drives to it?

Thanks for taking the time to read. If you can shed light on any of these issues or point me to some documentation about this stuff I will be very grateful.

 

[Lorne]

First reserch what OS you want, Setup cant be ported from one to the other without starting over again due to filing systems (MS or Lenix).

What you want to set up sounds good, Thats allot of space for home use (Alot of porn, jk) good size for a file server or dorm use.

A 1HD OS with 3HD raid 5 workspace sounds good, Back your main accasionaly (This is the set for file servers I prefered, For many reasons)
Or OS and data all on the 3HD R5, All fast all safe, Its just harder to switch OS on the fly (MS mainly, I dont dabble in Linex)

R5 controllers, Alot of differences there, Some alow splicing extra drives some you have to start from scratch, Most will recontruct a replaced drive on the fly some have to be forced, Time on reconstruct varies as with splicing.
Ive seen some controllers splice 1 or 2 new HD's and not reconstruct them as it wasnt nessasary and just resumed and some holding the system up till done.
Alot depending on depending on wether hardware or software dependent.

As for other software Im not sure, Im just the hardware tech, The software techs call me otherwise.

 

[Nothinman]

I haven't dove into the fileserver world much, but I hear Samba is a good solution? I would like multiple user accounts (on both Windows [Vista32/64bit & XP] and Linux platforms) with designated storage space to basically mount network drives and use them as they would local storage. Can Samba do all of this?

Samba implements the SMB/CIFS filesharing protocols, like Apache implements HTTP, so if you want to use Windows filesharing on Linux then Samba is your only real choice.

Say I'd like to put in the raid controller and connect 3 1TB SATA drives to it, for 2TB worth of storage space (3TB minus 1TB in RAID5, I think it's the same for RAID6 no?).

RAID6 loses you two drives worth of space since it does double the parity.

1. Is there a way to use an OS independent fs on the card? What if I stop liking my linux server and want to put the raid controller and the 3 drives in a Vista machine. ext3 won't work, right? Is there a filesystem to solve this? I'd like the RAID OS independent (I guess if I'm using something like Samba to handle the actual file server stuff it won't really be OS independent but it would still be a pleasant fiction to at least be able to put the card in another linux system and have everything working just like it was before.)

No. FAT is the only thing close to OS independent and you really don't want to use that.

2. Do these cards have good monitoring software? Lets say 1 of the drives fail. Will the card tell me immediately and which one? Will the RAID still operate (maybe not well performance wise) with one broken drive (that's the whole point, right?) How painful/less is it to put in a new 100% working drive and get the RAID to rebuild it? Does the raid software take care of this?

Depends on the card, personally I'd probably just use Linux software RAID and let mdadm monitor things.

3. Lets say the card has room for 4 drives and I want to add another 1TB drive to the raid, for 4TB minus 1TB storage after I've ran the 3 drive RAID for a year or so. Will this be just plug-and-play? Are these cards smart enough to extend the RAID as I keep adding more drives to it?

Again it depends on the card. And again with software RAID you can extend and reshape the array to your heart's content, even with drives attached to other controllers.

 

[Fullmetal Chocobo]

If you get a dedicated RAID controller, you can use it on any supported OS. I can pull my RAID controller (Areca ARC-1220) out of this machine, and put it in the Server 2003 box, load the RAID drivers, and all of my data is there. If you are doing a Linux box, be sure you have linux drivers on hand, and I would add the card first with a single hd that has nothing on it. Reason being if this is your first time doing an operation, you want to be sure you know what you are doing with it. Once it is up and running, shut down the machine, pull the single HD out, put in your array, and you are up and running.

In working with 1TB hard drives, if you have 5 disks, RAID 5 will give you 4TB of space (disregarding 2^10 vs 10^2 stuff); RAID 6 will give you 3TB since it has two parity calculations compared to RAID 5's 1 parity calc.

If you get a dedicated RAID controller, make sure it supports online expansion. This will allow you to do what you mentioned--adding a drive to the RAID array. Keep in mind, you have to rebuild the array, and it takes a while to do so. But you don't have to backup the data, destroy the array, create the new array with the extra disk, and restore...

 

[Saicho]

Thanks all for the replies.

The 1TB drives were just an example to make the calculations easy =)

@Fullmetal Chocobo: what fs are you using on your RAID with the Areca card?

@Nothinman: Can you go into some of the pros and cons of the Linux software RAID vs a dedicated RAID controller card?

 

[Fullmetal Chocobo]

Originally posted by: Saicho
Thanks all for the replies.

The 1TB drives were just an example to make the calculations easy =)

@Fullmetal Chocobo: what fs are you using on your RAID with the Areca card?

@Nothinman: Can you go into some of the pros and cons of the Linux software RAID vs a dedicated RAID controller card?

RAID card is currently on Win XP box (everything NTFS). I will moving this card to a Server 2003 box sometime this week, so we'll see how that goes. I'm not expecting any problems though.

 

[Nothinman]

@Nothinman: Can you go into some of the pros and cons of the Linux software RAID vs a dedicated RAID controller card?

The only real downsides are:

If you use a RAID level that requires parity calculations like RAID5 those get done on the host CPU instead of a dedicated one on the controller. If the box isn't really CPU bound this shouldn't be an issue.

You'll have to learn how to use mdadm to manage the software RAID. But Linux software RAID coupled with LVM on top is awesome.

The pros would be the virtually infinite configurability, just about anything you can think of can be done.

As long as your hardware supports hotswap you can replace faulty drives on the fly, add/remove drives from the array. ext3 and XFS support online growing so you can add a drive, reshape the array and grow the filesystem with 0 downtime.

You can add drives to the array from any controller, mixing PATA, SATA, SCSI, etc whatever you may have handy.

You can drop all of the drives from an array into just about any Linux box and have the array come up.

 

[Saicho]

Amazing, and save $600. This box will not be CPU bound.

What are the benefits of the RAID controller then? If the hardware RAID controller is OS bound by filesystem, then is there a real gain?

 

[Nothinman]

What are the benefits of the RAID controller then? If the hardware RAID controller is OS bound by filesystem, then is there a real gain?

Every hard disk is bound to whatever filesystem you put on it, there's no way around that.

IMO the main benefits of a good hardware RAID controller are:

A little more simplicity, you setup the array in the firmware and it just appears as one drive to the OS. If you're lucky you can manage the array with some tool from the manufacturer but those are kind of hit or miss. And the manufacture tools usually add more complexity since they vary by manufacturer and are usually closed source so I'd probably just prefer to learn mdadm once instead of a half-dozen different closed tools.

Battery backed cache for the entire array. Most of the bigger ones will have a good amount of memory on them to cache writes and a battery so that if power gets cut off unexepectedly the writes can be flushed to disk.