Support for SATA RAID under Linux?

[paulsiu]

There are a large number of motherboard these day that come with SATA Raid on the motherboard, but are these functionality available in Linux? I tried googling, but the Linux SATA FAQ I found seemed to indicate that support for these built-in RAID is probably beta or non-existent partly because they are not true hardware RAID. If I want RAID, I would have to buy a hardware RAID controller?

Paul

 

[drag]

Don't use onboard raid on motherboard unless your using a server motherboard (probably going to cost 400+ dollars) designed to do RAID 5 or something like it.

All those RAID 0/1 onboard controllers are mostly marketting gimmicks. Very little real-world benifit.

If you want a inexpensive RAID setup, use Linux software RAID. It's easier to setup (some installers support setting up RAID), it's more stable/safer, and probably faster. (those onboard stuff run mostly in software anyways.. like Winmodem/software modems do, just with propriatory drivers)

Most of them will work, more or less, but it's not worth dealing with IMO.

 

[n0cmonkey]

Linux software raid will work as well or better than the onboard "raid" devices. Those onboard things are just bios assisted software raid anyways and provide no benefits. Avoid Adaptec.

 

[paulsiu]

So Linux Software Raid will work just as well as the XP BIOS ones as far as cpu utilization goes since they all use the CPU for RAID. Hmmm... I guess only RAID 1 and 0 are supported. If I want RAID 5, I would have to get one of those expensive hardware RAID controller?

By the way, does the software RAID work with SATA or is it only with PATA?

Paul

 

[drag]

Linux software raid is top-level stuff.

It's probably the best your going to find and is benchmarked to be faster then even expensive hardware-based raid setups.

The reason you'd want a expensive hardware RAID setup isn't so much for speed, you get other features.. such as the SCSI controllers it has, the ability to do hotswap/hotspares, speed of rebuilding a array after losing a hardware, that sort of thing.

If you think about it, software RAID makes sense.. your comparing the speed from a embedded style 200-400mhz proccessor on a raid controller vs a 2000-3000mhz CPU.

Software RAID is significantly handicapped in capacity by the speed of the PCI bus right now. The 32bit 66mhz PCI bus that is used in current x86 machines has a maximum bandwidth of 133MB/s. Around 100MB/s realisticly.

A harddrive usually gets about 45MB/s. Newer ones can go a bit faster, but I am talking about sustained speeds. So after about 4-5 harddrives you pretty maxed out your systems internal bandwidth... your going to have to replicate volumes, and any writes have to go to 2 or more harddrives simitaniously to maintain parity and all that stuff. Above that your going to saturate the bus and loose performance.


Personally I use a RAID 5 software array for my home folder. It's in a dedicated PC, that that is all it does... it's connected to my main desktop by gigabyte ethernet.. the network is a cluster filing system designed for secure networks for clusters. Allows for migrating proccesses back and forth between computers and shared directories and such.. (check out OpenSSI if your curious).

The RAID 5 array consists of 2 Maxtor 7200rpm/8meg cache 120gig harddrives running on a SATA to PCI adapter. The 3rd hardrive is a older Western Digital 120gig/7200rpm/8meg cache PATA drive running from the onboard IDE controller. (with PATA, only use harddrives that are 'MASTER' setup, using slave drives is OK, but performance will suck.)

All 3 harddrives are partitioned with one big 110gig partition. (or something like that.. you know harddrive sizes vary a bit) and that is what I use to build the array.

*Note that this computer is booted off of the network, via PXE... If this computer had to deal with booting itself up I'd have to have a seperate /boot partition for the bios to read the kernel off of.. Only needs to be 20megs or so, but I make mine 100-200megs for comfort.

Ontop of the RAID device, I have it in one big LVM (logical volume managment) volume group, and that is divided up into volumes, which are basicly partitions, but can be moved around and resized and such without rebooting them. (although it depends on the filing system itself to weither or not they can be resized on the fly, I use Ext3, which I dont' think supports that..)


So basicly I have:
3 harddrives used to create one RAID 5 array. 2 SATA, 1 PATA.
Ontop of the raid array I use LVM to divide up into different 'partitions' which can be moved, resized, or deleted based on my needs at any given time.

It's just something I am screwing around with so that I know that I know howto do it.

LVM is nice because you can add harddrives to the volume group and get more capacity, or take away harddrive space.. or take a partition from another harddrive and just add that to the volume group. Stuff like that.

It's all kinda cool in a geeky sort of way.


Linux software RAID is very nice. It's fast, flexible, and has lots of usefull features. You can have RAID 0/1 or RAID 5 or even weird ones like RAID 4 or RAID 10 and stuff like that. Its IDEAL for creating cheap file servers that need massive amounts of space for storing data.

Then you can have hotswappables, and stuff like that.. but the hardware itself needs to support it. And until SATA gets more established that's realy just scsi.

 

[filterxg]

I'll be building my fileserver in a few weeks and I did intend to use a hardware RAID but I'm reconsidering.

As I am planning it. It will have 6 HDs. 2 IDE in RAID 1, 2 IDE LVM, and 2 SATA LVM. SATA support is not native to my motherboard but I already have the PCI card for it, it does not have RAID.

The computer is slow, an 850 T-Bird, so a hardware solution is probably still best, even with the headache. What do you guys think?

 

[drag]

You have to understand that 'hardware raid' isn't realy that affordable.

It's got to include it's own embedded proccessor, it's own RAM and all sorts of expensive stuff. The cheapest your going to find something like this is going to be around 200 or 300 dollars... (pretty sure). Anything cheaper is going to be software RAID anyways.. instead of just having the controller work you'd just use special drivers.

Think Winmodem vs Real modem. The onboard raid stuff and cheapy raid 0/1 controllers are mostly 'WinRaid'.


Also for that file server and a 850mhz proccessor.. How important is speed? If it's just a personal server to hold movies and such then it realy doesn't have to be that fast. You copy files over to it, copy files back when you want to deal with them. (network would probably be the bottleneck for that stuff.)
Speed is realy only going to be a issue if you have multiple users and stuff that has to access it all the time.

So what you'd probably want to do is setup all those drives under a single RAID 5 arrangement.

6 harddrives, say 200gig, would end up with a single RAID image of around 1TB. Stick LVM on top of that if you want to divide up the space, or you can partition it like a normal harddrive.

Although instead of getting all six drives, depending on you budget get a DVD burner or two so that you can make long-term storage for important files and such. Anytime you put thing on a harddrive your going to risk loosing it to a electrical problem and even harddrive failure (although RAID 5 reduces it by quite a bit you'd have to loose 2 drives to get damage to your data.. but if one drive starts f-ing up it can replicate the corrupt data to it's parity and you'd loose your data that way. RAID is no substitute for real backups).

Also I'd get a nice high-quality UPS for it, too.

 

[filterxg]

Thanks. I thought the 850 would be enough, just didn't know how much a Linux Software RAID array would tax the system . From the other posts I'll figure it'll be fine.

As for everything else, about 50GB worth would be backed up both by some kind of redundancy and by DVD's. This is my work files. Some movies and all my music files I'll also backup on DVD but I don't need redundancy (read DIVX). The rest of the space is for HD content recorded from my Digital Cable Box. So that needs neither redundancy nor backups.

 

[drag]

I think that it would put a pretty big load on that system then..

IDE drivers are pretty CPU intensive by themselves, since they all use CPU power to run the controller, and you have six of those.. And then you have software RAID on top of that.

I don't know if it will max out the CPU.. but it's not that hard to max out the CPU on certain filing system benchmarks just with one or two harddrives.

So it's not going to be a speed demon or anything like that...

saving media from a TV can be tricky, depending on your recording it. If your recording it directly to your file server speed can be a issue, you don't want it slowing down so much that the recording gets out of sync with the actual video feed. But it doesn't sound like your doing that.

I'd probably try it out with the 850, but be prepared to upgrade the motherboard and cpu if it's possible and you run into speed problems. You wouldn't need much.. just the slowest AMD cpu you can find new would be plenty.. (like a 1.8ghz duron is like 50$ at newegg.)

I don't think that you'll run into problems.

 

[drag]

here is something that is kinda funny.

Linux SATA RAID FAQ

Also usefull is Linux's SATA support page (those that support ''hotplug" are attractive..)

It describes the current status for Linux support on SATA controllers

Also for special note:

4. Vendor support
Completely open chipsets
I make an effort to highlight vendors that support their chipsets by posting hardware documentation publicly (open hardware). In an effort to provide positive reinforcement, I also make an effort to give priority to performing engineering tasks on open hardware.

Open chipsets:
Silicon Image 311x
Intel ICH5/ICH6
AHCI: Intel ICH6-R/M, upcoming SiS
ADMA: Pacific Digital Talon, others?

I think the Vitesse/Intel 31244 docs are public as well, but I need to check on that.

Note that this page is for bleeding-edge support. Currently Release Canadate kernels... 2.6.12-rc and 2.4.30-rc

 

[paulsiu]

Thanks for the explanation on why software RAID is faster. Hot pluggable server hardware isn't all that important. There isn't going to be 300 people screaming when the server goes down. If one of the RAID5 drive goes down, I assume that you can rebuilt using the software RAID?

It's also cool that you can mix SATA and PATA. I notice a lot of Windows RAID can't do that.

How fast is the RAID 5 vs. single drive? What I want is to setup a RAID 5 array and a backup volume that I'll off-site.

Paul

 

[drag]

Raid 5 speed is so so. It's fast for reads, but not much faster for writes. Even slower in some cases... this is because the system has to write to at least 2 drives to preserve parity plus do the calculations, even with very nice hardware writes tend to be a bit slow. For reads it's like mirror-Raid.

On my system I did a bunch of benchmarks and they were not faster at all over a single harddrive and I was a bit dissapointed But that is only with 3 drives. If I had like 5 I would expect to see some improvements in read performance. I think that for multi-user activity I'd see better performance even with just 3 drives.

And for rebuilding the array.. Yes.

Linux doesn't support hotplug-able SATA yet anyways.. In the future it will though.


You'd have to take the machine down (if the drive didn't take it down) and replace the drive, then boot up and then add the new drive. Then it should rebuild fairly automaticly. Keep in mind though it can take a long time...

For instance when I initialy installed my array it took 3-4 hours to have it get built up all the way. When rebuilding it should be accessable.. I think.. but it's going to be very slow with all the disk activity and such.

Also you can setup a 'hot spare' drive so that it will take over automaticly when you rebuild.

Also keep in mind this is just me fooling around with stuff. From what I know it all makes sense, but I am not a expert. For your reference you can find the Software RAID howto at:
http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html

and a bit about recovery:
http://www.tldp.org/HOWTO/Software-RAID-HOWTO-8.html

However using Linux Software RAID in production is fairly common and it is mature. Most modern installers allow you to configure software RAID from the install menu, too (and LVM). At least for Debian Testing you can do this.

If you want more perfect uptime, so that you know for certain that unless you loose power you'll have aviability, the next step up would be high-quality hardware raid device OR high aviability-style clusters. Basicly 2 servers sharing the same IP address and are mirrors of each other and communicate thru a special network filing system. So that basicly you'd have harddrive RAID 5 on each computer and then the computers be RAID 1-style mirrors of each other. So that if you need to take one down you have the other one to carry the load. (just a FYI)