Looking at SATA III RAID Controllers

[Cerb]

ZFS addresses a lot of issues that have nothing to do with RAID, overall RAIDZ is a relatively small part of ZFS.

Sure, but nothing else it has is relevant, assuming the OP's Windows setup is good enough, except for either drive count or read+write speeds, and RAID's inability to scale well to newer drives was one of the main reasons ZFS was made.

That said, the performance of RAIDZ isn't really better than a similar configuration of software RAID (similar amount of RAM writeback cache, perhaps SSD writeback cache).

Write performance is much improved over software RAID 5 or 6. HW controllers that are whole miniature servers on cards, and cost it, can be really fast, but software RAID runs up against the basic problem that it needs both to use RAM as a file cache, general IO cache, and RAID stripe cache, all the while not having a good way to reliably delay and coalesce writes. RAID 5/6 are consistently slow due to RMW, which requires fairly expensive controllers to mask well, for all but the simplest cases, like copying single big files. RAID-Z's big performance advantage is simply not needing to do all the in-place RMW.

 

[aigomorla]

@aigo What kind of differences will I be noticing?

I dont think u will notice a difference unless u do massive direct transfers.
Even then i think u'll still be bottlenecked on your mechanical drives before u get bottlenecked on the actual SATA2 lane.

I remember when i was setting up raid.. people telling me 6 raptors was still under the saturation point in SATA2.

Infact i can only think of a handful of people who has actually saturated SATA2.

Me being one of them with 3xSSD's in R0.



OP if u ask what type of setup i have..
I have a FreeNAS box... with 6 x 2TB in Raid-Z.
The freenas is on a i3 530 /w Zotac ITX board (has an intel gig nic) /w 16GB of DDR3.
FreeNAS i heard is way more ram sensitive, then CPU senstivie, and everyone told me set max and forget.
Im using the ICH10R on FreeNAS. so no dedicated Raid controller.
Raid-Z allows for 1 drive to fail... and rebuild time is arround 5-6hours.
Freenas is expandable if u do it the right way, as in replace each drive with 1 larger after u rebuild.
So i could change my 2TB to 3TB one at a time and rebuild, and not lose my data.

Its not hard to setup... hit up youtube.. and start googling FreeNAS setup.
The thing about freenas is its more software dependant then hardware dependant.
In a dedicated RAID controller, if ur array dies.. YOU MUST FIND THE IDENTICAL RAID CARD or u will lose your array.
In freenas.. u have to just find a board or controller which is FREENAS compatiable.. and rebuild the ZFS directory.

Also freenas works off a USB... so u can use all 6 sata ports for drives.

In short OP... when u get to large amounts of data... BACKUP becomes an issue... lolol...
Im only running at a total of 12-15TB on all my NASes... however backup on those guys is a NIGHTMARE.. :O
Even tho im partially protected by RAID-Z, its still NOT ENOUGH.

When u build a large storage array, you need to see:
1. Is it easy to backup.
2. Is it safe / redudundant.
3. If the world should end somehow... and everything on your network goes down, can it be recovered.

You should definitely avoid RAID5 with consumer drives. When a drive fails and the array tries to rebuild you have a high chance of getting at least one read error from one of your drives. At a minimum you will lose data, at worst you will lose the entire array.

Dedicated CARDs.. IE RAID CARDS are even more of a headache on drives with power save / sleep states.
Also yes, OP make sure ur drives are NAS compatible... or was designed to be on 24/7.
Theres others issues like TDLR on drives which make RAID a nightmare due to the drive sleeping when it shouldnt.

 

[youshotwhointhe]

Sure, but nothing else it has is relevant, assuming the OP's Windows setup is good enough, except for either drive count or read+write speeds, and RAID's inability to scale well to newer drives was one of the main reasons ZFS was made.

The additional flexibility and ability to detect and recover from corruption should be of interest to anyone putting in the effort to create a RAID array.

Write performance is much improved over software RAID 5 or 6. HW controllers that are whole miniature servers on cards, and cost it, can be really fast, but software RAID runs up against the basic problem that it needs both to use RAM as a file cache, general IO cache, and RAID stripe cache, all the while not having a good way to reliably delay and coalesce writes. RAID 5/6 are consistently slow due to RMW, which requires fairly expensive controllers to mask well, for all but the simplest cases, like copying single big files. RAID-Z's big performance advantage is simply not needing to do all the in-place RMW.

You are just plain wrong. Top of the line hardware RAID controllers stick two PPC cores and 1GB of RAM on a card with a battery. There is no magic involved. There is nothing these cards do that you can't do with software RAID. With software RAID you can easily dedicate some RAM to act as the stripe cache. It is this stripe cache that allows you to do the read/modify/write without hitting the disks too much. And RAIDZ has exactly the same performance characteristics as an equivalently configured standard RAID 5/6/7.

 

[Cerb]

The additional flexibility and ability to detect and recover from corruption should be of interest to anyone putting in the effort to create a RAID array.

And without RAID, you get just some checking, which is fairly little gain for a lot of money and time invested (a separate computer). Meanwhile, if going to the effort of a separate computer, may as well do it right, and BTRFS still seems to be too much of a moving target.

You are just plain wrong. Top of the line hardware RAID controllers stick two PPC cores and 1GB of RAM on a card with a battery. There is no magic involved. There is nothing these cards do that you can't do with software RAID. With software RAID you can easily dedicate some RAM to act as the stripe cache. It is this stripe cache that allows you to do the read/modify/write without hitting the disks too much.

Yet you run into the age old problem: large stripes handle read-ahead well, but that cache then needs to be many times larger the data, and writing a new file will almost always require reading stale data. Small stripes lead to needing less memory for that, but universally more HDD seeks. ZFS can write w/o needing to read them at all, nor have them in cache, allowing it to not choke like RAID 5. Rarely will the speed drop close to that of a single drive. You also have the problem of what to do with data in an unknown state after a bad shutdown, which typically triggers a long check of the whole array, during which performance is crap, no matter what kind of load you give it.

And RAIDZ has exactly the same performance characteristics as an equivalently configured standard RAID 5/6/7.

In the case of plain md RAID, only if everything is already in cache, and there aren't many flushes. When the pessimistic time comes, it is a time, not 1 problem write followed by 99 writes all from cache. BTRFS should become Linux' native answer, in time, just like ZFS was Solaris' (and then FreeBSD's, and finally now Linux' with ZoL).

In the case of Windows...Windows is basically the reason today, that RAID controllers are still a market. The OP would need to either change OSes (more likely, build another computer), change RAID type (lost space), or spend on controllers, and possibly an SSD cache drive, too, to make it faster, without sacrificing redundancy (RAID 0 + backups).

 

[aigomorla]

if u need HDD advice for RAID cards... im really loving the seagate NAS series.
And they wont break your wallet like enterprise series does.

 

[ShadowValentine]

Er, okay. So uh, RAID 5 isn't the best idea for what I want to do?

Recap: I have 5 WD 1TB Caviar Blacks and 3 WD 2TB Caviar Greens. I am out of Sata ports for three hard drives, which are currently 1TB Blacks. Also, for a separate reason I do not know, one of my hard drives keeps unassigning its drive letter upon shutdown/boot. I'm looking for a way to run them all simultaneously, and was previously recommended to RAID 5 both sets of drives. The two remaining Cav Blacks are currently in RAID 0.

I run Win7 64-bit. I use an ASUS Formula III Rampage, and have 12GB RAM. I also have two SSDs, one which is currently holding the operating system and the other which is not really doing much right now (It has a small amount of programs on it which I can reinstall / move elsewhere)

It looks like RAID 10 needs an even amount of drives for that to work, which I don't (exactly) have, unless you're recommending I use it for both the Cav Blacks and Greens.

 

[Cerb]

Performance-wise. RAID 5 isn't that great, and writing is just plain slow in Windows, with MS' or Intel's (well, chipset, anyway), no matter what. It allows you to lose up to 1 drive, and read fairly fast. Rebuilding, though, takes forever, these days.

RAID 0 increases bandwidth and space, by alternating stripes on each drive in the array. You can get the full read and write speed of however many drives you have. If one drive fails, there went your data.

RAID 1 mirrors the contents of the drives. Some implementations allow nearly RAID 0 read speeds, but that's not universal.

RAID 10 gives 1 to (n/2) drives-worth of failure capacity (if 2 in the same mirror go, that's it), by being a RAID 0 or RAID 1 mirrors. It's inefficient per drive, but around as fast as a RAID 0 of 1/2 the drives, and offers redundancy.

RAID 5 creates parity data of one drive's-worth, allowing you to lose one drive. RAID 6 spans this to 2 drives. However, that requires that each of the other drive's stripes be used to calculate new parity data every time there is a change, resulting in typically very slow write speeds in Windows (unless you get a real HW RAID controller, with cache), and often slow mixed read/write speeds universally.

--

So, the thread has veered more than little off. What are you using these drives for?

 

[ShadowValentine]

Just personal use at home. Games, movies, music, etc.

 

[Cerb]

If write performance is not an issue, RAID 5 will give you the best use of your space, without a single drive going out ruining everything. The primary limitation will be the drives, using SW RAID. But don't be thinking that alleviates the need for backups. If most of the data can be re-downloaded or otherwise recreated easily enough, then that's no big deal.

However, make sure to get a card that supports RAID itself, unless you're running Windows Server, because I'm pretty sure your Intel RAID won't make use of add-on ports.

Also, if it's mostly media, seriously consider buying or making a NAS, instead (you can make one about as cheap as buying, but have to do some research and planning, though the purchased units typically have a backplane with an indicator LED, which could make your life easier on a single-drive failure). You might get limited by GbE (20-40MBps small files, 70-100MBps large files), but it will be separate from your main OS, so anything breaking your OS won't affect it.

 

[aigomorla]

Recap: I have 5 WD 1TB Caviar Blacks and 3 WD 2TB Caviar Greens.

are they all full?
could u shuffle data over to begin with to start the raid array on the drives?

Because u cant magically raid with data on drives and expect data to survive.

but having off drives makes it kinda ugh... cuz windows RAID in most cases uses the lesser of the drives as base when it scales... meaning... if u have 1 tb with 2tb, that 2tb turns into a 1tb drive.

Shaddow can we tempt you to go on youtube and look up freenas?
http://www.youtube.com/watch?v=LlSiN6DbVrI

do you have any spare parts laying arround?
id take this upgrade oportunity to build a NAS and start removing drives off your system.

You could start with 5 x 1TB in ZFS RAID-Z... and then slowly replace the 1TB with the 2TB drives... ONE at a TIME... and still hold RAID-Z = RAID-5 recovery.
However u couldn't add another 1TB and hold recovery. Raid-Z is still pretty fast... u'd get about 50-80mb/s from source to NAS on 5x1TB array.
So its not as bad as u think it will be, it will still be noticibly faster then usb2.0 and wifi. (lolz)

But possibly instead of adding a RAID Controller... possibly.. cheap 6 sata motherboard + celeron Cpu + 8gigs of DDR3 (freenas is RAM hungry) + FreeNAS => better solution overall for both limited protection, and gaining more sata ports?

 

[ShadowValentine]

Yep, I have spare space to shuffle the data on the drives to start RAID.

Unfortunately I don't have any spare parts to build a FreeNAS box. If I'm understanding this video correctly, this is similar to DropBox, where it's stored externally and data is uploaded/downloaded as necessary? If so, then that would eat up my bandwidth / data usage (Unfortunately, I have capped data usage).

If the read/write speed in Raid 5 does not have a noticeable difference to the hard drives unRAIDed, I think I would prefer to keep all my drives in my system. If the Intel card I listed earlier won't do, are there any recommendations for RAID cards?

 

[Cerb]

Unfortunately I don't have any spare parts to build a FreeNAS box. If I'm understanding this video correctly, this is similar to DropBox, where it's stored externally and data is uploaded/downloaded as necessary? If so, then that would eat up my bandwidth / data usage (Unfortunately, I have capped data usage).

No, it would be more like, once set up, having a disk sitting at the other end of your network cable. It will not use any ISP resources, except obviously for an image to install from, and reading an install guide or two.

 

[aigomorla]

as cerb said it uses your internal network.

it doesnt use ISP... unless u setup a FTP on it, and u want FTP access.

The freenas box sits in a closet.. with ventilation..

u dont even need a full case either... u can just lay things out in a cardboard box and run it that way.

 

[ShadowValentine]

Hmm. So will it basically be like having a raid array on my system, except it's located externally?

Any chance of being linked to a site that sells NAS boxes?

 

[ShadowValentine]

If I were to run my drives strictly off my motherboard (ASUS Rampage III Formula) using software RAID, would it be better to run the WD Cav Blacks (1TB ea) in Raid 0 or Raid 5? Using either 3 or 4 of them.

 

[Soulkeeper]

If I were to run my drives strictly off my motherboard (ASUS Rampage III Formula) using software RAID, would it be better to run the WD Cav Blacks (1TB ea) in Raid 0 or Raid 5? Using either 3 or 4 of them.

With 4 drives:
Raid 0 with that many drives is asking for trouble imo
I'd personally look at Raid 10 with 4 drives.

Theoreticals:
raid5 = 3/4 total capacity, 3x read speed, 1x write speed
raid10 = 1/2 total capacity, 4x read speed, 2x write speed

http://www.raid-calculator.com/default.aspx
dog slow writes are common with raid5 and more than 3 disks.
It's only a good idea with 3 disk imo.
4 is the min for raid 10

 

[ShadowValentine]

Should I run two Raid 0s then?

 

[Emulex]

how about just jbod. that is the most reliable for non-server drives.

works pretty good for me.

 

[smakme7757]

I would definitely try to keep things simple. It seems your slightly over your head here and I'm not trying to insult your intelligence .

If i was you i would buy an empty NAS box - A Network Attached Storage device without any hard drives. As an example you could buy one of the new Netgear ReadyNAS or QNAP storage devices. Then you fill it up with your disks and attach it to the network.

Once that is done you will be able to configure it with a web interface, like your router. Then you access and transfer all your data over the network.

The bonus:
- It's not part of your main system so it's available 24/7
- It's consumer friendly. No settings up operating system, managing RAID
- The unit will use the best RAID level available and will most likely support hot swap
- You won't have to rebuild a failed RAID on the system you use everyday.
- It's a small little unit you hide in a corner somewhere only requiring some air flow.

Negatives:
- The data is only available over your network, so if you don't have Gigabit it can be a little slow.

It's the easiest most user friendly way to do this.

Should I run two Raid 0s then?

In RAID 0 your data is split over the drives. So little chunks of your data are spread over all the disks. If you loose a disk then you loose the chunks. Then the file can't be created and used.

Short answer. If one disk dies you loose everything.

 

[ShadowValentine]

@smakme Yeah, a little xD Trying to follow though.

Thanks for the breakdown of +- on NAS, that makes it a lot clearer. Unfortunately my network is kind of crappy, so I don't want to put unnecessary strain on it.

I'm running one RAID 0 array currently for two of my Cav Blacks, and I was just wondering, as one of the hard drives I have connected is being temperamental (http://forums.anandtech.com/showthread.php?t=2336125) I might do as a friend suggested and take it out, convert it into an external and use the sata port to either add to the current array (Yes, I know I'll have to take everything off, disassemble the array then reassemble it. That's the 3-drive Raid 5 I'm considering) or remove my DVD drive and create a separate RAID 0 array.

It makes things simpler in the long run; I haven't had any failures on my two-drive RAID so far, I don't have to buy/build a NAS box that although would be convenient, I don't really have space or need for an external box (Sounds like it's sort of like a server only accessible from inside the network, and uploads/downloads/streams data along the network?).

Correct me if I'm wrong though =o