SAN design questions

[Brazen]

Ok, so we just bought a EMC CX300 through Dell and they refused to sell it to us without having their consultants install it and set it up. I'm thinking of changing a few things, but I have a few questions that I can't find through google or EMC's site.

First off, the EMC guy chopped up the array into 5 drive and 9 drive Raid 5 groups. We have 30 disks so he set up 3 5-drive groups, 1 9-drive group, and a single drive for global hotspare. I would have thought to just set it up as one 29-drive Raid 5 group but he said you only want to use 4+1 and 8+1 Raid groups for performance reasons. Is this true? I haven't been able to confirm this anywhere.

2nd Question: about where do people start out with snapshot size for a typical file server?

 

[Genx87]

It makes sense, think about it, when you write to a raid 5 device you are writing to the entire array. It could be very disk intensive to write bits to 29 drives . Of course when reading you could get some very nice performance.

Also the more disks you add to the array the higher the chance for failure. In a RAID 5 array 2 disks go down and it is toast. What do you think the chances are in a 5 or 9 disk array for two disks to shat vs a 29 disk array?

I have only been exposed to these devices on a high lvl. I have seen a netapp in person and worked a little with it. I think the one we had, had about 160 disks in it. And of course was a NAS, not a SAN. But I think the basics are there, just a different interface.

And I am assuming the software of the EMC will present all those groups as a single volume?

 

[Brazen]

Originally posted by: Genx87
It makes sense, think about it, when you write to a raid 5 device you are writing to the entire array. It could be very disk intensive to write bits to 29 drives . Of course when reading you could get some very nice performance.

Well I guess that is the question, isn't it. It seems to me that with more disks you have less to write to each disk, so shouldn't performance be higher? And the EMC guys point was not that fewer disks are better, but that it be either 5 or 9 disks (which he kept refering to as 4+1 and 8+1, though I always thought the +1 was refering to a hotspare, be he used all 5 or 9 disks in the array).

Also the more disks you add to the array the higher the chance for failure. In a RAID 5 array 2 disks go down and it is toast. What do you think the chances are in a 5 or 9 disk array for two disks to shat vs a 29 disk array?

Plus there is the 1 global hotspare, and risking 3 out of 30 disking going down is long enough shot that it is acceptable to me. I could even set 2 global hotspares, and then 4 disks would have to go down, but EMC just recommends 1 global hotspare per 30 disks.

And I am assuming the software of the EMC will present all those groups as a single volume?

Yes, but due to the nature of Raid 5 we only get about 3 TB of usable storage with the current setup. With a single 29 drive array, we would get about 3.7 TB of usable storage. That's a bit of a difference and we really need that extra 0.7 TB

 

[netsysadmin]

Are you sure when he was reffering to 4+1 an 8+1 he didnt mean the fifth and ninth drives are setup for dedicated hotspares for that array? Then if you setup one global hotspare you will reduce the chance of losing two disks in one area since they are automatically replaced.

John

 

[Brazen]

Originally posted by: netsysadmin
Are you sure when he was reffering to 4+1 an 8+1 he didnt mean the fifth and ninth drives are setup for dedicated hotspares for that array? Then if you setup one global hotspare you will reduce the chance of losing two disks in one area since they are automatically replaced.

John

Well, ok now that is what I thought it means when it says 4+1 and 8+1, that 4 and 8 drives respectively are used in the array and 1 is used as a hotspare, but no he DID NOT set it up like this, all 5 drives and all 9 drives are used in the array, and the array sizes confirm this. The only hotspare is the 1 global hotspare assigned to the entire enclosure.

It sounds like you just confirmed that my understanding of the terms are correct and and HE was using them incorrectly. This does not bode well for my confidence in his SAN design... (not to go off on a rant, but I have mentioned in these forums a few times about the typical ineptitude of consultants).

 

[spidey07]

Brazen,

From what I've seen that is standard best practice for storage. Especially for that model.

Then again I'm no storage guy so I could be completely off base.

 

[Brazen]

Originally posted by: spidey07
Brazen,

From what I've seen that is standard best practice for storage. Especially for that model.

Then again I'm no storage guy so I could be completely off base.

What do you mean by "that?" Using 5 and 9 disk arrays? Or is it actually supposed to be 4 and 8 disk arrays? Or is it any number in between?

 

[spidey07]

I just pegged our storage guy and asked him about it. He said "we've got a couple dozen of those units...normally just use 4 and 8 drive raid 5 with a spare or two. bascially just follow best practices."

 

[Brazen]

Originally posted by: spidey07
I just pegged our storage guy and asked him about it. He said "we've got a couple dozen of those units...normally just use 4 and 8 drive raid 5 with a spare or two. bascially just follow best practices."

Ok, thanks, I guess that answers the 4 and 8 vs 5 and 9 issue. So apparently our storage consultant did mess it up. I would to see a paper on best practices. I can't find just a "CX300 best practices guide" or just a "SAN Raid and LUN best practice guide."

 

[spidey07]

well the dude I talked to referred to it as 4+1 and 8+1.

maybe we're getting caught in semantics.

 

[Brazen]

Originally posted by: spidey07
well the dude I talked to referred to it as 4+1 and 8+1.

maybe we're getting caught in semantics.

No, the +1 is referring to the hotspare, which we will use 1 global hotspare for all the Raid groups, but it sounds like then the individual Raid groups should have 4 or 8 drives plus 1 hotspare (which will be the global hotspare). So in this case it would be more like 4,8,8+1 or something like that.

 

[Brazen]

Ok, I have found EMC documentation that makes it clear that the +1 is not referring to the hotspare. It refers to the wasted disk, and the first number is the "effective disks." So, disregarding any hotspares, a two-disk RAID 1 would be 1+1, a six-disk Raid 10 would be 3+3, and a five-disk Raid 5 would be 4+1.

And by the way, the EMC document also says that 5 - 9 disks per Raid Group is recommend. It specifically says the idea that it must be either 5 (4+1) or 9 (8+1) is a common misconception, anywhere between those values is ok.

We have 30 disks, so I ended up chosing Raid Groups of 7 disks (6+1) and created 4 equal groups, with 2 global hot spares. However much storage I want to present to the host, I divide it by 4, create 1 LUN in each RG and combine them into a metaLUN. This is also per reccommendations in the best practices, to maximize performance and ease of expansion.