question about SAN

[SelArom]

hi, I'm doing a little research for a paper on storage area networks (SAN) and I was wondering if anyone can clarify somethings for me:

how exactly is the interface to the data presented to the servers (or other devices) accessing the data? that is, doesn't it need some kind of operating system to faciliate data access, and if so how is this different from just having a file server?

I read that the SAN supports block transfers of data rather than files, and that this is supposed to improve throughput, but how exactly does that work? you still have to send the same amounts of data to send a complete file, so where is the improvement?

that's all I can think of for now. It seems like an intersting thing, abstracting storage into its own network. I just don't get how much of a benefit this really is.

any thoughts are appreciated! thanks!

-SelArom

 

[Nothinman]

SANs are just a special kind of file server. They obviously need some kind of OS but it's usually some small custom RT OS or a custom Linux or other unix build. I don't really know any specifics, our SANs are all attached via fiber and I don't have to configure them on the fiber switch side.

As for the block transfers, if you're using something like iSCSI for the protocol it will still have the overhead of using the network for the transfer but you're essentially issuing raw SCSI commands across the network so the overhead of protocols like SMB/CIFS, FTP, etc is avoided. So while you will have to transfer the same amount of file data there might be less protocol data.

 

[SelArom]

cool thanks for your reply! I figured that it had to be there but the sites I was researching made such an abstraction of it that they made it seem like it "just works".

does the iSCSI and AoE protocols mean that you can basically mount these SAN storage devices onto a computer as a network drive? how does that work with multiple people accessing the same part of storage? or is this not what they are designed to do?

thanks again for your reply!
-SelArom

 

[SelArom]

one more question: is there any studies on the performance of SAN devices? like # of connections, throughput, speed, using it as a network boot kind of thing. etc? I can't seem to find any specifics on this kind of information.

in case anyone is wondering, my paper is already done. I would like to know more details about this technology

-SelArom

 

[Nothinman]

does the iSCSI and AoE protocols mean that you can basically mount these SAN storage devices onto a computer as a network drive? how does that work with multiple people accessing the same part of storage? or is this not what they are designed to do?

AFAIK they show up as a regular block device, the fact that they're on the network is completely abstracted away. As for two or more boxes accessing the same device at the same time, I imagine it's possible but would result in corruption if there's no coordination. For instance, we use fiber SANs for an Oracle cluster and we have to use ocfs for the filesystem because it's designed to coordinate the access on all of the nodes in the cluster. I imagine that if you just used a normal filesytem and mounted them on both machines at the same time they'd trample all over each other.

 

[SelArom]

Originally posted by: Nothinman

does the iSCSI and AoE protocols mean that you can basically mount these SAN storage devices onto a computer as a network drive? how does that work with multiple people accessing the same part of storage? or is this not what they are designed to do?

AFAIK they show up as a regular block device, the fact that they're on the network is completely abstracted away. As for two or more boxes accessing the same device at the same time, I imagine it's possible but would result in corruption if there's no coordination. For instance, we use fiber SANs for an Oracle cluster and we have to use ocfs for the filesystem because it's designed to coordinate the access on all of the nodes in the cluster. I imagine that if you just used a normal filesytem and mounted them on both machines at the same time they'd trample all over each other.

does that mean that they can be mounted as drives? when you say they show up as regular block devices, how exactly does that appear to someone wanting to access them? to explain what I mean, if I want to access a drive on my file server, I either mount the drive or access it by the server name and share \\server\videos or something like that. or I can view my network places and it shows a list of available shares... how does that contrast with how SAN works?

thanks for your reply!

-SelArom

 

[ktwebb]

Yes, the LUN (Logical Unit Number) will show up as a drive. If the LUN is presented while your in the OS you will likely have to scan. Either a command line SAN Scan or if your in windows you'd scan from disk manager.

Space is chucked out from the san, masked, and presented, typically to an HBA, a host bus adapter. AKA fiber card. The fiber card will attach to a fiber switch, the fiber channel fabric. The fabric sits between the HBA and the actual SAN device

You can also, if your hardware/software supports it, boot from the SAN. Currently I am moving all our VMWare esx servers over to a SAN boot device instead of the internal IDE or SCSI hard disk. Basically you boot into the HBA bios, enable it, and enable the boot device option, scan for the LUN and away you go. Depends on the SAN as to how easy that is. Pretty straightforward using our IBM shark. Sort of a PITA for the EMC DMX. But doable and a much better solution than a single point of failure IDE drive. (the slot for the second mirrored drive is occupied by the HBA)

I may be mistaken since I am not actually on the SAN team, but I believe the management front end for the shark is AIX, though everything is done pretty much through a GUI utility.


May be others way to implement specifically. This is how all the seperate fabrics are implemented in my environment. Not a bad career choice if your so inclined. Megabucks spent on a SAN, thus the guys responsible for upkeep, maintenance and managment follow suit.

 

[SelArom]

Originally posted by: ktwebb
Yes, the LUN (Logical Unit Number) will show up as a drive. If the LUN is presented while your in the OS you will likely have to scan. Either a command line SAN Scan or if your in windows you'd scan from disk manager.

Space is chucked out from the san, masked, and presented, typically to an HBA, a host bus adapter. AKA fiber card. The fiber card will attach to a fiber switch, the fiber channel fabric. The fabric sits between the HBA and the actual SAN device

You can also, if your hardware/software supports it, boot from the SAN. Currently I am moving all our VMWare esx servers over to a SAN boot device instead of the internal IDE or SCSI hard disk. Basically you boot into the HBA bios, enable it, and enable the boot device option, scan for the LUN and away you go. Depends on the SAN as to how easy that is. Pretty straightforward using our IBM shark. Sort of a PITA for the EMC DMX. But doable and a much better solution than a single point of failure IDE drive. (the slot for the second mirrored drive is occupied by the HBA)

I may be mistaken since I am not actually on the SAN team, but I believe the management front end for the shark is AIX, though everything is done pretty much through a GUI utility.


May be others way to implement specifically. This is how all the seperate fabrics are implemented in my environment. Not a bad career choice if your so inclined. Megabucks spent on a SAN, thus the guys responsible for upkeep, maintenance and managment follow suit.

wow you are totally over my head but I get the gist of what you're saying, and I thank you for shedding some light on this interesting technology! what kind of performance does SAN get as compared to like a regular file server or NAS? how many people can a SAN support?

can SAN be used for things like web/ftp servers or just strictly for storing raw data?

-SelArom

 

[Nothinman]

does that mean that they can be mounted as drives? when you say they show up as regular block devices, how exactly does that appear to someone wanting to access them?

I'm a linux guy, so when I hook up a fiber device they show up as a normal SCSI device (/dev/sdX) and can be partitioned, formatted and mounted like a regular drive. I would assume they appear similar in Windows. Since AoE is ATA I would guess that they would show up as normal ATA/IDE drives.

to explain what I mean, if I want to access a drive on my file server, I either mount the drive or access it by the server name and share \\server\videos or something like that. or I can view my network places and it shows a list of available shares... how does that contrast with how SAN works?

On the server they would appear as normal disks, from there you would have to share them and clients would see them as any other share on that particular server.

but I get the gist of what you're saying, and I thank you for shedding some light on this interesting technology! what kind of performance does SAN get as compared to like a regular file server or NAS? how many people can a SAN support?

You're thinking about the SAN incorrectly. Think of a SAN disk as any other disk, it's just connected to a server via ethernet, fiber, etc instead of a SCSI or IDE cable. You then use that disk in the server like any other. How many people the SAN supports is usually a factor of how many the server exporting it can support. What SANs get you is a central point to manage all of your storage, from there you can chunk it out to different servers as necessary and back up all of the data from a central point.

A NAS is also just another type of file server, but they attack the problem from the opposite direction. You get a NAS with X amount of disk space and you likely can't upgrade them very much and it's a completely seperate entity from the rest of your servers and NAS devices. Essentially a NAS is a 1 or 2U server with a special management console added on top of Windows or Linux. A few years ago we bought one from Dell and it was just a normal 1U server with Windows 2000 server with all of the disks setup in software RAID, nothing special.

can SAN be used for things like web/ftp servers or just strictly for storing raw data?

Essentially a SAN is a big hard disk, you still need to share the disk from a server so if you want to export the data via HTTP or FTP you need to get another box to handle that portion of the job.

 

[ktwebb]

Originally posted by: SelArom

Originally posted by: ktwebb
Yes, the LUN (Logical Unit Number) will show up as a drive. If the LUN is presented while your in the OS you will likely have to scan. Either a command line SAN Scan or if your in windows you'd scan from disk manager.

Space is chucked out from the san, masked, and presented, typically to an HBA, a host bus adapter. AKA fiber card. The fiber card will attach to a fiber switch, the fiber channel fabric. The fabric sits between the HBA and the actual SAN device

You can also, if your hardware/software supports it, boot from the SAN. Currently I am moving all our VMWare esx servers over to a SAN boot device instead of the internal IDE or SCSI hard disk. Basically you boot into the HBA bios, enable it, and enable the boot device option, scan for the LUN and away you go. Depends on the SAN as to how easy that is. Pretty straightforward using our IBM shark. Sort of a PITA for the EMC DMX. But doable and a much better solution than a single point of failure IDE drive. (the slot for the second mirrored drive is occupied by the HBA)

I may be mistaken since I am not actually on the SAN team, but I believe the management front end for the shark is AIX, though everything is done pretty much through a GUI utility.


May be others way to implement specifically. This is how all the seperate fabrics are implemented in my environment. Not a bad career choice if your so inclined. Megabucks spent on a SAN, thus the guys responsible for upkeep, maintenance and managment follow suit.

wow you are totally over my head but I get the gist of what you're saying, and I thank you for shedding some light on this interesting technology! what kind of performance does SAN get as compared to like a regular file server or NAS? how many people can a SAN support?

can SAN be used for things like web/ftp servers or just strictly for storing raw data?

-SelArom

SAN is storage space. What do you want to use space for? That is what you can do with it. Web server, ftp servers, whatever. It's not really a question of can an FTP server use a SAN. Can your ftp server use more space available to it? Then the answer is yes.

We have many types of servers that use SAN resources. Our SQL farm has a something like a terrabyte presented to it. That is cumulative. There are many LUN's. Some presented as 20GB LUN's, other 100, 200GB, whatever.

The other major use of our SAN space is virtual machines. VMWare ESX running on an IBM BLade. At the moment ESX 2.5.3 (it's a customized Red Hat 7.2) is loaded on IDE disks on the blades. There is an HBA on that blade as well, fiber attached to one of our SAN fabrics. The Host OS runs on the IDE drive. The Guest OS's, the VM's, run on vmfs volumes partitioned from SAN volumes. VM's run virtually everything from A to Z in our data center. Webservers, Appro, Laserpro, FTP, Domain controllers and on and on so as a sideways answer to your question, they are all SAN supported. For the record we still have plenty of physical servers. Some of them with fiber cards using SAN volumes as well.

 

[Goosemaster]

When using iscsi for a SAN over ethernet, how does the HBA find the LUN? IP address (TCP/IP) ?

 

[SelArom]

wow thanks so much for the insightful information! I think i get the basic idea, that SAN is a way of abstracting storage onto a network... I think I would have to see the actual implementation to get a deeper understanding.

as another question, how are these SAN's usually managed? is it similar to how you usually do http://192.168.0.1 to access the interface for a router?

thanks again everyone, you guys rock

-SelArom

 

[spidey07]

next - direct to t-pe.

 

[ktwebb]

Originally posted by: Goosemaster
When using iscsi for a SAN over ethernet, how does the HBA find the LUN? IP address (TCP/IP) ?

No experience with ISCSI. This isn't a bad faq.

http://www.sanrad.com/iscsi.asp?IP-SAN=231

 

[Goosemaster]

Originally posted by: ktwebb

Originally posted by: Goosemaster
When using iscsi for a SAN over ethernet, how does the HBA find the LUN? IP address (TCP/IP) ?

No experience with ISCSI. This isn't a bad faq.

http://www.sanrad.com/iscsi.asp?IP-SAN=231

cool. that was a great read.

 

[her209]

Originally posted by: ktwebb
We have many types of servers that use SAN resources. Our SQL farm has a something like a terrabyte presented to it. That is cumulative. There are many LUN's. Some presented as 20GB LUN's, other 100, 200GB, whatever.

The other major use of our SAN space is virtual machines. VMWare ESX running on an IBM BLade. At the moment ESX 2.5.3 (it's a customized Red Hat 7.2) is loaded on IDE disks on the blades. There is an HBA on that blade as well, fiber attached to one of our SAN fabrics. The Host OS runs on the IDE drive. The Guest OS's, the VM's, run on vmfs volumes partitioned from SAN volumes. VM's run virtually everything from A to Z in our data center. Webservers, Appro, Laserpro, FTP, Domain controllers and on and on so as a sideways answer to your question, they are all SAN supported. For the record we still have plenty of physical servers. Some of them with fiber cards using SAN volumes as well.

Can you assign VMs LUNS from the SAN?

 

[FreshPrince]

Originally posted by: ktwebb

Originally posted by: SelArom

Originally posted by: ktwebb
Yes, the LUN (Logical Unit Number) will show up as a drive. If the LUN is presented while your in the OS you will likely have to scan. Either a command line SAN Scan or if your in windows you'd scan from disk manager.

Space is chucked out from the san, masked, and presented, typically to an HBA, a host bus adapter. AKA fiber card. The fiber card will attach to a fiber switch, the fiber channel fabric. The fabric sits between the HBA and the actual SAN device

You can also, if your hardware/software supports it, boot from the SAN. Currently I am moving all our VMWare esx servers over to a SAN boot device instead of the internal IDE or SCSI hard disk. Basically you boot into the HBA bios, enable it, and enable the boot device option, scan for the LUN and away you go. Depends on the SAN as to how easy that is. Pretty straightforward using our IBM shark. Sort of a PITA for the EMC DMX. But doable and a much better solution than a single point of failure IDE drive. (the slot for the second mirrored drive is occupied by the HBA)

I may be mistaken since I am not actually on the SAN team, but I believe the management front end for the shark is AIX, though everything is done pretty much through a GUI utility.


May be others way to implement specifically. This is how all the seperate fabrics are implemented in my environment. Not a bad career choice if your so inclined. Megabucks spent on a SAN, thus the guys responsible for upkeep, maintenance and managment follow suit.

wow you are totally over my head but I get the gist of what you're saying, and I thank you for shedding some light on this interesting technology! what kind of performance does SAN get as compared to like a regular file server or NAS? how many people can a SAN support?

can SAN be used for things like web/ftp servers or just strictly for storing raw data?

-SelArom

SAN is storage space. What do you want to use space for? That is what you can do with it. Web server, ftp servers, whatever. It's not really a question of can an FTP server use a SAN. Can your ftp server use more space available to it? Then the answer is yes.

We have many types of servers that use SAN resources. Our SQL farm has a something like a terrabyte presented to it. That is cumulative. There are many LUN's. Some presented as 20GB LUN's, other 100, 200GB, whatever.

The other major use of our SAN space is virtual machines. VMWare ESX running on an IBM BLade. At the moment ESX 2.5.3 (it's a customized Red Hat 7.2) is loaded on IDE disks on the blades. There is an HBA on that blade as well, fiber attached to one of our SAN fabrics. The Host OS runs on the IDE drive. The Guest OS's, the VM's, run on vmfs volumes partitioned from SAN volumes. VM's run virtually everything from A to Z in our data center. Webservers, Appro, Laserpro, FTP, Domain controllers and on and on so as a sideways answer to your question, they are all SAN supported. For the record we still have plenty of physical servers. Some of them with fiber cards using SAN volumes as well.

boot from SAN ftw! instant D.R. site. When you're not in disaster mode, all your machines in D.R. now becomes one huge test lab. You can't do that with physical servers

 

[Nothinman]

as another question, how are these SAN's usually managed? is it similar to how you usually do http://192.168.0.1 to access the interface for a router?

It would depend on the vendor, model, etc. Ours is managed from a terminal, literally a VT520 connected via serial cable. You can buy manangement consoles for them, but they're extremely expensive and usually web interfaces and point/click GUIs just slow you down and get in your way once you understand the product.

 

[SelArom]

thanks for all the insight fellas! I've got my presentation today and some questions I'm anticipating from my audience are:

1) how do I make a SAN? not the specifics, but rather a high level overview of what is involved. the way I understand it, you need some kind of dedicated device with a hard drive and os that runs the SAN software. but how do I expose that for other people to use?

2) what kind of speeds do SANs average? is it feasible to use one as a hard drive and get performance just as good? if so, can a lot of people use it, like say 100 people all with the same mapped drive on their pc?

if anyone has any personal experiences with this kind of stuff I'd really appreciate your insight. thanks again!

-SelArom

 

[ktwebb]

Originally posted by: FreshPrince

Originally posted by: ktwebb

Originally posted by: SelArom

Originally posted by: ktwebb
Yes, the LUN (Logical Unit Number) will show up as a drive. If the LUN is presented while your in the OS you will likely have to scan. Either a command line SAN Scan or if your in windows you'd scan from disk manager.

Space is chucked out from the san, masked, and presented, typically to an HBA, a host bus adapter. AKA fiber card. The fiber card will attach to a fiber switch, the fiber channel fabric. The fabric sits between the HBA and the actual SAN device

You can also, if your hardware/software supports it, boot from the SAN. Currently I am moving all our VMWare esx servers over to a SAN boot device instead of the internal IDE or SCSI hard disk. Basically you boot into the HBA bios, enable it, and enable the boot device option, scan for the LUN and away you go. Depends on the SAN as to how easy that is. Pretty straightforward using our IBM shark. Sort of a PITA for the EMC DMX. But doable and a much better solution than a single point of failure IDE drive. (the slot for the second mirrored drive is occupied by the HBA)

I may be mistaken since I am not actually on the SAN team, but I believe the management front end for the shark is AIX, though everything is done pretty much through a GUI utility.


May be others way to implement specifically. This is how all the seperate fabrics are implemented in my environment. Not a bad career choice if your so inclined. Megabucks spent on a SAN, thus the guys responsible for upkeep, maintenance and managment follow suit.

wow you are totally over my head but I get the gist of what you're saying, and I thank you for shedding some light on this interesting technology! what kind of performance does SAN get as compared to like a regular file server or NAS? how many people can a SAN support?

can SAN be used for things like web/ftp servers or just strictly for storing raw data?

-SelArom

SAN is storage space. What do you want to use space for? That is what you can do with it. Web server, ftp servers, whatever. It's not really a question of can an FTP server use a SAN. Can your ftp server use more space available to it? Then the answer is yes.

We have many types of servers that use SAN resources. Our SQL farm has a something like a terrabyte presented to it. That is cumulative. There are many LUN's. Some presented as 20GB LUN's, other 100, 200GB, whatever.

The other major use of our SAN space is virtual machines. VMWare ESX running on an IBM BLade. At the moment ESX 2.5.3 (it's a customized Red Hat 7.2) is loaded on IDE disks on the blades. There is an HBA on that blade as well, fiber attached to one of our SAN fabrics. The Host OS runs on the IDE drive. The Guest OS's, the VM's, run on vmfs volumes partitioned from SAN volumes. VM's run virtually everything from A to Z in our data center. Webservers, Appro, Laserpro, FTP, Domain controllers and on and on so as a sideways answer to your question, they are all SAN supported. For the record we still have plenty of physical servers. Some of them with fiber cards using SAN volumes as well.

boot from SAN ftw! instant D.R. site. When you're not in disaster mode, all your machines in D.R. now becomes one huge test lab. You can't do that with physical servers

Booting from SAN in the Test/Dev Lab (EMC DMX). Will be booting from SAN once the Shark is replaced in the production environment. That project will begin when the Shark replacement is delivered. rougly a month. I have about 3 of the 26 Production blades booting from SAN. Once the VM's are up the I/O isn't that bad but reboots of ESX or bringing VM's up creates too much of a performance hit. Which really isn't very much at all. Says what kind of condition or shark is in however. Replacement will be high end so eventually all our blades/esx instances will be BootFromSan enabled. All our SAN's replicate to identical hardware at the Dark site.

BootfromSan would be unecessary and frankly, not even a consideration if the IBM bladecenters weren't so poorly engineered/conceived for local disks. Well, at least for enterprise environments with HBA's in the expansion slot. If the expansion slot doesn't have a card in it, mirroring to a second drive that normally goes in that location is fine.

 

[ktwebb]

Originally posted by: SelArom
thanks for all the insight fellas! I've got my presentation today and some questions I'm anticipating from my audience are:

1) how do I make a SAN? not the specifics, but rather a high level overview of what is involved. the way I understand it, you need some kind of dedicated device with a hard drive and os that runs the SAN software. but how do I expose that for other people to use?

2) what kind of speeds do SANs average? is it feasible to use one as a hard drive and get performance just as good? if so, can a lot of people use it, like say 100 people all with the same mapped drive on their pc?

if anyone has any personal experiences with this kind of stuff I'd really appreciate your insight. thanks again!

-SelArom

1. You don't. You contact a vendor and purchase it through those channels. I suppose there may be some uber geeks that have some homemade SAN device. Generally speaking you would be making a very high dollar purchase if your implementing a SAN. Tens to hundred thousands of dollars.

2. Depends on the hardware your running and how it's implemented. Drives, interface, fabric switch and HBA's at the end point. no one answer to that. And the second part of the question has been asked and answered as basically, it's not a valid question. Sounds like your just not getting the concept. I would suggest googling SAN technology. Might help you digest it better.

 

[SelArom]

Originally posted by: ktwebb
1. You don't. You contact a vendor and purchase it through those channels. I suppose there may be some uber geeks that have some homemade SAN device. Generally speaking you would be making a very high dollar purchase if your implementing a SAN. Tens to hundred thousands of dollars.

2. Depends on the hardware your running and how it's implemented. Drives, interface, fabric switch and HBA's at the end point. no one answer to that. And the second part of the question has been asked and answered as basically, it's not a valid question. Sounds like your just not getting the concept. I would suggest googling SAN technology. Might help you digest it better.

thank you for replying, and I guess I don't quite get it. I don't understand how SAN is any different than a file server. if it is just network storage, what is its advantage and difference between the traditional file server solutions?

-SelAorm

 

[SelArom]

did some googling and found this helpful article. I'm still shady on most of the details but if anyone else is in the dark about SAN like I am, they might enjoy this article:

http://www.enterprisestorageforum.com/technology/features/article.php/981191

-SelArom

 

[Nothinman]

1) how do I make a SAN? not the specifics, but rather a high level overview of what is involved. the way I understand it, you need some kind of dedicated device with a hard drive and os that runs the SAN software. but how do I expose that for other people to use?

Usually you buy one, but if you want you can setup a Linux box with some iSCSI software or nbd.

2) what kind of speeds do SANs average? is it feasible to use one as a hard drive and get performance just as good? if so, can a lot of people use it, like say 100 people all with the same mapped drive on their pc?

It depends on the medium used i.e. fiber, ethernet, etc. Throughput can be higher than a normal disk but usually latency will be higher since you have to go through the SAN device.

I don't understand how SAN is any different than a file server.

Think of a SAN as a really complicated SCSI controller. It's a disk server, not a file server.

 

[FreshPrince]

san speeds differer from device to device, but typically you would see about 960M bytes/sec and up to 2.8G bytes/sec ...