Fibre Channel

[thornc]

Hi,

Can someone explain the inner workings of FC and SAN?? I've been looking around and I can't seem to find a simple, to the point explanation of the important concepts...

Thanks in advance.

 

[rjain]

Are you asking for an explanation of how fiber optic cabling itself works or how the protocols work? General hardware may be a better place to ask about the protocols.

 

[thornc]

About how everything works... optics and protocols...

 

[rjain]

Keeping to the topic of this forum, I'll get you started on the optics.

Basically, the fiber optic cable guides the light signal from the emitter to the receiver. Some of the things the cable needs to do in order to be useful are:

1. Get the light from the emitter to the reciever. This is done by constructing the cable such that the light reflects off the outer boundary of the cable back towards the center. This allows the cable to curve around corners or in loops in case of slack.

2. Not lose the power of the light.

3. Not distort the light signals. If they are distorted too much, one bit will overflow onto another and the bit pattern won't be able to be reconstructed.

If you look at documentation for oscilloscopes, you'll see many of the measurements that are conducted to determine how well the signal transfer occurs.

 

[thornc]

I know the basics of snell's laws and refraction! The point that most confuses me is 1.

How do they make the cable go around corners, without loosing signal! a 90 degrees curve (pi/2 rads) seems to be enough to change the refractions of the cable.... or isn't it??

(thanks for the replies so far rjain!)

 

[rjain]

It's reflected off the interface between the glass core of the cable and the sheathing around that cable. But a cable that stretches across 1/4 of the world has a 90 degree curve, just like a cable that is taped to a corner of the wall. It's an issue of the lowest radius of curvature that can be tolerated.

 

[Mday]

Originally posted by: thornc
I know the basics of snell's laws and refraction! The point that most confuses me is 1.

How do they make the cable go around corners, without loosing signal! a 90 degrees curve (pi/2 rads) seems to be enough to change the refractions of the cable.... or isn't it??

(thanks for the replies so far rjain!)

look for "internal refraction" in fiber optics. basically, you have several concentric tubes of "fiber" with different coefficients for refraction, and the light doesnt exit the fiber at the walls, only the ends. if you think about how the fibers are bent, you'd know the answer to the curve question.

the light does lose "intensity" as the length of the fiber increases, but you can use optical amps to take care of that.

 

[CanOWorms]

The cladding is doped to have a slightly lower refractive index than the core so that the light is totally reflected from the cladding.

 

[Possessed Freak]

Edit: Hit the enter before replying, ooops...

Fibre Channel has nothing to deal with Fiber Optics. FC is basically scsi taken to a higher level. It supports 126/127 devices on a single chain versus the 15 allowed on regular scsi. FC makes a loop to all the devices and ends back at the controller. Think of it as the token ring of hard drives. If you want, I can dig up old links to the very specifics of fibre channel and just how it works. About 2 years ago, I attempted to make a FC hookup using nothing more than a few resistors and some cat-5 cabling. Unfortunately I read the schematic wrong and burnt out my 2 drives and controller. Having set constraints on monetary funds allocated to each project, even though I corrected my mistake, I could not retry the project.

 

[JW]

Fiber (e-r) refers to physical media. Optic cable used for networking, storage connections, whatever etc.

Fibre (r-e) Channel is a protocol that can run over either copper or fiber connections.

FC has three main topologies: 1: Point-to-point between two and only two devices. 2: A fabric model using a switch, with each connection having full bandwidth. 3: The one that PossFreak referred to is Arbitrated Loop (FC-AL). It connects up to 127 devices in a loop, sharing the bandwidth. But even though it's a loop, it doesn't use token passing. A device has to arbitrate for control of the loop. Typically you'll use a hub in this topology.

In the fabric model, it's also possible to connect an AL to a port on the switch for a hybrid situation.

-jw

 

[thornc]

Thanks for all the info guys. Sorry I didn't answer sooner, but I had no connection available.

JW, so there are several topologies. Interesting, which one is the one used in a SAN. FC-AL?? Usually they connect the client machines to a switch and then the switch to the disk arrays! So that might be the hybrid situation you are talking!

 

[Fallen Kell]

Usually SANs (Storage Area Networks) are basically a JBOD (Just a Bunch Of Disks), connected to a high speed network, in current technologies, a Fibre Channel network and have some basic additional properties in terms of access controls and management, which usually involve a specialized controler system connected to the disks which does access control and route all the data requests to the disks.

FC can actually be over copper wire or optical cable. Adapters on the ends of the cables allow for both types to be connected with current generation FC switches (basically these are just like regular network switches, but use the FC protocol).

To explain the inner workings in more detail would mean needing to focus on a particular SANs product. There are several major vendors for SANs equipment with the most PR going to EMC, but in reality they are really no longer the people to look at in terms of price/performance as they charge a very high premium for their equipment. All the SANs retailers do things slightly different in terms of how they control the access to the disks as well as maintainence and administration. Most have telnet/command line controls to do the configuration as well as some having a GUI interface.

To be honest, a SAN is an interesting idea depending on what you need to use it for. If you are just sharing out storage for user data and home area locations, a SANs is most likely not the best idea. The true power of a SAN is its mirroring and reliability ratings, but this comes at a very high price. For instance with mirroring and RAID5 (standard config), you will need to buy 6TB to get only 2TB of actual usable space (that is a lot of overhead cost).

 

[Matthias99]

I actually work for one of those SAN equipment vendors. While the high-end products cost a lot, some customers *need* to put petabytes of data into a single high-performance box. Most such vendors sell smaller boxes (still in the hundreds of thousands) for smaller solutions. If you're looking for storage for a home network, you really want a NAS (Network Attached Storage) system -- which we also sell.

Internally, our systems have a large number of FC hard disks organized into multiple arbitrated loops for redundancy purposes -- so if one drive goes bad, it doesn't knock out the whole array (we also have multiple controllers that can be attached independently to each loop, to guard against controller problems). You interface to the box as a whole through either a point-to-point or switched FC network (or through IBM's ESCON/FICON protocols, but I won't go there. Suffice it to say it's like Fibre Channel for mainframes.) We're also supporting iSCSI (SCSI run over Gigabit Ethernet). Externally, the box just looks like a whole lot of SCSI devices to whatever you plug into it. One advantage of a system like this as opposed to a JBOD array is that there is a lot more flexibility in creating virtual devices and doing things like snapshotting and remote synchronization automatically. You also don't have to worry about getting equipment from multiple vendors working together.

And there are different levels of mirroring and protection. Not everybody needs what you described -- RAID5 by itself offers single-disk failover, and can be configured with 3-8 disks per raid group (with 8 disks per group, you need 8 disks to hold 7 disks worth of data). There's also straight mirroring (RAID0), which gives a 2:1 ratio of raw disk space to usable space, and provides single-disk failover, but also improves performance (since you can read from both disks simultaneously and you don't have the parity overhead of RAID5).

There's an O'Reilly book ("SAN and NAS Technologies" or something like that) which would probably tell you more than you ever wanted to know about all of this.

 

[thornc]

Nice replies,
Not elaborate a bit on my problem...
I'm looking for a storage solution to be used as temporary (1 to 5 days) storage for a big ammount of data, around 4 to 8 TeraBytes.
The problem is not security but acceptable performance, the computers processing the data to be stored are putting out around 175GB of data each hour (50MB/sec), and this need to be stored untill some other computer takes it and puts it into a more permanent storage.

I've looked at iSCSI and NFS/CIBS over Gigabit Network, but I needed the help understanding this FC thing.
I just need to come up with an acceptable solution in terms of performance/price before sending it up to the managers.
Thanks for all the help so far.