Fiber optics distribution in a datacenter

[spidey07]

I hope to get some good feedback here because I'm getting frustrated. Our datacenter is quite large and cabling is becoming cumbersome. We've got fiber everywhere - from gigabit ethernet to fiber channel switched storage centers. My question is this - is it better to run fiber patch panels in each rack to a central distribution panel? Simply patch what you need to at the dist panels.

Right now all fiber cabling (not my choice or design BTW) is simply to run a two pair patch cord directly between the fiber channel switches to the host. Getting quite messy with 50+ hosts on SANs and dozens on gigE. All cabling is subfloor trays.

What I wanted to do was treat the fiber just like copper. Each rack (5-9 hosts) would contain a 24 strand patch panel. From this panel use a single 24 strand distribution cable to the MAIN distribution panel (numerous 24 or 48 port panels). Then use patch cords on the MAIN to connect racks to gigE ports or Fiber channel ports.

Any better ideas? All racks are 19" enclosed (darn) and space is a premium. I would like to come up with a standard rack configuration for Power, Fiber, Copper (2x30 amp 240 VAC power, 24/48 copper patch panel, 24 fiber patch panel)

All fiber is 62.5 u multimode. All host interfaces are SC but I'm open to ST or RT-MJ for the panels/patch.

I've been out of the physical layer for so long, help a brother out will ya?

Cheers!

 

[shadow]

is the distance great enough on those gigE to be worried about signal loss induced by more physical connections?

I really love centralized approaches, yet I am still a little fuzzy as to your overall layout, the egress from your network would be where you have your fiber channel switches and where you want to put the MAIN distribution hub (yeah I know weak word, so sue me ).

I also take it that you have all your switching equipment at one place as well.

I would definately suggest RT-MJ, or VF-45 for the patch panels due to their small size relative to SC and ST.

Okay, if I've understood everything properly I can make one more setup, run the cabling like you have suggested, distribution to each rack from the MAIN, but (this is the skimpy shadow talking here...) scratch the panels and simply organize the cabling well, use different colors for different section of your data center and make some sort of hooks / holders on the walls for the free wires. At the racks cut the cable to the appropriate length and zip-tie to the frame.

Terminate now or later, either way you can switch to panels later down the road if you want to.

ahh, either way, I hope this helps a bit

 

[spidey07]

Thanks shadow.

My problem comes in this form: LAN switching gear-gigE, in two racks (takes up two racks), four EMC fiber SAN switches in another, four Storageworks SAN switches in yet another. About 40 host racks total. So I have 40 host racks that need fiber connectivity to possible 4 other racks. Sounds like cross connect time to me?

I just don't like the idea of patch cables running this way and that, not labeled, no fricking clue what goes to what. I was hoping that a central distribuition rack for all fiber connectivity could alleviate some of the cable management nightmare i've got now.

I'm really looking for different opinions from people who have done large scale Datacenter fiber cabling.

My hats off to shadow for his opinion...anymore? (of course your idea is good shadow, i'd like to see some more opions)

cheers!
spidey

ps - man do I love vendor dinners. mmmm....filet mignon/cabernet.....mmmmm

 

[shadow]

awwww, your kind words make me blush!

I had breadsticks myself



btw did you check out my thread about the networking classes?

 

[spidey07]

Yeah, I saw the post. Really didn't think of much to add. There are so many disciplines.

The basics of IP and the OSI model are keen. Routers/switches/hubs and the like are bare minimums.

 

[shadow]

it's a hard question to answer well.

 

[CTR]

Hey Spidey I feel your pain. We have colocated equipment in some cities where we have to do that kind of equipment-jumper-panel-horizontal-panel-jumper-equipment runs. You've got to make some rack labels and get some cable tags. Create a naming standard for your racks, if you haven't done so already. Then label your equipment by its rack location and mounting position. Then put tags on each end of your cable, with both sides of each tag filled in. That means you can look at either tag and know exactly what interface on what equipment in which rack it is coming from and going to. Like Shadow said, color-coding helps too but if you already have a ton of cabling in place that is usually not an option.

Tagging both ends of every cable takes FOREVER but it saves you tons of time when it comes to troubleshooting. And it SCALES

 

[spidey07]

Well, if accepted practice is to simply label both ends (i've already preached that to the NOC cabling guys) then that's what I'll do. But man are there a lot of fiber patch cables running under the floor. Must be 100 or so (96 port of SAN switch 64 ports of gigE)

thanks a bunch fellas!

But the patchpanel would scale better

 

[CTR]

Whoah, there big fellah! I'm saying use as many patch panels as you can get away with, but label both ends of every piece of cable, even patches. I'm guessing you don't have any problems with db loss on these connections, right?

 

[spidey07]

Don't think loss is a problem according to the microtest, but then again i've only tested a few.

But, if I were to install this whole cabling structure there would be four couplers in line and four crimp style terminations (horizontal runs). Add three patch cables to the mix...rack, crossconnect, rack.

Is this too much? All 62.5u MM used for fiber channel and gigE.

thanks again

 

[CTR]

Sounds like you are pushing it, especially when you are talking about "crimping" fiber. Do you mean to tell me you don't have a fusion splicer?!!

Best way to find out if it works is try it. But one bad crimp and you are screwed. To get around this we buy all of our fiber runs pre-made. It sucks to have to get it measured out and then wait for the order to arrive, but at least we don't worry about local techs screwing up a termination.