How fast is gigabit?

[Shinare]

Yes, I realize its 1000Mbps but in reality how fast is it? I ask this because I am not seeing what I am expecting on my network.

I have 6 48-port gigE switches (Linksys SRW2048) and have them configured in a way that two of the switches have 3 fibre gigE connections between them in a LAG (Link Agrogate Group) between two floors, and there are two other switches on each floor with their own 2x 1Gb copper LAGs connecting them to the "main floor switch" for that floor.

Anyway, I do a file copy (a large directory full of files) from a workstation to another workstation on the same switch, both of us have gigE NICs and both are showing as 1Gb Full Duplex connections on the switch. The file copy is running about 1-7 mB/s which to me is waaaaaay slower than 1000. So I do a file copy from one server to another server (running W2k3R2) who are both on another switch (the main floor switch for floor 2) and I see the same speed. (the servers also have 1Gb NICs in them.

Am I expecting the wrong thing here, or is there something dredfully wrong with my setup?

 

[Madwand1]

Yes, something's wrong. Even dirt-cheap consumer gigabit can do much better.

http://forums.anandtech.com/messageview...atid=36&threadid=1840646&enterthread=y

 

[Jeff7181]

I'm not a professional, but here's my .02 anyway...

Theoretically Gigabit tops out at 125 MBytes/sec, which is still more than the majority of hard drives can keep up with. In short, there is something wrong with the setup. Even a 100 Mbit LAN should be capable of at least 10 MBytes/sec (theoretical max being 12.5). Keep in mind that lots of small files will transfer slower than one big file, fragmented files can't be read as fast as contiguous files, and writing data is typically slower than reading data. So if the destination drive is highly fragmented it could be slowing down because of that, and maybe also because if you're sending lots of small files, there's going to be more overhead as far as updating the file system.

*EDIT* I would probably try multiple file transfers between different computers. Like, 2 or 3 pairs of computers transferring files between each other... or find a workstation with two hard drives, and have two other workstations send files to each drive and see how your bandwidth looks then. If it still stays between 1-7 MB/sec I would assume there's an issue with the LAN itself... if each pair is able to maintain 1-7 MB/sec, or the workstation with two hard drives and two other workstations sending data to it jumps to 2-14 MB/sec than I'd assume it's just a case of slow computers not being able to keep up.

 

[netsysadmin]

The word Servers these days could almost me anything equipment wise. What is the specs on your servers and are they running SCSI drives?

John

 

[spidey07]

Make sure the NICs and switch are set to autonegotiate speed/duplex. Make sure you're using good cable.

Check for errors on the switch port or the NIC.

Something is very wrong if it is running that slow.

If need be run ethereal or a sniffer on one of the stations to see what is going on.

 

[Shinare]

servers are 5 Dell PowerEdge 2850s with 2x Dual Core 2.8GHz Xeon CPUs with 4GB RAM each. All 5 servers are connected to a Fiber Attached SAN system running 15x 10kRPM 147GB U380 drives in RAID5 via 2 load ballanced Qlogic fiber HBA's in each server. They are connectred to the LAN using a single 1GB integrated NIC.

 

[Smilin]

Using jumbo frames or no?

When you trace it out with netmon or ethereal, what is the MTU of the frames going by?

 

[spikespiegal]

My dual 2ghz Xeon servers running 15K SCSI (RAID 5 / no SAN) can easily push a file on gigabit at around 30mBYTES per sec. That's significantly faster than yours, so 'Houston, we have a problem'.

As for the SCSI comment above, I'm getting tired of hearing "Servers need SCSI" because it's not true. If you are incurring a lot of write transactions, then by all means go SCSI. For read transactions, I prefer a dense array of SATA. Cheaper, less heat, cheaper, quiter, less proprietary, cheaper, etc.