Gigabit performance: Onboard vs. PCI-E NIC

[djtodd242]

OK, I recently threw a gigabit switch into my network and attached my PCs to it as opposed to directly to my router. Both of my systems have Asus boards with onboard gigabit ethernet (nVidia chipsets).

It's definitely much faster than before, but I found that it's not nearly as fast as I think it should be. When copying 1.2G files from system to system the speed bounces around a fair bit, usually between 130-250Mb/s.

Both systems are XP SP3 with fairly new, recently defragged drives. One thought was that I was running into a disk interface bottleneck. The other was that the onboard NICs aren't as good as one would hope.

Would buying a pair of PCI/PCI-E NICs likely improve performance?

 

[JackMDS]

Originally posted by: djtodd242

Would buying a pair of PCI/PCI-E NICs likely improve performance?

Not for regular use.

250Mb/sec. I.e. 30MB/sec, transfer is common to Giga Peer-to-peer network that uses client OS.

Look at the TCP/IP stack, increasing the RCwin to 522720 might add some momentum to the "speed".

http://www.ezlan.net/giga.html

 

[Fullmetal Chocobo]

Originally posted by: JackMDS

Originally posted by: djtodd242

Would buying a pair of PCI/PCI-E NICs likely improve performance?

Not for regular use.

250Mb/sec. I.e. 30MB/sec, transfer is common to Giga Peer-to-peer network that uses client OS.

Look at the TCP/IP stack, increasing the RCwin to 522720 might add some momentum to the "speed".

http://www.ezlan.net/giga.html

Would using a Server 2003 domain server speed things up?

 

[JackMDS]

Some what, the client OS is still sitting on one side.

 

[Madwand1]

Originally posted by: Fullmetal Chocobo
Would using a Server 2003 domain server speed things up?

Yes, partially, but SMB transfers from 2003 to 2003 don't approach saturating gigabit IME.

Moreover, the generation of the OS is more important than the client/server distinction IME (assuming non-default tuning options in some cases though). I've seen SMB pushes from Vista to XP-64 and 2003 approach gigabit saturation, but never pulls, and not pushes from XP-32 and 2003 / 2003 R2, nor pushes to XP-32. (XP-64 and 2003 are basically the same generation, newer than XP-32, but of course older than Vista.)

Vista to Vista, I've approached gigabit saturation over SMB (2.0) for pushes and pulls.

Here's a graph showing some performance variations due to change of source OS, with the remote computer running Vista:

http://i89.photobucket.com/alb...smb-transfer-vista.png

Here are some details of a push from a Vista client to XP-64, showing 117.2 MB/s sustained. Pushes to 2003 performed similarly. Pushes to XP-32 hit ~80 MB/s at best.

E:\tools>dir n:\test\test0\10.gb
Volume in drive N is nvr533264
Volume Serial Number is BC8B-9694

Directory of n:\test\test0

08/24/2006 06:01 PM 10,000,000,000 10.gb
1 File(s) 10,000,000,000 bytes
0 Dir(s) 567,842,832,384 bytes free

E:\tools>time 0 nul
The current time is: 10:39:41.85
Enter the new time:
E:\tools>xcopy /y n:\test\test0\10.gb \\intel-xp64\n\test\test9
N:\test\test0\10.gb
1 File(s) copied

E:\tools>time 0 nul
The current time is: 10:41:07.21

 

[djtodd242]

Well, after doing a bit of poking around and some real research I went and ordered the intel NICs anyhow.

Mostly because while my onboard NIC in my workstation and my switch support jumbo frames, my older nForce 4 board in the "TV PC" doesn't. I also upgraded my cabling from CAT5 to CAT6, I doubt it did anything, but 2 cables set me back a whopping $12.

I also found some registry tweaks that did help a bit.

http://craig.backfire.ca/pages/computers/gigabit

But I do realize that like what you guys are saying, I'm probably getting about as close to the best I can hope for. I consider the Intel NICs to be a long term investment, as they'll most likely be better than anything we ever get onboard, so the cards will move from upgrade to upgrade over the years.

 

[JackMDS]

Originally posted by: djtodd242
Well, after doing a bit of poking around and some real research I went and ordered the intel NICs anyhow.

Mostly because while my onboard NIC in my workstation and my switch support jumbo frames, my older nForce 4 board in the "TV PC" doesn't. I also upgraded my cabling from CAT5 to CAT6, I doubt it did anything, but 2 cables set me back a whopping $12.

I also found some registry tweaks that did help a bit.

http://craig.backfire.ca/pages/computers/gigabit

But I do realize that like what you guys are saying, I'm probably getting about as close to the best I can hope for. I consider the Intel NICs to be a long term investment, as they'll most likely be better than anything we ever get onboard, so the cards will move from upgrade to upgrade over the years.

This registry "Trick" (which is called Jumbo frames) would work only if you hardware support Jumbo frames. Even then it might messup your Internet connection Coz Cable needs 1500 and DSL 1492.

In otherwords you spend money on hardware that probably does nothing to increase traffic, and you use a "Trick" that probably slows down your Internet.

 

[djtodd242]

Actually, my internet connection doesn't seem to be affected at all. I've been running with jumbo frames for about 3 days, and as an example, downloading from usenet is still at the same 4.5Mb/s as usual. I haven't noticed any other slowdowns with HTTP or email either.

I can't speak for anyone elses setup though. I imagine it works fine as I'm using a gigabit switch that does support jumbo frames, and while I'm sure my router doesn't it's hanging off the switch as well.

 

[JackMDS]

Ok, sounds good Enjoy. :thumbsup:

 

[Madwand1]

Originally posted by: djtodd242
http://craig.backfire.ca/pages/computers/gigabit

For Windows, all of that can be done more simply with a single setting:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"TcpWindowSize"=dword:40000

(Where 40000 is just the hex value of the setting provided in the above link, without further testing.)

A reboot is needed for this setting to take effect.

If you have a significant amount of memory, in some cases, further performance improvement can be seen with the following setting:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management]
"LargeSystemCache"=dword:00000001

This setting is on by default if you're running a Windows Server with "Maximize data throughput file sharing" or some similar option (the wording might change across versions), which is not visible in the UI for client OS versions.

 

[djtodd242]

I got the Intel Pro 1000 PT NICs today, and just turned Jumbo Frames on in the device(s). While overall the performance is about the same, it's much less "bursty".