Poor gigabit transmit performance with socket A chipset

[Madwand1]

The problem is that I get at best ~ 400 Mb/s transmit performance as measured by iperf (isolated from disk, etc). Receive performance is fine (> 900 Mb/s). CPU utilization is around 25% while sending, and around 50% while receiving.

Can anyone tell me why the KT600 setup's GbE transmit performance is so poor? Is there anything that I can do about this?

--------------------------

E.g. the following shows bi-directional performance, first transmitting, then receiving:

E:\tools\bench\iperf>iperf-orig -c 192.168.0.100 -l 1M -t 12 -i 3 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.100, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1780] local 192.168.0.157 port 1098 connected with 192.168.0.100 port 5001
[ ID] Interval Transfer Bandwidth
[1780] 0.0- 3.0 sec 147 MBytes 411 Mbits/sec
[1780] 3.0- 6.0 sec 145 MBytes 405 Mbits/sec
[1780] 6.0- 9.0 sec 146 MBytes 408 Mbits/sec
[1780] 9.0-12.0 sec 145 MBytes 405 Mbits/sec
[1780] 0.0-12.0 sec 584 MBytes 407 Mbits/sec
[1960] local 192.168.0.157 port 5001 connected with 192.168.0.100 port 2406
[ ID] Interval Transfer Bandwidth
[1960] 0.0- 3.0 sec 356 MBytes 996 Mbits/sec
[1960] 3.0- 6.0 sec 354 MBytes 990 Mbits/sec
[1960] 6.0- 9.0 sec 355 MBytes 993 Mbits/sec
[1960] 0.0-12.0 sec 1.38 GBytes 992 Mbits/sec


-------------------

Here's the setup:

Athlon XP 2600+ (~ 1.9 GHz, 333 MHz FSB)
1 GiB DDR 400 C2 1T
Asus A7V600 (KT600)
Intel Pro 1000 / MT Server (PCI-X running on PCI)
XP Pro SP2 / Vista RC2
WD 80 GB PATA
AGP video
Other PCI slots empty
Onboard 3com PCI GbE NIC disabled
Latest BIOS, chipset drivers, etc.
Cabling interchanged with another
Switch changed with another

NIC, TCP/IP parameter tweaks tried, 9K jumbo frames enabled for bulk of tests; also tried disabling it.

-----------------

The receiving computer in this case is a X2 3800+, and it can easily hit > 900 Mb/s bi-directionally with similar computers.

Onboard 3com NIC has similar performance & problem (receive OK, transmit poor).

The same pattern holds with other receivers.

A similar, slower computer -- Athlon XP 1800+ (~1.5 GHz, 266 MHz FSB) running on A7V8X (KT400) with a slower NIC (Marvell PCI) -- has better transmit performance (~ 640 Mb/s).

The same Intel PCI NIC in a different computer (X2 3800+) running a similar OS similarly configured can hit > 800 Mb/s bi-directionally (don't recall exactly atm).


Also posted the same question here: http://episteme.arstechnica.com/eve/forums/a/tpc/f/469092836/m/495009541831

 

[cmetz]

Via chipsets suck for I/O. Their PCI implementation has always been problematic. The socket A platform also sucks for I/O (something AMD fixed pretty well in the 939/940 platform). Add that up and what do you get? Poor performance.

With the same gigabit NIC, my lab AMD XP 2500+ / Nforce2 machines get about half the netperf bandwidth of P4 2.8GHz / 875 machines. Yes, that's slightly an apples and oranges comparison but the point is that those machines are similar in application performance but radically different in I/O performance.

 

[Madwand1]

Thanks for the reply. I guess I'll have to take the "VIA chipsets suck for I/O" answer, as that's how it looks. Also funny that the KT600 looks even worse than the earlier KT400.

 

[spidey07]

I'm not a PC hardware guy but I'll take a stab....

Something is going on with the bus and the transfer from RAM/proc/NIC. It's gotta cross those paths.

<---backs out of thread because I don't know jack about PC technology...just throwing darts.

Madwand, you seem to really be into this so I've posted in High Tech and excellant research paper I found the other day on 1 and 10 Gbs NICs and what influences their throuputs. PIC bus timings, frame size, interframe gap....tons of good stuff.

 

[Madwand1]

The funny thing in this case is that I somewhat eliminated the RAM/proc/NIC, and came up with worse performance with better gear. I think this shows here that another variable is coming into play, and I guess that's the chipset / PCI implementation as cmetz said -- something consumers wouldn't know anything about in terms of performance unless someone does such measurements which factors out the RAM/proc/NIC/etc. And the newer chipset performed worse -- again, pretty counter-intuitive.

Here's an article that I came across on the topic.

http://www.dssnetworks.com/v3/FAQs.asp#Relation

Some of the material is a little dated, but it still had material that I wasn't aware of previously. I wonder how well it ties in with other findings and the HT paper.

Thanks, I'll read it (as far as I can..).