Infiniband latency

[kobazik]

I got two hosts with Voltaire HCA500Ex-D (MT25408 ConnectX Mellanox) 10Gbit cards connected to a Cisco SFS7000d IB switch via CX4 3m cables.

I'm really concern about latency which is higher than on 1Gbit ethernet connection using same hosts.

[root@localhost ~]# ibping -G 0x0008f104039a5589 Pong from host-a.(none) (Lid 3): time 0.238 ms Pong from host-a.(none) (Lid 3): time 0.291 ms Pong from host-a.(none) (Lid 3): time 0.320 ms Pong from host-a.(none) (Lid 3): time 0.290 ms Pong from host-a.(none) (Lid 3): time 0.335 ms Pong from host-a.(none) (Lid 3): time 0.281 ms

Most people are having 0.040ms - 0.050ms results using same IB cards.

Cisco IB Switch and cards are running latest firmware.

Any ideas?

 

[brshoemak]

I think you'd be a bit better off posting your question in another forum like the Arstechnica Networking Matrix forum.

Many of the questions on this forum don't involve higher end networking setups like Infiniband or FCoE.

Not saying anything bad about this forum at all, just saying that if I have a question regarding things like BGP, fiber channel, etc. I'm more likely to get answer at another forum.