Solved! What Are The Major Differences Between Mellanox VPI And Mellanox Ethernet NICs?

[TryingToVirtualizeStuff]

Hello,

My company wanted to purchase a NIC with dual 100 GbE ports utilizing QSFP28 that must work with Windows 11 Pro.

From what I can see, Nvidia's Mellanox ConnectX-5 cards seem to meet my company's criteria.

However, the card that I was initially planning to purchase : MCX516A-CDAT

NVIDIA Mellanox: End-to-End Interconnect Networking Solutions

The leading supplier of end-to-end Ethernet and InfiniBand intelligent interconnect solutions and services.

store.nvidia.com

...is completely out of stock (and won't be in stock for months).

Whereas the following card (which seems similar) : MCX556A-EDAT

NVIDIA Mellanox: End-to-End Interconnect Networking Solutions

The leading supplier of end-to-end Ethernet and InfiniBand intelligent interconnect solutions and services.

store.nvidia.com

...is currently in stock.

That said, what are the major differences between the two cards, as it looks like the EDAT, which supports VPI, should work with both Ethernet and Infiniband.

Whereas the CDAT, only works with Ethernet (plus uses on PCIe 3.0 x16).

It seems like the EDAT offers more functionality/throughput, for the same price, but I'm worried that I'm missing something (as Nvidia would never offer more functionality for the same price ).

Thank you,
Nelson

 

[Fallen Kell]

Well, first off, both cards are PCIe 4.0 16x slot cards. They are that so you can actually get 2 ports that can be 100gig. A PCIe 3.0 would not be able to support two ports as it tops out at 128gigabit in either direction (i.e. 128gigabit to the card and 128gigabit from the card).

You already mentioned the other difference, the VPI card can be configured to run as infiniband (in fact, it is most likely configured in that mode out of the box and you need to use their firmware utilities to have it instead change to ethernet/network mode so that you canuse it under Windows as a network card). The VPI cards also tend to have a few other features built into the silicone such as different hardware offload engine functionality, but you need to do a deep dive into Nvidia’s documentation and feature sets for each model to see those things (Nvidia typically has a pdf overview of the card generation listing all the possible features in that generation of cards and then at the bottom has a table that matches the various models of cards in the generation with the feature set).

 

[TryingToVirtualizeStuff]

Well, first off, both cards are PCIe 4.0 16x slot cards. They are that so you can actually get 2 ports that can be 100gig. A PCIe 3.0 would not be able to support two ports as it tops out at 128gigabit in either direction (i.e. 128gigabit to the card and 128gigabit from the card).

You already mentioned the other difference, the VPI card can be configured to run as infiniband (in fact, it is most likely configured in that mode out of the box and you need to use their firmware utilities to have it instead change to ethernet/network mode so that you canuse it under Windows as a network card). The VPI cards also tend to have a few other features built into the silicone such as different hardware offload engine functionality, but you need to do a deep dive into Nvidia’s documentation and feature sets for each model to see those things (Nvidia typically has a pdf overview of the card generation listing all the possible features in that generation of cards and then at the bottom has a table that matches the various models of cards in the generation with the feature set).

Hello Fallen Kell,

Thank you very much for your input.

A few more questions for you (to confirm some things):

1. Can I assume the VPI card supports both Infiniband and Ethernet?

2. If so, can I assume it's preferable to go with the VPI card (to use Ethernet now and then in the future just reconfigure the card for Infiniband)?

In other words, what would be the point of purchasing the Ethernet card (as it's the same price as the VPI Infiniband card), if the VPI card can do both Ethernet and Infiniband?

Thank you,
Nelson

 

[mxnerd]

Maybe this helps?

Mellanox ConnectX-5 VPI 100GbE and EDR InfiniBand Review

Our Mellanox ConnectX-5 VPI 100GbE and EDR IB review shows why this PCIe Gen4 capable 100GbE and 100Gbps EDR InfiniBand adapter is in a class by itself

www.servethehome.com

 

[TryingToVirtualizeStuff]

Maybe this helps?

Mellanox ConnectX-5 VPI 100GbE and EDR InfiniBand Review

Our Mellanox ConnectX-5 VPI 100GbE and EDR IB review shows why this PCIe Gen4 capable 100GbE and 100Gbps EDR InfiniBand adapter is in a class by itself

www.servethehome.com

Hello MXNerd,

The article was definitely helpful; thank you for that.

Regards,
Nelson