Solved! What Are The Major Differences Between Mellanox VPI And Mellanox Ethernet NICs?

Jan 13, 2022
52
1
11
Hello,

My company wanted to purchase a NIC with dual 100 GbE ports utilizing QSFP28 that must work with Windows 11 Pro.

From what I can see, Nvidia's Mellanox ConnectX-5 cards seem to meet my company's criteria.

However, the card that I was initially planning to purchase : MCX516A-CDAT


...is completely out of stock (and won't be in stock for months).

Whereas the following card (which seems similar) : MCX556A-EDAT


...is currently in stock.

That said, what are the major differences between the two cards, as it looks like the EDAT, which supports VPI, should work with both Ethernet and Infiniband.

Whereas the CDAT, only works with Ethernet (plus uses on PCIe 3.0 x16).

It seems like the EDAT offers more functionality/throughput, for the same price, but I'm worried that I'm missing something (as Nvidia would never offer more functionality for the same price ;) ).

Thank you,
Nelson
 

Fallen Kell

Diamond Member
Oct 9, 1999
5,771
295
126
Well, first off, both cards are PCIe 4.0 16x slot cards. They are that so you can actually get 2 ports that can be 100gig. A PCIe 3.0 would not be able to support two ports as it tops out at 128gigabit in either direction (i.e. 128gigabit to the card and 128gigabit from the card).

You already mentioned the other difference, the VPI card can be configured to run as infiniband (in fact, it is most likely configured in that mode out of the box and you need to use their firmware utilities to have it instead change to ethernet/network mode so that you canuse it under Windows as a network card). The VPI cards also tend to have a few other features built into the silicone such as different hardware offload engine functionality, but you need to do a deep dive into Nvidia’s documentation and feature sets for each model to see those things (Nvidia typically has a pdf overview of the card generation listing all the possible features in that generation of cards and then at the bottom has a table that matches the various models of cards in the generation with the feature set).
 

Fallen Kell

Diamond Member
Oct 9, 1999
5,771
295
126
Well, first off, both cards are PCIe 4.0 16x slot cards. They are that so you can actually get 2 ports that can be 100gig. A PCIe 3.0 would not be able to support two ports as it tops out at 128gigabit in either direction (i.e. 128gigabit to the card and 128gigabit from the card).

You already mentioned the other difference, the VPI card can be configured to run as infiniband (in fact, it is most likely configured in that mode out of the box and you need to use their firmware utilities to have it instead change to ethernet/network mode so that you canuse it under Windows as a network card). The VPI cards also tend to have a few other features built into the silicone such as different hardware offload engine functionality, but you need to do a deep dive into Nvidia’s documentation and feature sets for each model to see those things (Nvidia typically has a pdf overview of the card generation listing all the possible features in that generation of cards and then at the bottom has a table that matches the various models of cards in the generation with the feature set).
 
Jan 13, 2022
52
1
11
Well, first off, both cards are PCIe 4.0 16x slot cards. They are that so you can actually get 2 ports that can be 100gig. A PCIe 3.0 would not be able to support two ports as it tops out at 128gigabit in either direction (i.e. 128gigabit to the card and 128gigabit from the card).

You already mentioned the other difference, the VPI card can be configured to run as infiniband (in fact, it is most likely configured in that mode out of the box and you need to use their firmware utilities to have it instead change to ethernet/network mode so that you canuse it under Windows as a network card). The VPI cards also tend to have a few other features built into the silicone such as different hardware offload engine functionality, but you need to do a deep dive into Nvidia’s documentation and feature sets for each model to see those things (Nvidia typically has a pdf overview of the card generation listing all the possible features in that generation of cards and then at the bottom has a table that matches the various models of cards in the generation with the feature set).
Hello Fallen Kell,

Thank you very much for your input.

A few more questions for you (to confirm some things):

1. Can I assume the VPI card supports both Infiniband and Ethernet?

2. If so, can I assume it's preferable to go with the VPI card (to use Ethernet now and then in the future just reconfigure the card for Infiniband)?

In other words, what would be the point of purchasing the Ethernet card (as it's the same price as the VPI Infiniband card), if the VPI card can do both Ethernet and Infiniband?

Thank you,
Nelson
 

ASK THE COMMUNITY