How is this possible - 8 PCIe x16 slots in a 2CPU mobo?

[petreza]

1.

I see that both Supermicro and Tyan have 2CPU motherboards with 8 PCIe x16 slots (plus a few smaller ones). How is this possible when the E5-2600 v2 and v3, respectively, have 40 lanes per CPU for 80 total?

The Supermicro motherboard comes in two pieces and is X9 version for the E2600v2 CPUs:
X9DRG-OTF-CPU

The Tyan motherboard is one piece and supports the newer E5-2600v3 CPUs:
(Google Translate from Japanese)
S7079
original link:
http://dualsocketworld.blog134.fc2.com/blog-entry-412.html

Obviously that is made possible by the 4 chips under the 4 heatsinks which are easily visible in the Tyan picture and in the manual of the Supermicro. But what are those? How do they work? When chips like that are used to make a single x16 to dual x16, can you further spit one of the new x16s to 2 x8 or some other configuration?
(PS. I found out that the chips in the Supermicro are PEX 8747 - some good info in the pdf maunal.


2.

Is there a place where I can get the motherboards only (plus the power supplies)?

Both motherboards are only available as barebone systems:
Supermicro X9DRG-OTF-CPU - SYS-4027GR-TRT - aprox. $5000.00
Tyan S7079 - FT77C-B7079 - aprox. $4000.00

That is way out of my budget but knowing how much a rack case like that costs it might be significantly cheaper to get the motherboard separately and build my own case.

Why would you need a motherboard like that, you may ask? - One big system for a home virtualization lab (and other uses) that will have a lot of PCIe cards (video, network, storage, USB) being passed-through to the various VMs. I understand that I can get 2 X10DRG-Q - $500 each - to achieve similar capacity but then I have to get 4 CPUs.

Thank you!

 

[radeson]

They have PLX chips expanding the overall PCI lanes like this 1 CPU workstation motherboard here which runs 16x/16x/16x/16x:

http://www.newegg.com/Product/Produc...-263-_-Product


If they dont have PLX chips then the max lanes is determined by the CPU. Like most single CPU boards run 16x/8x/8x/8x, or 8x/8x/8x/8x, or for 28 lane CPU's 16x/8, or 8x/8x/8x,

Without PLX chips more lanes is impossible

 

[Wall Street]

Also, socket 2011 chips have many more PCIe lanes than socket 1150 chips.

 

[zir_blazer]

1.

I see that both Supermicro and Tyan have 2CPU motherboards with 8 PCIe x16 slots (plus a few smaller ones). How is this possible when the E5-2600 v2 and v3, respectively, have 40 lanes per CPU for 80 total?

The Supermicro motherboard comes in two pieces and is X9 version for the E2600v2 CPUs:
X9DRG-OTF-CPU

Check the Manual pages 15 and 16, you have a diagram there that explains what is connected where. They rely on FOUR PLX PEX8747 on the expansion card. The Processors do a 16x/16x/8x arrangement, and the two pairs of 16x are connected to each PEX8747.
According to marketing they're PCIe 16x slots, however, you still have 16x from the Processor to the PLX chip, so you can't really use 32 Lanes worth of Bandwidth from both cards to the CPU. I recall some Threads about these chips and I don't recall any conclusive evidence that they are actually that useful. The only good thing is that at least for your purpose, you won't have issues since these PLX chips seems to not annoy the passthrough capabilities in any way, there was an older chip which did the same thing of "adding" PCIe Lanes, the nVidia nForce 200, which was a pain to work with.

 

[petreza]

Thanks!

Anyone have an idea where these motherboards are sold separately (not in barebone) - especially interested in the Tyan as it is newer generation. Google does no help. Or should I contact Tyan directly?

 

[piasabird]

Some motherboard manufacturers beefed up their motherboards for bit coin mining. They were using the video cards to do bit coin mining and they wanted the maximum number of video cards possible.