A few questions about PCI/E lanes and bandwidth and all that...

[ChorniyVolk]

1. If an X58/1366 motherboard only has 36 PCIE 2.0 lanes, then why do some motherboards support X16/X16/X8 for splitting between 3 cards? Onboard chip or something? If so, why don't the onboard chips support an extra X8 for the last one and get it to 2 X16 slots?

2. How many different types of PCI slots are there? PCI, PCIE 1/2.0 which has X1, X4, X8 and X16? Or is there more I missed? They do make those 4 types of slots for both PCIE 1 and 2.0 right?

3. I looked at a chart if how an X58 motherboard works, and it has the northbridge chip handling the PCIE 2.0 lanes, 36 split into whatever combos a board has, but I don't understand how the PCIE 1 lanes are handled by the southbridge chip, could someone explain?

4. Is it true that the current motherboards that use SATA 6GB/s and USB 3.0 have chips that handle those slots, and those chips take bandwidth away from the PCIE 2.0 lane? And that's why you can't do x16/x16/x8 on the x58 motherboards that have the 2?

5. A bit off topic, but I see on that same chart that the southbridge chip only supports up to 6 SATA 3GB/s ports, so how do motherboards that have more than 6 do it? A chip put in bu the manufacturer? If #4 is true, then do those extra SATA 3GB/s ports steal bandwidth from other areas as well?

Thanks.

 

[lopri]

1. Some boards may have additional PCIe switch (such as NF200) to provide extra bandwidth per slot. Think of it as, well, a switch. Or port multiplier. In theory they can help inter-GPU communication, but the aggregate bandwidth to the chipset remains the same.

2. Yes.

3. ICH10R provides total 6 lanes (Gen 2.0) on its own. It doesn't 'steal' the bandwidth from X58. It connects to X58 via DMI, though.

4. Yes, those extra controllers use PCIe lanes provided by the chipsets (either X58 or ICH10R) and it is by design. Unused lanes are kind of like wasted lanes.

5. Again, it isn't stealing. A board manufacturer can configure available PCIe lanes for whichever way it thinks makes sense, and you choose a board that provides the most suitable configuration for your usage.




As you can see, the ICH10R provides total 6 PCIe lanes, and it is there to be used. A board can be designed to provide one x4 slot, and use the other two lanes for network or auxiliary storage controllers. Some boards may provide two x1 slots, and use 4 lanes for USB 3.0, etc.

 

[ChorniyVolk]

1. So why is it usually limited to 16/16/8 instead of 16/16/16? Because the chip is just amplifying the x4 left at the end to x8, and increasing it to x16 would require a much stronger and more expensive chip?

Actually, that's how it is isn't it? You have to buy an expensive motherboard to use each slot at x16 because they put in a different chip which pushes the last lane to multiply it by 4 to get it to x16/x16/x16, and some boards split the 36 into 4 slots and then increase the ones who can't get to x16 with a chip of their own, or something along those lines right?

4. Why don't they just take the bandwidth from the x6 PCIe lanes? Maybe have one or two x1 slots and use the rest for USB 3.0 and SATA 6GB/s instead of using the PCIE 2.0 lanes? Does PCIe 1 just not provide enough bandwidth, or as much as the amount the lanes they take from PCIe 2.0 have? Why can't they then use a chip to amplify the <36 lanes to the bandwidth of 36 or more to match other boards? Or even amplify the PCIe 1 x6 lanes to provide the sufficient bandwidth? Is the problem all about how much it'll cost to do any of those?

6. So other boards can do more than 36 lanes, but these boards do less. Do they replace the chip that amplifies the bandwidth of the 36 lanes with a chip that handles USB 3.0 and SATA 6GB/s, and then not have any space/resources on the motherboard to stick the amplifying chip?

Thanks.

 

[ChorniyVolk]

1. So why is it usually limited to 16/16/8 instead of 16/16/16? Because the chip is just amplifying the x4 left at the end to x8, and increasing it to x16 would require a much stronger and more expensive chip?

Actually, that's how it is isn't it? You have to buy an expensive motherboard to use each slot at x16 because they put in a different chip which pushes the last lane to multiply it by 4 to get it to x16/x16/x16, and some boards split the 36 into 4 slots and then increase the ones who can't get to x16 with a chip of their own, or something along those lines right?

4. Why don't they just take the bandwidth from the x6 PCIe lanes? Maybe have one or two x1 slots and use the rest for USB 3.0 and SATA 6GB/s instead of using the PCIE 2.0 lanes? Does PCIe 1 just not provide enough bandwidth, or as much as the amount the lanes they take from PCIe 2.0 have? Why can't they then use a chip to amplify the <36 lanes to the bandwidth of 36 or more to match other boards? Or even amplify the PCIe 1 x6 lanes to provide the sufficient bandwidth? Is the problem all about how much it'll cost to do any of those?

6. So other boards can do more than 36 lanes, but these boards do less. Do they replace the chip that amplifies the bandwidth of the 36 lanes with a chip that handles USB 3.0 and SATA 6GB/s, and then not have any space/resources on the motherboard to stick the amplifying chip?

Thanks.

 

[lopri]

1. Where do you see x16/x16/x8 usually? There are only handful of boards with NF200, iirc. Again, if you look at the diagram above X58 provides total 36 PCIe 2.0 lanes, and they are usually configured x16/x16, x16/x8/x8, or x8/x8/x8/x8.

So there are total 42 lanes provided by X58 platform. (36 lanes from the north bridge, and 6 lanes from the south bridge) They are all Gen 2.0. I think you're confusing the first generation PCI Express (Gen 1) and the second generation. (Gen 2) Let me refer you to an old sticky.

http://forums.anandtech.com/showthread.php?t=286944&highlight=frequently+asked+questions+pci+express

Actually, that's how it is isn't it? You have to buy an expensive motherboard to use each slot at x16 because they put in a different chip which pushes the last lane to multiply it by 4 to get it to x16/x16/x16, and some boards split the 36 into 4 slots and then increase the ones who can't get to x16 with a chip of their own, or something along those lines right?

Well you do need to pay more for more bandwidth, yes, but not in the way you describe. You have total 42 lanes available on X58, and 24 lanes available on P55 (x16 from the CPU and x8 from the PCH). You have 42 lanes available on 790FX and 22 lanes available on 790X. They are all second generation PCI Express. X58 is more expensive than P55, and 790FX is more expensive than 790X.

Now, you may find the boards advertising "True x16/x16/x16" configuration - those are usually the ones that add the NF200 chips. Examples diagrams:



Or,



Now, with NF200

 

[ChorniyVolk]

Well like the EVGA boards, the E758, E760 etc.

http://www.newegg.com/Product/Produc...82E16813188039

1. Alright, so then the south bridge's lanes can't be used for video cards because the cards need a fast connection to the cpu via just through one middle-man, the north bridge, but going through the south chip then north and then cpu would be way too slow, right?

South bridge does 6 PCIe 2.0 lanes then? So would you explain how the E758 split that between it's PCIE 1.0 X1 lane, and its 2 PCI lanes?

#4 stays the same, but with this new knowledge I'll modify #6:

6. So other boards can do more than 36 lanes on the PCIE 2.0 , but these boards do less. Do they replace the NF200 chip with a chip that handles USB 3.0 and SATA 6GB/s, and then not have any space/resources on the motherboard to stick the NF200?

And if the south bridges lanes are for gen 2, why doesn't that chart say 6 PCI Express 2.0 x1 lanes, and not just 6 PCI Express x1?

 

[tcsenter]

The Southbridge lanes are PCI-E 1.x. Intel's been using the 500MB/s as an aggregate figure. e.g. P35 + ICH9 and P965 + ICH8

 

[lopri]

Well like the EVGA boards, the E758, E760 etc.

http://www.newegg.com/Product/Produc...82E16813188039

It's a misprint or mistake. The board does x16/x8/x8, not x16/x16/x8.

http://www.anandtech.com/mb/showdoc.aspx?i=3471&p=2

1. Alright, so then the south bridge's lanes can't be used for video cards because the cards need a fast connection to the cpu via just through one middle-man, the north bridge, but going through the south chip then north and then cpu would be way too slow, right?

Yes.

South bridge does 6 PCIe 2.0 lanes then? So would you explain how the E758 split that between it's PCIE 1.0 X1 lane, and its 2 PCI lanes?

The chipset supports up to 6 lanes, but it doesn't mean a board has to use them all. Said that, it is very likely the board is already using the other lanes for auxiliary gigabit controller, eSATA controller, etc.

And if the south bridges lanes are for gen 2, why doesn't that chart say 6 PCI Express 2.0 x1 lanes, and not just 6 PCI Express x1?

tcsenter is correct. The PCIe lanes, despite being Gen 2, from the south bridge runs at Gen 1 speed. It's kind of like running SATA 3.0 Gbps disks @1.5 Gbps speed.

The PCI Express lanes are version 2.0 but Intel decided to limit their speed to PCIe 1.x specs at 2.5GT/s. Why? We think it is because the DMI link continues at 1GB/s in each direction, which means a decent 6Gb/s SAS/SATA RAID card and a few upcoming 6Gb/s drives (SSDs anyone) could easily saturate the link.

http://www.anandtech.com/weblog/showpost.aspx?i=641