confused about crossfire and PCIe modes

[Special K]

I am reading about ATI's crossfire feature right now and there is one spec that I can't seem to find a clear definition for.

I was reading the specs for Gigabyte's GA-EP45-UD3P motherboard (link) and came across the following:

1 x PCI Express x16 slot, running at x16
1 x PCI Express x16 slot, running at x8
(The PCIEx16 and PCIEx8 slots support ATI CrossFireX technology and conform to PCI Express 2.0 standard.)

Also, in anandtech's review of the motherboard (link), it says:

Expansion Slots

2 - PCIe 2.0 x16 (x8 dual CrossFire)

Here is what I don't understand:

1. What is the difference between x8 and x16? What exactly do those numbers refer to?
2. Why is the second PCI express x 16 slot running at x8, and why does it run at x8 if it's listed as an x16 slot?
3. Every motherboard I've seen so far supports crossfire at x8. Do any support it at x16?
4. How big of a performance difference is there between crossfire at these different link speeds?

I have searched for these answers but can't find a clear explanation. Any advice would be appreciated.

 

[chizow]

Originally posted by: Special K
1. What is the difference between x8 and x16? What exactly do those numbers refer to?

That's the difference in electrical lanes, meaning 8 lanes vs 16 lanes of electrical signaling. Number of lanes ultimately translates into total available bandwidth. Each link/electrical lane is capable of 0.25 GB/s for PCIE 1.0 for a total of 4 GB/s bandwidth for x16 electrical lanes. This bandwidth is doubled for PCIE 2.0 to 0.5 GB/s for a total of 8 GB/s bandwidth for x16 electrical lanes.

2. Why is the second PCI express x 16 slot running at x8, and why does it run at x8 if it's listed as an x16 slot?

The x8 link bandwidth may be a chipset/pcie controller limitation. The chipsets may only allow for a total of 16 electrical lanes, so it splits them equally at x8 to allow for additional functionality, like SLI or CF. The boards will support the physical x16 form factors because those sizes and form factors are fixed to hardware, even though they will only run x8 electrical lanes in certain boards.

3. Every motherboard I've seen so far supports crossfire at x8. Do any support it at x16?

You're probably looking at P45 or 750i boards. Higher end boards with more expensive chipsets or additional PCIE controllers support multiple full bandwidth x16 lanes. X58 supports both CF and SLI at 2x16, X38 and X48 support 2x16 for CF only, and 780i/790i support 2x16 for SLI only. Many of those support additional PCIE slots with x8 or x16 depending on chipset or additional controllers. All of those chipsets are PCIE 2.0, which has double the bandwidth as PCI 1.0/1.1.

4. How big of a performance difference is there between crossfire at these different link speeds?

Its been shown PCIE 2.0 at x8 results in good results with CF, although there is some small benefit from 2 full x16 lanes. As faster individual cards are introduced, the x8 at PCIE 2.0 will become increasingly insufficient.

Here's a pretty good comparison done at THG. Basically it shows 2x8 @ PCIE 2.0 is good enough for Crossfire. Avoid anything below x16 @ PCIE 1.1 (equivalent bandwidth to PCIE 2.0 x8) or you'll start seeing performance degradation.
PCIE Comparison at THG

I have searched for these answers but can't find a clear explanation. Any advice would be appreciated.

Wiki guide on PCIE
That does a pretty good job of giving more detail if you're interested.

 

[nitromullet]

It's too bad that AT forums doesn't allow us to rate posts.... Very good and thorough answer, Chizow.

 

[Denithor]

If you take a good look at the bench results in that Tom's report you'll see that crossfire scaled the worst on the 965P and P35 chipset boards when the highest resolution + AA were used. That's because those two chipsets only provide an x4 bandwidth to the second GPU and that's at the PCIe 1.1 bandwidth. So when a lot of data has to pass through the pipeline, it chokes.

 

[geno]

As someone who recently strived to find a true x16 + x16 board, be forewarned. Many spec sheets and ads will list "2x16 PCI-E slots", this does not mean they will operate at 2x16! It doesn't mean they won't either. What I'm saying is, be sure to look further into any motherboard which touts this feature and be sure it supports 2x16 operation, not just the 2 physical x16 slots.

It can be confusing. My motherboard's manual says it does x16 + x8, but I know for a fact this isn't the case and it does do 2x16 in SLI.

My point is, do you homework on whatever you decide on to be sure it supports the modes you're looking for. Find feedback from current users who can confirm what modes really work and which don't, since some motherboard suppliers just aren't 100% clear on the supported features.

 

[Denithor]

Generally if you want true dual x16 slots you've gotta cough up the bucks for one of the "X" chipset boards on the crossfire side (975X, X38, X48, X58) or find a board that explicitly states dual x16 full-speed slots in SLI mode.

Usually the specifications list the modes the slots run in when multiple cards are inserted. When in question - ask the manufacturer before buying.

 

[Special K]

Originally posted by: chizow

Originally posted by: Special K
1. What is the difference between x8 and x16? What exactly do those numbers refer to?

That's the difference in electrical lanes, meaning 8 lanes vs 16 lanes of electrical signaling. Number of lanes ultimately translates into total available bandwidth. Each link/electrical lane is capable of 0.25 GB/s for PCIE 1.0 for a total of 4 GB/s bandwidth for x16 electrical lanes. This bandwidth is doubled for PCIE 2.0 to 0.5 GB/s for a total of 8 GB/s bandwidth for x16 electrical lanes.

2. Why is the second PCI express x 16 slot running at x8, and why does it run at x8 if it's listed as an x16 slot?

The x8 link bandwidth may be a chipset/pcie controller limitation. The chipsets may only allow for a total of 16 electrical lanes, so it splits them equally at x8 to allow for additional functionality, like SLI or CF. The boards will support the physical x16 form factors because those sizes and form factors are fixed to hardware, even though they will only run x8 electrical lanes in certain boards.

3. Every motherboard I've seen so far supports crossfire at x8. Do any support it at x16?

You're probably looking at P45 or 750i boards. Higher end boards with more expensive chipsets or additional PCIE controllers support multiple full bandwidth x16 lanes. X58 supports both CF and SLI at 2x16, X38 and X48 support 2x16 for CF only, and 780i/790i support 2x16 for SLI only. Many of those support additional PCIE slots with x8 or x16 depending on chipset or additional controllers. All of those chipsets are PCIE 2.0, which has double the bandwidth as PCI 1.0/1.1.

4. How big of a performance difference is there between crossfire at these different link speeds?

Its been shown PCIE 2.0 at x8 results in good results with CF, although there is some small benefit from 2 full x16 lanes. As faster individual cards are introduced, the x8 at PCIE 2.0 will become increasingly insufficient.

Here's a pretty good comparison done at THG. Basically it shows 2x8 @ PCIE 2.0 is good enough for Crossfire. Avoid anything below x16 @ PCIE 1.1 (equivalent bandwidth to PCIE 2.0 x8) or you'll start seeing performance degradation.
PCIE Comparison at THG

I have searched for these answers but can't find a clear explanation. Any advice would be appreciated.

Wiki guide on PCIE
That does a pretty good job of giving more detail if you're interested.

Thanks chizow, that is some very good information that answers my questions. Based on that THG article, it seems the P45 performs similar to the X48 in Crossfire mode, making the X38/X48 boards not worth the premium over a P45 board. I'm trying to spec out a mid-range gaming system and was concerned about the potential upgradeability of different platforms. I would likely start with a single 4870.

Also, it seems that a single 4870 X2 is equivalent in performance to 2 4870's in Crossfire. I assume that a single 4870 X2 card would realize the full speed of the PCIe 2.0 slot, right?

I read other articles on THG and it seems they are now even recommending the core i7 for mid-range systems. I realize that by going with the P45 I would be limiting my CPU upgrade options, but going with the core i7 seems like it would add an extra $200-$300 to the total cost.

THG spec'ed out a mid-range core i7 system for ~$1200 awhile back:

link

I'm not sure if the core i7 really shows much of an improvement over the core2 duo/quads when it comes to gaming, however.

 

[chizow]

Originally posted by: Special K
Thanks chizow, that is some very good information that answers my questions. Based on that THG article, it seems the P45 performs similar to the X48 in Crossfire mode, making the X38/X48 boards not worth the premium over a P45 board. I'm trying to spec out a mid-range gaming system and was concerned about the potential upgradeability of different platforms. I would likely start with a single 4870.

Also, it seems that a single 4870 X2 is equivalent in performance to 2 4870's in Crossfire. I assume that a single 4870 X2 card would realize the full speed of the PCIe 2.0 slot, right?

I read other articles on THG and it seems they are now even recommending the core i7 for mid-range systems. I realize that by going with the P45 I would be limiting my CPU upgrade options, but going with the core i7 seems like it would add an extra $200-$300 to the total cost.

THG spec'ed out a mid-range core i7 system for ~$1200 awhile back:

link

I'm not sure if the core i7 really shows much of an improvement over the core2 duo/quads when it comes to gaming, however.

Np, glad it helped. The 4870X2 would get the same bandwidth whether in an X48, P45, X58 etc, that's correct.

As for which platform to choose, multi-GPU solutions based on current high-end cards (GTX 260 or 4870 1GB) do absolutely require the fastest CPU solutions in order to see significant scaling gains from CF or SLI over a fast single GPU solution. There are significant benefits from Core i7 on a clock-for-clock basis over Core 2 or Phenom II with multi-GPU, so if you're definitely planning on going multi-GPU in the future it may be worth the increased price right now.

The main difference in price for i7 is the difference in DDR3 and mobo price. The difference for DDR3 has come down a good bit though and the difference between an i7 920 and 9650 actually favors the i7 now.

Here's a price build going off what they had in the THG guide:

i7 920 $250 (Microcenter price including tax and shipping)
6GB DDR3 $140
Mobo $250

A comparable P45 build might be
Q9650 $330
4GB DDR2 $40
Mobo $150

You can see the difference in price isn't nearly as extreme as it used to be now that DDR3 has come down a bit. Again, if you're thinking of going CF for sure in the future I'd consider the faster platform. Not only will it serve you better now, it'll be more upgradeable going forward also. I'm personally waiting for P55/Lynnfield as I multi-GPU isn't a priority for me, but X58 also has the additional benefit of supporting both CF and SLI.

 

[Denithor]

It will cost you $100-200 more for a top-notch i7 build versus a high end C2Q build.

It's definitely worth it if you're build a multiGPU setup. That page focuses on TripleSLI GTX 280 but the other pages give a lot more detail. That extreme setup simply is the best to show you the full advantage of an i7 965 (3.2GHz) versus a QX9770 (3.2GHz) and an E8400 (3GHz). Significant, to say the least.

So if you're planning a multiGPU setup from the start, you really shouldn't even consider anything but i7 for your build.


EDIT: And one further note - you can easily find a $200 X58 motherboard and $100 6GB DDR3-1333 memory kit with a little scouring of the internet. Making the pricing even more competitive. Just make sure you get a good aftermarket cooler as they make a huge difference in thermal levels for your CPU.

 

[Special K]

What would either of you recommend if I just wanted a single graphics card in the ~$250 range? Should I go with a fast dual-core CPU, a quad core, or the i7? This would be for a mid-range gaming system using a 24" monitor. Basically I'm trying to get the best bang for the buck here, without severely limiting the future potential of the system.

 

[chizow]

Originally posted by: Special K
What would either of you recommend if I just wanted a single graphics card in the ~$250 range? Should I go with a fast dual-core CPU, a quad core, or the i7? This would be for a mid-range gaming system using a 24" monitor. Basically I'm trying to get the best bang for the buck here, without severely limiting the future potential of the system.

For CPU choice, the first question is whether or not you plan to overclock. If you do, you'll want to put aside some money for a good cooler like Denithor said. If you don't plan to overclock, a fast Dual Core will give you the best bang for the buck as it'll typically be clocked similarly at about half the price. If you're willing to overclock though, a Q9650 or i7 will always be better than a Dual.

There's certainly a few games that will benefit from a Quad now (GTA4, FC2, UE3.0 games), and you can see here that even a fast GPU in the $250 range will benefit and scale with faster CPUs in modern games. This trend should continue as both Nvidia and ATI have begun optimizing their drivers for multithreaded performance and more PC games get ported from consoles.

GTA4 - 13 CPU round-up

COD4 + GRiD - Intel CPU Clock for Clock Comparison @ 2GHz

COD5 - 12 Intel and AMD CPUs

Far Cry 2 - various speeds

Left 4 Dead - various speeds

If you're concerned at all with future potential of the system, you'll want to go i7, as LGA775 and DDR2 is basically a dead platform. There's very little left in the way of upgrades and I doubt there's going to be any new CPUs that will offer performance that isn't already achievable through overeclocking. Personally, if I were building new right now and didn't have any ties to older hardware, I'd definitely go with the i7/DDR3 build if it were only $100-200 difference in price.