Tesla system - PCI-e confusion - nforce 200 required?

[pcunite]

I will be building a Tesla Personal Supercomputer system and wish to populate 4 PCI-e slots with GeForce 280 or Tesla boards. I will not be needing SLI support.

Requirements:
* Intel X58 + Xeon 3500 for ECC RAM.
* 4GB/sec transfer speed on each PCI-e slot (is this x16 Gen 2?)
* 24GB ECC RAM support

Based on my requirements it seems that nForce 200 is what provides 4 PCI-e x16 Gen 2 slots?

Is this true?:
http://www.nvnews.net/articles...58_chipset/index.shtml

 

[jaggerwild]

Hey PC,
YOu will want to do a check on each as some are reporting they are not getting TRI-SLI working on certain boards.
The EVGA Classified has 4 PCIexpress slots this is a top of the line mother board @ a premium proce as well, it does have life time warranty. There are others as well, again do the home work though......
EVGA CLASSIFIED

:beer: Regards!

 

[pcunite]

Well I don't need SLI so it is okay. PCI-e x16 x8 electrical Gen 2 is 4GB/sec so I am good there.

 

[Blazer7]

If you are serious about building a Tesla supercomputer then you've already decided to spend big on graphics boards. If that's the case it makes no sense to go cheap on the board.

I think that this is what you are looking for -> ASUS P6T7 WS SuperComputer

Workstation class X58 motherboard with 2 x nF200, support for CUDA parallel programing, XEON cpus, 24GB ECC mems & 7 PCI-E v2.0 x16 (3 x 8 or 16, 3 x 8, 1 x 16).

 

[sgrinavi]

Originally posted by: Blazer7
If you are serious about building a Tesla supercomputer then you've already decided to spend big on graphics boards. If that's the case it makes no sense to go cheap on the board.

I think that this is what you are looking for -> ASUS P6T7 WS SuperComputer

Workstation class X58 motherboard with 2 x nF200, support for CUDA parallel programing, XEON cpus, 24GB ECC mems & 7 PCI-E v2.0 x16 (3 x 8 or 16, 3 x 8, 1 x 16).

That's the one I was going to suggest too. I've been trying to come up with some sort of justificaton to switch to that one myself LOL.

BUT, I don't know if it supports all those two slot cards. Have you considered just using one of Nvida's suggested builds?

 

[pcunite]

The Asus uses the nForce 200 which I do not feel is reliable or necessary. nVidia recommends (for Intel based system) the AsRock X58 SuperComputer board which is all that seems to be out there without nForce and four PCI-e slots. But one of the slots will revert to x8 when three are populated (maybe two revert, not sure yet). But x8 is still 4GB/sec which is plenty fast.

 

[aigomorla]

P6T6 WS revolution

eVGA Classified

and too bad if you dont need SLI, its manditory feature on X58 board.

 

[Blazer7]

Originally posted by: pcunite
The Asus uses the nForce 200 which I do not feel is reliable or necessary. nVidia recommends (for Intel based system) the AsRock X58 SuperComputer board which is all that seems to be out there without nForce and four PCI-e slots. But one of the slots will revert to x8 when three are populated (maybe two revert, not sure yet). But x8 is still 4GB/sec which is plenty fast.

The X58 offers 36 PCI-E lines. That's 2 x16 + 1 x4 or 4 x8 max so no, your assumption about the AsRock reverting the last slot to x8 when the other 3 are populated is not correct. According to AsRock the blue slots are x16/x8 capable while the orange slots are only x8. This means that the board will function all 4 slots @x8 when all 4 are populated. If I'm right the board could also function @ 1 x16 + 2 x8 with the 4th slot (2nd orange) disabled.

The nF200 is not necessary if you are not going after top notch GPUs or if you don't care about upgrading your GPUs in the future. That said I fail to see a ?supercomputer? that relies on 4 x8 max especially when there are single GPU cards that stress the PCI-E v1.1 x16/PCI-E v2.0 x8 to its limits and dual GPU cards that even take a performance hit. Have in mind that nV's G300 is coming before the end of the year and according to everyone out there that's nV's first serious attempt at GPGPU/CUDA computing. So if you're serious about building a Tesla supercomputer it doesn't make much sense to avoid nF200 boards. As for your comment regarding the reliability of the nF200 I don't get it. The nF200 is not perfect (If memory serves it brings some unwanted latencies into play) but it has never been accused of being unreliable. Furthermore have in mind that AsRock is owned by ASUS and it's more or less their cheap line of boards. ASUS is using the nF200 in their high-end workstation mobos. That speaks of the reliability of the chip.

*** edit ***

AFAIK the AsRock does not support XEON cpus or ECC mems.

 

[pcunite]

Originally posted by: Blazer7

I fail to see a ?supercomputer? that relies on 4 x8 max

We won't be using even 4GB/sec really I just wanted that as a minimum. We are interesting in the computing not memory bandwidth at this point. We could get away with using the Geforce GTX 280.

As for your comment regarding the reliability of the nF200 I don't get it. The nF200 is not perfect (If memory serves it brings some unwanted latencies into play) but it has never been accused of being unreliable.

Frankly it is just hard to know what is quality or crap anymore. Without stringent metrics it all based on people's perceptions half the time. We have one too!

AFAIK the AsRock does not support XEON cpus or ECC mems.

Yes, I beleive your right on this board. Hmmm

 

[Blazer7]

Originally posted by: pcunite

We won't be using even 4GB/sec really I just wanted that as a minimum. We are interesting in the computing not memory bandwidth at this point. We could get away with using the Geforce GTX 280.

With supercomputers it is always good to plan ahead. You'll have no problem with the 280/285 and even with the 295 at this point but is 4GB/sec going to be sufficient for next-gen GPUs? G300/GT300 are going to be far more powerful and that's your logical upgrade path correct?

Frankly it is just hard to know what is quality or crap anymore. Without stringent metrics it all based on people's perceptions half the time. We have one too!

I know what you mean. If you were looking for SLI/CF I would have proposed a board without the nF200 but in your case the nF200 actually makes sense, at least from my point of view.