PCIe 3.0 4x x16 Motherboards

[Kepakiano]

Hi there,

I'm building a system which should be able to render to twelve monitors and serve as a CUDA machine. I would like to put nVidia cards into the system (the system runs on linux and the AMD drivers are... not that good), so - since nVidia limited the output of cards running on linux in the drivers to three displays - I need four of them.
Now I'm looking for motherboards with four fully connected PCIe 3.0 slots. I found the ASRock X79 Extreme11 (http://www.anandtech.com/show/6228/...iew-pcie-30-x16x16x16x16-and-lsi-8way-sassata) and the Asus P9X79-E WS (http://www.anandtech.com/show/7613/asus-p9x79e-ws-review/). The reviews on both boards are rather good, but people (e.g. on newegg) say that they are unstable. Does anyone know if this is real issue or something that can be solved just by updating the BIOS?

Concerning the two PLX chips: Do they in any way reduce CUDA performance? Will I be able to address the combined RAM of CPU and GPU using CUDA's unified memory?

Putting four GTX 780s into the system will produce a lot of heat. People building coin mining rigs use PCIe risers to fit more cards into their systems. Would that be an option for me too or do these risers somehow hurt the PCIe standard or even the stability of the connection?

Thank you for your time.

 

[VirtualLarry]

Concerning the two PLX chips: Do they in any way reduce CUDA performance? Will I be able to address the combined RAM of CPU and GPU using CUDA's unified memory?

There are no products on the market yet that support CUDA unified memory.

Putting four GTX 780s into the system will produce a lot of heat. People building coin mining rigs use PCIe risers to fit more cards into their systems. Would that be an option for me too or do these risers somehow hurt the PCIe standard or even the stability of the connection?

They will only hurt your performance if you use PCI-E x1 to x16 risers.

 

[wand3r3r]

You can use 16x to 16x or 8x to 8x risers which shouldn't lose any speed.

1x would certainly lose bandwidth.

 

[mfenn]

There are no products on the market yet that support CUDA unified memory.

Ever since CUDA 3.0 IIRC, you can have a unified virtual memory space between the CPU and GPUs. It's not true unified memory in the sense of being DMA bus masters, the CPU is still helping you copy, but it looks the same to a programmer.

OP, tell us a little more about your application. Do the GPUs communicate with each other or is it more of a master-worker type of paradigm with the CPU handing out work? What is are the sizes of the inputs and outputs?

 

[VirtualLarry]

Ever since CUDA 3.0 IIRC, you can have a unified virtual memory space between the CPU and GPUs. It's not true unified memory in the sense of being DMA bus masters, the CPU is still helping you copy, but it looks the same to a programmer.

OP, tell us a little more about your application. Do the GPUs communicate with each other or is it more of a master-worker type of paradigm with the CPU handing out work? What is are the sizes of the inputs and outputs?

I thought I had read that GPU-CPU unified memory would be here with Maxwell. If the CPU is simply copying the RAM back and forth behind the scenes, then it's not truely unified yet, is it
?

I know that AMD's HSA unified memory in Kaveri, doesn't require separate memory copies, AFAIK.

 

[mfenn]

I thought I had read that GPU-CPU unified memory would be here with Maxwell. If the CPU is simply copying the RAM back and forth behind the scenes, then it's not truely unified yet, is it
?

That's pretty much what I said? "It's not true unified memory in the sense of being DMA bus masters, the CPU is still helping you copy..."