Question Single or dual RTX 4090 setup?

[jd13386]

Hello everyone. I've recently been issued several RTX 4090s from MSI and need to start thinking about how PCs should be constructed around them for my research group.

These are intended to be remote access workstations, so no peripherals or monitors are required, and they'd be running CentOS Linux 7 exclusively with no need to tweak performance settings beyond the manufacturer's defaults.

I'm after advice on whether or not I should be housing 2 of these GPUs per chassis, or if I should go for a dedicated machine per GPU instead. I'd also mention that as far as the work goes, each GPU would be handling unrelated tasks.

These devices are being built with public money, so cost-effectiveness is one of the primary concerns, but reliability and equal performance issues also come into play and can be used to justify a higher expenditure if necessary.

I'm not overly familiar with the hardware side of computing, but presumably there are additional costs to the dual GPU setup, perhaps in terms of the mainboard and PSU? I'd imagine a more expensive cooling solution may also be needed to avoid each GPU negatively impacting on the other's performance.

Any recommendations you have on the matter would be greatly appreciated.

 

[igor_kavinski]

I'm not overly familiar with the hardware side of computing, but presumably there are additional costs to the dual GPU setup, perhaps in terms of the mainboard and PSU? I'd imagine a more expensive cooling solution may also be needed to avoid each GPU negatively impacting on the other's performance.

You are right about that. Dual GPU would need better case airflow and a higher wattage PSU. The motherboard would also need to have at least two PCIe 4.0 x16 slots spaced far enough apart to install the cards without any issues. If you have the physical space available to go with a single machine for each GPU, that would be the most cost effective way.

What are your workloads like? Are they primarily GPU bound or does the power of the CPU also come into play? If you don't know, it would be prudent to get one machine with a mid tier CPU and one with a higher end one and then benchmark them with your workloads. That will tell you whether it would make sense for you to spend the extra money on the high end CPUs with more cores.

 

[Brainonska511]

If you have two machines, are you planning on setting it up as a cluster so the work gets split between the two evenly, or would people have to choose which machine to use?

I have a friend who's lab has a machine with 4 4090s - they use it for processing cryoEM data and a few other calculation intense tasks.

You would want:
-Big enough PSU to power the GPUs
-a good processor to take advantage of the GPUs and assist whatever you're doing
-good airflow case
-ideally, the 4090s coolers would be more slot like, instead of exhausting into the case, so you could fit them and run them without overheating them

 

[igor_kavinski]

OP can also go with a pre-built: https://www.titancomputers.com/Titan-A499-OCTANE-PRO-AMD-Ryzen-Threadripper-Pro-p/a499-pro.htm

That way, just gotta pay the money which should be less than $20,000 for one workstation wih 4 x 4090s.

 

[Brainonska511]

Given it is a grant paying for the equipment, it could be better to buy some sort of pre built. A lot of institutions have weird minimum spend requirements on equipment, which actually makes it easier to buy something that costs more.

 

[jd13386]

Thanks very much for your input.

To answer your questions, the workloads we have here are very much GPU bound. A typical job takes around an hour to complete on our older 1080Tis, with only about 10s of that being CPU time, so the mid-tier CPU suggestion from Igor sounds quite sensible to me. I'd only have to ensure that it's future-proofed up to 5 years.

My group is probably too small to set up a dedicated cluster to distribute the work. As we all fit in a single office, the plan at the moment is for each member to develop code on their already existing devices, then sort out distribution with other members on the whiteboard first if they intend to move from local testing to remote production with the 4090s.

I guess the pre-built option won't be possible anymore. As Brainonska notes, institutional spending rules can get a little weird. In my case, the timing of the grant announcement coupled with the financial closure periods of the universities meant that all of us were forced to either spend this entire year's allocation of grant money within a single week or return it. In a mad panic, I asked our now overworked supplier to just max out the number of 4090s with the budget, so now I have 5 expensive paperweights already sitting on my desk gathering dust. Next year will be less stressful with the accounts opening in February when I can start building around them.

 

[igor_kavinski]

so now I have 5 expensive paperweights already sitting on my desk gathering dust.

Test them one by one in some system to ensure they are working fine. It would suck to find out months later that there is something wrong with one or all of them. Run your typical workload on each for a few hours to see that they are stable. Also, be wary of the GPU connector melting issue (not sure if you got ones with old connector or new one).

Read up on the issue here: https://www.igorslab.de/en/rest-in-peace-12vhpwr-connector-welcome-12v-2x6-connector/

Other links:

NV 12VHPWR issues revisited

He gets a 4090 almost every day with a melted connector. He also says it's common for the connector to fully snap off 4xxx cards. These videos are only a few days old. Why aren't tech sites reporting these ongoing incidents? What the heck is going on here? This abomination of a connector...

forums.anandtech.com

Secret change to the sense pins starting from the GeForce RTX 4070 FE and no one noticed it | igor´sLAB

Sometimes you are blind or just don’t look closely enough. This surprises me, frankly, with the mass of reviewers, but then again, I wasn’t any better. After having examined some headers of the…

www.igorslab.de