Hi all
I want to build a PC that can handle training and inferencing of LLMs that are in the ~ 70B parameter category (eg Llama 3.1 70b) locally. I was going to put it off until RTX 5090/blackwell comes out, but unfortunately in the last week my current Skylake-X PC is on its last legs (faulty RAM modules, random power downs, etc) so I thought I might as well build the backbones until RTX 5090 comes out. I also want to be able to play games on the computer.
Essentially I would need to maximise the amount of VRAM in my system (need about 70GB). This would mean getting two RTX 5090 and reusing my current RTX 3090. I am assuming the RTX 5090 will draw about 450W each, and my current RTX 3090 draws 400W at peak. This means I would need a motherboard that has three PCIe slots of sufficient speed, a very beefy PSU and a big case. CPU performance is somewhat important, but not critical (8 - 12 cores should be plenty, ST performance is important. I don't think I need 16 cores. ). Training over the cloud is unlikely an option due to the nature of my training data. A 'Quadro' card like the RTX6000 Ada 48GB cost AUD 15k a pop, if I can get my hands on them, and is probably not a realistic option budget wise.
The system (that I will build now) is (the parts listed here are relatively fixed) :
CPU - Ryzen 9 9900X or 9700X (depends on pricing). Intel is out because the performance of training and inferencing will likely drop substantially if it is scheduled to E-cores (yes I can reschedule them manually but it is painful to do so), not to mention the Raptor Lake debacle.
RAM - 64GB, eg: this . I may need 128GB (will see how it works out later).
Cooler: thinking of reusing my existing Noctua NH-D15S cooler if I can, otherwise I will get a similar cooler.
HDDs: I have three 3.5" HDDs, 1 2.5" SSD and 1 NVMe SSD (Samsung 960 Pro) and will be reusing them
These are the parts that I am not quite sure about and would appreciate advise:
MB: Asus X670E-E - this has the best overall performance over the PCie slots (ie no PCIe 3.0 x1 electrical slots!).
Case: Phanteks Enthoo 719 D-RGB Full Tower Case (here). Ideally I would like the Enthoo Elite but it is way out of my budget. Vertical GPU mounting is a must especially if the cards are triple slots. I may need a riser card which I will get down the track.
PSU: Seasonic Prime PX 1600 (here).
My budget for the parts above is about 3-4k AUD, excluding the RTX 5090(s) of course. Obviously the less the better. Water cooling is not necessary. Everything else not mentioned will be reused.
Would appreciate feedback about the build, but in particular the PSU and case.
TIA.
I want to build a PC that can handle training and inferencing of LLMs that are in the ~ 70B parameter category (eg Llama 3.1 70b) locally. I was going to put it off until RTX 5090/blackwell comes out, but unfortunately in the last week my current Skylake-X PC is on its last legs (faulty RAM modules, random power downs, etc) so I thought I might as well build the backbones until RTX 5090 comes out. I also want to be able to play games on the computer.
Essentially I would need to maximise the amount of VRAM in my system (need about 70GB). This would mean getting two RTX 5090 and reusing my current RTX 3090. I am assuming the RTX 5090 will draw about 450W each, and my current RTX 3090 draws 400W at peak. This means I would need a motherboard that has three PCIe slots of sufficient speed, a very beefy PSU and a big case. CPU performance is somewhat important, but not critical (8 - 12 cores should be plenty, ST performance is important. I don't think I need 16 cores. ). Training over the cloud is unlikely an option due to the nature of my training data. A 'Quadro' card like the RTX6000 Ada 48GB cost AUD 15k a pop, if I can get my hands on them, and is probably not a realistic option budget wise.
The system (that I will build now) is (the parts listed here are relatively fixed) :
CPU - Ryzen 9 9900X or 9700X (depends on pricing). Intel is out because the performance of training and inferencing will likely drop substantially if it is scheduled to E-cores (yes I can reschedule them manually but it is painful to do so), not to mention the Raptor Lake debacle.
RAM - 64GB, eg: this . I may need 128GB (will see how it works out later).
Cooler: thinking of reusing my existing Noctua NH-D15S cooler if I can, otherwise I will get a similar cooler.
HDDs: I have three 3.5" HDDs, 1 2.5" SSD and 1 NVMe SSD (Samsung 960 Pro) and will be reusing them
These are the parts that I am not quite sure about and would appreciate advise:
MB: Asus X670E-E - this has the best overall performance over the PCie slots (ie no PCIe 3.0 x1 electrical slots!).
Case: Phanteks Enthoo 719 D-RGB Full Tower Case (here). Ideally I would like the Enthoo Elite but it is way out of my budget. Vertical GPU mounting is a must especially if the cards are triple slots. I may need a riser card which I will get down the track.
PSU: Seasonic Prime PX 1600 (here).
My budget for the parts above is about 3-4k AUD, excluding the RTX 5090(s) of course. Obviously the less the better. Water cooling is not necessary. Everything else not mentioned will be reused.
Would appreciate feedback about the build, but in particular the PSU and case.
TIA.
Last edited:
Facebook
Twitter