Work-Gaming workstation GPU/OS virtualization

[Verifguru]

I've just finished my high-end home workstation hardware build, and now its time to install the SSD with software.
My use-case of this workstation is two fold. Use linux server for my small team (max 4 users) development work in machine
learning and parallel algorithms, remotely, as well as let my kids play their high-end graphics games.
This means, apart from OS level virtualization, I'll be needing GPU virtualization as well.

Here are my build details:
a) CPU: Ryzen 3900x (12 Core)
b) GPU1: NVidia RTX 2080 Ti
c) GPU2: NVidia GTX 1050 Ti
d) Motherboard: x570 (Asus Crosshair VIII Hero Wifi)
e) RAM: 32GB G.Skill (3600 MHz) DDR4,
f) SSD: Sabrient Rocket 1 TB NVME

I have the following requirements, but I'm no expert in this area, so I seek your help:

1] Use a linux host (Ubuntu ?)
2] One VM (VM1) running Ubuntu LTS server, for my team to remotely connect to, where most of development work can be done
3] One VM (VM2) where Windows 10 Pro Workstation edition will serve as gaming system
4] GPU sharing/switching between VM's using containers concept i believe(or anything else), without any restarts, to achieve:
a) Script based switching of full GPU1 to either VM1 or VM2 when full performance of GPU is desired by either
b) Script based sharing of GPU1 between VM1/VM2 when full performance of GPU1 is not really desired by either
c) GPU2 to be used by host Ubuntu primarily, and which can be shared with VM1 when much performance is not required

Please provide suggestions in terms of Virtualization (KVM ? But performance wise slower I’ve read, compared to ESXI) host OS, GPU virtualization technique, that can provide maximum performance with least latency possible

 

[Kaido]

unRAID + LimeTech:

LimeTech – Virtualization Host

lime-technology.com

 

[VirtualLarry]

4] GPU sharing/switching between VM's using containers concept i believe(or anything else), without any restarts, to achieve:
a) Script based switching of full GPU1 to either VM1 or VM2 when full performance of GPU is desired by either
b) Script based sharing of GPU1 between VM1/VM2 when full performance of GPU1 is not really desired by either
c) GPU2 to be used by host Ubuntu primarily, and which can be shared with VM1 when much performance is not required

I don't know if this is possible that easily with a consumer-edition GPU. The GPU makers (AMD and NVidia), make specialized "server" GPUs that support virtualization and GPU sharing. (Think cloud clusters.) You might have to pay several thousand for one of those.

 

[_Rick_]

I don't know if this is possible that easily with a consumer-edition GPU. The GPU makers (AMD and NVidia), make specialized "server" GPUs that support virtualization and GPU sharing. (Think cloud clusters.) You might have to pay several thousand for one of those.

Yes, you can get 4 non-Tesla Tu104's for the price of one Tu104 which allows you to assign vGPU profiles - more or less statically.
If you need the video outputs and have the space, it shouldn't be an issue. If you're space-limited and can do without the extra ports, or use remote thin clients, then the Teslas come into play.
We run some 200 desktops off of just 20 or so servers - there the Teslas are worth it, even after the Grid licensing costs.
Since you already have GPU, I'm afraid you'll have to exclusively assign them to the VMs.
If you want to use the 2080Ti for machine learning, then I'd recommend running a service (maybe just a Jenkins agent, maybe a tensorflow executor) on that machine, and calling it over the network, from the Ubuntu VM.
In general though, for machine learning tasks, I'd rather spend a few hundred dollars on Amazon/Azure/Google/whatever cloud instances.

I would also add a ton more RAM to that machine...

 

[Kaido]

Yes, you can get 4 non-Tesla Tu104's for the price of one Tu104 which allows you to assign vGPU profiles - more or less statically.
If you need the video outputs and have the space, it shouldn't be an issue. If you're space-limited and can do without the extra ports, or use remote thin clients, then the Teslas come into play.
We run some 200 desktops off of just 20 or so servers - there the Teslas are worth it, even after the Grid licensing costs.
Since you already have GPU, I'm afraid you'll have to exclusively assign them to the VMs.
If you want to use the 2080Ti for machine learning, then I'd recommend running a service (maybe just a Jenkins agent, maybe a tensorflow executor) on that machine, and calling it over the network, from the Ubuntu VM.
In general though, for machine learning tasks, I'd rather spend a few hundred dollars on Amazon/Azure/Google/whatever cloud instances.

I would also add a ton more RAM to that machine...

Nice! I'd love to hear more! I haven't done anything with vGPU's for a couple years & just picked up a few GRID/Tesla cards off eBay to goof around with for my work lab. I went through VMware, Citrix, and Hyper-V & settled on Hyper-V. What host OS are you using?

 

[_Rick_]

Nice! I'd love to hear more! I haven't done anything with vGPU's for a couple years & just picked up a few GRID/Tesla cards off eBay to goof around with for my work lab. I went through VMware, Citrix, and Hyper-V & settled on Hyper-V. What host OS are you using?

For the desktops, it's Citrix's Hypervisor. Aren't they Xen based?
Ironically, we use ESXi for our server VMs, so there's no one hypervisor to rule them all.
Actually, since Docker for Windows runs on Hyper-V, we theoretically use that as well, on developer workstations

 

[Kaido]

For the desktops, it's Citrix's Hypervisor. Aren't they Xen based?
Ironically, we use ESXi for our server VMs, so there's no one hypervisor to rule them all.
Actually, since Docker for Windows runs on Hyper-V, we theoretically use that as well, on developer workstations

I was a VMware guy for the longest time. Didn't much care for Citrix outside of HDX. One guy I consult with got me onboard the Hyper-V train a year or two ago...I was very resistant, but ended up really liking it from an implementation & usability POV. I'm just getting back into the vGPU game & it's kind of a mess on Hyper-V, as far as standalone VPC's & Terminal Servers go throughout the various host versions of the OS, compatibility, and evening naming systems for the vGPU implementation. And a lot of the GRID/Tesla stuff is locked up inside of NVIDIA, so I'm debating how deep down the rabbit hole I want to go, at least outside of RemoteFX etc. (which under 2019 is only accessible via Powershell now afaik).

But I do a lot of weird little projects for companies that could sometimes really benefit from a centrally-connected GPU-enhanced hosting system. I've had reasonable success with stuff like HP's RGS software, which works surprisingly nearly as well as PCoIP hardware cards & whatnot. And I think things will only get better as we dive into 5G cellular & hosting services like Shadow Tech, Liquid Sky, Strata, Paperspace, and Geforce Now. Programmers can be remarkably clever with enough funding involved & a solid team to work with, so I expect that they'll figure things out as smoothly as how Netflix works these days...4K streaming with nary a hiccup over the past several years.

For me, as always, it's not just about the capabilities, but about access to the information...it's so dang confusing to get a simple one-paragraph high-level explanation of what's available & how to put it into action. My own personal setup guides are a cinch on Hyper-V & were like that before when I was doing VMware environments...here's the idea (virtualization with say GPU acceleration), here's the elements (terminal servers or standalone desktops or remote apps, with or without GPU acceleration on a shared or dedicated card, or on Hyper-V, with RemoteFX or DDA or GPU-D or whatever mumbo jumbo they're calling it these days), and here's the checklist to implement it. I could teach a six year old how to do it with that approach, in like 10 minutes. But, things are always evolving, so it's a research project as always haha!

I think it would be cool to do in-house. Like setup an unRAID system with Lime Tech & some video cards, then stick an Intel Compute stick on each monitor & TV and stream a graphically-accelerated desktop as needed anywhere...HTPC, CAD at home, CGI, video, photo editing, gaming, emulation, whatever. Especially if they could make it turnkey & just buy like a Synology lunchbox from Staples, stick some hardware in, and have magical access to everything. Nerd's pipe dream, but a fun idea!

 

[_Rick_]

With "cheap" long distance (>1.5 m ) 10Gbit interconnects, I'm surprised, there's not more of that being done. Linus Tech Tips did (one of) their setup with long hdmi and usb cables. But KVMoIP should scale up, with just a little bit more front-end intelligence (technically any smart tv should be able to thin-client these days).
The only thing stopping me from going full 10Gbit is that I don't have a closet for an actively cooled switch. The price is really reasonable, especially compared to some custom active/optical usb bridges...

Actually, going off on a weird tangent: What if RDMA tech were to become mainstream, so you could just read data from a remote framebuffer? Talking of pipe dreams

I'm still not entirely convinced that VMs are really an upgrade at home, as usually density isn't the challenge. The opposite is usually true: cooling a high density server at home is a challenge, while maintaining three 3U chassis with just a single CPU and reasonable stats is way more bang for the buck. With decent remote management tooling, as you'd have in a datacenter these days, and boot-time optimized BIOS, as well as decent suspend-to-ram functionality, I'm pretty sure you could scale up by adding "blades" quite easily. Sure, you can't as easily resize your physical hardware, but at least a single-point-failure doesn't take your whole IT setup down.

 

[Kaido]

With "cheap" long distance (>1.5 m ) 10Gbit interconnects, I'm surprised, there's not more of that being done. Linus Tech Tips did (one of) their setup with long hdmi and usb cables. But KVMoIP should scale up, with just a little bit more front-end intelligence (technically any smart tv should be able to thin-client these days).
The only thing stopping me from going full 10Gbit is that I don't have a closet for an actively cooled switch. The price is really reasonable, especially compared to some custom active/optical usb bridges...

Actually, going off on a weird tangent: What if RDMA tech were to become mainstream, so you could just read data from a remote framebuffer? Talking of pipe dreams

I'm still not entirely convinced that VMs are really an upgrade at home, as usually density isn't the challenge. The opposite is usually true: cooling a high density server at home is a challenge, while maintaining three 3U chassis with just a single CPU and reasonable stats is way more bang for the buck. With decent remote management tooling, as you'd have in a datacenter these days, and boot-time optimized BIOS, as well as decent suspend-to-ram functionality, I'm pretty sure you could scale up by adding "blades" quite easily. Sure, you can't as easily resize your physical hardware, but at least a single-point-failure doesn't take your whole IT setup down.

Yeah, and even with that, stuff like Shadow Tech (Cloud DaaS) is getting so good that you don't really even need that stuff at home anymore. I have a $13/mo account with them & while it's not perfect, the performance is pretty dang amazing!

Shadow Cloud Computing

Cloud Gaming verwandelt deine Geräte in einen leistungsstarken High-End-PC. Wähle das Setup zum Streamen deiner Videospiele oder Software

shadow.tech

I got into wireless backhaul a few years ago, starting with Ubiquiti's Loco series, which are AWESOME!

Long distance outdoor (500+ ft) wired or wifi options

Long Story: My uncle is looking to get internet access in his home, problem is the cable company won't run a cable to the house because of distance. They will not run a line over 300ft from the road. My uncle lives about 600ft behind my grandfather's house on the other side of an open field...

forums.anandtech.com

It's $99 for a pair that has a 3-mile wireless range. The more pricey stuff does 1.4 Gbps over 8 miles:

UISP airFiber 24 GHz Bridge - Ubiquiti Store

AF-24 delivers up to 1.5+ Gbps throughput at a range of up to 13+ km.

store.ui.com

The crazy thing is how reliable this stuff is. I do a bunch of little projects (shops & barns & detached garages & man-sheds & stuff) with the Loco units & they've lasted through blizzards & years of operation flawlessly. I just replaced one that had been operation for 3 solid years because of a literal computer bug...it wasn't recycling on the PoE, so I went up on a roof to check & it was filled with spiders lol.

That, and you can get some pretty decent gaming laptops these days, so you don't really even need a desktop to do high-quality gaming anymore. The world has changed!