$30k PC with 7 GPUs can play Crysis

[TechBoyJK]

https://www.youtube.com/watch?v=LXOaCkbt4lI

What a beast!

 

[moonbogg]

It runs crisis 3, but at low FPS. That rig sucks! JK rig is awesome for families with like 14 kids so they can take turns. He could probably sell that rig to like someone with a lot of kids, like Oprah or Bill Cosby, or that octo mom broad.

 

[Artorias]

That's ridiculous. 1550W+ usage at max load. Bonkers!

 

[Markbnj]

Awesome I can order my copy now.

Heh, watched the vid. What an awesome build. I love that Caselabs rig, and the GPU cooling setup... unbelievable.

 

[Carfax83]

It runs crisis 3, but at low FPS.

Low FPS? I thought the FPS was pretty good given the resolution for each monitor..

He picked a good game to showcase the rig, because Crysis 3 can use up to six threads..

 

[TheELF]

He picked a good game to showcase the rig, because Crysis 3 can use up to six threads..

Then he should have shown (the intro to) welcome to the jungle,the rest of the game is pretty much a dual threaded corridor crawler.

 

[Sonikku]

Very Back to The Future-ish.

 

[futurefields]

What a great computer right guys? Thing is sweet. Wish I was Linus!

 

[DigDog]

what?

but, are they crossfired? or does every user only access 1? also, PCIE lanes?

 

[redzo]

That's 1 GPU for each VM. It should end up about like almost 100% bare metal performance. Hypervisors are very efficient now days. It should "feel" like using 7 distinct gaming pc's, 7 different OS instances, 7 etc..

 

[DigDog]

somehow, i don't find that interesting. i would be more interested in how fast a 7x crossfire is .. not that i would ever use one.

 

[RampantAndroid]

It's interesting in the sense that you could get 7 people gaming on a single system - at 1500W, that's not bad, I think. At 30k however...that's bad.

 

[XavierMace]

To be fair, it could have been done substantially cheaper. They had 32Gb of RAM per VM. They could have cut that in half which would have allowed 16Gb DIMM's rather than 32Gb. That alone would drastically reduce the price. Ditto for the SSD's, way cheaper options available.

 

[TheELF]

Yes,but to quote the mythbusters "If it's worth doing, it's worth overdoing"

 

[Kaido]

Dang, I watched that whole video with my jaw hanging down

 

[Silverforce11]

It's interesting in the sense that you could get 7 people gaming on a single system - at 1500W, that's not bad, I think. At 30k however...that's bad.

You can do it a lot cheaper with other monitors, non-ECC ram (gaming rig doesn't need that), different CPU (each of those Xeons costs heaps) etc..

But it aint his money, so go for broke!

 

[Possessed Freak]

Did they hook 7 keyboards/mice up to the machine for any type of true proof of concept? What about 7 sets of discrete sound?

Kind of useless and mis-titled without that.

 

[TechBoyJK]

yes they had 7 km's hooked up.

not sure what they did about sound.

 

[Kaido]

So one of the things that REALLY stood out to me was the GPU pass-through. This is something I've been trying to figure out how to do for a variety of projects. This is HUGE! I had no idea unRAID had this feature:

http://lime-technology.com/virtualization-host/

I've never seen anything work this well before...this is big, big, BIG!

Traditional virtual machine implementations tend to focus on interacting with those VMs through remote graphics connections. While convenient, remote graphics just don’t yield the same user experience as a locally attached monitor, mouse and keyboard. And accelerated graphics are required to make sure media playback is smooth and gaming graphics quality is high. That’s where unRAID 6 truly stands apart in allowing you to utilize a GPU (graphics card) as well as locally attached input devices to directly interact with a VM, and completely blur the lines between virtual and physical machine user experiences.

...

Our implementation of KVM includes modern versions of QEMU, libvirt, VFIO*, VirtIO, and VirtFS. We also support Open Virtual Machine Firmware (OVMF) which enables UEFI support for virtual machines (adding SecureBoot support as well as simplified GPU pass through support). This allows for a wide array of resources to be assigned to virtual machines ranging from the basics (storage, compute, network, and memory) to the advanced (full PCI / USB devices). We can emulate multiple machine types (i440fx and Q35), support CPU pinning, optimize for SSDs, and much more. Best of all, these virtualization technologies won’t impact the reliability of the host operating system.

...

Specifically for PCI device passthrough, your system must support Intel VT-d or AMD-Vi. You can validate support for Intel VT-d through the Intel ARK site, and searching for your CPU, but AMD does not provide such a means to search for support (you can try here). unRAID also supports the pass through of USB devices such as keyboards, mice, webcams, and more. This makes it easy to create virtual desktops that you can interact with using locally attached devices on the system (USB device pass through does not require the use of Intel VT-d capable hardware).

This is an interesting tidbit from the Youtube comments:

He's using unRAID. Just like with any other Virtual Machine setup (e.g. using VMware), you can connect physical devices (like your USB keyboard and mouse) to a certain VM. In most VM solutions, you can dedicate a certain device to a VM, using it's unique fingerprint. With unRAID, you can even dedicate physical connections (e.g. the first USB port on the top) to a VM. So you can physically split you connections over your VMs

For me at least, this is a whole new ballgame...I've never seen a system that has this much dedicated hardware support & also this much specific-port support. Lime Technology has a blog post on VM gaming:

http://lime-technology.com/gaming-on-a-nas-you-better-believe-it/

What I don't know is how they get it to the end-user. Stuff like RDP requires RemoteFX to make heavy video & 3D streaming even remotely useful, and then you're starting to talk Hyper-V, 10GbE, etc. OnLive tried this (with lag) & there's a bunch of various VDI/PCoIP/etc. technologies that do video compression for streaming & whatnot, but no easy, solid solution is currently available, at least not without spending tons of green on infrastructure & licensing (Citrix, VMware, Hyper-V, etc.). This their other video, they have two players hardwired to one computer using virtualization:

https://www.youtube.com/watch?v=LuJYMCbIbPk

According to the NAS gaming post above, there's not too heavy of a hit from going with a pass-through VM solution, which is awesome. The problem is distribution - how you get that content to people. I work with engineers a lot & this would be a really cool solution for cubicle pods & branch offices where you have five or ten workers & could centrally-manage them off a single system. I suppose you could do things like 100' HDMI cables, Ethernet-USB extenders, and so on to make it work; I'm just really curious about actual implementation via hardware, thin client, remote desktop, etc.

Super super super neat though!

 

[Kaido]

Did they hook 7 keyboards/mice up to the machine for any type of true proof of concept? What about 7 sets of discrete sound?

Kind of useless and mis-titled without that.

Apparently with unRAID, in addition to GPU pass-thru, you can select specific USB ports per VM as well, both of which are extremely powerful features. So option number one is to pop in a PCI/e sound card & pass the whole card through, or alternatively, just pass through a dedicated USB sound card. Plenty of excellent USB DAC's out there, although personally, I just use the little $10 Syba USB sound cards, they sound way better than most onboard soundcards. Sounds like you could get a handful of those to do audio pass-through. Only thing would be you'd have to have everyone in the same room for a physical connection hookup...hmm. Interesting project for sure!

Edit: unRAID said 'you don't pass through ports, you pass through devices. Hubs don't matter then.' The only issue right now is if there are devices with the same vendor & ID, which is something they're working on.

 

[rivethead]

$30K and he still had to end up using double-sided tape to mount the SSDs......that made me lol.

 

[Kaido]

$30K and he still had to end up using double-sided tape to mount the SSDs......that made me lol.

But he DID do a nice job mounting them in an aligned fashion

I can't tell you how many times I've done that...built like a $5,000 super engineering or video editing computer for someone & forget some moronic part that just makes you go AHHHHH! lol

 

[XavierMace]

So one of the things that REALLY stood out to me was the GPU pass-through. This is something I've been trying to figure out how to do for a variety of projects.

This feature has been available with many virtualization engines for some time, you just have to have hardware that support it.

What I don't know is how they get it to the end-user. Stuff like RDP requires RemoteFX to make heavy video & 3D streaming even remotely useful, and then you're starting to talk Hyper-V, 10GbE, etc. OnLive tried this (with lag) & there's a bunch of various VDI/PCoIP/etc.

That's where the pass through comes in. They aren't using RDP/ICA/etc, it's being directly output through the video cards/USB ports just like a regular computer.

Pass though is great for stuff like this. It's less useful in a business setting where you're not wanting to dedicate hardware to a single VM as that's kinda defeating the purpose of using VM's in the first place. Save for passing through a RAID controller to virtualize your storage server.

 

[freeskier93]

So one of the things that REALLY stood out to me was the GPU pass-through. This is something I've been trying to figure out how to do for a variety of projects. This is HUGE! I had no idea unRAID had this feature:

http://lime-technology.com/virtualization-host/

I've never seen anything work this well before...this is big, big, BIG!



This is an interesting tidbit from the Youtube comments:



For me at least, this is a whole new ballgame...I've never seen a system that has this much dedicated hardware support & also this much specific-port support. Lime Technology has a blog post on VM gaming:

http://lime-technology.com/gaming-on-a-nas-you-better-believe-it/

What I don't know is how they get it to the end-user. Stuff like RDP requires RemoteFX to make heavy video & 3D streaming even remotely useful, and then you're starting to talk Hyper-V, 10GbE, etc. OnLive tried this (with lag) & there's a bunch of various VDI/PCoIP/etc. technologies that do video compression for streaming & whatnot, but no easy, solid solution is currently available, at least not without spending tons of green on infrastructure & licensing (Citrix, VMware, Hyper-V, etc.). This their other video, they have two players hardwired to one computer using virtualization:

https://www.youtube.com/watch?v=LuJYMCbIbPk

According to the NAS gaming post above, there's not too heavy of a hit from going with a pass-through VM solution, which is awesome. The problem is distribution - how you get that content to people. I work with engineers a lot & this would be a really cool solution for cubicle pods & branch offices where you have five or ten workers & could centrally-manage them off a single system. I suppose you could do things like 100' HDMI cables, Ethernet-USB extenders, and so on to make it work; I'm just really curious about actual implementation via hardware, thin client, remote desktop, etc.

Super super super neat though!

Posts like this give me a good chuckle. Linus' rig is very cool, but the underlying technology is hardly new. unRAID is using KVM, which has been around for a while now. Lime-Tech only appears to be the first packaging all this into a nice consumer friendly OS.

There's lot of really cool stuff going on with virtualization but a lot of it takes place at the data center level or behind closed doors.

 

[moonbogg]

Did they hook 7 keyboards/mice up to the machine for any type of true proof of concept? What about 7 sets of discrete sound?

Kind of useless and mis-titled without that.

Good question. Can the sound card or onboard audio provide sound for all 7 virtual rigs? Even if it can produce enough voices or number of sounds, would it actually work? I want to see 7 people playing Crysis 3, each recording their gameplay on FRAPS at the same time, otherwise SHENS.