When will computer games look completely realistic?

[bryanl]

I recently saw this music video that resembles a first person shooter game, and it made me wonder why computer games still don't look nearly as realistic:

http://www.youtube.com/watch?feature=player_embedded&v=Rgox84KE7iY#at=75

How much more computer power would be needed to achieve something similar?

 

[colonelciller]

check this out:

http://youtu.be/3FIviB90Ih8

I see 2 things to work on... 1 is the animation side of things, the other is the graphics quality, textures, detail levels etc. I think we're getting pretty close with the animations, but for the detail levels we're nowhere near powerful enough.

Photorealistic CG takes a long time to render and even more time in post-production to add the imperfections of the real-world.

In theory you could create filters that would add additional rendering passes to the images and render those in real time (each filter a separate pass or series of passes), but to have that happen in real-time and be photo-realistic would be a tremendous computational challenge i believe.

essentially we'd need to get to where we can render movies in real-time on desktop machine with a single GPU. I guess that people who say "computers are powerful enough already" don't want to see us get anywhere near to this kind of realism in gaming... too bad.

 

[Zodiark1593]

Ehh, I'm not too into realism myself, but even within the past few years, CG takes months to render a full length movie on a server farm. Consider that very few have similar compute resources the movie industry possesses, it will take a very long time for that kind of performance to trickle down to even high end gaming GPUs.

 

[Rifter]

Honestly i think they will find a way to make your brain do the rendering before they come up with CPU's/GPU's that can handle it.

As in i think they will find a way to jack into your brain and convince it that you are in a game and your brain would do all the rendering, kinda like a virtual reality.

 

[Turbonium]

I see 2 things to work on... 1 is the animation side of things, the other is the graphics quality, textures, detail levels etc. I think we're getting pretty close with the animations, but for the detail levels we're nowhere near powerful enough.

Honestly, I see the biggest obstacle being animations, particularly those of humanoids/humans.

Even looking at blockbuster movies, you can usually tell there is something "unnatural" about CG humanoids and their movement. Our brains are just really good at picking up natural vs. unnatural movement in that regard.

Also: face animations during speech. Even the best movies do a pretty bad job of emulating it - they end up looking freakish and artificial.

 

[Midwayman]

The issue isn't the rendering so much. We can already do some pretty damn realistic rendering in real time. The issue is in animation- but not even in making lifelike animations. Its in scripting and blending procedural animation. Basically you need some pretty high level AI to make animations work and look like they are behaving intelligently in real time. You can fake it to a certain extent with motion capture animation and some intelligent scripting. However what you really need is to simulate the AI in characters heads and then have them drive their skeleton appropriately for the environment.

Playing back a realistic animation in real time is very doable and really a matter of just throwing hardware at it. Getting a game that reacts smoothly and realistically is a much tougher problem.

 

[wirednuts]

well, i think movie cgi has clearly made it to that point. when its done right, you simply cant tell its not real.

the problem with games, is youre trying to simulate physical reality so well that your eyes cant perceive the difference, and it has to do that over and over 60 times a second no matter what motion happens. that means every shadow, every nuance must be spot on. In movies, you get the chance to go over and over it before releasing it, filtering it step by step until you can tell its perfect. in games, you must account for every possible scenario in front of you at once and its basically impossible even on supercomputers.

 

[RPD]

Honestly, I see the biggest obstacle being animations, particularly those of humanoids/humans.

Even looking at blockbuster movies, you can usually tell there is something "unnatural" about CG humanoids and their movement. Our brains are just really good at picking up natural vs. unnatural movement in that regard.

Also: face animations during speech. Even the best movies do a pretty bad job of emulating it - they end up looking freakish and artificial.

http://en.wikipedia.org/wiki/Uncanny_valley

 

[colonelciller]

I see 2 aspects to uncanny valley.

1 is the animation
2 is the photorealism

With the animations there are huge strides being made using cameras that spatially map the 3D movements of actors. The new tech has the ability to do this mapping without needing to paint or affix "dots" to the actors faces and bodies. This tech is improving extremely fast and the prices of the gear is dropping very fast. So with the animations I personally believe that we will be there quite soon. LA noir is an example of some amazing progress made possible by this tech, and hollywood techniques will continue to leak into the gaming world.

the other part of animation is the way objects, clothing, hair, water, wind, etc... behave. The only way to make it look like it does in the real world is to use physics... and that's very cpu intensive. The tech is there though, however our home systems are woefully pathetically underpowered for such calculating on the fly (as you'd need in a game). This is why I'm always befuddled by comments that computer power far surpasses the needs of current gamers. In my opinion we're nowhere close, and we will never have a game that even attempts to bridge the gap towards more realism until there is hardware to run it on.

On the other hand, even the best facial animations & object animations will be rendered null and void as to bridging uncanny valley if the textures and photorendering into aren't there... and right now they are right on the cusp in the hollywood setting.

 

[Midwayman]

I see 2 aspects to uncanny valley.

1 is the animation
2 is the photorealism

With the animations there are huge strides being made using cameras that spatially map the 3D movements of actors. The new tech has the ability to do this mapping without needing to paint or affix "dots" to the actors faces and bodies. This tech is improving extremely fast and the prices of the gear is dropping very fast. So with the animations I personally believe that we will be there quite soon. LA noir is an example of some amazing progress made possible by this tech, and hollywood techniques will continue to leak into the gaming world.

Markerless motion capture has some severe limitations, particularly in the case of LA Noir. First up is that it only really can recreate something that is actually present. It works more like a 3d camera than motion capture that can be used to drive non-human meshes. LA Noir in particular has an extremely small capture volume and requires actors to recreate their performance twice- once for body motion and once for the facial. This is combined in post. Its moving away from where state of the art is going. Avatar otoh used head mounted cameras and 2d face tracking to be able to capture both performances at once. This leads much more tightly integrated performances. Even in a composited animation like golem from the lord of the rings, they moved to capturing the motion capture data at the same time as the live action film so they wouldn't have to redo scenes multiple times.

the other part of animation is the way objects, clothing, hair, water, wind, etc... behave. The only way to make it look like it does in the real world is to use physics... and that's very cpu intensive. The tech is there though, however our home systems are woefully pathetically underpowered for such calculating on the fly (as you'd need in a game).

This is relatively easy. We understand how to do this. We only need faster hardware.

On the other hand, even the best facial animations & object animations will be rendered null and void as to bridging uncanny valley if the textures and photorendering into aren't there... and right now they are right on the cusp in the hollywood setting.

Its really the animations that are the hard part. We've done very believable still models for awhile. Also the uncanny valley isn't so much about the models. You can have decidedly non-human models and perfect animation and not worry about the uncanny valley. The animation side of the scale is the important one to consider. Its why many animated films and games deliberately use less than human models.

 

[lagokc]

Imagination Technology just this year released ASICs that can do ray-tracing in real time for CAD software using a method I doubt can be applied to objects moving in real time. It will be a few years before GPUs have the power to do that sort of thing to a moving scene while maintaining 60fps.

 

[colonelciller]

Markerless motion capture has some severe limitations, particularly in the case of LA Noir. First up is that it only really can recreate something that is actually present. It works more like a 3d camera than motion capture that can be used to drive non-human meshes. LA Noir in particular has an extremely small capture volume and requires actors to recreate their performance twice- once for body motion and once for the facial. This is combined in post. Its moving away from where state of the art is going. Avatar otoh used head mounted cameras and 2d face tracking to be able to capture both performances at once. This leads much more tightly integrated performances. Even in a composited animation like golem from the lord of the rings, they moved to capturing the motion capture data at the same time as the live action film so they wouldn't have to redo scenes multiple times.



This is relatively easy. We understand how to do this. We only need faster hardware.



Its really the animations that are the hard part. We've done very believable still models for awhile. Also the uncanny valley isn't so much about the models. You can have decidedly non-human models and perfect animation and not worry about the uncanny valley. The animation side of the scale is the important one to consider. Its why many animated films and games deliberately use less than human models.

great post Midwayman
very informative!

 

[colonelciller]

Imagination Technology just this year released ASICs that can do ray-tracing in real time for CAD software using a method I doubt can be applied to objects moving in real time. It will be a few years before GPUs have the power to do that sort of thing to a moving scene while maintaining 60fps.

Just looked it up: http://www.pcper.com/news/Graphics-...nation-Shows-Series2-Ray-Tracing-Accelerators

WOW! the youtube vid of the tech is amazing: https://www.youtube.com/watch?v=1Zc1FJkbwIc

Very cool... and the prices are incredibly reasonable.
Does this completely eviscerate AMD & Nvidia in the workstation graphics space?

 

[lagokc]

Very cool... and the prices are incredibly reasonable.
Does this completely eviscerate AMD & Nvidia in the workstation graphics space?

The Imagination card isn't a replacement for the GPU, it's an additional processor that only does ray tracing similar to the physics cards that were available a few years back only did physics. Your workstation still needs an nVidia/AMD GPU in addition to the $800/$1500 ray tracing board if you want to do that sort of thing.

 

[exdeath]

Screw graphics, when are games going to be fun and original and have a captivating unforgettable single player story mode again? An M4 and grenades and red barrels can only be so realistic looking... as can the straight hallways you run down...

 

[colonelciller]

The Imagination card isn't a replacement for the GPU, it's an additional processor that only does ray tracing similar to the physics cards that were available a few years back only did physics. Your workstation still needs an nVidia/AMD GPU in addition to the $800/$1500 ray tracing board if you want to do that sort of thing.

the more i think about this board the more exciting it is. do you know if it can be paired with a "consumer level" card such as the 7950, and have that combination then give equal (actually better) performance than a single "professional" (overpriced) card!?

 

[lagokc]

the more i think about this board the more exciting it is. do you know if it can be paired with a "consumer level" card such as the 7950, and have that combination then give equal (actually better) performance than a single "professional" (overpriced) card!?

The way I understand it, the card doesn't actually do the ray tracing calculations in real time (that would require tremendous processing power which the card does NOT have). What it does is it creates the rays for a static scene (imagine a CAD drawing, an architecture program, something that doesn't move) and then uses its large amount of RAM to store those positions in a database so that they don't actually have to be recalculated in real time. The lightsource and rays don't move, all the card has to do is look up what the lighting should look like from the camera position.

So put your pants back on and don't expect it to be useful for prettying up your games.

 

[reallyscrued]

Photorealistic scenes in real-time are completely possible within our lifetimes if we leave human movements and expressions out of the scene.

I was argue that image is near photorealistic already. Ray tracing and radiosity light effects are going to be the next big leap in realism, until then, everything's gonna look like an updated version of FarCry 3.

 

[mindless1]

^ That doesn't look near photorealistic to me, but if there was random dust and dirt particles added, it would come closer. IMO that's one of the problems with CGI, not enough randomness, errors, imperfections, etc. That and the tendency to go overboard with color saturation, and not get the lighting / reflections right.

 

[wsaenotsock]

I actually think we are there with Textures and modeling, and potentially animation-- since animation quality is just a function of time spent, and there is really no limit there.

The most obvious weak point in my opinion that gives away CG has always been lighting. Direct lighting is pretty good, but the diffused lighting is incredibly difficult to do properly. Only ray tracing has been able to do this well, and from what I understand, the performance needs to increase to see this technology in a game.

 

[jaqie]

Simple answer: when a single commodity built computer available for general sale can render a physics-accurate representation of a glass of water in motion on a per molecule basis in realtime, with proper lighting including reflection, refraction, lensing, and other such things. Once we can do that in realtime on a commodity computer things will begin truly looking up.

Reason: that volume of water rendered and lit properly per molecule is not required, but take one one thousandth of that resolution applied to the glass and use that power on the rest of the scene, whatever it may be, and that will give enough computing budget for lighting and physics for game makers to apply current compute-saving tactics to to make games that look to us completely real compared to real scenes filmed and shown on the 1080p monitors we use today.

 

[Blanchflower]

How much more computer power would be needed to achieve something similar?

My prediction would be 80 to 120 years from now.....

 

[sm625]

We don't even have the right technology. The human eye takes data from millions of rods and cones and assembles it holographically into a data stream to be sent down the optic nerve. The amount of processing that occurs here may not be as high as the amount of processing that occurs on a rendering farm, in terms of computations per second, but the data is processed much more efficiently.

Until we start designing our GPUs in this way we will simply not be able to create photorealistic interactive environments.

I bet someone from 20 years in the future would laugh at our current tech as surely as we would laugh at 3D processing from 20 years ago (Remember Starfox on SNES with its custom DSP? lol) The funny thing is we havent really progressed that far. We are still dealing with pixels and polygons rather than waves and interference patterns.

 

[nightspydk]

When I played some of my first games on PC like bards tale, I had no real gfx, but just a text based game like monkey island etc. Then I had to imagine the most of it and it made it so much more real.

Just think about it our imagination is amazing.

 

[Compman55]

Back in the days of 486 DX4-100 playing quest for glory 4 shadows of darkness, me cousin and I speculated that by the time computers got to 786 processors they would look real. Man were we wrong.