Realism in games. When does it take for a 3d game to look identical to a movie?

[VERTIGGO]

I'm somewhat of a "realismopheliac". I like to play a good game like Half Life 2 or Battlefield 2 and forget that I'm just clicking away at a machine. I haven't heard much discussion about this in particular, though I know that most gamers crave more and more realistic game animation. I'd like to hear what you would tell the developers if they asked you what to improve.

My thoughts:

Personally, I'm a resolution guy. I find that one of the biggest reminders that I'm staring at a screen is the jaggies from faulty anti-aliasing, or simply low resolution. This is one thing you cannot find in real life! Consequently, I run at 12x10 and anxiously wait the future of LCD technology and the gradual increase in resolution supported by video processors.

There are several aspects of games that I think make up the majority of the performance/quality issue (of course if I forgot something, let me know):

LIGHTING
Recent games seem to have mastered this (BF2, Doom3) to a degree beyond the other aspects of those games.

GEOMETRY
Normal mapping improves the end result, but is only a mask for this largely CPU related limitation. The number of polys must increase before edges and curves will look smooth.

TEXTURE
This is one of my peeves. Ever see a realistic looking brick wall or rock face, and running over close to it and notice that it's just a bunch of fuzzy colors? While finer texturing is always a good thing, I think there is a petition-worthy opportunity to add "distance texturing" to the world of Trilinear/Anisotropic filtering. What I mean by this is a real time texture change from near to far. This means that when you approach a wall, the texture actually improves, and you can inspect it very closely and it still looks real, perhaps to the point where, for instance, you can see a fly that is landed on the wall; and it's not blurry. At the same time, shifting your view to a larger distance displays lower quality textures, but which still look real from where you are. This is somewhat related to the Bilinear/Trilinear issue, with different texture quality at distances. Possibly I'm only asking for an extension of that process to include extremely small distances.

COLOR SHADING
I'm sure everyone has their opinion on fog, and smoke, though most games have rendered these extremely effectively, within the limitations of resolution.

And a multitude of features that can't always be seen in a screeny, but add tremendously to the realism and immersion of gameplay:

PHYSICS -
SPECIAL EFFECTS -

SOUND EFFECTS -
Another peeve. Especially as it applies to gunshots in FPS. Far too many games simply have cheesy weapons sounds. I mean, how many different sounds does a real M16 make? I fire an M4 regularly, and it's an adrenaline rush. Battlefield Vietnam's carbine (m4) felt like an airsoft in comparison the sounds from Far Cry and Battlefield 2. Personally, I think the best weapons sound track to date was in Soldier of Fortune 2. Not quite as realistic as BF2, but gave that awesome feeling when blasting away with a shotty or chugging away with an M60. The BF2 sounds are pretty accurate considering you would need a 10000 watt speaker system to fully enjoy the violence of the explosions and gunfire, but the lame clicking sound of a battlefield 1942 or Doom 3 machine gun just leaves you feeling insignificant and helpless!

MUSIC -

VOICE ACTING
Did someone say Far Cry?
Actually I love the game, but the speech and corny plot/storyline really detract from its lush realism.

I'll think of more sometime...

 

[VERTIGGO]

oops, correction: "Realismophile"

 

[xSkyDrAx]

Look at the ps3 and xbox360 screenshots.

 

[InfiniteLurker]

Originally posted by: VERTIGGO


TEXTURE
This is one of my peeves. Ever see a realistic looking brick wall or rock face, and running over close to it and notice that it's just a bunch of fuzzy colors? While finer texturing is always a good thing, I think there is a petition-worthy opportunity to add "distance texturing" to the world of Trilinear/Anisotropic filtering. What I mean by this is a real time texture change from near to far. This means that when you approach a wall, the texture actually improves, and you can inspect it very closely and it still looks real, perhaps to the point where, for instance, you can see a fly that is landed on the wall; and it's not blurry. At the same time, shifting your view to a larger distance displays lower quality textures, but which still look real from where you are. This is somewhat related to the Bilinear/Trilinear issue, with different texture quality at distances. Possibly I'm only asking for an extension of that process to include extremely small distances.

Doesn't the Unreal engine try to do something similar to this already? I thought I had read that somewhere... I'm sure it won't show a fly on the wall though... yet

 

[Fern]

In addition to your remarks, I would add:

- Get the dang lips in synch with voice

- More realistic facial expressions (and what's up with the blocky heads?)

- Accurate looking body movements/motion

- Details! Highly rendered soda cans, "trash" occaisionally blowing in the breeze when outside (think foliage in FarCry). Rooms shouldn't be sparsely "decorated", there should be many "normal" objects laying around that one can pick up and throw etc.

Fern

 

[fishbits]

You make good points in things that can be improved on the technical end, but another question is "Should 'photorealism' be the goal?" I think in many cases it could be a disappointment compared to a more stylistic approach. Always tickles me how the great-looking games of yesteryear look so awful technically today, but they did a good job of inspiring our imaginations to fill in the details.

As for sound, often when folks fire guns they wear hearing protection because it's that freakin' loud. Now of course we could simply turning down the volume, but then in a "realistic" soundscape, normal audio cues would be inaudible when everything is scaled so that the sounds of rockets and guns are bearable. I'm guessing there'll always have to be that trade-off there, but maybe the user could be allowed to select how much dynamic range there will be in the audio dept.

Next question of course is "How much does it all cost, and are you willing to pay that price?" The price comes in hardware, it comes in the software title, and it comes in taking months and more extra for the game to be developed. What if a gaming company decided to go for a revolutionary leap in these things, yet didn't make their money back? Still overall, "better looking, more realistic" games are more appealing, so I don't think the developers will ever stop trying to make things look and feel more immersive. Look at the wealth of surround sound out there now, high dynamic lighting and transparency AA with the 7800s, etc. Good things will keep coming along, but never fast enough to keep up with our appetites

 

[M00T]

You need artists... lots of artists to use their eyes and point out aspects which could add/remove reality.

 

[elkinm]

I think the biggest bottleneck towards photorealistic games is the programming and the rendering method. How to code a game and how to code the colors to make them look realistic.

As far as graphics, the textures need to be improved to be realistic and true 3D which does exist already.
And the colors need to be improved.

 

[Concillian]

Originally posted by: VERTIGGO
My thoughts:

Personally, I'm a resolution guy. I find that one of the biggest reminders that I'm staring at a screen is the jaggies from faulty anti-aliasing, or simply low resolution. This is one thing you cannot find in real life! Consequently, I run at 12x10 and anxiously wait the future of LCD technology and the gradual increase in resolution supported by video processors.

You will be waiting for something more than that, you will need to wait for a total paradigm shift.

Current display technology is of no higher resolution (in terms of dpi) than monitors made 10 years ago. Sure you can get monitors capable of displaying large sizes, but their true resolution is not all that different from a good 15-17" monitor a decade ago.

To make things truly realistic it will take reducing the size of the pixels quite signifcantly. Think of a laser printer at 300 DPI. Print a black and white photo out on a 300 DPI laser printer and a 600 DPI laser printer and there will be a difference, yet displays are still stuck at ~100 DPI.

Currently if you go from a 17" 1280x1024 display to a 24" 1920x1200 display there is no reason to expect that you would need AA any more or less than you did at 1280x1024. Because these monitors have almost the same size pixels (0.264mm vs. 0.27mm). It only works to remove the need for AA if you set the larger monitor further back, so they have similar effective size.

I would love to see some true high resolution displays. But I have a feeling we will be waiting a LONG time. I'd guess something on the order of 300 DPI will be necessary to get pretty close to realistic with other 'trickery' like AA. But that would be ~3840x3072 on a 17" display. That's not happening anytime soon.

 

[fishbits]

Funny, when I watch a DVD on my non-HD TV, the jaggies aren't bad at all. Yes, smaller pixels are an improvement, but they don't seem to be the limiting factor in today's games (and hardware) when we're looking for "photorealism." There's plenty of places where realism would break down between the software, cpu and graphics card before the monitor could even think of being a factor. As great as the games look, swapping out the monitor isn't going to make the characters from Doom, Unreal, Far Cry, WoW etc look meaningfully more photorealistic.

 

[luigi1]

Its a puzzle. TV is low fi as compaired to a puter moniter. Isn't it? Still TV is more realistic than the best of games but if you display the game on a TV you lose detail. I really think weve got the power now but we have the paradigm of a monocrome graphic card as baggage were caring around. Why are we talking about shaders and light sources and pixel pipelines when hue and brightness is all we need. Thats how a TV works. Once apon a time it was only brightness. then some smart guys added hue. Now we have pixel pipelines and geom engines when all you need is hue and brightness.

 

[Concillian]

While I do not disagree that there are other factors more important. I think you missed that I was addressing a single point of his post, which is why I quoted only one part of it.

One thing about the example you used. I have to ask if you watch DVDs on your TV from 15-20" away? Because that's how far I sit from my monitor when I game. And a TV at that distance looks quite bad. It's kind of like how my car is pretty dirty right now, but if I stand 20 feet away it doesn't really look all that bad. The distance people sit from TVs afford the ability for them to be significantly less detailed. It won't make current games look better to sit further away because they are designed to be viewed from 18" away or so.

Nothing will make the characters from Doom, Unreal, Far Cry or WoW more realistic because they were not designed to be realistic. Of course it takes a cohesive effort from artists as well as having the hardware and coding to support it, and those games were not artistically done to look photorealistic. IMO the artistic perspective is the most difficult, as even animators for movies with more than 1/60th of a second allocated for rendering each frame have a difficult time making things look realistic.

Mostly I meant to point out how most people are concerned about what we currently call resolution, but they really aren't getting much real quality improvement. They're either adding size to add more immersion or they are putting things further away to enhance the 'dirty car' effect I noted earlier.

 

[HaelWho]

Originally posted by: xSkyDrAx
Look at the ps3 and xbox360 screenshots.

A lot of the very high quality screens you see (The infamous Ferrari screenshot that was one of the first screens released) are rendered screenshots and can't really tell you about the system's performance. For example, nearly any PC can render a high quality picture, but there's not really any way that the computer can render frames of a game at that quality fast enough to make it realistic. Basically, all it would be is just frame...frame...frame...frame...frame...frame...etc...

 

[h7o]

I hear you on the resolution. I love how everything looks so much nicer at high resolutions. I'd like to see something like 4000x3000. I think that's doable with OLED. SEDs too but I think they'll be for TVs only. OLEDs can go up to 300 dpi. I think we'll get there in ten years.

 

[Bona Fide]

Max Payne tried photorealism with people's faces. They got real actors/actresses to act out the parts and then rendered them into the game.

We all saw how [horribly] that worked out. No, thanks.

But, many games are working towards breathtaking realism. A few...

-----

Far Cry - When I first fired up the game, the scenery BLEW ME AWAY. If you play this game at at least 12x10 with max settings and 64-bit enabled, you'll be amazed...I couldn't believe it. The CryEngine is going to become as legendary as the Quake 3 and Unreal engines.

Forza Motorsport - Reflection mapping, realistic modeling. This is the true Gran Turismo killer. One race and you'll see the visual brilliance. All reflection maps are 100% accurate, and you can actually see the crowd's faces on your hood when you fly by them.

Black & White - Quite possibly the best A.I. I have seen in a game thus far. Cannot wait for BW2 to come out. Your creature can LEARN, for crying out loud!

 

[Kaspian]

One of the things that is holding back the developers is the hardware currently available to the average user.

 

[fishbits]

One thing about the example you used. I have to ask if you watch DVDs on your TV from 15-20" away?

Actually I can, as my TV is quite on the small side by today's standards. And with its 640x480ish resolution, it looks much more photorealistic than any real-time game engine out there. Same as DVDs look much more photorealistic when viewed on my monitor up close than any game out there run at a higher resolution. No, I'm not trying to claim it's ideal or without defects, but that it's again more photorealistic since the image source signal is more photorealistic in the first place.

I hear you on the resolution. I love how everything looks so much nicer at high resolutions. I'd like to see something like 4000x3000.

If our monitors were limited to say 800x600 but the rendering software and hardware capable of generating full-motion real time scenes of the quality of just shooting with a good camcorder, it would be such a multigenerational leap that gamers would gasp and weep with joy over for years to come. Meanwhile, current models and textures displayed even at 4000x3000 wouldn't matter at all besides for the diminishing returns that the added anti-aliasing would provide. Pong at 4000x3000 is still pong. Higher resolutions look better, but we need more polys, better lighting, better textures, better animations etc before the signal hits the monitor if realism is what we're truly after.

 

[BFG10K]

Doesn't the Unreal engine try to do something similar to this already

Yes, it's called detail texturing. Very few games actually implement this feature, the most notable being the original Unreal series and the Serious Sam series.

 

[SonicIce]

realism sucks. as long as the game is fun. if u want realism buy a simulator game.

 

[Jeff7]

- Accurate looking body movements/motion

This is difficult. We know exactly how a normal human looks when walking. Duplicating it with a computer is very difficult - people will easily note that something doesn't look quite right. They're not sure exactly what it is, but something's not right.
For movies, they solve this by using motion-capture and an actor. The computer character can just be mapped to the captured movements.
Computer games can't do that, unless it's a purely scripted character motion, which doesn't make for good gameplay when it's overused. Someone would have to know exactly how a person moves, and how to correct for things that are "not-quite-right."

As for sound, often when folks fire guns they wear hearing protection because it's that freakin' loud. Now of course we could simply turning down the volume, but then in a "realistic" soundscape, normal audio cues would be inaudible when everything is scaled so that the sounds of rockets and guns are bearable. I'm guessing there'll always have to be that trade-off there, but maybe the user could be allowed to select how much dynamic range there will be in the audio dept.

Yeah, there's a reason that we play games, instead of just going down to a shooting range.
I personally don't like noise - I hate DVDs, like the Matrix, which use the "full Dolby dynamic range". Whispers sound like whispers, so you turn it up to hear what's being said, but then gunshots (or even things like a door closing) are loud enough that the neighbors will call the police because of the gunfight next door. All about balance.
That's why I like watching DVDs on the computer - the players generally let you choose just how dynamic the range gets.


Visual realism - I think that some movie studios use some kind of motion blur too, which does help. Watching some movies before this was properly implemented, certain CG sequences look jumpy. Some parts of the first Shrek movie were like this, most notably in that Karaoke part, with Robin Hood singing - some of his rapid movements just looked choppy.

 

[VERTIGGO]

Actually, Far Cry implements motion blur, but it's not used unless the character moves rapidly as in falling.

 

[SonicIce]

Originally posted by: VERTIGGO
Actually, Far Cry implements motion blur, but it's not used unless the character moves rapidly as in falling.

acually it looks like its always on but its very subtle

 

[asm0deus]

the animatrix (edit: final flight of the osiris) was very photorealistic but it took thousands of hours to render so you're looking at a huge technology gap in trying to render it in realtime.

 

[VERTIGGO]

- Get the dang lips in synch with voice

- More realistic facial expressions (and what's up with the blocky heads?)

- Accurate looking body movements/motion

- Details! Highly rendered soda cans, "trash" occaisionally blowing in the breeze when outside (think foliage in FarCry). Rooms shouldn't be sparsely "decorated", there should be many "normal" objects laying around that one can pick up and throw etc.

Fern

damn straight! anyway, why is body movement so hard? I've studied the way a human walks, and its not complicated. its mostly the twist effect, and the irregular up and down rhythm that is totally lost on so many developers. it would be easy to simulate a casual stride, because that rhythm can be hard coded and look great.

 

[VIAN]

I'd say we need a lot more years. Yup.

We would need to run almost full AF, which we don't, and possibly many more samples. AA has to be kicked up a notch to what Nvidia has. A nice mix of SuperSampling and Multisampling. But much higher than that. 2-4 times better. Run it at 1280x1024. High Quality textures. Amazing lighting. A lot of poly power. motion blur filmy appearance.

Possibly in a decade.

if you want to take it a step further and have it shown in 3d with glasses, then, possibly 2 decades.

And I still think it's far from creating a real, breathing world. This is just for photorealism.