Why not Spline the Edges?

[imported_lev]

Example Image

In recent years I've been blown away at the amount of realism and detail that is being introduced into real-time 3d. Take a look at the example image. The features on the face are remarkably detailed with beautiful lighting (not to mention a stone cold stare). But the profile is all sharp lines and jagged edges. The jaggies can be smoothed out using antialiasing, but why can't the sharp lines be smoothed out? We already have the normals at each vert. Why not fit a third order poly between those vertices? The only real issues are finding those critical edges and filling in the void with something meaningful.

I'm somewhat of a beginner in computer graphics, so maybe there is an obvious reason why this would be difficult that I don't see. First we need to find the critical edges. What I consider critical are the edges which the pixel depth on the edge to the pixel depth off the edge forms a delta above a specified threshold. For instance, in the image the edge on the head in relation to the back wall. I think we would need the projected vertex locations and normals. Maybe also have a flag of wether or not the vertex should be considered in the smoothing process. A depth of which to include in the smoothing (maybe smoothing out all the critical edges in the scene would be very computing intensive). Then we can stretch the already textured polys. I'm thinking this would all be a post process. One problem I would worry about is what happened to the the pixels on the back wall that were covered by a pointed edge. At what point was it discarded? Can I detect the critical edge and save some of the runner up pixels along each edge?

Is this even possible with the vertex shader and pixel shader seperate? Would this be only be possible in directx 10 type hardware? Am I way off and looking at this completely wrong?

I remember a technology used briefly to interpolate a higher poly model from a lower poly model. That seem excessive now. Keep the lower poly count, but smooth out the rough edges, that's my thoughts.

Maybe this is wildly out of place, but it was just something that has been nagging at me for a while and I didn't know of a better place to post this.

I appreciate any input (good or bad ).

Thanks for reading

 

[40sTheme]

Hmmm... I have no idea, but there are games with smoother faces than Quake. Quake isn't exactly the example of amazing graphics. They're great, but definitely not the best. Crank up the graphics with Oblivion and get some realistic faces mods, and most of the issues with faces are resolved. I know Crysis has almost no jagged model edges except for the ones that are supposed to be there (i.e. edges on clothes). Take a look at some Crysis images and see. Can't really show you the Oblivion stuff; I can' t run it on max, and there are probably no images with the mods.

 

[josh6079]

Without tweaking anything, the Oblivion faces look like crap. Quake 4 has some of the best graphics I have ever seen. Their faces are better than any other game IMO, including Sources.

 

[Nightmare225]

Originally posted by: josh6079
Without tweaking anything, the Oblivion faces look like crap. Quake 4 has some of the best graphics I have ever seen. Their faces are better than any other game IMO, including Sources.

Are you kidding me? Source has worse faces than Quake 4? Don't make me laugh... :disgust:
http://nfhl.stsland.ru/image/hl2/npc/alyx.jpg

 

[Dethfrumbelo]

Yeah, interesting how the smoothing makes the poly faces within the visible parts of the mesh look smooth, but the edges at the visible exterior of the mesh are not smoothed and remain straight lines from vertex to vertex. The only way to smooth those outside edges would be to add additional verticies, and changing the basic shape of the mesh can be problematic after other operations have already been applied on top - deformation/scaling/smoothing/texture unwrap/etc. Doing something like this on-the-fly is pretty intensive to boot.

There are some well-known 3D modeling programs that can smooth the geometry by adding polys and can also turn a high poly model into a low poly model, but again, it's a pretty CPU intensive operation.

Current rendering methods are geared to treat models as verticies with direct/straight lines between them creating flat faces. Maybe the next step is to develop more complicated surface descriptions and set the hardware up to deal with that level of mathematics. Surfaces would need to be rendered as mathematical curves and arcs, instead of a series of connected verticies. Ultimately, this is where it should go, otherwise you're always dealing with the brute force method of adding more veritices/polys to achieve nicer curves.

 

[extra]

Didn't/doesn't ATI have some sort of tech on their cards or drivers that could do stuff like this? I seem to remember it talked about, but no one really used it. Anyone remember what it was called? Was it something related to this?

 

[xtknight]

Originally posted by: lev
Why not fit a third order poly between those vertices? The only real issues are finding those critical edges and filling in the void with something meaningful.

Performance. It's not that they couldn't do it, but that means more polys for the engine to render, and it already has tens/hundreds of millions per scene. Just imagine that even adding one poly inbetween the two would multiply the work by 1.5 (excluding non-model polys).

I guess they could just make a higher-poly model before it is sent to the engine, to lower a tad bit of the load. I don't know how hard it would be for the engine to selectively interpolate.

Originally posted by: extra
Didn't/doesn't ATI have some sort of tech on their cards or drivers that could do stuff like this? I seem to remember it talked about, but no one really used it. Anyone remember what it was called? Was it something related to this?

ATI TruForm? It's a tessellation technique, and now that I think about it maybe it did do this. It's quite intensive though. NVIDIA didn't coin a specific term for it (the GeForce GPUs support N Patching (HOS (High Order Surfaces))).

Search for "Bezier Curve Tesselation" in this NVIDIA pdf. It hints at the possibility of this existing in Direct3D 10 games. I believe that's the exact same thing as N Patching that all GPUs support in D3D9/OGL.

I believe TruForm is implemented in RTCW. ATI had a demo showing a low-poly shark model and what TruForm did to it.

 

[Matt2]

Originally posted by: Nightmare225

Originally posted by: josh6079
Without tweaking anything, the Oblivion faces look like crap. Quake 4 has some of the best graphics I have ever seen. Their faces are better than any other game IMO, including Sources.

Are you kidding me? Source has worse faces than Quake 4? Don't make me laugh... :disgust:
http://nfhl.stsland.ru/image/hl2/npc/alyx.jpg

Source looks too "cartoony".

her eyes are too far apart, she looks like she has down syndome and she has very limited facial detail.

I like the Doom 3 engine much more than Source. Source just looks... fake...

 

[Trevelyan]

Originally posted by: Matt2
Source looks too "cartoony".

her eyes are too far apart, she looks like she has down syndome and she has very limited facial detail.

I like the Doom 3 engine much more than Source. Source just looks... fake...

Honestly I've developed an appreciation for both Doom and Source engines.

I think Source does best with outdoor, well lit environments. The water and landscaping are fantastic. I also think their faces are the best, simply because of the EXPRESSIONS engine they have running for the characters. Seriously, what other game do the characters show the wide range of emotion as Alyx in Episode 1?

But, I do like the dark, inside environment that Doom offers. I think the engine may be more optimized to be GPU, rather than CPU dependent.

Both are great engines, but my chips falls with Source simply because of what Valve has done with it.

 

[imported_lev]

Please no arguing about which game has better graphics. I'm impressed by all the games listed.

From the images I've seen of crysis, they have the same problem. Admittedly the faces look nice (if you view the ign images you will know what I am talking about), but that's mostly from the higher poly count. Some of the other images look nice, but mostly due to the depth of field blurring effect. Other images don't look quite as nice on the edges. And the problem is very obvious with leafy vegitation (the overgrowth).

Example 1

Example 2

 

[imported_lev]

I was thinking it might be possible to not draw polygons in 3-space, but rather spline the edges as a 2D post process with some information of the vertices. Namely the projected locations and the normals (only the x and y components would be needed). Since we already have the normal, it saves calculations needed to make the spline.

 

[imported_lev]

Originally posted by: Dethfrumbelo
There are some well-known 3D modeling programs that can smooth the geometry by adding polys and can also turn a high poly model into a low poly model, but again, it's a pretty CPU intensive operation.

This is true, and that's how we get a normal map. It makes everything look amazing , except at the edges .

 

[imported_lev]

Originally posted by: xtknight
Search for "Bezier Curve Tesselation" in this NVIDIA pdf. It hints at the possibility of this existing in Direct3D 10 games. I believe that's the exact same thing as N Patching that all GPUs support in D3D9/OGL.

Thanks for the link. I'll read through that tomorrow.

 

[josh6079]

Originally posted by: Nightmare225

Originally posted by: josh6079
Without tweaking anything, the Oblivion faces look like crap. Quake 4 has some of the best graphics I have ever seen. Their faces are better than any other game IMO, including Sources.

Are you kidding me? Source has worse faces than Quake 4? Don't make me laugh... :disgust:
http://nfhl.stsland.ru/image/hl2/npc/alyx.jpg

Like I said, IMO this kind of face when casually playing in the game in these kinds of situations impress me more than Source's did. It's fine if you don't agree. To me they just have more realistic facial shadows and skin.

Would Z-buffer improve facial textures if it could be done in 32-bit rather than 24 or 16? Eventually I think a nice improvement would include facial wounds when at certain health barriers and in OpenGL applications I wonder about how fine tuned of a Z-buffer depth we would need in order to really bring out the geometry of a good cut.

 

[zephyrprime]

The thing is, everything the video card does is actually super simple mathematically speaking. It doesn't seem that way, but it is. Having written a very meagre software rendering, I've seen this first hand. Just calculate out the transformation of a line from 3d space onto a 2d plane and you will see that it actually results in a very simple formula. Just try that for a bezier curve and you will want to shoot yourself.

However, the idea of doing stuff to smooth out edges isn't bad. You just have to be cleaver about it. Hardware is providing more power so stuff like this is becoming possible. I was looking at the xtknight's link and here's one way I thought of to smooth out edges without too much computational burden:

The idea is to create more polygons like with ATI's truform but with a twist. Use Nvidia's idea for silouhette detection to figure out the important part of the models to add polygons to. This way, the computational burden is modest compared with Truform.

 

[josh6079]

Originally posted by: zephyrprime
The thing is, everything the video card does is actually super simple mathematically speaking. It doesn't seem that way, but it is. Having written a very meagre software rendering, I've seen this first hand. Just calculate out the transformation of a line from 3d space onto a 2d plane and you will see that it actually results in a very simple formula. Just try that for a bezier curve and you will want to shoot yourself.

However, the idea of doing stuff to smooth out edges isn't bad. You just have to be cleaver about it. Hardware is providing more power so stuff like this is becoming possible. I was looking at the xtknight's link and here's one way I thought of to smooth out edges without too much computational burden:

The idea is to create more polygons like with ATI's truform but with a twist. Use Nvidia's idea for silouhette detection to figure out the important part of the models to add polygons to. This way, the computational burden is modest compared with Truform.

That does seem like a good idea. Although I think it would still give a significant frame hit depending on how many silouhette detections designate the polygons. Still, good idea zephyrprime. :thumbsup:

 

[zephyrprime]

Originally posted by: josh6079That does seem like a good idea. Although I think it would still give a significant frame hit depending on how many silouhette detections designate the polygons. Still, good idea zephyrprime. :thumbsup:

Yeah, it could give a significant frame rate hit altough I don't really have any specific insight into how much of a hit it would incur. I guess this sort of thing would be done in the geometry shaders. The problem would be that edge detection would have to scan through all the trangles on the screen and compute a cross product for each triangle to get a surface normal and then calculate a dot product of the surface normals of adjacent triangles to calculate the angle between triangles. Does that take a long time? I have no idea. I think surface normals are calculated already for hidden surface removal but I'm not sure. If that's the case, then edge detection wouldn't be too bad. The actually extrapolated tesselation would be pretty fast I thinkg because the silouhettes on the screen are actually only a small percentage of given scene.

 

[avi85]

Originally posted by: josh6079
Without tweaking anything, the Oblivion faces look like crap. Quake 4 has some of the best graphics I have ever seen. Their faces are better than any other game IMO, including Sources.

How do you do that? I've always hated how crappy the faces look on oblivion (relative to the rest of the game).

 

[imported_lev]

xtknight,

Thank you very much for the link. It seems like the Silhouette Detection and the Bezier Curve is unrelated. From the looks of the slides, it seems like the speaker was jumping from topic to topic. But it would be cool if they decreased the poly count in the cow, did the silhouete detection, stored the projection vertices and applied a bezier spline. Without hearing the speaker, it's hard to figure out why these are new effects are possible with d3d10 (somewhat implying they are not possible or impractical using d3d9).

Other forum with similar question

I found this by searching silhouette and some other things. I couldn't think of the word silhouette earlier, so I called the silhouette edges critical edges. A part of me hates the fact that Joe Bloggs asked this question two year ago . Maybe his idea is now possible with the silhouette detection and some interpolation (or cheap blurring effect). I'll credit Mr. Bloggs in the patent

Check out DogmaD's post. He added some clarity for me.

 

[zephyrprime]

Originally posted by: levWithout hearing the speaker, it's hard to figure out why these are new effects are possible with d3d10 (somewhat implying they are not possible or impractical using d3d9)

It's because of the geometry shaders which are new with DX10. Now doing all sorts of geometry calculations at rendertime is possible.

Extruded shadow volumes should be especially important I think.

 

[Dethfrumbelo]

I agree with the OP that this is the most glaring deficiency present in games (rendering) and should have been given more serious attention. 16XAA, HDR, etc, but we're still looking at ugly, jutting polys in 2006.

If the DX10 specs do nothing else, this alone would be a big step forward.