MLAA= A post processing effect that helps get rid of jagged edges with a slight bluing of the image. It's best left off, and only turned on with games that don't have supported for MSAA.
Edge AA= Another type of AA that uses shaders to help improve the image quality of MSAA. For the most part, it doesn't improve image quality much over normal MSAA, and has a large performance hit with games that tax the shaders. It's best to just play with it and see if it works for you.
Adaptive AA= A setting for OpenGL and DX9 games that mimics Nvidia's Transparency AA. It will anti-alias transparent parts of the image, such as foliage and fences, to increase image quality. It's called adaptive because the driver decides whether to use MSAA or SSAA, and it doesn't have too much of a performance hit.
AMD Optimized Tessellation= AMD will include a profile with the drivers where they decide on a tessellation level and override the developers wishes. It's best as of now to use the Use Application Levels option.
Subsurface Format Optimization= A setting that can help with performance with some older DX9 games by changing a texture format. In actual practice, you won't see any kind of difference in performance or quality.
