Okay, let me get a little more precise on that matter:
First, it's not clear whether or not the AFTester always determines the correct amount of samples. Now comes the a little bit more complicated reason:
The analysis is (most likely) based on letting the algorithm sample a special texture for each pixel. The result that is then forwarded to the frame buffer gets analyzed by the cpu, and from this analysis, the most likely sample count is determined. The concept works because in the filter kernel, each texel sample has an adaptive weight and distance to the other sample along the long axis of the ellipsis that is laid into the pixel footprint (the "shadow" of the pixel in texture space), and if you adjust a floating point texture in multiple passes and let it filter, you get information about the sample count by comparing the texture input with the filter output. What this algorithm straight out misses are samples that are very close to each other with very similar weights. After all, that's not so dramatic as those samples do not contribute the most to the filter result, but it might have a strong influence on the final results. That, of course, does not render those analyses completely pointless, but you must consider quite an error in the measurement, but certainly not to the point where it's useless.
Let's come to the banding and the possible reason again.
The literature implementation of angle invariant AF reconstructs the pixel footprint in the texture space by using texture gradient calculations and then lays an ellipsis into this footprint. Along its long axis, AF requests the additional samples, filtering in the orthogonal direction is mostly covered by mipmapping.
Now, the length of the long axis divided by the length of the short axis yields the AF ratio. If the the long axis is 3 times longer than the short axis, it takes 4 trilinear samples (=32 texel reads). 4 samples, because traditionally, only powers of two were allowed (2:1,4:1,8:1...), but I think it is save to say that nowadays they're, if at all, restricted to even ratios.
The reason for this power of two ratios is simply that you want to give each sample the same weight, and this, in computer language, just implies a bitwise shift instead of a true division if you keep to the ratios. But TMUs have become more powerful over the years, and as my sources state, they are in most cases restricted to even ratios.It is, however, key to keep to the Nyquist theorem (sampling frequency >= double the information frequency), to avoid unnecessary flickering.
Yet, AMD and NV do not fully obey the mathematical law. To save performance and bandwidth, they do not keep to the reference implementation, but instead have developed more intelligent sample distributions that need less samples. And here is the first key point: If those analyses are correct, there should be something wrong in that part of the algo that calculates the right sample amount.
Here's what happens on HD5K:
1. They have abviously shifted transition a little too far, resulting in flickering and moiree.
2. Strangely, after the transition they first increase the sample rate but then l!ower it again!. That's totally irrational behaviour. Why should it take less samples further in depth where texture distortion is bigger? This is either a bug in AMDs hardware or it is related to what I want to explain now.
Generally, the principle alone leaves potential for banding, as it's probably not possible to increase the sample rate fully progessively, to always stay just above the Nyquist level. The problem is solved by
- giving those additional samples after each transition different weights in an adaptive manner
- progresively diverging sample positions
- a combination of both
Let me draw a little picture:
2 trilinear samples along the long axis just before the algo switches to the next level
o------------o
next level, 4 samples (O means two samples, the more black the more weight)
O-----------O
more texture distortion
o-o-------o-o
even more distortion
o---o---o---o
This is just one of many possible solutions (you can imagine how tricky it can be to distribute 16 samples intelligently). This is the next key point.
There might be something wrong or a bug in the routine that determines the weights, as this algorithm MUST work together with the one that determines how much "optimization" is going on, as the weight and position of each sample always depends on the amount of samples used.
And this might also the reason for strange results in the AFTester. A bad sample distribution might lead to wrong estimates about the amounts of samples used. So this might be another explanation for what we see in the AFTester, and it would still explain why those harsh transtions in those Texel-Analysis shots conincide with the banding.
Apart from that, you're (Xarick) obviously right that it's not a big deal in terms of most games apart from TM, but AMD
1. has been totally quiet about this before the Barts launch and straight out rejected any request for explanations or whatever
2. then not only promised to fix a bug for Barts, but told us that they're giving us a new feature
3. and at last failed to do what they can do with higher AF levels, which means that they have the bug for yet another generation, plus a potential for even more flickering
I know that AMD has to make money, but what they have done with Barts regarding AF is certainly not "the way it's meant to be played", to say it with NV terms.
Facebook
Twitter