I guess you're referring to:
Except that's not exactly true. It might be, depending on the processors and binaries, but certainly not in all situations. Processors that are not compatible will obviously not be running the same binary, making the above statement seem a bit odd as nothing suggests limiting to x86. Even then, some binaries can have multiple code paths depending on processor features or strengths, or a program may have multiple binaries to accomplish the same. Doing the same task with AVX or AVX 512 will change the number of instructions involved. And that's just the pre-compiled stuff. JITed code could end up quite different given an intelligent JIT compiler that plays to the strengths of whatever processor it's running on.
A few caveats:
If you're running special codepaths for different levels of AVX you're going to dynamically load and for the most part recompile everything. The same way you can't compile a 32 and 64 bit executable into the same program, you can't set a certain compiler flag to do both AVX and even AVX2 in the same dll. (AVX and AVX2 are very similar) (Sandy Bridge to Haswell)
Doing the same task with AVX or AVX512 will have similar instructions but AVX512 will issue less instructions to get the same work done. An exception to this is operations that require permutations. A 2 lane AVX path is one pass. For a 4 lane AVX512 path is 4 passes. Plus additional in lane element manipulation for both AVX and AVX512. (1 to 5 passes)
There is an additional penalty for mixing modes. Running SSE3 mixed with AVX for example will require register flushes in the upper lane.
JIT will probably never replace the standard compiler and library system. It has its place in many housekeeping tasks, but traditional code is still the norm.
For the most part AVX2 isn't going anywhere and AVX512 will probably find its way into a few libraries. Its niche is still high performance computing and even there it is far from the norm in code.