Unfortunately, a lot of those instructions (VNNI) are used for inference. Which strikes me as the biggest issue of dropping AVX-512 support. It's not the 512b vector length per se, but the other instructions bundled in.
My understanding of AVX was that cracking it once (i.e. from 256b to 2x128b, or 512b to 2x256b) is doable without too much effort, but cracking it twice (512b to 4x128b) is disproportionately more complicated. Might be hearsay, but we'll have to see what "NextMont"/Meteor Lake does.
Imo, Intel should at least backport VNNI instructions and such to 256b width. Wouldn't fully solve the problem, but would at least work as a stopgap measure.