We put our optimizations through the test mill regularly. If you know of a specific workload that suffers due to avx512 usage, don’t… hide it. Please share this information with us so we can investigate and adjust for it!
However, to try, you can always sudo rm /usr/lib64/haswell/avx_512/libfoo.so.1; sudo ldconfig temporarily, and swupd verify --fix will bring those files you removed back (be careful to remove the right ones and not everything in /usr/lib64.