Are compilers not keeping up with hardware advances?
Sorta. It's more like compiler makers are slackers or are poor.
Microsoft is a slacker. The PII was out for like 2 years before they released a patch for Visual C++ that let it use SSE. On the other hand, the folks that make GCC are poor since they don't have any budget at all! Borland C++ is more concerned with preserving market share and fighting Microsoft than worrying about performance.
The situation is so bad that Intel has had to resort to releasing their own C++ compiler. But not many people use this. For me, it has some compatibility problems too.
However, compiler makers are only responding to their customers - i.e. software companies. Software companies rely more on hardware advances than on low level efficiency to achieve adequate performance. There's never enough time to dick around with assembler code for the sake of good performance.
Personally, I think software companies shouldn't bother with low level performance unless they really need to. Few programmers know how and the code that is produced is too arcane which makes it difficult to manage. However, compiler makers should be held to a higher standard since their code is depended upon by so many other products. Visual C++'s memfill code is a joke. It's just a while loop that copies stuff one byte at a time. What awesome 8-bit technology.
Also, I think most performance problems are more a matter of poor design and poor algorithm selection/invention than a lack of low level optimizations.