A combination of waiting for people to get on board with vista/7 (the lack of a good vista launch really hurt), the availability of DX10+ HW, development time for the new engine, driver support, knowing when to leave their low end users (along with just the cost of supporting many more types of hardware), the list goes on.
For each company its unique as they themselves decide when to go to DX10+.
Software lags behind HW.
If people have modern hardware to run games like BF3, its a non-issue to install a new OS. People who are on old hardware aren't even in the target market anyway.