Pretty much everything sits on PCIe right now. Raid cards are PCIe, the ICH10R is DMI which basically uses 4 PCIe lanes... In reality, if you have 2x PCIe 2.0 x16 cards even on an X58, you run out of PCIe lanes quickly (4 DMI for IC10R, 32 graphics, 4 remaining).
That means on the X58, and having two PCIe 2.0 x16 lanes going, you have 4 lanes leftover which is 2GB/s of bandwidth for anything else. (Remember the ICH10R has 2GB/s through DMI).
On the P55, in contrast, you have 16 lanes from the processor (GPU) + DMI (2GB/s) to the P55. Meaning, even if you replaced the P55 with a SATA 3 monster, you aren't getting more than 2GB/s. Also, if you added PCIe drives off the CPU lanes, your GPU now gets x8 bandwidth since you are splitting the x16 channel.
I think a big reason we still have cabled drives is that PCB costs would be fairly high if you moved to PCIe drives. For those with one drive, it isn't an issue, but for those that want to have multiple drives, you quickly need more slots which means expanding board size. SATA/SAS connectors are small!!!