Basically it comes down to a matter of focus, and inertia. The folks who know the most about cameras (and using them) are the professional photographers and extreme hobbyists.
Bodies come and go, but lenses are forever. The number of lenses available for Nikon and Canon is much larger than for any other system. I don't just mean how many different lens types are manufactured; I mean how many are out there in the world, and how easy it is to get your hands on them. I don't think I've ever seen somebody in a photography forum with a serious collection of high-end Sony or Pentax glass. Not even enough to match my meager Canon collection, which is just 3 relatively cheap L zooms and 3 non-L primes. Let alone the dudes with multiple f/2.8 telephotos, multiple f/1.2 and f/1.4 L primes, etc. (same with Nikon although I am not as familiar with the exact lenses) These are all lens collections built up over years of work in the case of pros, and years of saving in the case of hobbyists like myself.
The used market is an important consideration, as is the new market. How many stores carry Canon and Nikon? Best Buy carries a couple of "L" lenses, last time I checked, and a couple of pricey Nikon lenses too. Dedicated local photo shops often have almost the full lineup of Canon and Nikon in stock to rent or purchase. That's what the pros use, so that's what they stock. So if some dude with a Rebel walks into the store then they can look and try out all the Canon lenses they have.
But even that extremely important consideration also just a part of the equation. How easy is it to get a replacement lens if yours goes bad, or to send it in to get serviced with quick turnaround. How many of your peers are using each system, who among your mentors uses each system. You will find almost all serious pros in the most widespread/popular forms of photography (event photography, sports photography) using Canikon, and it just filters down from there.
That's what the pros use, they give their advice to the extreme hobbyists, the extreme hobbyists give their advice to the lesser hobbyists, the lesser hobbyists give their advice to the soccer moms.
IMO it's very much a trickle-down effect of a significant number of pros being tightly locked-in to Canon and Nikon. It affects many parts of the market. On a personal note, I have personally bought much of my gear from pros and former pros, ranging from newspaper beat photographers to studio portrait photographers to semi-retired dudes who somehow make a living taking photos for CD covers and other arts-related projects.
People who make their living with their cameras are pretty resistant to change. There's got to be a major benefit to switching, because all kinds of subtle things (like the position of knobs and switches) can screw with your head. A completely-different-from-the-ground-up AF system is not something that a sports or wedding photographer wants to mess with. There will be too many misfocussed shots during the learning curve. And "Almost zero lag" is not good enough for these people, it must be zero. There is not much room for error. Yes, EVFs are getting better. No, they are not good enough, and they may never be the equal of TTL viewfinders. The human eye is a sensitive instrument, and the immediacy of the image in the viewfinder of a DSLR is a wonderful thing. Absolutely zero lag is technically impossible to achieve, so it remains to be seen whether "fast enough" can be reached.
But, you argue, most everyday Joes aren't sports or event photographers. Go back and read what I wrote above. What the sports and event photographers use, determines pretty strongly what the everyday Joes are going to buy for themselves.
Now on to some specific points....
1) Lenses that have ring-type USM (HSM, SWM, etc.) have their motors built into the lens by necessity. They are not conventional "motors" at all, with rotating gears and such. The rotor and stator form a ring around the barrel of the lens. There is no way to implement ring-type USM with a motor in-body. Almost all of Canon's lenses over $300 or so use Ring USM. (It was more or less invented by Canon, and the adoption of the same technology by every other camera brand for its high-end lenses indicates the superiority of the technology. The speed and silence of a Ring USM lens is incredible.) Adding a motor in the camera body for the small proportion of lenses that could make use of them wouldn't make much sense to Canon, would it? Of course the bulk of lenses sold are cheap lenses that use standard motors, but it's a safe bet that the higher-end bodies will be used almost exclusively with USM lenses.
Furthermore (beyond the USM discussion, and assuming standard motors) moving the focus motors in-lens was seen as a positive innovation at the time. Building the motor inside a camera body is a one-size-fits-all solution. Whether it's moving the light elements in a kit lens or hefty elements in a 28-70mm f/2.8, it's the same motor. It's not adapted to the lens at hand. When it's built into a large lens, the motor can be large. When it's built into a small lens, the motor can be small.
2) I agree that sensor-shift is a great thing, and I wish that Canon would adopt it as a supplement to in-lens Image Stabilization. However, from what I've read, the best in-lens IS implementations can still beat out the best sensor-shift implementations. However, the "works with any lens, even primes" aspect of sensor-shift is hard to beat.
3) and 4) SLTs are new territory. I don't think you're going to see the pros moving to them anytime soon (except for videographers, for whom an expensive DSLR+lenses is still probably quite a bit cheaper than the $100k+ pro stuff they're used to using). In some cases, yes, they make sense. I agree that the technology is exciting and it is opening new possibilities. (I would still not recommend shooting into the sun though, it can damage your sensor just the same as it can damage your eyes!)
Facebook
Twitter