You guys are forgetting one major area where wear takes place when you downshift, and that's in the entire exhaust system, mainly the muffler.
The increased backpressure that the engine is making when it's revving high with no load is actually pretty bad on the muffler/cat/resonator(if your car has one).
For me, I shift down when slowing down, but recently I've been doing that less as I know that $10 brake pads are cheaper to replace than a $80 muffler, even though the brakes will need changed about twice as many times as the muffler as a trade off.