I read an interesting article a while back. It suggested, among other things, that the reason why 16:9 is a popular aspect ratio (math factoring reasons aside) is because the average aspect ratio of human vision is roughly 16:9. (Differs from person to person of course) Once I heard that I started noticing little things here and there, and frankly that sounds believable to me. At the very least, 16:9 is closer than 4:3.
Now, as for watching things with letterbox bars, what it comes down to is whether or not you're getting the whole picture. 4:3, 16:9, 2.35:1, doesn't make any difference to me. All I care about is getting the most picture; NOT the highest resolution, but the most stuff in frame. (Unless the filmmaker wants it to be at a certain AR, e.g. The Truman Show was intended to be shown 4:3, being that it's about TV) For example, a number of movies shot back in the late 70s and early 80s were filmed in standard 4:3, but then cropped for final big-screen presentation to 1.85:1. For those flicks the fullscreen DVDs actually give you more picture; there was even a lawsuit over it, as the widescreen editions of a number of those films specifically advertised that they offered a "wider, more complete picture", when the opposite was true.
Personally, I like to straddle the fence by watching most of my movies on my computer, which is equipped with two monitors: one running 1600x1200, (4:3) and one running 1680x1050. (16:10) Fullscreen stuff goes on the fullscreen monitor, widescreen stuff goes on the widescreen monitor. But that's me.