It really only seems good at the start because the library's smaller. However, what I started to realize is that I only play one or two games at a time, so who cares if I only own 3 or 4 at first? I mean, I own about 40 games on my 360 (and that's just the disc ones, not counting the 15-20 digital ones from Games with Gold or the dozen or so XBLA titles), yet all I was playing before I got my One was Halo 4. Before that, I was only playing Black Ops II. Before that, it was one of those, with the occasional Borderlands 2 mixed in.
Where backwards compatibility would be useful is those whose friends are slow to adopt. I only got my One because I got it as a graduation gift, because I didn't want it enough (or have the money) to buy it right now, so I would have probably waited until the end of the year. That meant I'd never really get to play games with my friend who got a One. Now that I have a One, I can't play games with my cousin or brother-in-law online, as they're on the 360. Having backwards compatibility, and still being able to play Halo or Black Ops with them would be nice, but it's not the biggest deal. If they get the new Xbox, then we'll just play the new games, and the backwards compatibility becomes useless.
Really, it's a great starter idea, but after about a year, it's now a mostly-meaningless feature.