I have some litecoins, but I don't have the resources to throw into serious bitcoin mining with ASICs. I don't mind litecoin mining on my home machines now especally since it's getting colder.
I think most of us here view it the same way as investing: it takes a lot of funds to turn a profit, consumes time to stay abreast of developments, and is potentially (or very in the case of altcoins) speculative. Then there's the input cost (name electricity).
I'm especially wary of altcoins right now because they are more susceptible to manipulation than larger, more stable "real" currencies, but they are becoming more stable and balanced as the markets mature and governments and law enforcement start to accommodate them and treat them like hard currency.
Whoever though up of the "mining" metaphor had it exactly right. Just isn't worth it for everyone to enter the game.