Well it's easy to say that now, but we don't know where we'll be in two years. Smart phones have a long ways to go to even match yesterday's desktop performance.
Well software only uses so much resources unless you expect them to keep bloating it up.
The issue with Android was that it wasn't optimized and that 600mhz, 1ghz phones were not sufficient. That's why when we jumped to dual core it was still like oh my god my phone is usable now. The 4=>4s change wasn't that big because iOS is more resource efficient.
But even with Android I think we're at a place where quad is nice but it isn't going to be that big of a difference. You can see it being faster, but it's not the difference between usable and slow as hell.
Thus even on my SGS2, I don't feel that far behind my gf's SGS3. She feels my phone is rock solid and fast as hell at times. Maybe because I'm rocking CM9 and a custom kernel (can't wait til CM10 gets the hwcomposer fixed and mem leak resolved). I could upgrade but at that point I'm upgrading for the larger screen and nothing more.