Surfing the net and occasional OS lags are the main reasons why I gave my iPhone 4 to my wife. Ignoring LTE for the moment, it just renders slowly in Safari. But as mentioned, even in iOS 6, it had occasional OS lags of a few seconds. The lags are still infrequent in iOS 7, but happen a bit more often than in iOS 6. A few seconds here and there aren't going to kill me, but it's still damn annoying.
For the little I've tried the 5, those lags are gone. Safari rendering is also much, much faster.
However, my wife is like you. She thinks the iPhone 4 is fine in iOS 6, and worlds better than the crappy feature phone she used to have. She also thinks the iPhone 4 is fine in iOS 7 too. Her only complaint is having to learn all the UI changes in iOS 7, and the colour scheme. She particularly doesn't like the white background for text messages and to be honest, I'm not a big fan of that either. However, she has no complaints about the speed.
P.S. With regards to listening to music. One thing I noticed was that if I was playing music in the background, the iPhone 4 in iOS 6 would lag more than it usually did. I don't know for the iPhone 5, but I'm told for the iPhone 5 it's not a significant issue, which makes sense since it's much faster clocked and is dual-core. This was not a major issue in iOS 5 on the iPhone 4 BTW, IIRC. It became more noticeable in iOS 6. I have not tried that in iOS 7.