My first smartphone was the S2. I loved that friggin thing, had so much fun with it until my clumsy ass broke it. After the S2 I had the Note i717, and I have now upgraded to the S4. I highly recommend the S4, unless you are set on trying a different brand.
So far I have found the camera to be amazing, noticeably better than the S2 and Note, which were already awesome as far as I'm concerned. I mean, it's a phone. A phone! Here is one of the first pics I took with my S4:
http://img42.imageshack.us/img42/2679/sookierawr.jpg. I don't know why, but I was just kind of blown away by how it looked.
It sucks in low light like every other cell phone camera I've seen, but the 'night mode' that it does have is vastly improved from what I was used to, so it's still pretty cool.
With a quad core and 2 gigs on jelly bean it's pretty zippy, still not 100% lag-free but a huge improvement over my S2 and Note regardless of which ROM I had on there. Honestly the damn thing flies. Finally I don't actually have the urge to put a custom ROM.
I've posted many times about how the Note can be used one-handed and fits fine in my pockets, and size never an issue - i stand by all of that, but in comparison the S4 is very light, very thin, and even easier to use during one handed operation (obviously). It has a very comfortable and satisfying feel in the hand.
Battery life is substantially better than the Note - and I was more or less pleased with its performance. The Note II may be better than the S4 though in that regard. The S4 has a better screen than the original Note - 1080p, a bit brighter and a bit more crisp, but losing the size of the Note takes some getting used to. Horizontal web browsing on 5" is ... meh, almost pointless when you're used to the Note.
So I would say if you spend a lot of time with media consumption, go for the Note II as the size will be much better for watching videos, playing games, or anything that the bigger screen might provide a more pleasurable experience for.
If you don't spend a considerable amount of time consuming media or doing tons of browsing horizontally, then the S4 might be a better fit overall.
If you want to explore other large devices like the Note, I think LG has a nice one, the Optimus G Pro (I think). If you want to try something like the S4 but not Samsung, maybe explore the HTC One. A buddy has one and it's really, really nice too!