I traded my S4 GPe w/ my co-worker's new N5 yesterday just to see if I liked the N5 better. Didn't see any difference in performance. The screen on the N5 was noticeably brighter than the S4 on max brightness but the colors seemed slightly washed out compared to the AMOLED (I prefer the over saturation on AMOLED). N5 obviously felt more solid but I wasn't sure if I liked the slight increase in thickness and weight.
When I had the GNex/N4, I thought the soft buttons was the best thing in the world for an Android phone. With that being said, I've come to realize that I actually prefer hard buttons on the S4 for some reason. It just feels more "right".. Can't really explain

Just a personal preference I guess.
N5 is a very solid phone for the price. I would probably buy it over the GPe right now if I needed a phone. I thought I'd feel more guilty for dropping ~$700 for the GPe after holding the N5. Surprisingly, I don't feel like I got ripped off w/ the GPe in any way.