Just picked up a Core m3 SP4 for work because Microcenter was selling em for $699. I swear you wouldn't be able to tell this had a Core M in it, it's that smooth/fast. Also, best part, 100% fanless! No noise or crazy heat. So far, I'm in love with my SP4. Got a SP4 type cover coming it. Borrowed a co-workers SP3 type cover, did not like. KB flexed pretty bad and the keys are not spaced at all.
I will say that 1) IMHO my iPad Air2 is still a better tablet/media consumption device, so this won't be replacing it, and 2) it should've came with 8 GB RAM. Battery life has been pretty solid so far. Averaging 7-8 hours. I'm VERY happy with it, especially for the price I paid.
Edit: You don't need the regular Surface 4 if you can get an SP4 at this price point with the pen. I think the main reason why this tablet is so smooth is the lightning fast NVMe storage.