The implemenentation is DRASTICALLY worse. The note 4 has similar shutter response to the iphone 5s. The 6 is much much better at moving objects.
If OP wants to sacrifice photo quality that is one thing. He says he does not. I don't know how you can reccomend an android phone to somebody that walks in saying "shutter speed, auto focus, performance" and say Note 4. Note 4 takes more detailed still pics in good light, that's about all it does better than the 6 plus.
Take all the reviews, and average them. The iphone is a vastly superior phone for taking pics and everyone here knows it.
Real life ownership of both phones jives with reviews in my opinion. The iPhone 6 has a fantastic camera, but so does the Note 4. There is no substantial difference in daily use.
Note 4 - superior daylight photos (detail) and better front camera
IPhone 6 - simpler, more intuitive camera app
Equal in real world usage - low light and focus speed.
I've had great success in using both phones in our family in all mixed conditions. If you really had to pick one, I'd say the iPhone still focuses a tiny bit faster while low light pics are better in the Note 4, but again real world usage is basically the same.
You'll be very happy with either camera.