Live Photos format: .JPG mixed with 960x720 .MOV at 12-15 fps --- Now HEIC image + 1080p HEVC h.265

[Eug]

http://www.macrumors.com/2015/09/22/iphone6s-live-photos-file-details/

Because of the way Live Photos work, combining a MOV file with a JPG file, the full Live Photo experience will only be viewable on iOS devices that run iOS 9, the Apple Watch with watchOS 2, and Macs running OS X El Capitan. Sending a Live Photo by email or directly to non-supported devices, the Live Photo will be stripped of the MOV component and sent as a regular JPG.

However, when importing Live Photos to your Mac in Yosemite's Photos app, they're imported as a separate JPG and MOV file. When TechCrunch's Matthew Panzarino sent a Live Photo to our iPhone 6 running iOS 9, the Live Photo was viewable (with motion) in iOS 9. When imported into Yosemite's Photos, the image was split into the JPG and MOV components. Importing into Photos on OS X El Capitan generates a single photo in the library, and double clicking it plays the video portion of the Live Photo.

---

The videos are 4:3 960x720, which is a little bit disappointing actually. We had heard comments that they were based on JPG, which obviously they aren't (which makes sense given the limited storage used). I guess the Live Photos are JPG, but the actual videos are not. I still don't know the actual video format, but assuming they're just H.264 I would have hoped for say 4:3 1440x1080. Actually, I'd prefer H.265 1440x1080, but that would cause problems with older iOS 9 compatible devices.

I guess I'll be updating to El Capitan sooner rather than later, to maintain Mac compatibility with Live Photos. I'll have to check out the VPN compatibility soon, as that's the only thing that's holding me back from updating on my MacBook Pro and my iMac.

Also, I hope Live Photos survives iMessaging. It would make sense that they would not work as texts, but it would be a shame if iMessaging didn't maintain the integrity of Live Photos. I wonder if iMessage knows what OS the recipient is on. I would assume so, but I'm not sure. All of our iOS devices in active use are currently on iOS 9.1 beta, so that's OK, and my wife's iPhone 6s will be too. The problem is that we have iMessage active with some Macs too, and those are still Yosemite.

---

EDIT: It's now up to 29.97 fps (iOS 10.2), and 1080p. See my post here:

https://forums.anandtech.com/thread...fps-at-1440x1080.2448989/page-3#post-38805342

---

EDIT: As of June 2017, my prayers have been answered. They are now 1440x1080 HEVC on certain devices, albeit paired with HEIC images now instead of JPEG. There is also the option of leaving it has 1440x1080 h.264 with JPEG though.

This came nearly 2 years after I first posted this message, but better late than never!

 

[Rakehellion]

That file size is pretty big for h264. It might be motion JPEG.

 

[Eug]

If motion JPEG that makes sense, and is along the lines of what we were postulating before. It would also mean each image is effectively independent, Which would be good.

 

[Rakehellion]

h264 has high overhead and can't be played backwards. I guess we'll find out tomorrow.

 

[mmntech]

I was sort of expecting the videos would be a higher resolution than that. Bit rate is pretty high though. Works out to be 9 mbits/s. Plus it's only shot at half the frame rate of conventional video. So it could still be Motion JPEG to ensure the best quality, with the lower res used to save on space. My money is on h.264 though.

I'm sure there will be some sort of app that comes out that will allow you to share these with non-supported devices.

 

[Midwayman]

I'm sure there will be some sort of app that comes out that will allow you to share these with non-supported devices.

Facebook has said they will support it by the end of the year.

 

[Eug]

As expected, Live Photos survives iMessaging but does not survive emailing. I iMessaged my iPhone 5s and the video stays intact. Just press and hold to see the video. Once it has been received in your iMessage thread, you can also share it to your photo gallery too. I'm a little disappointed though that there is no option to somehow include the video in an email. You can't even send it as separate JPEG and MOV files.

This is a pretty good reason IMO for iPad 2 / iPad mini / iPhone 4S users to upgrade to iOS 9.

This is also a pretty good reason (besides speed) to finally move my mother-in-law off the iPhone 4.

Note that if you take a burst set of pix, there is no Live Photo. It's just a regular burst set.

 

[Eug]

Heh. My wife took my kid to a birthday party and the other moms asked her about her new iPhone and Live Photos. She took a bunch, but couldn't remember how to play them. She said she felt like a complete tool.

So, while they're easy to use once you get the hang of it, it's not like everyone will know how to use them right off the bat. Further, since non-Live photos and burst photos don't always have the label visible, it's not always easy to know if it's a live photo or not. The UI needs to be tweaked a touch to make it simpler to navigate.

However, for me, my main beef with Live Photos is the format implementation. It's embedded H.264 at 960x720 and 12 fps.

My preference would be H.265 at 1440x1080 at 24 fps. iPhone 7? By the time the iPhone 7 comes out, all iPhones sold by Apple would be H.265 compatible, assuming Apple decides to keep only the previous 2 generations around (iPhone 6, iPhone 6s, and iPhone 7) for sale. It gets a little more complex with the iPads though.

BTW, at least on my iMac Core i7 it's easy to play the files backwards. Doesn't break a sweat, at least with these mid-resolution files.

 

[Rakehellion]

My preference would be H.265 at 1440x1080 at 24 fps.

Then the Live Photo would be three times bigger than the photo itself. Might as well just take a video.

I just took a Live Photo selfie and the photo was 874k while the video was 2.5MB. I have tons of space so I probably won't disable this feature yet, but damn.

 

[Eug]

Then the Live Photo would be three times bigger than the photo itself. Might as well just take a video.

I just took a Live Photo selfie and the photo was 874k while the video was 2.5MB. I have tons of space so I probably won't disable this feature yet, but damn.

Not really. You could probably get away with a video being say twice as large, considering H.265 is almost twice as efficient as H.264. 6 Mbps would likely be fine.

This might actually be acceptable if the iPhone 7 gets 32 GB base storage.

If it really has to be 3X as large for 1440x1080 at 24 fps, then I'd still recommend 24 fps, but go with 960x720. The 24 fps is more important than the resolution, since 720p is actually half decent.

If my Live Photos were 5-7 MB each, I'd have no problem with that. It's nice to have the video, but I don't want my stuff to be primarily videos. I want real photos, and having the short video attached to is a nice bonus.

Overall I think this format is an evolving one. This first iteration is just that, a first iteration. It's going to get better with time.

 

[Rakehellion]

What are you doing with Live Photo that you need so many pixels? It's like people arguing that the iPhone needs a 16 megapixel camera when 99% of those photos are just going to Instagram anyway.

 

[Eug]

What are you doing with Live Photo that you need so many pixels? It's like people arguing that the iPhone needs a 16 megapixel camera when 99% of those photos are just going to Instagram anyway.

What do you mean? It's not as if 1080p is asking a lot. Since when is 1440x1080 "so many pixels"?

If you look at these files on say an iMac, the quality is just so-so for the video. It'd likely look significantly better with 1080p, at least if they can manage to keep the encode quality decent, and the 24 fps would go a long way to make these feel less gimmicky. For that I'd be more than willing to have the video portion double in size, which should be possible with h.265. With h.264 it'd be more like quadruple the size.

Remember also that you can turn off the feature if you don't want it. Furthermore, there is nothing stopping Apple from having two tiers of Live Photos video quality.

I can see Phil going on stage in Sept. 2016 to promote Live Photos 2 exclusive to the iPhone 7 and iPhone 7 Plus, allowing for higher resolution video at twice the frame rate to achieve a more "immersive" feel for Live Photos. However for those who want to save space, they can either set the phone to Live Photos 1 mode, or else turn it off completely.

BTW, Photos in Yosemite separates these out as .m4v files, not .mov files. Yes, it doesn't matter for Macs, but occasionally other equipment doesn't know what m4v means.

 

[Rakehellion]

What do you mean? It's not as if 1080p is asking a lot. Since when is 1440x1080 "so many pixels"?

If you look at these files on say an iMac, the quality is just so-so for the video. It'd likely look significantly better with 1080p, at least if they can manage to keep the encode quality decent, and the 24 fps would go a long way to make these feel less gimmicky. For that I'd be more than willing to have the video portion double in size, which should be possible with h.265. With h.264 it'd be more like quadruple the size.

Remember also that you can turn off the feature if you don't want it. Furthermore, there is nothing stopping Apple from having two tiers of Live Photos video quality.

I can see Phil going on stage in Sept. 2016 to promote Live Photos 2 exclusive to the iPhone 7 and iPhone 7 Plus, allowing for higher resolution video at twice the frame rate to achieve a more "immersive" feel for Live Photos. However for those who want to save space, they can either set the phone to Live Photos 1 mode, or else turn it off completely.

BTW, Photos in Yosemite separates these out as .m4v files, not .mov files. Yes, it doesn't matter for Macs, but occasionally other equipment doesn't know what m4v means.

You completely dodged the question: Why is 960x720 not good enough?

And really, I don't see a lot of people on board for quadrupling the size of every photo on disk for what's no more than a gimmick.

 

[Eug]

I answered the question already. On my iMac the 720p video just looks so-so. It's right in the post you quoted above.

And like I said, with h.265, the video portion of the file only needs to double most likely, not quadruple. Quadrupling it would be required with h.264, and I agree that's not realistic for 2016.

 

[Rakehellion]

What you're neglecting is that this is metadata piggybacking on top of an image file. If you want a movie, just take a movie.

 

[Eug]

What you're neglecting is that this is metadata piggybacking on top of an image file. If you want a movie, just take a movie.

I don't want a movie. I want a better quality Live Photo. I don't see why that is so hard to understand.

BTW, this is like the arguments 10 years ago that 720p is all we needed for video.

 

[Rakehellion]

I don't want a movie. I want a better quality Live Photo. I don't see why that is so hard to understand.

BTW, this is like the arguments 10 years ago that 720p is all we needed for video.

This isn't 10 years ago. It also isn't 10 years from now. High quality Live Photos is mostly a bad thing.

There will probably be third party apps with improved Live Photo capture.

 

[Eug]

This isn't 10 years ago. It also isn't 10 years from now.

Ten years from now? This is a feature I want TODAY, and the technology is already here. In fact, the technology was here ten years ago, ironically.

Remember also, if you don't like Live Photos sizes, you can simply turn off the feature, although what I envision is a two tier Live Photos setup, with "high quality" 24 fps 1080p and a "standard quality" 12 fps 720p. We already have multiple video settings on the iPhone, so there shouldn't be anything stopping Apple from having multiple Live Photos settings too.

But since we can't have it now, I'd be happy with it next year with the iPhone 7.

High quality Live Photos is mostly a bad thing.

I guess you don't realize how ludicrous and shortsighted that sounds.

There will probably be third party apps with improved Live Photo capture.

Not a very good solution most likely, even if it does happen.

 

[Childs]

You completely dodged the question: Why is 960x720 not good enough?

And really, I don't see a lot of people on board for quadrupling the size of every photo on disk for what's no more than a gimmick.

Its always better to capture at the highest resolution possible. I have a crap load of photos and movies that I took in the early days of digital cameras and they dont really hold up on modern resolution screens. In 5 years 4K will probably be ubiquitous like 1080p is now.

It might look fine in 5 years, but if given the option I would even take live photos at 4K. Its better to resize down later than resize up. There really is no reason to not make the resolution configurable. If the phone can take 4K video a 4K live photo shouldnt be a problem. Yeah the files will be bigger, but I'd be fine with that. If its configurable everyone is happy.

 

[Eug]

Its always better to capture at the highest resolution possible. I have a crap load of photos and movies that I took in the early days of digital cameras and they dont really hold up on modern resolution screens. In 5 years 4K will probably be ubiquitous like 1080p is now.

It might look fine in 5 years, but if given the option I would even take live photos at 4K. Its better to resize down later than resize up. There really is no reason to not make the resolution configurable. If the phone can take 4K video a 4K live photo shouldnt be a problem. Yeah the files will be bigger, but I'd be fine with that. If its configurable everyone is happy.

Yeah, in terms of detail, the video part of the Live Photos look fine on the iPhone 6s and my iPhone 5s when looking at the entire image. When you zoom in though, the reduction in quality shows, although most people won't be doing that too often.

On an iPad Air 2, you start to notice the reduction in quality even for just regular full-image viewing, but for the most part it's OK (unless you zoom in).

However, on my iMac, it really does look mediocre.

For all of the above though, the lack of a 24 fps frame rate really is a problem. 12 fps is simply too low.

So again, a good compromise for the iPhone 7 would be to have a default setting of "Live Photos 1" which is the current 960x720p @ 12 fps, with an option of "Live Photos 2" new to 2016 with 1440x1080p @ 24 fps. And then in 2018 with the iPhone 8, Live Photos 3, with 2880x2160p @ 24 fps.

 

[Rakehellion]

Ten years from now? This is a feature I want TODAY, and the technology is already here. In fact, the technology was here ten years ago, ironically.

Bottom line is, most people don't want their photos to be 12MB a pop. Hell, this is part of the reason they were so reluctant to upgrade their camera from 8 megapixels.

 

[Rakehellion]

I guess you don't realize how ludicrous and shortsighted that sounds.

It isn't short sighted. I'm thinking of 4 months from now when my 64GB phone is full of nothing but metadata. I gave you the example of where the current system already quadruples the size of front-facing images.

 

[Eug]

It isn't short sighted. I'm thinking of 4 months from now when my 64GB phone is full of nothing but metadata. I gave you the example of where the current system already quadruples the size of front-facing images.

And I already told you that h.265 chops off a huge chunk of those file sizes, as compared to h.264. Realtime hardware h.265 encoding is already supported by the iPhone 6 from 2014. So it's not as if it would be a huge stretch to think Apple might make h.265 the basis of Live Photos 2 with the iPhone 7 in 2016.

 

[Rakehellion]

And I already told you that h.265 chops off a huge chunk of those file sizes, as compared to h.264. Realtime hardware h.265 encoding is already supported by the iPhone 6 from 2014. So it's not as if it would be a huge stretch to think Apple might make h.265 the basis of Live Photos 2 with the iPhone 7 in 2016.

h265 chops off about 30% of the file size, which is insignificant considering you just increased file size by 400%. Also, hardware decompression doesn't exist on older devices.

 

[Eug]

h265 chops off about 30% of the file size, which is insignificant considering you just increased file size by 400%. Also, hardware decompression doesn't exist on older devices.

Your post here is at best very misleading.

1. One reason it is misleading is because of the way you used the percentages. Even if your 30% number is correct, it's not 400% - 30% = 370%. It's 400% - 30%(400%) = 400% - 120% = 280%.

2. This only applies to the MOV/m2v portion of the file. The JPEG portion of the file remains the same size.

Let's take a real world example from my kid's classmate's birthday party. When I imported this file into Photos in Yosemite 10.10.5, it was split into two files, an m2v file of 1.1 MB**, and a JPG file of 2.2 MB. That makes the combined file size 3.3 MB.

So, if the video file quadrupled in size (if it were h.264 1080p at 24 fps), but then shrank by 30% from that (because of h.265), that means the new video file size would be 2.8*1.1 MB = 3.1 MB

Add the 3.1 MB m2v to the original 2.2 JPG file, and you get a total file size of 5.3 MB. Overall file size is 5.3 MB/3.3 MB = 1.61X compared to a "regular" iPhone 6s Live Photo.

Now even if we're generous and say another Eug's-Live-Photo-2 might be 2X a current Live Photo or even 2.5X, that doesn't seem bad to me at all, for vastly improved video. I'd have no problem shooting a kids' birthday party full of Eug's Live Photo 2 shots, since it's likely most of these would be in the 5-7 MB range, with all of them well under 10 MB. Note though that I'd probably buy the 128 GB version of the iPhone 7. Given that I'd probably have > 80 GB free on that thing after syncing, that represents at least over 10000 photos using Eug's Live Photos 2 standard. So, I don't see the problem here. BTW, as I said earlier, I would expect, or at least hope, that the iPhone 7 comes with 32 GB minimum.

3. However, that 30% compression savings estimate from h.265 you state is a conservative estimate. I've read the savings from h.265 can be anywhere from about 30% to about 50% when compared to h.264, but with 40% being a reasonable average for similar quality. For example this paper states 39%:

http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf

So, armed with all this information, I'd say that improved video quality for Live Photos is an inevitability. Unless Live Photos dies a miserable death altogether, it will undoubtedly improve, and sooner rather than later. I am hoping for 2016, although I wouldn't be completely surprised if it was 2017. I would be very surprised though if Apple waited until 2018 to improve the video quality of Live Photos.

---

**Note: As mentioned, when I imported the file into Yosemite I got a 2.2 MB JPG file and a 1.1 MB m2v file (with audio intact). However, when I upgraded to El Capitan, the database was updated and on screen the photo and video portion were combined into one single file. However, if then re-exported that same file to the desktop, what the program spit out was two separate files again, but this time a 2.2 MB JPG file and a 1.5 MB MOV file. I don't know what the 0.4 MB difference comes from, since the video looks and sounds identical.

---

tl;dr version:

A 1080p h.265 24 fps Live Photo would probably usually be about 2X the size of a current 720p h.264 12 fps Live Photo, NOT 4X the size.