How to Encode Video for Mobile Use

There are hundreds of mobile devices out there, and it's basically impossible to support all of them. But the good news is that mobile devices are getting better.

Modern smartphones can actually play high-quality video, and smartphone use is increasing. That's not to say that 3GP is over, or that everyone has a smartphone. But smartphone use is growing, and not surprisingly, smartphone users are more likely to watch video on their phones.

So if you want to support 90%+ of mobile devices, you need at least two video types: 3GP + MPEG-4 for less sophisticated devices and h.264 + MP4 for smartphones. That's good news, really. One output video can cover all of your smartphone users: iPhone/iPad/iPod, Android, and (for the most part) Blackberry. You can include PSP, PS3, and Xbox 360 for good measure.

Of course, while one universal smartphone output can take care of most smartphone users, you can do better with multiple mobile outputs. For example, the iPad has a native resolution of 1024x768—five times higher than the 480x320 on earlier iPhones. So if you encode your video at 480x320, you'll be missing out on the near high-def capabilities of the iPad.

Fortunately, you can target mobile devices well using a handful of standard encoding profiles. Start with the Universal Smartphone Profile for wide compatibility. Then, add in an Advanced Smartphone Profile version for the more advanced devices and round out your mobile list with a legacy profile for widest compatibility (either our Legacy Smartphone Profile below or even a 3GP video for even wider compatibility).

Note that the following defaults are the starting point for these profiles. Brightcove Zencoder uses these settings by default, but you can replicate them easily enough in whatever encoding tool you're using.

Defaults

• Video: h.264, Level 3.0
• Baseline profile audio: AAC, 1-2 channels

1. Universal Smartphone Profile

This is a great starting profile for wide compatibility with modern smartphones. Plays on just about everything, though it doesn't take advantage of the higher resolutions and codec complexity possible on the newest crop of devices.

Plays On

• iOS: iPhone, iPad, Apple TV, iPod Touch, iPod Classic, iPod 5.5G
• Blackberry: Bold 9000, Curve 8910, 8900, 8520, Pearl 9XXX, Storm, Storm 2, Torch, Tour, Bold 9650 + 9700
• Android: All (?)
• Other: PSP (3.30+), PS3, Xbox 360, web

Doesn't Play On

• iPod 5G
• PSP (pre-3.30)
• Blackberry Curve 9330, 9300, 8530, 83XX
• Pearl 8XXX, 88XX

Settings

Defaults, plus:

• Audio_bitrate: 128 (or less)
• Audio_sample_rate: 44100 (or less)
• Size: 480x320
• Max_frame_rate: 30
• Video_bitrate: 1500 (or less)

1b. Universal Smartphone Profile B: Higher Resolution

This profile plays better on iPhone 4g, iPad, Apple TV, new iPod Touch, Droid, PS3, and Xbox, by increasing the video resolution. The extra pixels are wasted on older iPhones though, and make for a video that won't play on Blackberry and some Android phones.

Plays On

Everything above, minus Blackberry and maybe weaker Android devices.

Settings

Universal Smartphone Profile (above), plus:

• Size: 640x480

2. Advanced Smartphone Profile

Newer iOS devices allow higher resolutions and higher encoding complexity (which means better compression). In particular, iPad and Apple TV users shouldn't have to watch 480x320 video on their beautiful screens, so it makes sense to provide a higher quality version if you want to provide a good experience to these users.

Plays On

• iOS: iPhone 4G, iPad, Apple TV*, newer iPod Touch
• Android: Nexus One, Droid, maybe others (Note: Some users report trouble with 720p video)
• Other: PS3, web

Doesn't Play On

• iOS: iPod 5G/5.5G/Classic, iPhone 3GS and before, older iPod Touch PSP, old Apple TV*
• Blackberry: all
• Android: others
• Other: PSP, PS3, Xbox 360, web

Settings

Defaults, plus:

• H264_profile: main
• H264_level: 3.1
• Audio_bitrate: 160 (or less)
• Audio_sample_rate: 48000
• Size: 1280x720 (max) or 960x640 (iPhone 4 native)
• Max_frame_rate: 30
• Video_bitrate: 5000 (or less)

*2b. Advanced Smartphone Profile B: with Old Apple TV Compatibility

To support older Apple TV devices, use the Advanced Smartphone Profile setting, plus one of the following.

Settings

Advanced Smartphone Profile (above), plus either one of the following:

• Size: 960x540
• Max_frame_rate: 24

3. Legacy Smartphone Profile

This profile plays on the last major set of H.264-based mobile devices: notably, older iPods and some Blackberries. The tradeoff is significantly smaller video: 320x240, at no more than 768kbps.

Plays On

Everything above, plus:

• iPod 5G, PSP (pre 3.30)
• Blackberry Curve 9330, 9300, 8530, 83XX
• Pearl 8XXX, 88XX

Settings

Defaults, plus:

• Audio_bitrate: 128 (or less)
• Audio_sample_rate: 44100 (or less)
• Size: 320x240
• Max_frame_rate: 30
• Video_bitrate: 768 (or less)
• H264_level: 1.3

4. Legacy 3GP Profile A and B

Finally, a 3GP profile or two will extend support to many remaining mobile devices. Notably, you can use these on most of the same devices supported above under the Legacy Smartphone Profile. So if you're encoding a 3GP video at 320x240, you might not need to encode another H.264 video at 320x240. Note that 3GP video support is still in beta at Zencoder. Finally, note that these videos will look terrible, but that's the cost of supporting 3GP phones.

Plays On

Hard to say. There are thousands of types of 3GP devices, and each one is a little different. Consider these a starting point.

Summary

If you want to create mobile video, start with the Universal Smartphone Profile. For better quality, supplement this with Advanced Smartphone Profile video. For wider compatibility, add a Legacy profile or two using either MP4 or 3GP. It only takes 1-3 profiles to support most mobile devices.

Edits

Older iPhone/iPod devices ask for the "H.264 Baseline Low Complexity" profile. "Low Complexity" isn't an H.264 standard; it actually just means "only 1 reference frame." The jury is out on how much Apple devices really enforce this, but for true compatibility, you should probably use Baseline profile and limit reference frames to 1. You can do this at Zencoder with the new h264_reference_frames setting.

November 23, 2010: A few people have asked about Palm Pre video. The published specs for Palm Pre are very similar to other smartphones:

• 480x320 native resolution (with 640x480 supported)
• H.264, H.263, or MPEG-4 video
• MP3 and AAC audio (along with a few other codecs)

If these specs are accurate and comprehensive, then the Universal and the Legacy profiles above should work on Palm Pre.

January 24, 2011: In order to deliver 3GP video as an RTMP stream it needs to be "hinted." Add "hint": 1 to your API request to enable it.