Context-Aware Encoding: Improve Video Quality and Cut Costs

As streaming services scale up to meet the demand for more content across more screens, the storage and bandwidth costs of delivering video to those screens rises.

Efficiently delivering high-quality video at scale to a vast range of devices is central to what we do at Brightcove. Our Context-Aware Encoding (CAE) feature can drastically reduce the cost of storing and streaming video, while improving playback quality for viewers.

With CAE, the compression settings are tuned for each video using advanced content analysis algorithms. In most cases, this results in smaller video files (and smaller CDN bills) while maintaining visual quality.

Adaptive Bitrate Streaming - How We Got Here

Most video delivered over the internet today uses Adaptive Bitrate (ABR) streaming technologies, such as HLS and MPEG-DASH, to optimize video playback.

An ABR stream contains multiple copies of the same video, called “renditions,” which are encoded at different resolutions and bitrates. When a user presses the play button, the player receives a manifest that lists the renditions available for playback. The player chooses the appropriate rendition to play based on several factors, including the bandwidth currently available, buffer fullness, and the size of the playback window. As these factors change during the course of playback, the player can switch up to a higher-quality rendition or down to a lower-quality one, ensuring that the viewer gets the best possible video quality with minimal buffering.

Streaming services generally create a single encoding configuration for all of their content—a predetermined set of ABR renditions, often called a “ladder,” that is used to encode every piece of content. Deciding on the list of resolutions and bitrates to put into an ABR ladder is an inexact science. In some cases, ABR ladders can be tuned to a specific use case. Animation, for example, can be encoded at lower bitrates as the content is generally less complex.

The Problem With Static ABR

Reaching a wide audience forces streamers into a one-size-fits-all scenario, using a single bitrate ladder to encode a variety of content types for a variety of end-user devices. In most cases, a single ABR ladder is being used to encode HD sports content for TV playback as well as cartoons for playback on phones.

The problem? Using a single bitrate ladder often results in inconsistent quality between different types of content. High-complexity video such as sports content requires more bits to achieve an acceptable viewing experience than animation. For example, if you skew your ABR ladder towards higher bitrates to ensure your sports content looks great, you’re wasting storage and bandwidth when using that same profile to encode animated content. That wasted storage and bandwidth costs real money.

Optimizing ABR Streams With Context-Aware Encoding

That’s where Brightcove’s Context-Aware Encoding technology comes in. Instead of using one ABR ladder for all content, CAE analyzes each source video and intelligently builds a custom bitrate ladder for each piece of content. In addition, CAE takes into account constraints associated with the delivery network and device being used to view the content. It decides how many renditions are needed and what resolutions and bitrates to use for each, while still maintaining a consistent level of quality across all titles. This results in massive savings on storage and bandwidth costs, while improving the playback experience for users.

The chart below illustrates a typical static ABR ladder alongside an example of a context-aware ladder that was generated for a typical news clip.

CAE was able to deliver the same quality as the static ABR ladder using half as many renditions, and using lower bitrates or higher resolutions for each rendition. This has a positive impact on both playback performance and cost efficiency.

Better Playback Performance at Lower Bitrates

Using the static ABR ladder, a mobile 3G user with 1,000kbps of sustained bandwidth would peak at the 360p rendition at 900kbps. CAE determined that it could create a higher resolution 432p rendition at a lower bitrate - 777kbps. This rendition uses 13% fewer bits and actually delivers higher resolution and quality, as you can see in the below comparison.

See the difference? The sharper details of the context-aware clip are noticeable in the text at the bottom, as well as the man's hair.

Similarly, a user with an economy-priced residential internet plan supplying 3,000kbps of sustained bandwidth would top out at the 720p rendition under the static ABR ladder, while they would be able to stream at full 1080p using CAE, delivering a Full HD experience.

Reduced Storage and Bandwidth Costs

Encoding video to all eight renditions of the static ABR ladder generates a total of 14,750kbps of data. For a one minute video, that’s 885MB. The context-aware ladder on the right generates only 4,950kbps. So for the same one minute video, that’s only 297MB, which represents  a savings of 66% in storage costs.

Since the bandwidth used depends on the number of viewers and the duration of video viewed, bandwidth savings can be even higher than those for storage. Overall, we anticipate up to 50% savings on storage and bandwidth costs. By adjusting the bitrates of the renditions to the lowest rate necessary to maintain visual quality, viewers will generally consume less bandwidth during each viewing session, providing additional savings on bandwidth costs.