The Race for Next Generation Video Codec
The race is on for the next generation video codec. For the last 15 years, H.264 has been the unchallenged leader in video compression and streaming and has earned native support from all device and browser manufacturers. However, with the successor of H.264 - HEVC - the market scenario has changed. Not only is the adoption of HEVC slow but HEVC is also facing stiff competition from VP9, a free codec developed by Google.
Comparison of VP9 and HEVC
VP9 supports most of the features found in HEVC. Both the codecs employ similar techniques of compression such as larger block size, transform, intra & inter predictions and CABAC based entropy coding.
Video codec and device support
A device can support multiple codecs but there are two fundamental problems that prevent it. Firstly, supporting multiple codecs forces content providers to encode the same video twice which is both costly and memory inefficient. Secondly, it is troublesome for Original Equipment Manufacturers (OEMs) to provide native support for multiple codecs due to the associated design and manufacturing costs.
It is necessary to have native support for a codec because it enables a browser or a device to play encoded video out of the box without any need for extra software or plugin. In other words, the codec is bundled with the distribution software instead of a user downloading it as an add-in. Bundling such support software is an important factor for ensuring a smooth user experience. The content providers must ensure that the video compression format used by them is playable directly on the end user device to avoid showing annoying messages to the end user about unsupported video format.
A video codec can take the form of a software library (x265 or libvpx) running on CPU or it can be backed by a hardware acceleration unit for faster and efficient compression/decompression. The companies compressing videos using H.264 and the companies providing hardware or software support for decoding these compressed videos must pay a hefty licensing fee to MPEG LA. This is against the fundamental thought process of free software community, particularly open source community. Thus, the communities and service providers developing software for free use are in support of a royalty free codec.
Licensing Fee for H.264
MPEG LA offers an alternative license for the convenience of users enabling them to acquire patent rights necessary for the AVC/H.264 standard from multiple patent holders in a single transaction as an alternative to negotiating licenses with each of them individually. The licensing fee can be paid by Original Equipment Manufacturer (OEM), enterprise or by the end customer. A brief overview of the licensing fee is as follows.
OEM for PC but not a part of OS
OEM for PC OS
Until recently, no one was able to develop a codec with performance comparable to that of H.264. This status-quo was broken when Google released VP9 codec in late 2012 to compete with HEVC, the next generation codec after H.264. VP9 is shipped with Google Chrome and Android and it implies that millions of Google devices are ready for VP9 encoded videos. VP9 is gaining market because it is free.
Making a video codec is certainly not a child’s play and that is why it has taken years for someone to even question the leadership of H.264. Although Google introduced VP9, it is difficult to say with certainty that VP9 will deliver the same level of performance as H.265 without infringing on the H.264/H.265 patents that were carefully crafted to cover all encoding ideas a good compression standard may have. x265 and libvpx are software video codec libraries for H.265 and VP9 respectively. The analysis by Netflix on these video libraries shows that x265 outperforms libvpx for almost all resolutions except at 1080p where the performance gap narrows (or even reverses).
Even if libvpx performs inferior to x265, libvpx is free and open source. On the other hand, HEVC is expensive and full of uncertainties due to licensing issues involving a lot of parties who are trying to collect their share of the pie in the license. Content providers want to steer clear off this war so that their users get the best possible content without interruption. So VP9 becomes the obvious choice.
If VP9 succeeds in emerging as the market leader, all the existing videos would have to be encoded to VP9. If multiple video codecs co-exist, it will be a nightmare for both the end users and the content providers. They would have to maintain two different copies of the same video content and check device compatibility before playback. To bridge this gap, there have been discussions about transcoding the video on the fly to make it compatible with the specification of the end-user device. However, since the networks are not very fast, real time transcoding disrupts the user experience.
Microsoft and Apple are the major players driving codec decisions apart from Google. In the competition between HEVC and VP9, two prominent groups have emerged; the First group supports HEVC exclusively and is backed by Apple and the second group supports VP9 exclusively and is backed by Google. Google Chrome and Mozilla do not provide browser support for HEVC natively. Apple Safari does not support VP9 and thus its customers are unable to play 4K videos on YouTube’s website. In contrast, Microsoft has stayed neutral and has introduced support for both HEVC and VP9 in its Edge browser.
As far as the current market situation is concerned, Apple has already adopted HEVC in its hardware decoder/encoder unit. Other companies may follow the suit. However, Google stands as an exception and is likely not to support HEVC. At least for its major revenue drivers, YouTube and Android, Google will continue to push for making VP9 the codec of choice for the world. If Google fails, it ends up paying heavy license cost for HEVC, which would be unpalatable for Google.
AV1 and alliance for open media
The success of VP9 sparked renewed interest and confidence in the Alliance for open media to develop AV1 codec which is a successor to VP9. Apple joined the Alliance for open media in Jan 2018 and this step by Apple is welcoming for the development of open and free codec. In fact, AV1 is believed to be the unifying factor across all platforms. The odds are against HEVC which faces an uncertain future and is not being supported by Google Chrome and Mozilla.
Alliance for open media members
AV1 promises to give 30-50% compression gains over VP9. AV1 may push both VP9 and HEVC to oblivion and may emerge as the single most sought-after codec standard in the market. But as with every video codec standard, the industry must make a smooth transition to maintain backward compatibility. A lot of devices are already shipped with built-in support for VP9 and/or HEVC and transition to AV1 cannot be sudden. OEMs and end users, both prefer devices having technology that lasts longer. All said, during this brief transition period, it will be interesting to see what choices market leaders might make to keep their devices future proof and which codec emerges as the new market leader.
In Dec 2018, in a German lawsuit against Huawei by an MPEG LA patent holder, the judgement forced Huawei to decide between licensing H.264 technology from the MPEG LA group or exiting Germany and other markets served via the infringing phone. This shows the complex licensing landscape of video codecs where no single company can claim to hold all the key patents of a particular video codec.
Stay tuned for the next blog in this series for IP insights on video codecs.