Samsung to Enable Cinematic 8K on a Future Phone
At its Developer Conference, a team from Samsung outlined plans to enable more cinematic video from its smartphones, with the use of a new codec, the Advanced Video Professional (AVP) codec that it has developed.
The talk covered four areas:
- What is Professional video?
- Why do we need an APV codec?
- What is the APV codec?
- What are the benefits of the AVP codec?
Increasingly high performance from smartphone cameras have enabled much higher quality video and users and content creators increasingly want to be able to use sophisticated post processing including colour grading. If you are editing video extensively, you need better codecs than the currently typically used technologies such as HEVC, which are fine for distribution, but less optimized for post-production. The desire for extensive post-production processes is a key definer of professional video.
Why Do We Need an APV Codec?
Codecs such as HEVC have a ‘group of pictures’ (GOP) architecture, which makes video editing more complex than is ideal as sometimes all that is stored is the difference from previous frames.
A better type of codec would be one where each frame is self-contained – an intra-frame codec. Established codecs also tend to use a lot of complex calculation, so it can be hard to run them to produce high bit rate, high quality content, using a smartphone processor. Compressing to low bit rates can leave compression artefacts and degrade colour quality.
What is the APV codec?
The APV codec is designed for the compression of content on smartphones at relatively high (from several 100 Mbps upwards) bitrates. It is said to be a visually lossless coded – that is to say that although some information will be lost, the compression should not be visible. The codec has been optimized for smartphone processors. After content has been captured and processed in APV, the final content can be compressed with a traditional codec such as HEVC.
The codec uses ‘traditional’ process but in slightly different ways. There is:
- Partitioning – into tiles and macroblocks. The use of tiles allows for the exploitation of multi-processor systems, Samsung said. In the 16 x 16 macroblocks there are 8 x 8 transform blocks which can be used to compress very efficiently, the firm said.
- Prediction is used in the transform domain and Joshi said that the use of domain prediction eliminates a key throughput bottleneck in other codecs.
- High Throughput Entropy Coding uses a ‘zig-zag’ scanning approach and ‘hybrid variable length code’ (hybrid VLC) to allow the codec to maintain high throughput at high bitrates
Benefits of the Codec
Madhukar Budagavi then summarized the benefits of the APV codec which include
- Perceptually lossless video quality (for optimum editing)
- Low complexity – which is essential for running on a smartphone
- Intra frame only for simple editing
- Boosted chroma fidelity using 4:2:2 or 4::4:4 sub-sampling and 10 to 12 bits of depth
- Frame tiling and high throughput processing using a lightweight entropy coding scheme and no pixel domain prediction
- Support for bitrates from a few hundred Mbps to ‘a few Gbps’ to allow support for 2K, 4L and 8K content.
- Multiple steps of encoding and re-encoding without severe degradation – which is also important for editing and post-processing.
- Flexible/extensible metadata in the codec
Samsung performed a subjective comparison with ‘a conventional professional video codec’ and with HEVC and it seemed that the audience thought that APV at 1.2 Gbps matched the quality of the ‘professional video codec’ at 1.5Gbps.
Comparing with HEVC, the audience saw that HEVC at 200Mbps looked a lot worse than the APV at 1.2Gbps. However, running on a smartphone, Budagavi said, the HEVC codec cannot produce more than 200Mbps because of the complexity of the processing. Samsung showed the demo on a TV at the Developers’ Conference.
Objective metrics such as PSNR show that you need 20% less bitrate for the same quality which is a big saving in space taken by the captured content.
Conclusion
To meet the desire of consumers to capture better quality video for post-processing from smartphones, Samsung has developed APV which has better quality and uses 20% less processing, but can be run effectively on a smartphone.
Samsung said that the APV codec will be on a future Samsung phone, although it did not detail when this would be available.
Editor’s Comment
Of course, Apple already has support for its Prores codec in its iPhones and recently boosted the performance in its iPhone 15 Pro to 4K at 60fps using external storage, up from 4K 30fps on the iPhone 14 Pro (using internal memory). Crucially, however, Apple does not support 8K on the iPhone yet, while Samsung clearly stated that it will support 8K with the APV technology.
