Live Production 8K Video is Down to 48 Mbps With 40 Mbps Already in Spin Digital’s Sights
In our never-ending quest to explore the nascent field of live production 8K video encoding, we’ve been trying to understand whether the accepted fact that codecs get more efficient as resolutions increase is indeed true.
After reaching out to some of our favorite experts, Jan Ozer came back to us, addressing the question with his usual just-do-it approach where only actual data speaks. His analysis concluding that the truism is, in fact, valid when you express it in bits-per-pixel is available here.
In another approach to the same problem, Spin Digital’s Mauricio Alvarez Mesa returned to us with their take on the question.
As this was a piece for the 8K Monitor, Alvarez Mesa translated my question into what the real-world bitrate required for 8K is, and more specifically, what it is for 8K 60 Hz 10-bit HDR encoding.
Of course, the first answer to that question is: “well, that depends.” Doh! Ok, but on what?
Encoding Criteria
Mauricio named five of the most critical factors:
- The coding standard used: AVC/H.264, HEVC/H.265, AV1, VVC/H,266,
- The quality of video encoder implementation,
- The video use case, i.e., Offline (VoD) or live (live OTT, broadcasting),
- The type of content, e.g., talking heads, TV series, or live sports.
- Quality criteria.
Everything in this list is relatively self-evident except perhaps the Quality Criteria, so let’s look at that.
What are Video Quality Criteria?
Most ways of measuring the quality of an audiovisual experience fall into one of two categories. They are either “objective metrics” or “subjective experiments.”
Objective metrics
Technical analysis of the video usually generates these metrics automatically.
The well-known Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity Index (SSIM) have traditionally been used to measure video quality. However, these metrics do not correlate well with the Human Vision System (HVS). Other metrics such as VMAF or the ITU P.1204 standard correlate better with the perceived quality but currently only support up to 4K resolution. In particular, VMAF does not support HDR, which is a crucial element of the 8K format. Another metric – not yet widely adopted – based on the HVS proposed by Fraunhofer HHI called XPSNR1 supports 8K video.
Subjective metrics
Despite the constant progress of AI, nothing yet beats the human eye, and the best way to evaluate the quality of an encoded video at a specific bitrate is using formal subjective experiments. That means allowing humans to view the video and rate its quality. However, these experiments are expensive and time-consuming. As a result, subjective tests do not scale well as these cannot be performed frequently. Subjective tests are usually measured with a mean opinion score of 1 (very bad) to 5 (excellent).
Encoding Tradeoffs
One size doesn’t fit all, and Spin Digital illustrated the complex tradeoff between the criteria we defined above, leading to different required bandwidths with two use cases. Both use HEVC for live encoding of broadcast-like content.
Case 1: NHK STRL: 8K broadcast channel
NHK performed a study on the recommended bitrate for 8K broadcasting applications2. In this study, videos were encoded at different bitrates using a first-generation HEVC ASIC-based live encoder, and the perceived quality was evaluated through a subjective experiment. The authors define Broadcast-grade quality as achieving an average MOS of 3.5, with a minimum MOS of 3.0. The results showed that a bitrate of 85 Mbps was required to attain broadcast-grade quality. This value has since been assumed by part of the media industry as the “standard” bitrate for live 8K. But, as already mentioned, a first-generation encoder was used in the study.
Case 2: 8K live internet streaming
Within the context of the EU-funded Immersify project3, Spin Digital conducted a formal subjective test to measure the recommended bitrate for 8K in immersive spaces using a latest-generation CPU-based HEVC live encoder (Spin Enc Live4). The experiments concluded that broadcast-grade quality could be achieved at a bitrate between 25 to 40 Mbit/s, as shown below. Further tests performed using recent 8K TVs and newer content with HDR showed that a bitrate of 48 Mbit/s (about 40% lower bitrate than NHK) was enough for achieving the target broadcast quality. Other industry partners have also confirmed this value at demonstrations and live events5.
We asked Spin Digital’s Mauricio what tricks and optimizations their encoder includes to produce high-quality video at lower bitrates in real-time:
- Perceptually optimized encoding increases visual quality at the same rate using a human visual model that can run in real-time for 8K 60 Hz. With this option enabled, Spin Digital claims they can achieve the target bitrate of 48 Mbps even for more challenging content (e.g., sports).
- With various algorithmic optimizations, the encoder achieves high encoding speed while maintaining high quality by using statistical models for picture partitioning and mode decisions based on machine learning. Spin Digital believes that these statistical models can solve complex encoding problems without relying on neural networks or similar techniques.
A white paper comparing the compression efficiency and encoding speed of different HEVC encoders6 describes in detail some of Spin Digital’s experiments with 8K content.
Mauricio provided the picture above that compares different HEVC encoders using several 8K videos, showing very different quality-performance points (encoding speed in frames per second vs bitrate increase at the same quality). These Spin Digital results show that Spin Enc Live produces similar quality to offline encoders at significantly higher encoding speed and higher quality than real-time software and hardware-accelerated (e.g., GPU-based) encoders. A server with two Intel Xeon Platinum 8368 CPUs (2x 38* cores) and an RTX 3070 GPU (for NVENC) ran these comparison tests.
All taken together, Spin Digital currently recommends 48 Mbit/s for 8Kp60 10-bit HDR live content when using HEVC. Spin Digital believes that there is still a margin for improving the encoder but not significantly with HEVC.
Beyond HEVC
With Fraunhofer HHI, Spin Digital demonstrated the visual quality produced by a VVC offline encoder implementation (HHI’s VVenC) to that of a well-known HEVC encoder (x265)7. The demonstration showed that VVC at 25 Mbit/s produced similar quality to HEVC at 50 Mbit/s. However, for this level of bitrate reduction, VVC currently requires 20 times more computation than HEVC, which is impractical for live applications. Mauricio finally commented that a live VVC encoder using more practical computing resources could achieve around 15% bitrate reduction compared to an optimized HEVC live encoder, resulting in a bitrate of approximately 40 Mbit/s for live 8K encoding including 60 fps and HDR.
To fully confirm these bitrates for live 8K using the VVC codec, developers and service providers will need to work on codec optimizations, new quality metrics, and more subjective experiments.
* apologies, the first version of this article had an incorrect value of 28 cores.
References
- C. R. Helmrich, et al., “Xpsnr: A Low-Complexity Extension of The Perceptually Weighted Peak Signal-To-Noise Ratio For High-Resolution Video Quality Assessment,” ICASSP 2020, pp. 2727-2731
- Y. Sujito, S. Iwasaky, K. Chida, K. Iguchi, K. Kanda, X. Lei, H. Miyoshi, K. Kazu, “Video Bit-rate Requirements for 8K 120-Hz HEVC/H.265 Temporal Scalable Coding: Experimental Study based on 8K Subjective Evaluations”, APSIPA Transactions on Signal and Information Processing, 2019.
- S. Sanz, A. Nikrang, “Report on QA and Content Preparation Guidelines”, Immersify Project, October 2021: https://immersify.eu/home/guidelines-reports/qa-and-content-preparation/ Spin Digital, “8K HEVC Real-time Encoder (Spin Enc Live)”: https://spin-digital.com/products/spin_enc_live/
- Spin Digital, “Joint Press Release: Global 8K Live Streaming Showcase 2020”, January 2021: https://spin-digital.com/announcements/global-8k-live-2020/
- Spin Digital, “HEVC Real-time Software Encoder for 8K Live Video Applications”, December 2020: https://spin-digital.com/tech-blog/whitepaper-spin-enc-live/
- Spin Digital, “8K HDR Encoding and Playback using VVC”, July 2021: https://spin-digital.com/tech-blog/8k-vvc-encoding-and-playback/
