8K Ingest and Render 30 times faster
By guest author Keith Doughty, Vice President of Sales and Marketing at QuVIS
8K is ushering in new media presentation capabilities, including an expanded color range, deeper black levels, increased brightness, and alternative frame rates. These features enable the display and distribution of new classes of premium content, potentially transcending Digital Cinema Packages.
Serious desktop media processing tools have already exchanged their 8-bit signal processing heritage for more capable 16-bit and 32-bit floating point components. This provides adequate component and processing resolution to support very high dynamic ranges and expanded color spaces for HDR10 and HDR12 media. The remaining challenge is primarily increasing processing speed.
Consider rendering an 8K/24fps eDCP supporting full band HDR12 dynamic range in real-time on the typical desktop system. If a 16-bit per component source file, such as an uncompressed 16-bit TIFF file, is used, at least 4.8 Gbps of input bandwidth is required. While processing requirements for this format are light, the data rate exceeds the current IO capability of many desktop machines. If a popular compressed mastering format such as ProRes XQ is used, the input data requirement is reduced six-fold to 0.8 Gbps, which is better. Still, this compressed format now requires substantial processing overhead to decode the relatively massive entropy-encoded bit-stream. Not only does this consume time directly, but it also competes with other tasks for available compute resources slowing them too. A better IO and compute load balance is needed in formats for 8K desktop media production!
Media formats: The data rate priority problem
Traditional mastering formats use a rate priority compression approach where the format designer determines the maximum data rate, which is likely to be required for the most extreme case. There are several significant problems with this approach. The most burdensome is the resulting inefficiency because it operates at or near its worst-case data rate. A quality priority approach is the obvious answer but has been elusive because of the difficulty of discriminating between useful picture information and typically less useful noise information. Decades ago, a method was developed for image encoding, which sidestepped this information discrimination issue with an assured channel quality process that made the distinction between picture and noise somewhat irrelevant and, ironically, largely automatic.
The QuVIS Quality Priority Encoding (QPE) method was the first use of this approach which is very effective at maintaining and delivering content at a specified channel quality with data efficiency. Typical data rates are lower than with rate priority formats, quality is consistent, and because the channel quality is controlled, there is no catastrophic quality drop if extremely information-rich media shows up.
The media production process
The current process can be cumbersome when the goal is cinema-quality media production and delivery for high dynamic range 8K displays. Digital cinema was designed around a 12-bit dynamic range HDR12. Acquisition, working storage, and color work typically need more resolution than the final product. In this case, 14 to 16 bits of dynamic range is desirable. Even when dropping to a 12-bit media format such as ProRes XQ, about 800 Mbps is required for 8K/24p and 2 Gbps for 8K/60p. Uncompressed 16-bit TIFF files will be six times as big. Reading, writing, copying, transferring, and verifying media files at these data rates is time-consuming.
Editing is usually done using a lower-resolution proxy format. However, at some point, full-resolution output media must be rendered using full-resolution source media. On current desktop systems, it is not unusual for even 2K work to render much slower than in real-time. HDR 8K content requires much more processing, typically 16 to 32 times as much.
These results led to the QuVIS QIC project
We realized these new performance capabilities have much broader applications than our own Digital Cinema Package (DCP), eDCP, and IMF media export. General media production would be significantly facilitated by very fast ingest, rendering, and exporting to a media format capable of high dynamic range production, providing excellent quality and improved data efficiency. The QIC project focuses on enhancing the production media format’s speed, quality, and efficiency for all desktop media applications.
QIC Format efficiency improvements
Current popular HDR-capable mastering formats providing 12-bit to 16-bit component resolution require 2 to 12 Gigabyte/S for 8K at 60 fps. These data rates and associated processing times are a fundamental barrier to practical desktop processing. An upper mid-range desktop computer running popular media editing software is often limited to a few frames per second at 8K by the hardware IO capability and processing requirements. A dramatic improvement in compression efficiency is required to reduce file size, storage costs, IO bandwidth, and bit-stream processing overhead. Two proven and very effective methods are used to achieve a significant size reduction while maintaining image quality for 12-bit to 16-bit applications.
The first step is to change the mastering format philosophy from a fixed data rate, varying quality Rate Priority Encoding (RPE) method to a fixed channel quality, varying data rate Quality Priority Encoding (QPE) method. Although RPE methods are reasonable and pervasive in media communication, a QPE process is better suited to applications where quality is essential and rates vary. The reduction in data usage to achieve a specified quality target is significant. An RPE-based mastering format effectively operates at a worst-case data rate adequate to ensure the most complex image information ever expected would meet the desired quality. The QPE alternative implements an image channel quality process using sampling theory to precisely identify and enforce quality requirements for every aspect of the pass-band. Everything which passes through the channel is always encoded. Data rates vary depending on picture information at the specified channel quality and will be vastly lower than RPE rates when dealing with typical production media.
The second method achieves a substantial improvement in the effectiveness of the transform-based auto-correlation analysis. The better an image auto-correlation is “understood”, the smaller the bit stream needed to represent that image at a specified quality. A traditional mastering format divides an 8K image into many segments and performs an auto-correlation transform on each part in isolation. Intuitively, this is like trying to “understand” an 8K picture by looking at each piece of a 500,000-piece jigsaw puzzle. A much-improved alternative uses a full image transform aware of the entire image and frequency spectrum to achieve a much more comprehensive and practical “understanding”.
Summary of 8K Ingest and Render times
Originally the QIC plan targeted high-end hardware configurations with high CPU core count, high-end GPUs, and multiple GPUs expected to be needed for 8K HDR support. Software development focused on keeping these system resources busy to provide excellent CPU and GPU utilization over a wide range of core counts and GPU resources.
We were surprised and pleased to discover that mainstream desktop configurations benefit from the QIC approach much more than expected. Even four core systems have shown a dramatic improvement in export speed. Systems with reasonably large memory and fast IO with 6 to 12 cores and a single upper mid-range GPU are likely to be able to ingest, render, and export 8K HDR media at respectable speeds. Export speeds at 2K and 4K have been breathtaking.
Using an eight-core system with upper mid-range GPU, an Adobe Premiere 8K render and export to QIC format is real-time (slightly over 30 fps) and about 30 times faster than the same render targeted at ProResXQ. Higher core count systems exhibit a nearly linear increase in QIC render and export performance relative to core count.
The QIC codec with associated ingest, render and export will have general available in February 2023.
Stay Tuned for an update with some ongoing lab tests.
