Ateme: With VoD Driving 8K, AV1 Has a Cool Film-Grain Solution
We recently had the chance to catch up with Thomas Burnichon, the Los Angeles-based VP of innovation strategy of Ateme, the video compression and CDN specialist. We covered several topics on how the firm is developing its support of 8K with VoD driving 8K. One aspect caught my attention – and that (perhaps surprisingly in this digital age) is ‘film grain’.
Film grain
The choice of which film to use and how to process it for the production team of a film-based project is an important decision. As well as changing the color and ‘look’ of the film, how the film is exposed and processed can change the grain, which is the pattern of silver particles that make up a film-based image. The structure of the grain is, to use a technical term, ‘stochastic’ – that is, it is random but within a defined range depending on the film stock, exposure, and chemicals used for the development of the film. However, the random nature makes it a problematic visual feature to deal with efficiently in a codec. So, it’s vital to deliver the grain ‘look’ that the creator intended, but via a codec, it gets complicated.
This problem is not new. At the SID Display Week Business Conference back in 2014, when the world was making the last significant change to display resolutions with 4K and UltraHD the big topics, Mark Turner of Dolby said there was a challenge to scanning old movie content.
“When you scan a back catalog movie to 4K or 8K, you get high-definition film grain. That makes compression difficult as the film grain seems to be “noise” to the processing circuitry”.
AV1 to the rescue
Engineers have been addressing this challenge, and the developers of the AV1 codec added a process called Film Grain Synthesis (FGS) to remove the film grain before encoding. Its characteristics are stored in a ‘grain table’, and the grain is then added back in decoding. A similar tool existed in H.264 but was optional and was not widely deployed.
The grain is moved from the image itself to a metadata model capturing its characteristics. The « denoising » process must be done with care as it can have the unfortunate side-effect of blurring fine details. Renoising during the stream decode uses a block-based approach which is well suited to the processing abilities of most consumer devices. Robust quality control is required in workflows using FGS.
The FGS process allows for much lower bitrates and, depending on the content, Ateme has achieved compression gains of 30% to 50% for heavily grained content.
Why do you want to add film grain back to the content if it is “noise-like”, you might ask. Becasue the content creators like the look of film and film grain is part of that look, so must be added back.
AV1 is mainly used in VoD, where grain can often be present in movie content, unlike live broadcasting. As it happens, Burnichon told me that the bulk of 8K work Ateme sees is still in the on-demand area. Ateme’s live 8K work had been limited to large-scale trials for several years. The trial with The Explorers in 2020 is an example.
Start-ups like The Explorers have started showing the way with original production. Larger companies like Rakuten have previously committed to 8K catalogs. We can expect the treasure-trove of 70mm film stock to enter the significant VoD platforms in the coming years.
Thomas explained that for 8K, Ateme’s main commercial product is the file-to-file transcoding solution TITAN File. All kinds of VoD operators use it globally. TITAN File already supports 8K in HEVC and AV1. Ateme is working on adding VVC support to TITAN File.
Thirty per cent drops in 8K bandwidth on the horizon with VVC
I asked Thomas for an update on compression targets for 8K independently of the FGS solutions described above.
Ateme currently provides 8K contribution feeds at bitrates of around 180 Mbps. The short-term target is to bring these down to 120 Mbps using VVC.
Distribution bitrates for 8K go below 30 Mbps today for VoD in HEVC. Here the target is to descend below 20 Mbps once the new VVC codec is deployed.
Power requirement for encoding and decoding 8K
The impact of streaming on the environment is a critical topic, as illustrated by the Greening of Streaming initiative. Codec power consumption is a determining factor. Burnichon told me that new generations of codecs typically double algorithmic complexity, and the case is slightly worse for AV1 encoding. However, as the new codec is still mainly used in the VoD context, more complexity is not an issue as it is acceptable for coding to take some time, especially on cloud instances that can scale up and down on demand.
Decoding is also more complex, but the availability of hardware decoders is easing that issue too. Power requirements are much less significant as soon as dedicated hardware becomes available. There is probably much hardware optimization needed in the future for mains-powered devices, a subject the author is pursuing and will report back on. On mobile devices, the situation is different, and here too, we are investigating and will report back.
With its clever film-grain solution, AV1 can keep VoD driving 8K adoption for a while longer as we wait for live 8K to truly take off.
