Wavelet Beam Finds the Signal in the Noise
At IBC, we were talking to high resolution maven and camera maker, Pawel Achtel, about the challenge of defining resolution when video signals have been heavily processed (which is all of them given the realities of de-bayering, chroma sub-sampling and compression). “Ah!”, said Achtel, “You should talk to Wavelet Beam”. So we did. After an interesting but short discussion at the show, we followed up with an interview with Dirk Hildebrandt, the founder of the company.
Hildebrandt has a long history in engineering and electronics and worked on videotape systems in Germany for the company that became DFT, the film scanner maker. By coincidence, our next planned interview is with that company – we also talked to them at IBC. Hildebrandt also worked on quality measurement systems for cable providers which gave him a lot of experience in measuring video quality and understanding how it degrades.
First Product – Film Test Strips
The projects that he worked on after starting his own consultancy included creating film test strips for scanners so that the performance of the scanners could be characterized. The strips were created using an Arri 4K laser-based film recorder for the highest quality output. As well as testing the resolution, these film strips can reveal other issues in scanners as they can have all sorts of problems – some of which are not exposed without this kind of test material. “You can’t just rely on what you see”, Hildebrandt explained. “If you start with the sensor output”, he added “You’re lost”.
Hildebrandt worked with Achtel on de-noising of RAW files and he developed algorithms for noise analysis and processing. He likes the fact that Achtel’s camera stores the data from the sensor with as little processing as possible (although this means very large file sizes). The algorithms for de-noising and RAW processing are available to Wavelet Beam’s customers via a ‘software as a service’ approach. The software, called Iris is based on Linux and runs via CUDA on Nvidia GPUs.
Demanding Clients
He explained that some of his clients are very demanding in terms of quality with one particular one (no names) that tends to record artistic events with the highest quality of video possible and Dolby uncompressed audio. They see each of their performances as being ‘for the archives’. Wavelet Beam mostly performs its analysis and processing as an off-line service, but live processing is possible at 1080 30P with 8 bit output.
We talked about the challenge of dealing with noise and with grain (as we recently wrote in our article about the work of the Fraunhofer HHI). Hildebrandt explained to us that noise and grain are usually quite different in their impact on the signal and, if he can get to the content before any post-processing, his technology can deal with them separately and eliminate them effectively. However, if there has been significant manipulation, especially of the contrast of the image, they tend to blend together and that makes elimination more difficult.
AVI Film Grain Synthesis
We discussed the film grain analysis and synthesis in AVI and Hildebrandt pointed out that AV1 film grain processing is ‘2D’ only and was developed by Netflix and Google (we found this paper that explains it). That makes it OK for consumer use, he believes, but for professional and archival use, it is not really good enough. De-noising can be tough in some conditions, for example with fast moving cameras. AV1 film grain synthesis uses repeating patterns, although there is some mitigation of those effects, as discussed in the paper.
Once you have the content de-noised and with the grain clarified, you can then understand the true resolution of the image by looking at the high frequency content. There is a nice demonstration of the use of the analysis software to show the actual resolution based on a video of the moon on the firm’s website. In the past, Hildebrandt has measured the output from cameras rated at 8K that have just shown 5.6K actual resolution. On the other hand, the Achtel camera really does produce true 8K.
One aspect of the measurement is to measure the spatial energy in moving from one frame to the next and all sorts of factors can affect the true resolution including changes in focus. The data from the analysis can be exported as a csv file so that further investigation and management can be done.
Processing Should Match the Genre
We talked at some length about the value of metadata that identifies the genre of content. The processing that is then applied could be appropriate to that content. This is done in the case of ‘Filmmaker mode’, but it should be different from the processing for sports, for example, he suggested (and broadcaster Sky would agree – ed.). Â
Processing broadcast content can be a challenge with the installed base of cameras that capture in 1080i seeing processing to 720P for distribution. Even at the early stages, there is a lot of processing especially with older infrastructure, which is typical for public broadcasters. The result can be too little information and visual distortions such as ‘waxy faces’.Â
Wavelet Beam continues to work on developing ways to define video quality that might go beyond VMAF and simpler metrics such as PSNR.
