NHK Develops 30K Camera and 15K Display
Back at IBC 2024, NHK told us that it was working towards 32K 120fps video as this is the ITU target for 360 degree immersive VR resolution. We could see the logic for this, as in VR each eye tends to see just a small window into that resolution. Ideally., you want the kind of resolution that can minimize the ‘screen door effect’ – I.e. where the eye can see individual pixels. That level of resolution, depending on the field of view of the headset, is around 4K per eye. If the angle of view is 45 degrees, you would need 8 times as much resolution to fit to 360 degrees. Hence the need for around 32K.
The ITU has a free to download document, ITU-R BT.2123-1 that recommends a resolution of 30,720 x 15,360 (30K x 15K) for a full 360 degree image.
What we weren’t expecting to see so quickly was that NHK is actually trying this level of resolution. NHK has a research laboratory in Japan called STRL and at the end of May it held an Open House event. At this year’s event STRL had a demonstration of immersive video based on videos captured with a special 30K camera and with half of the image projected on a hemispheric surface with 15K resolution. This means that immersive video can be provided without the user needing a headset.
The camera system uses 6 sensors with a pentagonal prism. Five of the sensors are covering the horizontal image direction with one pointing upwards. The camera is described briefly here. The sensors each capture 9K x 7K images and these are stitched together to create the 30K stream. The high bandwidth of the system means that each sensor needs to be connected via Ethernet at 100Gb and six workstations are used to capture the video. NHK said that this method allows ‘broadcast-quality’ resolution across all captured areas of the image.
360 High End Cameras at the HHI
In the past, one of the key developers of this kind of high end 360 degree camera was the Fraunhofer HHI which developed the OminCam 360 from 2015 to 2018. The OmniCam used ten cameras, each capturing an image via a mirror segment. However, this earlier camera supported just 10K x 2K resolution – more than 20X fewer pixels than the NHK system. The Fraunhofer HHI has previously demonstrated its TiME Lab immersive projection environment. (The author was impressed when he visited the HHI)
The 15K Display
The researchers at NHK said that they have developed a display ‘equivalent’ to a 15K pixels by diagonally shifting the 8K image created by its 8K projector. That’s a well understood technique for boosting resolution in projection devices which can struggle to create high native resolution because of the small imagers used. We dug into this topic last year. We have reported that Sony’s device business has developed an 8K LCOS chip and JVC is reported to have a native 8K D-ILA LCOS chip that is used in some simulators, so NHK may even have had a choice of imagers available for this demonstration.
Of course, an advantage of a direct view display like the hemispheric version shown in Tokyo, compared to headset use is that the image does not need to be captured in a stereoscopic way to provide different feeds for each eye.
NHK said that it is planning to develop a “compact and practical 30K 360-degree shooting system by 2027” We’ll be watching for developments.
Although the demonstration at the Open House event was of projection, regular readers might remember that we reported at the start of the year that NHK is also working on seamless tiling concepts, so in the future such a hemispheric display could be developed using this technology. Having said that, it is three years since AUO showed the simulator dome based on microLED technology at Touch Taiwan.
XXX
