NHK Pushes Forward on Immersive Video
Although it’s billed as the 4K HDR Summit, the annual event that takes place in Seville also extends to other high resolutions and immersive technologies.
Tetsuya Hayashida is from NHK of Japan, which has presented regularly at the event, and he talked about the work that NHK has been doing to develop 360 degree video for high quality immersive experiences. We have already reported on some aspects of this project from meetings with NHK at IBC.
Still Challenges
Hayashida started by outlining that although there have been many advancements in technology, there are still challenges in creating truly immersive content with enough resolution and field of view as well as high frame rates and spatial audio. Although there are 360 degree cameras in the market, they are limited to 12K resolution which is not enough for UHD performance over the full field of view.
Displays for head-mounted displays (HMDs) have up to around 4K per eye but this falls ‘well short of the detail required’. Although large hemispherical (dome) displays exist, but they are too expensive for many applications and impractical for consumers. Because of these limits, NHK started to develop technology to overcome them. ITU-R recommendations were adopted as a starting point.
The parameters of the ITU recommendation are in the table above. Hayashida explained that NHK sees the 30K x 15K resolution recommendation as an extension of the existing 8K ITU standard. NHK also required that HD broadcasting quality had to be maintained in all directions. Viewers may be looking in different directions at the same time and so high quality has to be presented in every direction. To meet this need, NHK developed a 360 degree camera (seen at IBC) that could produce final resolution of the content at 30K resolution.
The Technical Details
In technical terms, the lens was required to have an MTF of 0.2 at the Nyquist frequency across the entire sphere. (Hayashida said that 0.2 is a typical value for an industrial camera lens). Stitching points and boundaries should be minimized (to avoid some of the challenges seen in the Abu Dhabi project that we recently reported on. Other requirements were for a frame rate of 60 Hz and a bit depth of 10 bits as well as global shutter operation (to avoid motion artifacts). Square image sensors were preferred to allow the maximum area to be captured from the lens.
NHK looked at a number of different configurations of cameras to optimize the different requirements and eventually a pentagonal prism, seven camera system was chosen based on a 9K x 7K industrial camera model. 30K resolution could be achieved in vertical and horizontal directions but for now, the downward view was not considered. Different lenses were needed for the horizontal and vertical directions.
A lens was developed for each of the cameras and a dedicated rig was developed to get the cameras very close and reduce the parallax error. It also helped to improve the stitching quality of the images. An optical transmission system was needed to connect the cameras each to a separate recording PC. The camera was compared to a Kandao Obsidian Pro industrial 12K 360 degree camera and Hayashida said that detailed comparisons showed the clear advantage of the NHK design.
Display Development
The group also developed a 15K hemispherical display based on a projection system using three 1.21 inch 8K UHD LCOS imagers with 4,500 lumens of output. A fisheye lens was fitted and by using a half-pixel shift, 15K effective resolution was achieved with a field of view of 180 degrees horizontally and 100 degrees vertically.
A second demonstration display was made using sixteen 4K UHD curved OLED panels in a 4 x 4 tiled configuration again with 180 degree view horizontally.
Both of these displays allowed more than one person to enjoy the immersive experience rather than the single viewer for a headset.
Location Shooting
The camera was taken to various sites around Tokyo including on a bus and on a boat. Content was successfully captured at each site including fine details and moving objects.
Hayashida then showed a video of the camera and some of the content. An ambisonic microphone was used to capture the audio. Audio was edited to better match the footage. The results were shown at the NHK STL Open House event in Tokyo in the summer of 2025.
At IBC, the demonstration was shown along with an HMD and a system to extract 8K video streams from the full 30K x 15K image. Overall, there was a very positive reaction to the technology, Hayashida said.
