Skip to Main Content
March 19, 2024

8K Enables Immersive Vivaldi 3.0 Project

One of the applications for 8K that has been gaining more interest, especially since the launch of the Apple Vision Pro, is the use of 8K to create virtual worlds for viewing in 3D headsets. When you are capturing a very wide field of view, you really need a lot of pixels to deliver a great experience.

Carlos J. Ochoa Fernández recently presented a webinar for the VRARA (of which he is President of the Madrid Chapter) on a project ‘Vivaldi 3.0’. The aim of the project is to develop an immersive universe for Antonio Vivaldi’s ‘Four Seasons’ based on a top quality performance by violinist Ara Malikian. The performance was captured in 8K using 360° video and with ambisonic audio. The content is intended to be used in education to inspire and inform youngsters and engage them in the music. Ochoa is heavily involved in education – he is the Co-Chair of VR/AR Education Committee and developed a previous project some four years ago. We spoke with him to learn more.

Original Project

The original project was with the Reina Sofía School of Music (Escuela Superior de Música Reina Sofía) in Madrid which is an elite musical academy that has a strong tradition of artistic performances. The project was conceived during Covid when students could not gather for classes and concerts at the School (concerts are a daily feature). There was a need to involve those from 13-18 and an innovative and creative approach needed to be tried.

A concert was recorded in 4K 360 with ambisonic sound and with the musicians around the camera and in baroque palace settings. The concert content was combined with other information about, for example, the workmanship and skill needed for instrument making. That content has been shown on relatively unsophisticated Pico headsets for four years now. However it has continued to engage students not only at the Reina Sofia School, but around the country.

Vivaldi 3 Image of headset with excited user

Vivaldi 3.0 Moves to a Higher Level

The Vivaldi 3.0 project moves to a much higher level of production values and content, once the concept of this kind of interactivity had been established as effective. Key to  the development was the involvement of virtuoso violinist Ara Malikian who donated his time to the project, which is supported by the Spanish government. Ochoa spent several days with Malikian both observing his interactions with musicians and others and recording two concerts. One of the concerts was with an audience of children and parents and the other was with just professional musicians.

The content was captured with Insta Pro2 8K 360º cameras and Zoom ambisonic recording equipment. The main camera was generally between the key violins and cellos. Audio was recorded from the centre. Ochoa found that allowing Malikian to freely talk about the pieces and his feelings was much more effective than the pre-prepared script, which was abandoned.

A Lot of Data

More than 20 Terabytes of files were created in the production process. However, these are not simply used as a fixed playback source, but are designed to be used in a virtual environment that is more like a game. (Ochoa managed to avoid the word ‘metaverse’ almost entirely in our conversation!). His team has created a digital world where viewers can move to enjoy different views and content, largely based on the relationship of the music to nature, reflecting the composer’s ideas.

The Vivaldi 3.0 content has been developed to be viewed on Meta Quest and Pico headsets. Later, Ochoa wants to bring the content to the WebXR platform to make it more widely available. One of the big challenges is in audio as the headsets tend to use spatial sound rather than ambisonics. To optimise the audio, Ochoa worked with a German company that was also involved in the immersive audio at the Sphere in Las Vegas.

The amount of data involved causes some practical challenges. Ochoa told us that there are the four video files for the Seasons (each of 5 gigabytes) and a further 20Gb of game data. He emphasized that 6 degrees of freedom is essential in the headsets to allow simple and easy to learn interactiveiy.

Although the project was developed in Spain, Ochoa is very interested in working with interested parties in other countries.

0 0 votes
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x