KitPlus - The TV-Bay Magazine

To view this page ensure that Adobe Flash Player version 11.1.0 or greater is installed.

were fully 3D audio compliant even back then so it’s nothing new. The challenge then was what to do with the 3D audio, how to play it back outside of a laboratory. The early use of 3D audio was not about being immersive but about the flexibility it gave you when steering around this microphone. You may only want a mono or stereo output from this 3D audio capture but it’s about being able to steer around and being able to reposition this microphone in post- production. This is a very important statement because this is where SoundField and virtual reality really start to gel together. So if you think about it, the way virtual reality is captured from a 3D video point of view means we can smoothly move around this space. SoundField B-format captures audio in exactly the same way allowing us to use exactly the same head tracking or positional data used to position the video to move the audio perfectly in sync. All we have to do for this is use the four SoundField B-format audio channels together with the video. The head tracking information, which is used to move the video around, can then be used to steer the audio. IF I’M WEARING VR GOGGLES I NEED TO HEAR SOUND BEHIND ME WHEN I TURN TO SEE WHAT THESE SOUNDS ARE THEN THEY SHOULD BE IN FRONT REALLY, SO WHAT ARE THE CHALLENGES IN DOING THAT AND MAINTAINING THIS AUDIO INTERACT? There are a lot of different ways you could do audio for VR, and as you progress down these different ways the experience will become more realistic for the consumer, which is the end goal. So first of all, we could just have a fixed stereo which doesn’t move with the video so you just lay down a stereo track like you would have always done for a standard video shoot. So a lot of the VR content out there is exactly that: you move your head and the audio stays completely static. Clearly this is not satisfactory and we are really missing out here, in the end it is the audio that will make you really believe you’re in a virtual reality. The second thing you could do is to use head tracking to play back stereo audio that is in line with what you’re seeing. As you move your head, the audio will pan around in sync with the video. At this point you won’t really hear a discrete source behind you over headphones because it’s just a stereo image which is facing forward - or if you do hear the sound you will not localise it behind you. Again, this is an improvement and there is more and more virtual reality material available that is done in this way but clearly it’s not the holy grail. The true holy grail is really about being able to recreate complete 3D audio over headphones. Binauralisation aims to do exactly this and mimics spatial cues generated by your head and your ears to trick your brain into hearing real 3D audio. This technology again has been around for a long time but has been fraught with challenges in that every person’s head and ears are different. When you measure a given person’s head the results are extremely convincing. However coming up with a set of measurements that work for a wide range of people has been challenging, but a lot of progress has been made in recent years. So this is where it starts to get very exciting. We can capture 3D audio using a SoundField B-Format microphone, we can use object-based audio to augment with other mono or stereo sources and we can playback 3D audio over headphones that can use the video head-tracking data to move them all in sync. Now we are really starting to be immersed in a virtual reality both from a video and an audio perspective! So from where I am standing it looks like the technology is available to go out and create truly immersive virtual reality experiences. All we need now is lots of creativity to make amazing content. KITPLUS - THE TV-BAY MAGAZINE: ISSUE 113 MAY 2016 | 51