VR and the importance of tracking


TV-Bay Magazine
Read ezine online
Download PDF
Download PDF

I would like to begin this article by clarifying what we at Shotoku mean when we talk about VR in live production. It’s not the production of immersive, 360 content where you need to wear a headset; we are talking about virtual studio (VS) and augmented reality (AR) work, such as placing graphics into a green screen environment or physical set. The technology used for this work is entirely different, though equally specialist – therefore it is important to understand the challenges of this kind of production in order to make informed kit choices.

One of the most important aspects of VR/AR production is accuracy, i.e. ensuring that the graphics are stable in the correct position relative to the set. If the tracking is not sufficiently accurate or starts to lose calibration, the graphics will start to drift and become unstable.

There are three main types of tracking system available today:

  • Mechanical tracking systems (encoder technology attached to or built into camera support equipment) capture vital camera positional information such as pan/tilt, height, X,Y floor position and zoom/focus parameters. This information is merged at the SPI (Serial Position Interface) box and feeds in real time via industry standard protocol to the graphics engine. At Shotoku we have built physical position tracking encoders into most of our products – including tripod heads, pedestals, cranes and robotics, providing a very wide choice of solutions.
  • Absolute external reference systems like Shotoku’s Free-d2 consist of a small upward-facing tracking camera mounted on the main camera, looking at a set of reference markers mounted to the studio ceiling or within the lighting grid. Lens data is combined with the video image and presented to the Free-d² processor which precisely calculates the camera’s 3D position and provides industry standard, frame-synchronized tracking data to any graphics engine.
  • Point of contrast referencing systems uses sensors attached to a camera to track features in the environment, thus providing positional information. The system needs to be able to see enough fixed points of contrast to maintain its tracking.

Some systems are more suitable for certain types of projects. If you are planning a live VR production, here are some factors to consider that might influence your choice of tracking technology.

1. Studio-based or outside broadcast?

Mechanical encoders and absolute systems both work well in a studio, but the latter are generally not flexible enough for location work. Instead you could use a point of contrast system; however, these systems can have difficulty performing under any change in lighting (e.g. the sun’s position changes, the sky clouds up, darkness falls during the production) making such systems less stable.

2. Green screen or hard set environment?

Systems that use sensors to track features in the environment rely on points of contrast; they can struggle in a green screen environment and may need to look away from the set to find their reference. A mechanical system may work better for a green screen production.

3. Cost vs Number of AR/VR channels

Mechanical encoding has the same cost per camera to be tracked. External tracking becomes more cost effective with multiple channels as a single installation of external targets will track multiple cameras. Obviously each camera needs a spotter camera but this is only part of the system cost.

4. Do you want to go handheld?

As encoders are embedded in the camera supports, mechanical systems are not suitable if you want to go handheld. Absolute systems take their references from the spotter camera and target tracking points, so as long as the spotter camera can see the markers, the main camera can be tracked regardless of how the camera is supported.

5. Moving between locations?

Absolute systems are by nature fixed installations and cannot be moved to another location. Mechanical equipment with built-in encoders can be moved around and set up as required.

6. Re-calibration requirements?

Mechanical systems by their very nature build errors over time, particularly if using pedestals prone to wheel slip on floors that are less than perfectly flat. When this happens the system needs to be recalibrated, which isn’t always easy on a live production. Speed is the key here, and Shotoku’s SPi-Touch two point calibration is one of the quickest systems around – it allows the operator to recalibrate in under a minute, quick enough to fit within a commercial break. Absolute systems very rarely drift as they do not have a mechanical interface, so they seldom need recalibration.

7. Tracking delay

Mechanical tracking systems output positional data in real time. Tracking delays must be considered, especially in cases such as lip-synching during a live event.

To choose a tracking solution that will cover most, if not all, of your requirements, talk to a tracking system manufacturer – preferably one that offers multiple types of system so they can be flexible in their advice. Tracking is a critical component in any VS/AR system so it pays to choose the right solution.


Tags: iss130 | shotoku | vr | live production | KitPlus
Contributing Author KitPlus

Read this article in the tv-bay digital magazine
Download PDF
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Sennheiser VR at IBC 2016

    Sennheiser VR at IBC 2016

  • Elemental Technologies VR at NAB 2016

    Elemental Technologies VR at NAB 2016

  • Virtual and Augmented Reality support with the Arrow Fx7 from Miller at NAB 2017

    Virtual and Augmented Reality support with the Arrow Fx7 from Miller at NAB 2017

  • Ross Video at BVE 2017

    Ross Video at BVE 2017

  • Streamstar Sports Live production solutions. All the tools for complete production at IBC2019

    Streamstar Sports Live production solutions. All the tools for complete production at IBC2019

  • Live Production with ChyronHego at NAB 2018

    Live Production with ChyronHego at NAB 2018

  • NewTek at IBC 2015

    NewTek at IBC 2015

  • Grass Valley at BVE 2015

    Grass Valley at BVE 2015

  • NewTek TalkShow at IBC 2014

    NewTek TalkShow at IBC 2014

  • PacTV Truck at NAB 2014

    PacTV Truck at NAB 2014

  • Grass Valley at BVE 2014

    Grass Valley at BVE 2014

  • Cobham: Mini RF Transmitters at NAB 2013

    Cobham: Mini RF Transmitters at NAB 2013

  • Cobham: RF Transmitter at NAB 2013

    Cobham: RF Transmitter at NAB 2013


Related Shows
  • Live production with Newtek at BVE 2015

    Live production with Newtek at BVE 2015


Articles
Original KVM or KVM over IP
Jochen Bauer Will the technology used in broadcasting solely consist of IP devices? For years, IP has been entering all areas of life. Especially control room applications as they are typically deployed in broadcasting benefit from the IP revolution in many ways. But an “IP-only broadcast world” is not yet here. Nevertheless, the trend clearly moves towards IP transmission, even though a large part of content production still uses traditional transmission paths. And therefore we continue to live in a hybrid world, using both original and IP-based technology. KVM experts Guntermann und Drunck still rely on both original KVM and KVM-over-IP™ to be able to offer their customers the best of both worlds.
Tags: iss139 | kvm | gdsys | guntermann and drunck | kvm-over-ip | Jochen Bauer
Contributing Author Jochen Bauer Click to read or download PDF
Keeping the Show on the Road
Andy McKenzie There is long-established saying in the media business that, if something goes wrong, at least nobody dies. It is almost true unless you happen to be a TV producer suffering a cardiac arrest because your primary video feed has gone blank during a high-budget programme.
Tags: iss139 | service | support | finepoint | maintenance | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF
Why MADI is Still Relevant
Stephen Brownsill While the original idea for MADI was to cater to a very narrow recording studio application, the standard remains a viable go-to multichannel audio technology. Beginning as a standard in 1991, MADI was first introduced to the world as digital production was beginning to come of age. MADI was put together in 1988 by Solid State Logic, AMS-Neve, Sony (DASH) and Mitsubishi (ProDigi) as a way to transport up to 56 channels of digital audio between large-format audio consoles of the day and digital multi-channel tape machines via 75-Ohm coaxial cables. Both tape-based machines have long since disappeared from the equipment landscape.
Tags: iss139 | madi | tsl products | aes10 | aes | dolby atmos | st-2110 | sam-q | Stephen Brownsill
Contributing Author Stephen Brownsill Click to read or download PDF
The Future of Broadcast Connectivity
Jamie Adkin The use of KVM equipment has been essential to meet the evolving needs of the broadcast industry for many years. Over that time, many in the industry have recognised the importance of using IP-enabled KVM to break down technological barriers and enable real-time access to visuals wherever and whenever they’re needed. These components are vital parts in live production environments in particular.
Tags: iss139 | adder | kvm | ip kvm | Jamie Adkin
Contributing Author Jamie Adkin Click to read or download PDF
Keeping Pace with the Content Revolution
Kevin Fitzgerald These are uniquely challenging times for broadcasters and their technical teams. Not only are they having to negotiate the move to IP-based infrastructures and the introduction of new formats and techniques such as 4K and HDR, they are also having to generate more content than ever before to support OTT and web services as well as traditional linear broadcast.
Tags: iss139 | streamstar | streaming | case 800 | ipx | ipx-3g | Kevin Fitzgerald
Contributing Author Kevin Fitzgerald Click to read or download PDF