New Upmixing Challenges and Strategies for Audio Post-production Today and Tomorrow


Jon Schorah TV-Bay Magazine
Read ezine online
by Jon Schorah
Issue 109 - January 2016
Following the successful introduction of Halo Upmix, NUGEN Audios new upmixing plugin, Creative Director and Co-Founder Jon Schorah reflects on the changing landscape and increasingly complex demands of the modern audio post-production environment.

The audio post-production engineer survives in a demanding and complex world where time is at a premium; yet, these professionals are required to deliver results at the highest quality levels as a matter of routine. Its essential to design concise and efficient workflows, and the production and delivery of surround mixes is no exception. Day-to-day TV programming is produced to a tight deadline and budget. While upmixing original stereo programming to surround has become routine, digital distribution is demanding an increasingly robust upmix for a number of reasons. This in turn adds new complexity and places additional demands upon a successful upmix process.

First and foremost, the process must be able to provide a good upmix from the original source material. Of course, good is somewhat subjective, but one simple tenet holds true - the upmix must respect the original source material, providing a naturally extended panorama that unfolds the original into a surround context. This is very important when the original source is a broadcast mix that is being translated for broadcast in surround. Here creative decisions have been made, and these preferences need to be respected and represented in the resultant upmix. A strong relationship between the original source material and the upmix not only respects creative intent but also delivers a coherent experience to the consumer.

Secondly, there is the issue of dialogue isolation. Most delivery specifications require a high degree of dialogue isolation into the centre channel for aesthetic purposes and consistency of delivery, but achieving this is not always a simple process. In many situations, separate stems are not available, and with archive restoration, the stems may not have existed in the first place. Historically, upmix processes have applied rather brute-force methods for achieving this end, resulting in unfortunate side effects for the mix as a whole. Recent increases in computing power, however, have led to great improvements in processes that facilitate this isolation. Techniques that go far beyond simple mid-side and frequency isolation are now possible, with minimal over-isolation and transparent transmission of directional content passed to the sides.

A third requirement has recently come into play, largely as a result of digital distribution methods. An upmix must, in most cases, be highly downmix-compatible. This means that a downmixed version of the upmixed audio (i.e. the upmix collapsed back down to stereo) must still sound very close, if not identical, to the original for all intents and purposes. There are many processes that can be used to generate a full-sounding upmix, but some of these can cause serious downmix compatibility issues. For original TV production, the 5.1 downmix needs to correlate very closely to the original stereo source, since the consumer stereo version is commonly produced at the set-top box level as a downmix of the broadcast surround. Even in the context of a new full surround mix, in which the downmix will not be directly related to a single upmixed audio source, it is still an important requirement to deliver a down-mixed track free from unnatural- sounding delay and phase artefacts.

The upmix process also needs to be flexible. Traditionally, the majority of cinematic releases have used a 7.1 surround configuration, with a 5.1 configuration for TV. For film audio engineers, a 7.1 upmix solution is perfectly adequate; however, many freelance engineers routinely offer services to both the film and television industries and therefore need greater upmix flexibility. For those with an eye on the future, the advent of object audio offers an enticing 9.1 bed track, including additional information for overhead loudspeaker arrays. According to one manufacturers technical white paper, in theory an object-based mix, including a 9.1 bed, can be automatically translated to 7.1, 5.1, and stereo formats allowing the audio professional to author once, optimise everywhere. This has the potential of introducing a universally translatable mix, but it remains to be seen how and if object audio will be fully established in practice.

Finally, there is the consideration of loudness. International standards now regulate the loudness of audio for television delivery across the world, and cinema sound is now moving towards similar recommended practices. This raises a potential problem, since a 7.1 original mix, 5.1 TV reversion, and stereo TV downmix of the same audio may not all have the same loudness. This would not be a major issue if each mix could be delivered discretely, because each mix could be normalised to loudness compliance with minor global offsets but this is not standard practice. For many home listeners, as weve noted, the 5.1 mix is delivered to the consumer STB where it is automatically downmixed into stereo. To further confuse the situation, the difference in loudness between the surround and downmix version is not a constant value from one program to another. Nor is the difference in loudness always in the same direction; it can be louder or quieter for a number of different reasons. Plus, the high degree of centre channel dialogue isolation often demanded for television broadcast can further exacerbate this loudness disparity. With a possible difference of several LU in either direction, in some situations we might see a re-introduction of loudness jumps from one program to the next simply due to differing loudness in the consumer downmix. Its critically important to gain an understanding of this difference in loudness during the upmix if delivery specifications are to be met; yet little research or recommended practice is currently available on the matter.

In summary, as much as any other aspect of audio production, upmixing now needs to be seen in the context of the digital content everywhere consumer revolution. Gone are the days of isolated production for a single playout scenario. Cinematic releases are routinely re-purposed for television, and premium television content broadcast in 5.1 is routinely downmixed into stereo and then further re-purposed for mobile listening. An understanding of these different processes and how the original mix will translate is an important part of the upmix decision if creative intent and consumer satisfaction are to be preserved throughout the distribution chain.


Tags: iss109 | upmixing | audio post | halo upmix | plugin | nugen | Jon Schorah
Contributing Author Jon Schorah

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • HHB and Soundfield at BVE North 2011

    HHB and Soundfield at BVE North 2011

  • Nugen Audio Loudness Meter, Halo Upmix and Halo Downmix at NAB 2018

    Nugen Audio Loudness Meter, Halo Upmix and Halo Downmix at NAB 2018

  • Nugen Halo Upmix and AMB Updates at IBC 2017

    Nugen Halo Upmix and AMB Updates at IBC 2017

  • Nugen Audio Halo Upmix at BVE 2016

    Nugen Audio Halo Upmix at BVE 2016

  • Nugen Audio at NAB 2016

    Nugen Audio at NAB 2016

  • NUGEN Audio at IBC 2015

    NUGEN Audio at IBC 2015

  • RTW at IBC 2014

    RTW at IBC 2014

  • Doremi at IBC2011

    Doremi at IBC2011

  • 7.1.2 Dolby Atmos support by Halo from Nugen Audio at NAB 2017

    7.1.2 Dolby Atmos support by Halo from Nugen Audio at NAB 2017

  • Nugen Audio at IBC 2016

    Nugen Audio at IBC 2016

  • Nugen Audio Loudness Toolkit at BVE 2016

    Nugen Audio Loudness Toolkit at BVE 2016

  • NUGEN Loudness Toolkit at NAB 2015

    NUGEN Loudness Toolkit at NAB 2015

  • NUGEN Audio at BVE 2015

    NUGEN Audio at BVE 2015

  • Nugen Audio at NAB 2014

    Nugen Audio at NAB 2014

  • NuGen Audio at IBC 2013

    NuGen Audio at IBC 2013

  • NUGEN Audio: Loudness Toolkit at NAB 2013

    NUGEN Audio: Loudness Toolkit at NAB 2013

  • Nugen Audio at IBC 2012

    Nugen Audio at IBC 2012


Related Shows
  • NAB 2013 Day 1

    NAB 2013 Day 1


Articles
Shedding Light on the Blackmagic Pocket Cinema Camera 4k BMCPP4K
Garth de Bruno Austin “What is it about light that has us craving it?” Is the question asked in the opening seconds of Garth de Bruno Austin’s latest short, The Colour of Light. Exploring this natural, human need as well as our innate desire to control it, Garth’s film showcases everyday people going about their lives in differing degrees of luminance, whether that be an artificial streetlight or a natural morning sunrise.
Tags: iss134 | blackmagic | cinema camera | 4k | cpp4k | Garth de Bruno Austin
Contributing Author Garth de Bruno Austin Click to read or download PDF
Using Wireless Transmission
Jeremy Benning Wireless acquisition is a staple of live sports, entertainment and reality shows where cable free capture permits shots not previously possible, for health and safety reasons, and gives the camera-operator greater artistic licence to roam. The same is increasingly true of narrative drama where cinematographers are keen to work handheld or Steadicam where that helps tell the story. Any equipment which frees their movement and time by being lighter, easier to use and reliable in performance is going to tick a lot of boxes.
Tags: iss134 | wireless | 4k | transmission | Jeremy Benning
Contributing Author Jeremy Benning Click to read or download PDF
Accelerated Workflows with eGPU
Mike Griggs From the UK’s National Trust to magazine publishers to manufacturers, digital content creator Mike Griggs has a wide and varied portfolio of clients for whom he creates 3D art, motion graphics and multimedia exhibits. A typical day might involve sampling birdsong near Virginia Woolf’s country estate or creating 3D animations for VR. To keep on top of these demands, Griggs wanted to take the full power of the GPU computing revolution on the road.
Tags: iss134 | sonnet | egpu | amd | post production | editing | Mike Griggs
Contributing Author Mike Griggs Click to read or download PDF
An Obituary to Timecode
Bruce Devlin - new A stoic and persistent character that stubbornly refused to change with the times, Timecode has finally passed on, but no-one has noticed. A long-lasting industry veteran, Timecode was brought into this world at an uncertain date in the late 1960s due to the needs of analogue tape workflows and the demand for synchronisation between audio and video devices. A joint activity between SMPTE and the EBU led to the work on Time and Control codes starting its journey to standardisation in the early 1970s.
Tags: iss134 | timecode | smpte | ebu | edit | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read
The Wireless Way to 4k
JP Delport DTC’s AEON group of products have been specifically designed for the 4K market. We encode with the more efficient HEVC algorithm, which means we are taking a 12G signal and compressing it to a bitrate that can be managed over an RF link. So what makes this a leading idea in the 4K revolution?
Tags: iss134 | wireless | 4k | transmission | JP Delport
Contributing Author JP Delport Click to read or download PDF