New Upmixing Challenges and Strategies for Audio Post-production Today and Tomorrow


Jon Schorah TV-Bay Magazine
Read ezine online
by Jon Schorah
Issue 109 - January 2016
Following the successful introduction of Halo Upmix, NUGEN Audios new upmixing plugin, Creative Director and Co-Founder Jon Schorah reflects on the changing landscape and increasingly complex demands of the modern audio post-production environment.

The audio post-production engineer survives in a demanding and complex world where time is at a premium; yet, these professionals are required to deliver results at the highest quality levels as a matter of routine. Its essential to design concise and efficient workflows, and the production and delivery of surround mixes is no exception. Day-to-day TV programming is produced to a tight deadline and budget. While upmixing original stereo programming to surround has become routine, digital distribution is demanding an increasingly robust upmix for a number of reasons. This in turn adds new complexity and places additional demands upon a successful upmix process.

First and foremost, the process must be able to provide a good upmix from the original source material. Of course, good is somewhat subjective, but one simple tenet holds true - the upmix must respect the original source material, providing a naturally extended panorama that unfolds the original into a surround context. This is very important when the original source is a broadcast mix that is being translated for broadcast in surround. Here creative decisions have been made, and these preferences need to be respected and represented in the resultant upmix. A strong relationship between the original source material and the upmix not only respects creative intent but also delivers a coherent experience to the consumer.

Secondly, there is the issue of dialogue isolation. Most delivery specifications require a high degree of dialogue isolation into the centre channel for aesthetic purposes and consistency of delivery, but achieving this is not always a simple process. In many situations, separate stems are not available, and with archive restoration, the stems may not have existed in the first place. Historically, upmix processes have applied rather brute-force methods for achieving this end, resulting in unfortunate side effects for the mix as a whole. Recent increases in computing power, however, have led to great improvements in processes that facilitate this isolation. Techniques that go far beyond simple mid-side and frequency isolation are now possible, with minimal over-isolation and transparent transmission of directional content passed to the sides.

A third requirement has recently come into play, largely as a result of digital distribution methods. An upmix must, in most cases, be highly downmix-compatible. This means that a downmixed version of the upmixed audio (i.e. the upmix collapsed back down to stereo) must still sound very close, if not identical, to the original for all intents and purposes. There are many processes that can be used to generate a full-sounding upmix, but some of these can cause serious downmix compatibility issues. For original TV production, the 5.1 downmix needs to correlate very closely to the original stereo source, since the consumer stereo version is commonly produced at the set-top box level as a downmix of the broadcast surround. Even in the context of a new full surround mix, in which the downmix will not be directly related to a single upmixed audio source, it is still an important requirement to deliver a down-mixed track free from unnatural- sounding delay and phase artefacts.

The upmix process also needs to be flexible. Traditionally, the majority of cinematic releases have used a 7.1 surround configuration, with a 5.1 configuration for TV. For film audio engineers, a 7.1 upmix solution is perfectly adequate; however, many freelance engineers routinely offer services to both the film and television industries and therefore need greater upmix flexibility. For those with an eye on the future, the advent of object audio offers an enticing 9.1 bed track, including additional information for overhead loudspeaker arrays. According to one manufacturers technical white paper, in theory an object-based mix, including a 9.1 bed, can be automatically translated to 7.1, 5.1, and stereo formats allowing the audio professional to author once, optimise everywhere. This has the potential of introducing a universally translatable mix, but it remains to be seen how and if object audio will be fully established in practice.

Finally, there is the consideration of loudness. International standards now regulate the loudness of audio for television delivery across the world, and cinema sound is now moving towards similar recommended practices. This raises a potential problem, since a 7.1 original mix, 5.1 TV reversion, and stereo TV downmix of the same audio may not all have the same loudness. This would not be a major issue if each mix could be delivered discretely, because each mix could be normalised to loudness compliance with minor global offsets but this is not standard practice. For many home listeners, as weve noted, the 5.1 mix is delivered to the consumer STB where it is automatically downmixed into stereo. To further confuse the situation, the difference in loudness between the surround and downmix version is not a constant value from one program to another. Nor is the difference in loudness always in the same direction; it can be louder or quieter for a number of different reasons. Plus, the high degree of centre channel dialogue isolation often demanded for television broadcast can further exacerbate this loudness disparity. With a possible difference of several LU in either direction, in some situations we might see a re-introduction of loudness jumps from one program to the next simply due to differing loudness in the consumer downmix. Its critically important to gain an understanding of this difference in loudness during the upmix if delivery specifications are to be met; yet little research or recommended practice is currently available on the matter.

In summary, as much as any other aspect of audio production, upmixing now needs to be seen in the context of the digital content everywhere consumer revolution. Gone are the days of isolated production for a single playout scenario. Cinematic releases are routinely re-purposed for television, and premium television content broadcast in 5.1 is routinely downmixed into stereo and then further re-purposed for mobile listening. An understanding of these different processes and how the original mix will translate is an important part of the upmix decision if creative intent and consumer satisfaction are to be preserved throughout the distribution chain.


Tags: iss109 | upmixing | audio post | halo upmix | plugin | nugen | Jon Schorah
Contributing Author Jon Schorah

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • HHB and Soundfield at BVE North 2011

    HHB and Soundfield at BVE North 2011

  • Nugen Audio Loudness Meter, Halo Upmix and Halo Downmix at NAB 2018

    Nugen Audio Loudness Meter, Halo Upmix and Halo Downmix at NAB 2018

  • Nugen Halo Upmix and AMB Updates at IBC 2017

    Nugen Halo Upmix and AMB Updates at IBC 2017

  • Nugen Audio Halo Upmix at BVE 2016

    Nugen Audio Halo Upmix at BVE 2016

  • Nugen Audio at NAB 2016

    Nugen Audio at NAB 2016

  • NUGEN Audio at IBC 2015

    NUGEN Audio at IBC 2015

  • RTW at IBC 2014

    RTW at IBC 2014

  • Doremi at IBC2011

    Doremi at IBC2011

  • VisLM Loudness Meter update from Nugen Audio at IBC 2019

    VisLM Loudness Meter update from Nugen Audio at IBC 2019

  • 7.1.2 Dolby Atmos support by Halo from Nugen Audio at NAB 2017

    7.1.2 Dolby Atmos support by Halo from Nugen Audio at NAB 2017

  • Nugen Audio at IBC 2016

    Nugen Audio at IBC 2016

  • Nugen Audio Loudness Toolkit at BVE 2016

    Nugen Audio Loudness Toolkit at BVE 2016

  • NUGEN Loudness Toolkit at NAB 2015

    NUGEN Loudness Toolkit at NAB 2015

  • NUGEN Audio at BVE 2015

    NUGEN Audio at BVE 2015

  • Nugen Audio at NAB 2014

    Nugen Audio at NAB 2014

  • NuGen Audio at IBC 2013

    NuGen Audio at IBC 2013

  • NUGEN Audio: Loudness Toolkit at NAB 2013

    NUGEN Audio: Loudness Toolkit at NAB 2013

  • Nugen Audio at IBC 2012

    Nugen Audio at IBC 2012


Related Shows
  • NAB 2013 Day 1

    NAB 2013 Day 1


Articles
Connect A Broadcast-Quality Miniature Camera To Your Computer!
Spencer Doran Upgrade any home working or live streaming setup with a broadcast-quality professional webcam
Tags: webcam | marshall | CV503 | USB | ZOOM | Streaming | Spencer Doran
Contributing Author Spencer Doran Click to read
An Argument for the Z Cam
Jake Ratcliffe Z Cam first dipped their toes into the cinema camera market back at NAB 2018 with the release of the E2 and since then they have proven themselves as one of the go-to camera manufacturers for affordable and unique camera solutions. Their family of cinema cameras has since evolved into a set of 6.
Tags: iss142 | cvp | cvpgroup | micro four thirds | e2c | zcam | full frame 6k | tilta | smallrig | 8sinn | zacuto | rvlvr | Jake Ratcliffe
Contributing Author Jake Ratcliffe Click to read or download PDF
State of the Nation and Virtual Social Distancing
Dick Hobbs - new Under normal circumstances, I usually make my plans for NAB in November time. It’s when the airlines tend to have sales. This year, for personal reasons, I did not get around to booking back then, and it was only at the end of February that I could start thinking about it.
Tags: iss142 | nab | kitplus show | iabm | darren whitehead | warner media | streaming | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF
KVM Trends for Broadcast Pros
Jochen Bauer G&D North America Inc., one of the world’s leading KVM manufacturers, presents their latest KVM trends 2020 for broadcast professionals. KVM systems play an essential role in broadcasting by forming the link between IT and broadcast structures and thereby offering operation across systems and independently from any platform.
Tags: iss142 | kvm | ip kvm | kvm over ip | 4k over ip | kvm matrix | remote access | Jochen Bauer
Contributing Author Jochen Bauer Click to read or download PDF
Why Renting is the New Buying
Ed Tischler To rent or to buy? This has long been a topic of debate in the world of property, with people having to choose between spending virtually all of their money up front or giving their money away each month with little chance of seeing a return.
Tags: iss142 | gravity media | ott | renting | Ed Tischler
Contributing Author Ed Tischler Click to read or download PDF