New Upmixing Challenges and Strategies for Audio Post-production Today and Tomorrow


Jon Schorah TV-Bay Magazine
Read ezine online
by Jon Schorah
Issue 109 - January 2016
Following the successful introduction of Halo Upmix, NUGEN Audios new upmixing plugin, Creative Director and Co-Founder Jon Schorah reflects on the changing landscape and increasingly complex demands of the modern audio post-production environment.

The audio post-production engineer survives in a demanding and complex world where time is at a premium; yet, these professionals are required to deliver results at the highest quality levels as a matter of routine. Its essential to design concise and efficient workflows, and the production and delivery of surround mixes is no exception. Day-to-day TV programming is produced to a tight deadline and budget. While upmixing original stereo programming to surround has become routine, digital distribution is demanding an increasingly robust upmix for a number of reasons. This in turn adds new complexity and places additional demands upon a successful upmix process.

First and foremost, the process must be able to provide a good upmix from the original source material. Of course, good is somewhat subjective, but one simple tenet holds true - the upmix must respect the original source material, providing a naturally extended panorama that unfolds the original into a surround context. This is very important when the original source is a broadcast mix that is being translated for broadcast in surround. Here creative decisions have been made, and these preferences need to be respected and represented in the resultant upmix. A strong relationship between the original source material and the upmix not only respects creative intent but also delivers a coherent experience to the consumer.

Secondly, there is the issue of dialogue isolation. Most delivery specifications require a high degree of dialogue isolation into the centre channel for aesthetic purposes and consistency of delivery, but achieving this is not always a simple process. In many situations, separate stems are not available, and with archive restoration, the stems may not have existed in the first place. Historically, upmix processes have applied rather brute-force methods for achieving this end, resulting in unfortunate side effects for the mix as a whole. Recent increases in computing power, however, have led to great improvements in processes that facilitate this isolation. Techniques that go far beyond simple mid-side and frequency isolation are now possible, with minimal over-isolation and transparent transmission of directional content passed to the sides.

A third requirement has recently come into play, largely as a result of digital distribution methods. An upmix must, in most cases, be highly downmix-compatible. This means that a downmixed version of the upmixed audio (i.e. the upmix collapsed back down to stereo) must still sound very close, if not identical, to the original for all intents and purposes. There are many processes that can be used to generate a full-sounding upmix, but some of these can cause serious downmix compatibility issues. For original TV production, the 5.1 downmix needs to correlate very closely to the original stereo source, since the consumer stereo version is commonly produced at the set-top box level as a downmix of the broadcast surround. Even in the context of a new full surround mix, in which the downmix will not be directly related to a single upmixed audio source, it is still an important requirement to deliver a down-mixed track free from unnatural- sounding delay and phase artefacts.

The upmix process also needs to be flexible. Traditionally, the majority of cinematic releases have used a 7.1 surround configuration, with a 5.1 configuration for TV. For film audio engineers, a 7.1 upmix solution is perfectly adequate; however, many freelance engineers routinely offer services to both the film and television industries and therefore need greater upmix flexibility. For those with an eye on the future, the advent of object audio offers an enticing 9.1 bed track, including additional information for overhead loudspeaker arrays. According to one manufacturers technical white paper, in theory an object-based mix, including a 9.1 bed, can be automatically translated to 7.1, 5.1, and stereo formats allowing the audio professional to author once, optimise everywhere. This has the potential of introducing a universally translatable mix, but it remains to be seen how and if object audio will be fully established in practice.

Finally, there is the consideration of loudness. International standards now regulate the loudness of audio for television delivery across the world, and cinema sound is now moving towards similar recommended practices. This raises a potential problem, since a 7.1 original mix, 5.1 TV reversion, and stereo TV downmix of the same audio may not all have the same loudness. This would not be a major issue if each mix could be delivered discretely, because each mix could be normalised to loudness compliance with minor global offsets but this is not standard practice. For many home listeners, as weve noted, the 5.1 mix is delivered to the consumer STB where it is automatically downmixed into stereo. To further confuse the situation, the difference in loudness between the surround and downmix version is not a constant value from one program to another. Nor is the difference in loudness always in the same direction; it can be louder or quieter for a number of different reasons. Plus, the high degree of centre channel dialogue isolation often demanded for television broadcast can further exacerbate this loudness disparity. With a possible difference of several LU in either direction, in some situations we might see a re-introduction of loudness jumps from one program to the next simply due to differing loudness in the consumer downmix. Its critically important to gain an understanding of this difference in loudness during the upmix if delivery specifications are to be met; yet little research or recommended practice is currently available on the matter.

In summary, as much as any other aspect of audio production, upmixing now needs to be seen in the context of the digital content everywhere consumer revolution. Gone are the days of isolated production for a single playout scenario. Cinematic releases are routinely re-purposed for television, and premium television content broadcast in 5.1 is routinely downmixed into stereo and then further re-purposed for mobile listening. An understanding of these different processes and how the original mix will translate is an important part of the upmix decision if creative intent and consumer satisfaction are to be preserved throughout the distribution chain.


Tags: iss109 | upmixing | audio post | halo upmix | plugin | nugen | Jon Schorah
Contributing Author Jon Schorah

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • HHB and Soundfield at BVE North 2011

    HHB and Soundfield at BVE North 2011

  • Nugen Audio Loudness Meter, Halo Upmix and Halo Downmix at NAB 2018

    Nugen Audio Loudness Meter, Halo Upmix and Halo Downmix at NAB 2018

  • Nugen Halo Upmix and AMB Updates at IBC 2017

    Nugen Halo Upmix and AMB Updates at IBC 2017

  • Nugen Audio Halo Upmix at BVE 2016

    Nugen Audio Halo Upmix at BVE 2016

  • Nugen Audio at NAB 2016

    Nugen Audio at NAB 2016

  • NUGEN Audio at IBC 2015

    NUGEN Audio at IBC 2015

  • RTW at IBC 2014

    RTW at IBC 2014

  • Doremi at IBC2011

    Doremi at IBC2011

  • 7.1.2 Dolby Atmos support by Halo from Nugen Audio at NAB 2017

    7.1.2 Dolby Atmos support by Halo from Nugen Audio at NAB 2017

  • Nugen Audio at IBC 2016

    Nugen Audio at IBC 2016

  • Nugen Audio Loudness Toolkit at BVE 2016

    Nugen Audio Loudness Toolkit at BVE 2016

  • NUGEN Loudness Toolkit at NAB 2015

    NUGEN Loudness Toolkit at NAB 2015

  • NUGEN Audio at BVE 2015

    NUGEN Audio at BVE 2015

  • Nugen Audio at NAB 2014

    Nugen Audio at NAB 2014

  • NuGen Audio at IBC 2013

    NuGen Audio at IBC 2013

  • NUGEN Audio: Loudness Toolkit at NAB 2013

    NUGEN Audio: Loudness Toolkit at NAB 2013

  • Nugen Audio at IBC 2012

    Nugen Audio at IBC 2012


Related Shows
  • NAB 2013 Day 1

    NAB 2013 Day 1


Articles
Review: JVC GY-HC900 Camera
Andy McKenzie JVC has a long history of developing relatively low price-point equipment, dating right back to its success in the 1970s with the VHS tape format. JVC products sometimes impact the broadcast mainstream but generally find greatest success in niche market segments such as news production where price and portability take priority over the no-comprise quality and robustness demanded by the leading studio-based and OB content producers.
Tags: iss138 | jvc review | camera | sdhc | sdxc | mpeg-2 | nab | gy-hc900 | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF
How the Womens World Cup was Boosted by Broadcast
John Griffiths As we write this article, England’s Lionesses have just beaten Norway 3-0 and are smashing viewing figure records, with 6.9 million viewers tuning in to watch them play Cameroon on BBC One. Rewind to the previous World Cup in 2015, and England’s group and early stage knockout games tempted up to 2.5 million viewers for each match. It’s safe to say that women’s football is finally having its moment; what was perhaps once considered a niche sport is finally gaining momentum in the mainstream space with broadcasters giving it the attention it deserves.
Tags: iss138 | wwc | world cup | spicymango | diversity | John Griffiths
Contributing Author John Griffiths Click to read or download PDF
Making the Workflow Flow
Bruce Devlin - new The toughest things about being the Standards Vice President (SVP) is that everyone expects standards to be the most important thing. In all the systems that I’ve designed and deployed over the years, I've yet to find any production workflow that is 100% standards based. True, the core technologies may well be standards based, but the overall workflow is made up of many technology pieces from open source code, through de-facto delivery specifications based upon SMPTE or Trade Association Specifications that in turn depend on full, International Standards to work. I can already hear some folks saying "In the good old days, everything used standards", but I beg to disagree.
Tags: iss138 | pye museum | pye-philips | smpte | ietf | ieee | w3c | aes | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF
TVFutures. No More Comfort Zone - Get a Placement
Michelle Brown I have often heard that any chance of getting work in the television industry is based quite heavily on who you know, and whilst that might indeed be true, I’ve also come to the realisation that it may also depends on what opportunities you seize and how far out of your comfort zone you might be prepared to push yourself.
Tags: iss138 | portsmouth uni | creative careers | placement | Michelle Brown
Contributing Author Michelle Brown Click to read or download PDF
State of the Nation: Wings, Fine Coffee and Fake Nudes
Dick Hobbs - new One of the many attractions of this time of year is that I get to meet with my fellow judges for the IBC Innovation Awards, and see what the industry sees as the most important, most forward-looking projects of the day. For me, this says what the key talking points are going to be in September, far more eloquently than the endless press releases from vendors which are already tumbling into my inbox.
Tags: iss138 | ibc | smpte 2110 | deepnude | cisco | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF