Full reference testing helps ensure quality audio delivery


Adam Schadle TV-Bay Magazine
Read ezine online
by Adam Schadle
Issue 101 - May 2015

With the ever-increasing amount and variety of programming, devices, screen sizes, quality levels, and delivery methods today, broadcasters and delivery networks must be able to measure the quality of audiovisual content - and their processes for delivering it - to ensure it will meet standards that satisfy audiences, subscribers, advertisers, and regulators. One of the most accurate methods for assessing changes in quality is full-reference testing, which compares original content to processed content. Full-reference testing produces very accurate and highly repeatable results at a much lower cost than subjective testing with human listeners. Astute content distributors recognize that fact and are using full-reference testing to help them determine the best combination of equipment and delivery schemes to use in their operations.

In terms of selecting an appropriate delivery bit rate, the workflow is fairly straightforward (Figure 1): An A/V source player feeds an uncompressed signal to the encoder's input. The compressed output of the encoder can be fed directly to the decoder's input and captured in real time to form one element in a set of full-reference test sequences. Additional outputs are then captured and added to the sequence set by making small-step changes in the encoder bit rate. You can create more full-reference test sequence sets by using different source content and repeating the process of capturing and storing the decoder output sequences. (This workflow can also be performed as a file process in place of real-time play/record functions.)

Once the full-reference comparisons are made for various bit rates, differences in audio quality caused by each bit-rate change can be measured in each sequence set using objective perceptual techniques such as PEAQ (Perceptual Evaluation of Audio Quality). The PEAQ model, an international standard for audio quality measurement, was calibrated to deliver scores that correlate with the mean opinion scores of a panel of expert listeners who took part in a formal subjective test. In this sense, PEAQ can be viewed as an average expert listener score. By plotting the scores for each set of sequences on a graph, it's easy to see the quality trend for each bit rate under comparison.

PEAQ is optimized to measure the perceptual quality of wideband (20 kHz) audio codecs and has been successfully used to measure the quality of other wideband audio devices such as analog-to-digital converters, digital-to-analog converters, and sample-rate converters.

The mean quality rating is either scored as a 0 to -4 or 1 to 5. The score is an objective differential grade (ODG), which represents the overall severity of the impairments in the test signal. Its meaning is defined on the rating scales of Figure 2.

This objective, full-reference test measurement will provide a quality value for the corresponding audio in each frame of video. These frame-by-frame quality values can be compared directly to the quality values from other elements of the same test sequence set (since they all used the same source sequence), or they can be compared to other sets that have been generated using the same source sequence at another bit rate. If objective measurements vary greatly from one bit rate to another in portions of the output sequences, then you can examine them more extensively using subjective techniques.

The full-reference test and measurement process for evaluating codec technologies is very similar to the one for choosing a bit rate, except that you change the codec engine for each test instead of the bit rate (Figure 3). After evaluating the first codec (A), simply insert a second codec (B) into the test workflow and start again. From there PEAQ measurements can be made to identify trends in the average sequence scores as a result of the different codecs.

By using full-reference testing and the PEAQ audio quality scoring model, content originators can get a clear idea of how close a processed audio signal is to the original, and how it is affected by changes in different delivery variables. The results of these tests help them to ensure a highly accurate measurement of quality for audio delivery.


Tags: iss101 | audio delivery | full reference | workflow | PEAQ | bit rates | Adam Schadle
Contributing Author Adam Schadle

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • ChyronHego show their PRIME Workflow at IBC 2018

    ChyronHego show their PRIME Workflow at IBC 2018

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Snell Advanced Media (SAM) 4k workflow at IBC 2015

    Snell Advanced Media (SAM) 4k workflow at IBC 2015

  • LIVEU Workflow solutions at NAB 2015

    LIVEU Workflow solutions at NAB 2015

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Streaming Media with Interra Systems at NAB 2017

    Streaming Media with Interra Systems at NAB 2017

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • ERA - Adobe Anywhere - at BVE 2015

    ERA - Adobe Anywhere - at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • Quantel LiveTouch at IBC 2014

    Quantel LiveTouch at IBC 2014

  • Quantel deal with AFP at IBC 2014

    Quantel deal with AFP at IBC 2014

  • Snell Kahuna Production Switcher at IBC 2014

    Snell Kahuna Production Switcher at IBC 2014

  • Telestream Wirecast and Switch at IBC 2014

    Telestream Wirecast and Switch at IBC 2014

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Broadstream Solutions at NAB 2014

    Broadstream Solutions at NAB 2014

  • Telestream Enterprise at NAB 2014

    Telestream Enterprise at NAB 2014

  • Telestream Switch at NAB 2014

    Telestream Switch at NAB 2014

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Yospace: Advert Insertion at NAB 2013

    Yospace: Advert Insertion at NAB 2013

  • Vimond Control Center at NAB 2013

    Vimond Control Center at NAB 2013

  • Telestream Vantage v5 at NAB 2013

    Telestream Vantage v5 at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Telestream at BVE 2013

    Telestream at BVE 2013

  • Guntermann and Drunck GmbH at BVE 2013

    Guntermann and Drunck GmbH at BVE 2013

  • Tedial at NAB 2012

    Tedial at NAB 2012

  • Marquis Broadcast at NAB 2012

    Marquis Broadcast at NAB 2012

  • Root6 at NAB 2012

    Root6 at NAB 2012

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • TMD at NAB 2012

    TMD at NAB 2012

  • PlayBox Technology at BVE 2012

    PlayBox Technology at BVE 2012

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Net Insight at IBC2011

    Net Insight at IBC2011

  • Tedial at IBC2011

    Tedial at IBC2011

  • IPV at IBC2011

    IPV at IBC2011

  • TSL at IBC2011

    TSL at IBC2011


Related Shows
  • Bubble and Squeak: Live and Loud from Monday at IBC 2013

    Bubble and Squeak: Live and Loud from Monday at IBC 2013


Articles
The Biggest Toy Shop in the World
Emma Morrison When Nigel Woodford started his career at BBC Wood Norton in 1962, television was black and white and BBC Two had not yet been launched. In 2018 Nigel will retire, and Richmond Film Services, the pro-audio equipment rental company set up by Nigel in 1973, can count numerous contributions to iconic moments in British cultural, sporting and film history over this time.
Tags: iss133 | richmond film services | audio rental | auction | liquidity | go-dove | Emma Morrison
Contributing Author Emma Morrison Click to read or download PDF
Ride along on the Tour de Tech at IBC
Lorna Garrett Finding your way through the halls and aisles of the massive IBC can feel a bit like attempting to complete all 21 stages of the Tour de France — but in far fewer days. But have no fear; your team at Garland is all geared up and ready to show you the best on show at this year's exhibition.
Tags: iss133 | garland | liveu | lu600 | media excel | teracue | wisi | broadcast wireless systems | bws | artel | ibc | Lorna Garrett
Contributing Author Lorna Garrett Click to read or download PDF
Creating Authentic Content That Counts
Frank le Mair We’re deep into the 2010’s and the way in which we consume content has changed forever. Broadcasters and content owners are fighting for eyeballs in a saturated market where consumers are watching their favourite shows on different devices across a number of platforms. To target millennials and younger generations, who generally consume short bursts of video on YouTube and social media - particularly Snapchat, Instagram and now IGTV - media companies are creating more and more authentic stories and are using platforms that are compelling for this demographic. Unlike generations before them, they have totally new video viewing patterns and ideals.
Tags: iss133 | insight tv | monster energy | amazon | millennial | Frank le Mair
Contributing Author Frank le Mair Click to read or download PDF
Peli Air 1507 Review
Phil Vinter Originally named after a bird that carries its precious cargo through the skies, it was, perhaps, only a matter of time before Peli released its Air range.
Tags: iss133 | peli | peli air 1507 | trekpak | Phil Vinter
Contributing Author Phil Vinter Click to read or download PDF
The Pace of Change
Dick Hobbs - new

The youngest human to stand on the moon (so far) was Charles Moss, the lunar module pilot of Apollo 16. Charlie had a wonderful claim: his father witnessed the Wright Brothers’ first flight at Kitty Hawk, and lived to see his son on the moon.

Does anything capture the speed of technological advance better than that? The whole of the history of powered flight in one lifetime.

Tags: iss133 | state of the nation | st2110 | st2110-10 | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF