Full reference testing helps ensure quality audio delivery


Adam Schadle TV-Bay Magazine
Read ezine online
by Adam Schadle
Issue 101 - May 2015

With the ever-increasing amount and variety of programming, devices, screen sizes, quality levels, and delivery methods today, broadcasters and delivery networks must be able to measure the quality of audiovisual content - and their processes for delivering it - to ensure it will meet standards that satisfy audiences, subscribers, advertisers, and regulators. One of the most accurate methods for assessing changes in quality is full-reference testing, which compares original content to processed content. Full-reference testing produces very accurate and highly repeatable results at a much lower cost than subjective testing with human listeners. Astute content distributors recognize that fact and are using full-reference testing to help them determine the best combination of equipment and delivery schemes to use in their operations.

In terms of selecting an appropriate delivery bit rate, the workflow is fairly straightforward (Figure 1): An A/V source player feeds an uncompressed signal to the encoder's input. The compressed output of the encoder can be fed directly to the decoder's input and captured in real time to form one element in a set of full-reference test sequences. Additional outputs are then captured and added to the sequence set by making small-step changes in the encoder bit rate. You can create more full-reference test sequence sets by using different source content and repeating the process of capturing and storing the decoder output sequences. (This workflow can also be performed as a file process in place of real-time play/record functions.)

Once the full-reference comparisons are made for various bit rates, differences in audio quality caused by each bit-rate change can be measured in each sequence set using objective perceptual techniques such as PEAQ (Perceptual Evaluation of Audio Quality). The PEAQ model, an international standard for audio quality measurement, was calibrated to deliver scores that correlate with the mean opinion scores of a panel of expert listeners who took part in a formal subjective test. In this sense, PEAQ can be viewed as an average expert listener score. By plotting the scores for each set of sequences on a graph, it's easy to see the quality trend for each bit rate under comparison.

PEAQ is optimized to measure the perceptual quality of wideband (20 kHz) audio codecs and has been successfully used to measure the quality of other wideband audio devices such as analog-to-digital converters, digital-to-analog converters, and sample-rate converters.

The mean quality rating is either scored as a 0 to -4 or 1 to 5. The score is an objective differential grade (ODG), which represents the overall severity of the impairments in the test signal. Its meaning is defined on the rating scales of Figure 2.

This objective, full-reference test measurement will provide a quality value for the corresponding audio in each frame of video. These frame-by-frame quality values can be compared directly to the quality values from other elements of the same test sequence set (since they all used the same source sequence), or they can be compared to other sets that have been generated using the same source sequence at another bit rate. If objective measurements vary greatly from one bit rate to another in portions of the output sequences, then you can examine them more extensively using subjective techniques.

The full-reference test and measurement process for evaluating codec technologies is very similar to the one for choosing a bit rate, except that you change the codec engine for each test instead of the bit rate (Figure 3). After evaluating the first codec (A), simply insert a second codec (B) into the test workflow and start again. From there PEAQ measurements can be made to identify trends in the average sequence scores as a result of the different codecs.

By using full-reference testing and the PEAQ audio quality scoring model, content originators can get a clear idea of how close a processed audio signal is to the original, and how it is affected by changes in different delivery variables. The results of these tests help them to ensure a highly accurate measurement of quality for audio delivery.


Tags: iss101 | audio delivery | full reference | workflow | PEAQ | bit rates | Adam Schadle
Contributing Author Adam Schadle

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Cinegy 8k daniel2 workflow from glass to glass at IBC 2019

    Cinegy 8k daniel2 workflow from glass to glass at IBC 2019

  • ChyronHego show their PRIME Workflow at IBC 2018

    ChyronHego show their PRIME Workflow at IBC 2018

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Snell Advanced Media (SAM) 4k workflow at IBC 2015

    Snell Advanced Media (SAM) 4k workflow at IBC 2015

  • LIVEU Workflow solutions at NAB 2015

    LIVEU Workflow solutions at NAB 2015

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Streaming Media with Interra Systems at NAB 2017

    Streaming Media with Interra Systems at NAB 2017

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • ERA - Adobe Anywhere - at BVE 2015

    ERA - Adobe Anywhere - at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • Quantel LiveTouch at IBC 2014

    Quantel LiveTouch at IBC 2014

  • Quantel deal with AFP at IBC 2014

    Quantel deal with AFP at IBC 2014

  • Snell Kahuna Production Switcher at IBC 2014

    Snell Kahuna Production Switcher at IBC 2014

  • Telestream Wirecast and Switch at IBC 2014

    Telestream Wirecast and Switch at IBC 2014

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Broadstream Solutions at NAB 2014

    Broadstream Solutions at NAB 2014

  • Telestream Enterprise at NAB 2014

    Telestream Enterprise at NAB 2014

  • Telestream Switch at NAB 2014

    Telestream Switch at NAB 2014

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Yospace: Advert Insertion at NAB 2013

    Yospace: Advert Insertion at NAB 2013

  • Vimond Control Center at NAB 2013

    Vimond Control Center at NAB 2013

  • Telestream Vantage v5 at NAB 2013

    Telestream Vantage v5 at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Telestream at BVE 2013

    Telestream at BVE 2013

  • Guntermann and Drunck GmbH at BVE 2013

    Guntermann and Drunck GmbH at BVE 2013

  • Tedial at NAB 2012

    Tedial at NAB 2012

  • Marquis Broadcast at NAB 2012

    Marquis Broadcast at NAB 2012

  • Root6 at NAB 2012

    Root6 at NAB 2012

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • TMD at NAB 2012

    TMD at NAB 2012

  • PlayBox Technology at BVE 2012

    PlayBox Technology at BVE 2012

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Net Insight at IBC2011

    Net Insight at IBC2011

  • Tedial at IBC2011

    Tedial at IBC2011

  • IPV at IBC2011

    IPV at IBC2011

  • TSL at IBC2011

    TSL at IBC2011


Related Shows
  • Post Covid-19 workflows for media & entertainment production.

    Post Covid-19 workflows for media & entertainment production.


Articles
The Future of Broadcast Technology
Manfred Reitmeier

Now that OTT and VOD have become more mainstream, many commentators talk about traditional broadcast methods, like terrestrial transmission, being a thing of the past. With so many new platforms and non-traditional content services carving out a growing slice of the market, you can be forgiven for thinking that linear over-the-air television is on its way out. The reality is that the industry must strike a balance between meeting consumers’ shifting habits and the business and operational needs of content providers.

Tags: Rohde Schwarz | 5g broadcast | Manfred Reitmeier
Contributing Author Manfred Reitmeier Click to read
A switch in time: how KVM can unlock the future of broadcasting
Chris Smeeton

One of the major changes for broadcasters during the pandemic has been the shift towards remote production; by no means a new phenomenon in an IP environment, yet accelerated under lockdown to accommodate travel and gathering restrictions. A 2021 report found that almost 40% of broadcast professionals now employ remote production, up 9% on the previous year.

Tags: KVM | ARGOSY | GDSYS | KVM Tech | Chris Smeeton
Contributing Author Chris Smeeton Click to read
Rohde & Schwarz integration with Adobe Premiere Pro
Ciaran Doran

Rohde & Schwarz SpycerNode storage & VENICE ingest are perfect partners for Adobe workflows

Tags: Rohde Schwarz | media | VENICE | SpycerNode | adobe | premier pro | Ciaran Doran
Contributing Author Ciaran Doran Click to read
Cloud integration: The only way is up
Francois Vaillant

In 2017 we saw the introduction of SMPTE 2110 and since then, the transition to IP has emerged as a priority for broadcasters looking to streamline their operations, decrease their footprint and effectively integrate with an industry in a rapid state of flux. A 2020 survey by Devoncroft found that almost 20% of broadcasters had already deployed SMPTE 2110, and over 25% were planning to do so. Then came the pandemic, precipitating the need for on-the-fly collaboration and accelerating the transition to remote, virtual and cloud production.

Tags: dejero | smpte 2110 | dazzl | streaming | Francois Vaillant
Contributing Author Francois Vaillant Click to read
The Cloud - a measured approach
Ciaran Doran

With the buzz of ‘cloud’ everywhere in our industry it would be natural to think that cloud is the only game in town. Isn’t it time to step back and consider very carefully how, or whether, you make that journey to the cloud?

Tags: cloud | rohde and schwarz | rohde | schwarz | Ciaran Doran
Contributing Author Ciaran Doran Click to read

Registration for the KitPlus Show at MediaCity is now OPEN with headline sponsor Blackmagic Design. The doors open from 10am to 4:30pm on 23rd June 2022. This is a FREE to attend event so please visit the Manchester venue page to see exhibitors, sessions and workshops, plus check out our Virtual Production Learning Zone from the Mo-Sys Academy at the show.