Full reference testing helps ensure quality audio delivery


Adam Schadle TV-Bay Magazine
Read ezine online
by Adam Schadle
Issue 101 - May 2015

With the ever-increasing amount and variety of programming, devices, screen sizes, quality levels, and delivery methods today, broadcasters and delivery networks must be able to measure the quality of audiovisual content - and their processes for delivering it - to ensure it will meet standards that satisfy audiences, subscribers, advertisers, and regulators. One of the most accurate methods for assessing changes in quality is full-reference testing, which compares original content to processed content. Full-reference testing produces very accurate and highly repeatable results at a much lower cost than subjective testing with human listeners. Astute content distributors recognize that fact and are using full-reference testing to help them determine the best combination of equipment and delivery schemes to use in their operations.

In terms of selecting an appropriate delivery bit rate, the workflow is fairly straightforward (Figure 1): An A/V source player feeds an uncompressed signal to the encoder's input. The compressed output of the encoder can be fed directly to the decoder's input and captured in real time to form one element in a set of full-reference test sequences. Additional outputs are then captured and added to the sequence set by making small-step changes in the encoder bit rate. You can create more full-reference test sequence sets by using different source content and repeating the process of capturing and storing the decoder output sequences. (This workflow can also be performed as a file process in place of real-time play/record functions.)

Once the full-reference comparisons are made for various bit rates, differences in audio quality caused by each bit-rate change can be measured in each sequence set using objective perceptual techniques such as PEAQ (Perceptual Evaluation of Audio Quality). The PEAQ model, an international standard for audio quality measurement, was calibrated to deliver scores that correlate with the mean opinion scores of a panel of expert listeners who took part in a formal subjective test. In this sense, PEAQ can be viewed as an average expert listener score. By plotting the scores for each set of sequences on a graph, it's easy to see the quality trend for each bit rate under comparison.

PEAQ is optimized to measure the perceptual quality of wideband (20 kHz) audio codecs and has been successfully used to measure the quality of other wideband audio devices such as analog-to-digital converters, digital-to-analog converters, and sample-rate converters.

The mean quality rating is either scored as a 0 to -4 or 1 to 5. The score is an objective differential grade (ODG), which represents the overall severity of the impairments in the test signal. Its meaning is defined on the rating scales of Figure 2.

This objective, full-reference test measurement will provide a quality value for the corresponding audio in each frame of video. These frame-by-frame quality values can be compared directly to the quality values from other elements of the same test sequence set (since they all used the same source sequence), or they can be compared to other sets that have been generated using the same source sequence at another bit rate. If objective measurements vary greatly from one bit rate to another in portions of the output sequences, then you can examine them more extensively using subjective techniques.

The full-reference test and measurement process for evaluating codec technologies is very similar to the one for choosing a bit rate, except that you change the codec engine for each test instead of the bit rate (Figure 3). After evaluating the first codec (A), simply insert a second codec (B) into the test workflow and start again. From there PEAQ measurements can be made to identify trends in the average sequence scores as a result of the different codecs.

By using full-reference testing and the PEAQ audio quality scoring model, content originators can get a clear idea of how close a processed audio signal is to the original, and how it is affected by changes in different delivery variables. The results of these tests help them to ensure a highly accurate measurement of quality for audio delivery.


Tags: iss101 | audio delivery | full reference | workflow | PEAQ | bit rates | Adam Schadle
Contributing Author Adam Schadle

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • ChyronHego show their PRIME Workflow at IBC 2018

    ChyronHego show their PRIME Workflow at IBC 2018

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Snell Advanced Media (SAM) 4k workflow at IBC 2015

    Snell Advanced Media (SAM) 4k workflow at IBC 2015

  • LIVEU Workflow solutions at NAB 2015

    LIVEU Workflow solutions at NAB 2015

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Streaming Media with Interra Systems at NAB 2017

    Streaming Media with Interra Systems at NAB 2017

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • ERA - Adobe Anywhere - at BVE 2015

    ERA - Adobe Anywhere - at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • Quantel LiveTouch at IBC 2014

    Quantel LiveTouch at IBC 2014

  • Quantel deal with AFP at IBC 2014

    Quantel deal with AFP at IBC 2014

  • Snell Kahuna Production Switcher at IBC 2014

    Snell Kahuna Production Switcher at IBC 2014

  • Telestream Wirecast and Switch at IBC 2014

    Telestream Wirecast and Switch at IBC 2014

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Broadstream Solutions at NAB 2014

    Broadstream Solutions at NAB 2014

  • Telestream Enterprise at NAB 2014

    Telestream Enterprise at NAB 2014

  • Telestream Switch at NAB 2014

    Telestream Switch at NAB 2014

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Yospace: Advert Insertion at NAB 2013

    Yospace: Advert Insertion at NAB 2013

  • Vimond Control Center at NAB 2013

    Vimond Control Center at NAB 2013

  • Telestream Vantage v5 at NAB 2013

    Telestream Vantage v5 at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Telestream at BVE 2013

    Telestream at BVE 2013

  • Guntermann and Drunck GmbH at BVE 2013

    Guntermann and Drunck GmbH at BVE 2013

  • Tedial at NAB 2012

    Tedial at NAB 2012

  • Marquis Broadcast at NAB 2012

    Marquis Broadcast at NAB 2012

  • Root6 at NAB 2012

    Root6 at NAB 2012

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • TMD at NAB 2012

    TMD at NAB 2012

  • PlayBox Technology at BVE 2012

    PlayBox Technology at BVE 2012

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Net Insight at IBC2011

    Net Insight at IBC2011

  • Tedial at IBC2011

    Tedial at IBC2011

  • IPV at IBC2011

    IPV at IBC2011

  • TSL at IBC2011

    TSL at IBC2011


Related Shows
  • Bubble and Squeak: Live and Loud from Monday at IBC 2013

    Bubble and Squeak: Live and Loud from Monday at IBC 2013


Articles
Sony HDC-4800 Review
Andy McKenzie First announced at NAB 2016, the Sony HDC-4800 is a studio camera system capable of shooting 4K/UHD at up to 8x or full HD at up to 16x. With a price point upwards of £250,000 it is a very high-end product with a wide feature set. In Sony's own words, "This is the future of live production, designed to satisfy the storytelling aspect of modern sports production.” Deliveries began in mid 2017 and, after careful preliminary evaluation, we invested in several systems for our hire fleet ahead of the FIFA World Cup in Russia.
Tags: iss134 | review | hdc-4800 | sony | finepoint | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF
What content providers need to know about OTT
Hiren Hindocha As OTT (Over-The-Top) technology has gotten more mature and established robust standards over the years, the concept of OTT monitoring is gaining popularity. With customer expectations soaring, it’s vital for OTT providers to deliver superior quality content. To deliver Quality of Experience (QoE) on par with linear TV broadcast, the entire system, starting from ingest to multi-bitrate encoding to delivery to CDN must be monitored continuously.
Tags: iss134 | ott monitoring | qos | logging | compliance | dash | atsc | cloud | Hiren Hindocha
Contributing Author Hiren Hindocha Click to read or download PDF
Protecting the continuity of transmission
Lorna Garrett Your viewers love you. You consistently bring them their preferred channels 24/7. They’ve come to rely on you for their viewing pleasure. They never miss cheering on their beloved sports teams. They’re the envy of their friends as they watch live concerts of their favourite bands. They gather the family around and catch up on their must-see shows. They don’t have a bad word to say about you.
Tags: iss134 | garland | gpl | streaming | artel | disaster recovery | Lorna Garrett
Contributing Author Lorna Garrett Click to read or download PDF
Shedding Light on the Blackmagic Pocket Cinema Camera 4k BMCPP4K
Garth de Bruno Austin “What is it about light that has us craving it?” Is the question asked in the opening seconds of Garth de Bruno Austin’s latest short, The Colour of Light. Exploring this natural, human need as well as our innate desire to control it, Garth’s film showcases everyday people going about their lives in differing degrees of luminance, whether that be an artificial streetlight or a natural morning sunrise.
Tags: iss134 | blackmagic | cinema camera | 4k | cpp4k | Garth de Bruno Austin
Contributing Author Garth de Bruno Austin Click to read or download PDF
The brave new world of software based production
Boromy Ung In today’s rapidly evolving broadcast industry, the only constant media organizations can truly count on is change — and the need to adapt as rapidly and cost-effectively as possible. One of the biggest agents of change is the IP revolution, driving broadcasters to migrate their operations to all-software solutions running on commodity, IT-based technologies.
Tags: iss134 | chyronhego | graphics | sports | ott | Boromy Ung
Contributing Author Boromy Ung Click to read or download PDF