QOS - Its time to take a more objective approach


Simen Frostad TV-Bay Magazine
Read ezine online
by Simen Frostad
Issue 83 - November 2013
Any business that fails to satisfy its customers is almost certain to fail if theres significant competition to it. Digital media businesses are very definitely competing in a market where the customer can switch quickly if not satisfied, so its vital for providers to know when their customers are getting a satisfactory service, and when they arent.

In the conventional wisdom QoS (Quality of Service) monitoring can provide diagnostic information on the devices in the delivery chain and the transports streams, while the QoE (Quality of Experience) element is intended to provide a user-centric evaluation of the service, pointing to any faults that have an impact on the users experience.

The reality though is that the conventional approach to QoE monitoring is based on some questionable assumptions, and has its origins outside the television/digital media industry. To monitor the experience of the user, so the thinking goes, you have to ask users to provide a subjective assessment. To gather the data from subjective assessments into a usable form, a methodology was developed and called MOS (Mean Opinion Score), specified by the ITU-T (International Telegraph Union) recommendation P.800. This was deployed when telcos were developing new infrastructure or, more recently, services such as VoIP. A panel of expert assessors would listen to the service and note their evaluation of quality while key phrases such as You will have to be very quiet, There was nothing to be seen, and They worshipped wooden idols were transmitted over the line. The experts would record their scores, assessing any impairments on a scale of 1-5, from imperceptible to very annoying. The scores would then be averaged and weighted using the kind of statistical manipulations common in the social sciences and market research.
The same basic concept of QoE monitoring later took hold in digital media monitoring and MOS spawned VideoMOS and AudioMOS criteria which brought opinion-scoring methodologies into the media arena. But media providers need to know what real users are experiencing, rather than panels of expert assessors. So MOS evaluation became robotized and algorithms were developed to simulate those subjective reactions from imperceptible to very annoying. The robots watch the service and the data is fed into the algorithmic engine, which attempts to simulate the subjective reaction of the viewer, with scores for factors such as jerkiness, blurriness and blockiness.

This might appear sophisticated, but its really a tortuous evolution of an opinion-gathering methodology developed for an entirely different industry, and as the word opinion in the term MOS suggests based on the subjective reactions of human assessors. Subjectivity is complicated: a human viewer watching a top-quality 1080i transmission of the Superbowl, followed on the same channel by the 1940s B&W movie Casablanca might be fully satisfied. Yet in a robotized QoE subjective assessment based on MOS criteria, the Superbowl would score highly, while Casablanca would be marked way down for blurriness, scratches and other artifacts, for its lack of resolution and color. Distorted by results like this, the data from this kind of QoE system becomes less reliable to the provider.

Its time to take a more objective approach. And it can be done, because there is now the technology to monitor the quality of the service continuously and direct from the users device connected TV, tablet or smartphone. The Objective QoE solution launched by Bridge Technologies at IBC 2013 will collect data from the entire delivery chain, from the head end right up to and including the viewers devices. So, rather than have a robot watching the service and simulating the response of a mythical average viewer, media providers will be able to get real data direct from the viewing devices. And what makes this new approach objective is that the QoE evaluation is made from empirical factors that lower the quality of digital media services such as lost packets, timeouts, and buffering. If you can accurately and continuously detect that a viewer is experiencing any of these errors, you no longer need to confect an algorithmic opinion about it, because you have a completely trustworthy, objective set of data on which to assess in real time the quality of each users experience.


Tags: iss083 | bridge technologies | quality of service | connected tv | ibc 2013 | digital media | service | quality | Simen Frostad
Contributing Author Simen Frostad

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Bridge Technologies QoE Monitoring with Mobile Videowall Display at IBC 2013

    Bridge Technologies QoE Monitoring with Mobile Videowall Display at IBC 2013

  • Bridge Technologies PocketProbe App at NAB 2013

    Bridge Technologies PocketProbe App at NAB 2013

  • Bridge Technologies at IBC 2012

    Bridge Technologies at IBC 2012

  • Bridge Technologies at IBC2011

    Bridge Technologies at IBC2011

  • SMPTE on BroadcastShow LIVE at IBC 2013

    SMPTE on BroadcastShow LIVE at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Softron Media Services on BroadcastShow LIVE at IBC 2013

    Softron Media Services on BroadcastShow LIVE at IBC 2013

  • SIS LIVE and their products ManPak and DriveForce at IBC 2013

    SIS LIVE and their products ManPak and DriveForce at IBC 2013

  • Audio Network on BroadcastShow LIVE at IBC 2013

    Audio Network on BroadcastShow LIVE at IBC 2013

  • Dejero on BroadcastShow LIVE at IBC 2013

    Dejero on BroadcastShow LIVE at IBC 2013

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Canon on BroadcastShow LIVE at IBC 2013

    Canon on BroadcastShow LIVE at IBC 2013

  • Harmonics Peter Alexander on BroadcastShow LIVE at IBC 2013

    Harmonics Peter Alexander on BroadcastShow LIVE at IBC 2013

  • Haivision on BroadcastShow LIVE at IBC 2013

    Haivision on BroadcastShow LIVE at IBC 2013

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Atomos with the Samurai Blade on BroadcastShow LIVE at IBC 2013

    Atomos with the Samurai Blade on BroadcastShow LIVE at IBC 2013

  • ChyronHego on BroadcastShow LIVE at IBC 2013

    ChyronHego on BroadcastShow LIVE at IBC 2013

  • Cinegy on BroadcastShow LIVE at IBC 2013

    Cinegy on BroadcastShow LIVE at IBC 2013

  • Trilogy on BroadcastShow LIVE at IBC 2013

    Trilogy on BroadcastShow LIVE at IBC 2013

  • Blackmagic Design on BroadcastShow LIVE at IBC 2013

    Blackmagic Design on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Calrec on BroadcastShow LIVE at IBC 2013

    Calrec on BroadcastShow LIVE at IBC 2013

  • Glensound on BroadcastShow LIVE at IBC 2013

    Glensound on BroadcastShow LIVE at IBC 2013

  • Zylight on BroadcastShow LIVE at IBC 2013

    Zylight on BroadcastShow LIVE at IBC 2013

  • Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

    Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

  • Carl Zeiss on BroadcastShow LIVE at IBC 2013

    Carl Zeiss on BroadcastShow LIVE at IBC 2013

  • Miranda on BroadcastShow LIVE at IBC 2013

    Miranda on BroadcastShow LIVE at IBC 2013

  • Orad Hi Tec Systems on BroadcastShow LIVE at IBC 2013

    Orad Hi Tec Systems on BroadcastShow LIVE at IBC 2013

  • Photon Beard on BroadcastShow LIVE at IBC 2013

    Photon Beard on BroadcastShow LIVE at IBC 2013

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Guntermann and Drunck on BroadcastShow LIVE at IBC 2013

    Guntermann and Drunck on BroadcastShow LIVE at IBC 2013

  • Brainstorm on BroadcastShow LIVE at IBC 2013

    Brainstorm on BroadcastShow LIVE at IBC 2013

  • Pilat Media on BroadcastShow LIVE at IBC 2013

    Pilat Media on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Nucomm on BroadcastShow LIVE at IBC 2013

    Nucomm on BroadcastShow LIVE at IBC 2013

  • Wohler Technologies on BroadcastShow LIVE at IBC 2013

    Wohler Technologies on BroadcastShow LIVE at IBC 2013

  • Nexidia QC on BroadcastShow LIVE at IBC 2013

    Nexidia QC on BroadcastShow LIVE at IBC 2013

  • SIS LIVEs Martyn Hopkins on BroadcastShow LIVE at IBC 2013

    SIS LIVEs Martyn Hopkins on BroadcastShow LIVE at IBC 2013

  • Atomos with the Ronin on BroadcastShow LIVE at IBC 2013

    Atomos with the Ronin on BroadcastShow LIVE at IBC 2013

  • Snell on BroadcastShow LIVE at IBC 2013

    Snell on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Primestream on BroadcastShow LIVE at IBC 2013

    Primestream on BroadcastShow LIVE at IBC 2013

  • NewTek on BroadcastShow LIVE at IBC 2013

    NewTek on BroadcastShow LIVE at IBC 2013

  • Wyplay on BroadcastShow LIVE at IBC 2013

    Wyplay on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Forbidden Technologies on BroadcastShow LIVE at IBC 2013

    Forbidden Technologies on BroadcastShow LIVE at IBC 2013

  • Atomos with the Datacolor Spyder on BroadcastShow LIVE at IBC 2013

    Atomos with the Datacolor Spyder on BroadcastShow LIVE at IBC 2013

  • Sound Devices on BroadcastShow LIVE at IBC 2013

    Sound Devices on BroadcastShow LIVE at IBC 2013

  • Cooke Optics on BroadcastShow LIVE at IBC 2013

    Cooke Optics on BroadcastShow LIVE at IBC 2013

  • SIS LIVEs Mark Shadbolt on BroadcastShow LIVE at IBC 2013

    SIS LIVEs Mark Shadbolt on BroadcastShow LIVE at IBC 2013

  • Cobham on BroadcastShow LIVE at IBC 2013

    Cobham on BroadcastShow LIVE at IBC 2013

  • Riedel Communications on BroadcastShow LIVE at IBC 2013

    Riedel Communications on BroadcastShow LIVE at IBC 2013

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Behind the scenes of the Teradek booth on BroadcastShow LIVE at IBC 2013

    Behind the scenes of the Teradek booth on BroadcastShow LIVE at IBC 2013

  • Wowza Media Systems on BroadcastShow LIVE at IBC 2013

    Wowza Media Systems on BroadcastShow LIVE at IBC 2013

  • Stream UK on BroadcastShow LIVE at IBC 2013

    Stream UK on BroadcastShow LIVE at IBC 2013

  • Cambridge Imaging Systems on BroadcastShow LIVE at IBC 2013

    Cambridge Imaging Systems on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Polecam talk high speed on BroadcastShow LIVE at IBC 2013

    Polecam talk high speed on BroadcastShow LIVE at IBC 2013

  • TSL Products PAM1, SAM1 and PAM PiCo at IBC 2013

    TSL Products PAM1, SAM1 and PAM PiCo at IBC 2013

  • Calrec Callisto at IBC 2013

    Calrec Callisto at IBC 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • DPA Microphones at IBC 2013

    DPA Microphones at IBC 2013

  • Gearhouse Broadcast HD OB trailer at IBC 2013

    Gearhouse Broadcast HD OB trailer at IBC 2013

  • Glensound with the USB Commentary Mixer at IBC 2013

    Glensound with the USB Commentary Mixer at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • RTW at IBC 2013

    RTW at IBC 2013

  • Sound Devices PIX260i Recorder at IBC 2013

    Sound Devices PIX260i Recorder at IBC 2013

  • Thomson Broadcast at IBC 2013

    Thomson Broadcast at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • Atomos Ninja2 Recorder at IBC 2013

    Atomos Ninja2 Recorder at IBC 2013

  • Atomos Ronin at IBC 2013

    Atomos Ronin at IBC 2013

  • Atomos new Samurai Blade at IBC 2013

    Atomos new Samurai Blade at IBC 2013

  • Autocue DSLR Prompter using iPad at IBC 2013

    Autocue DSLR Prompter using iPad at IBC 2013

  • Blueshape Batteries at IBC 2013

    Blueshape Batteries at IBC 2013

  • Datacolor Spyder on the Atomos stand at IBC 2013

    Datacolor Spyder on the Atomos stand at IBC 2013

  • i-Movix Slow Motion and D-Flicker at IBC 2013

    i-Movix Slow Motion and D-Flicker at IBC 2013

  • JVC GY-HM600 series camera updates at IBC 2013

    JVC GY-HM600 series camera updates at IBC 2013

  • Marshall Monitors at IBC 2013

    Marshall Monitors at IBC 2013

  • Metus MAM and Ingest at IBC 2013

    Metus MAM and Ingest at IBC 2013

  • Polecams range of accessories and minicams at IBC 2013

    Polecams range of accessories and minicams at IBC 2013

  • Tektronix at IBC 2013

    Tektronix at IBC 2013

  • TSL Products at IBC 2013

    TSL Products at IBC 2013

  • TSL Systems at IBC 2013

    TSL Systems at IBC 2013

  • Zylight LED Fresnel at IBC 2013

    Zylight LED Fresnel at IBC 2013

  • Pixel Power at IBC 2013

    Pixel Power at IBC 2013

  • Albis Technologies celebrating ten years at IBC 2013

    Albis Technologies celebrating ten years at IBC 2013

  • ATEME at IBC 2013

    ATEME at IBC 2013

  • Brainstorm with Aston demonstration at IBC 2013

    Brainstorm with Aston demonstration at IBC 2013

  • Brainstorm at IBC 2013

    Brainstorm at IBC 2013

  • Broadpeak at IBC 2013

    Broadpeak at IBC 2013

  • Cobham transmitters at IBC 2013

    Cobham transmitters at IBC 2013

  • Guntermann and Drunck CrossDisplay switching and CCD at IBC 2013

    Guntermann and Drunck CrossDisplay switching and CCD at IBC 2013

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Haivision live encoding HEVC at IBC 2013

    Haivision live encoding HEVC at IBC 2013

  • Multidyne SilverBack, BullDog and Spider at IBC 2013

    Multidyne SilverBack, BullDog and Spider at IBC 2013

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • Lighting cameraman Jonathan Harrison shows the Celeb and Tegra at IBC 2013

    Lighting cameraman Jonathan Harrison shows the Celeb and Tegra at IBC 2013

  • Blue Lucy Media at IBC 2013

    Blue Lucy Media at IBC 2013

  • ChyronHego Paint at IBC 2013

    ChyronHego Paint at IBC 2013

  • Cinegy at IBC 2013

    Cinegy at IBC 2013

  • ChyronHego Virtual Placement at IBC 2013

    ChyronHego Virtual Placement at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • Blackmagic Design with the new ATEM Production Suite at IBC 2013

    Blackmagic Design with the new ATEM Production Suite at IBC 2013

  • Softron Media at IBC 2013

    Softron Media at IBC 2013

  • Viaccess-Orca at IBC 2013

    Viaccess-Orca at IBC 2013

  • Quantel Pablo Rio at IBC 2013

    Quantel Pablo Rio at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Matrox Maevex at IBC 2013

    Matrox Maevex at IBC 2013

  • Matrox Avio at IBC 2013

    Matrox Avio at IBC 2013

  • Matrox Video with Monarch HD at IBC 2013

    Matrox Video with Monarch HD at IBC 2013

  • NuGen Audio at IBC 2013

    NuGen Audio at IBC 2013

  • IBC 2013

    IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Verizon Digital Media Services at IBC 2017

    Verizon Digital Media Services at IBC 2017

  • Omnitek Consultancy and Design Services at IBC 2017

    Omnitek Consultancy and Design Services at IBC 2017

  • SIS Live Anylive and Gateway Service Platform at IBC 2017

    SIS Live Anylive and Gateway Service Platform at IBC 2017

  • Volicon Media Intelligence service at IBC 2016

    Volicon Media Intelligence service at IBC 2016

  • ERA - Cloud Services - at BVE 2015

    ERA - Cloud Services - at BVE 2015

  • Prime Focus Technologies at IBC 2016

    Prime Focus Technologies at IBC 2016

  • Elemental Technologies - AWS acquisition at NAB 2016

    Elemental Technologies - AWS acquisition at NAB 2016

  • Elemental Technologies - Amazon acquisition at IBC 2015

    Elemental Technologies - Amazon acquisition at IBC 2015

  • Viaccess-Orca at IBC 2014

    Viaccess-Orca at IBC 2014

  • Autodesk at NAB 2013

    Autodesk at NAB 2013

  • Thear Technology at BVE 2013

    Thear Technology at BVE 2013

  • True Lens at BVE 2013

    True Lens at BVE 2013

  • Lens Adaptor at BVE 2013

    Lens Adaptor at BVE 2013

  • Thear Technology Limited at BVE North 2012

    Thear Technology Limited at BVE North 2012

  • Broadpeak at NAB 2012

    Broadpeak at NAB 2012

  • Thear Technology Limited at BVE 2012

    Thear Technology Limited at BVE 2012

  • Thear Technology Limited at ProVideo2011

    Thear Technology Limited at ProVideo2011

  • MTF at IBC2011

    MTF at IBC2011

  • Prime Focus at IBC2011

    Prime Focus at IBC2011


Related Shows
  • HbbTV: Connected TV for broadcasters

    HbbTV: Connected TV for broadcasters


Articles
Perimeter LED screens management
Nicolas Houel Opened in January 2016, Parc Olympique Lyonnais, also known as Groupama Stadium, is the new home of Olympique Lyonnais football club, one of the most popular clubs in France. Since its inauguration, the stadium was a host of UEFA Euro 2016, and was also chosen to stage, among other important events, the 2018 UEFA Europa League Final and football at the 2024 Summer Olympics.
Tags: iss133 | 3dstorm | graphics | groupama stadium | liveexpert | livecg | deltacast | Nicolas Houel
Contributing Author Nicolas Houel Click to read or download PDF
Why are trade shows still important
Mike Crimp First, the disclaimer: as the CEO of a rather large trade show, my opinion is likely to be skewed. However, I wouldn’t be here – now in my eighth year at IBC - if I didn’t truly believe that well-curated trade shows like IBC are a hugely important resource for our industry.
Tags: iss133 | ibc | innovation award | Mike Crimp
Contributing Author Mike Crimp Click to read or download PDF
Taking on a self employed placement year
Joshua Round The idea of being self-employed or freelancing has always been somewhat terrifying for me. There is a level of uncertainty and responsibility that comes with the freedom of being self-employed, the likes of which makes me wonder why I chose to give it a go for my placement year as part of my university course - (BSc) Television and Broadcasting.
Tags: iss133 | placement year | university | student | education | portsmouth | Joshua Round
Contributing Author Joshua Round Click to read or download PDF
Creating Authentic Content That Counts
Frank le Mair We’re deep into the 2010’s and the way in which we consume content has changed forever. Broadcasters and content owners are fighting for eyeballs in a saturated market where consumers are watching their favourite shows on different devices across a number of platforms. To target millennials and younger generations, who generally consume short bursts of video on YouTube and social media - particularly Snapchat, Instagram and now IGTV - media companies are creating more and more authentic stories and are using platforms that are compelling for this demographic. Unlike generations before them, they have totally new video viewing patterns and ideals.
Tags: iss133 | insight tv | monster energy | amazon | millennial | Frank le Mair
Contributing Author Frank le Mair Click to read or download PDF
How todays technology platforms can make AMAZING seem mundane
Bruce Devlin - new Some things never change. It's September and it's time for another IBC. Maybe I'm getting old and maybe I have selective memory about the past, but recently there hasn't been the buzz of years gone by. Sure, as you wander around the vast halls there will be lots of enthusiastic sales people trying to give you the best demo possible, but something has changed and to survive, I believe that vendors and customers need to change too.
Tags: iss133 | ibc | class | vulture | imf user group | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF