QOS - Its time to take a more objective approach


Simen Frostad TV-Bay Magazine
Read ezine online
by Simen Frostad
Issue 83 - November 2013
Any business that fails to satisfy its customers is almost certain to fail if theres significant competition to it. Digital media businesses are very definitely competing in a market where the customer can switch quickly if not satisfied, so its vital for providers to know when their customers are getting a satisfactory service, and when they arent.

In the conventional wisdom QoS (Quality of Service) monitoring can provide diagnostic information on the devices in the delivery chain and the transports streams, while the QoE (Quality of Experience) element is intended to provide a user-centric evaluation of the service, pointing to any faults that have an impact on the users experience.

The reality though is that the conventional approach to QoE monitoring is based on some questionable assumptions, and has its origins outside the television/digital media industry. To monitor the experience of the user, so the thinking goes, you have to ask users to provide a subjective assessment. To gather the data from subjective assessments into a usable form, a methodology was developed and called MOS (Mean Opinion Score), specified by the ITU-T (International Telegraph Union) recommendation P.800. This was deployed when telcos were developing new infrastructure or, more recently, services such as VoIP. A panel of expert assessors would listen to the service and note their evaluation of quality while key phrases such as You will have to be very quiet, There was nothing to be seen, and They worshipped wooden idols were transmitted over the line. The experts would record their scores, assessing any impairments on a scale of 1-5, from imperceptible to very annoying. The scores would then be averaged and weighted using the kind of statistical manipulations common in the social sciences and market research.
The same basic concept of QoE monitoring later took hold in digital media monitoring and MOS spawned VideoMOS and AudioMOS criteria which brought opinion-scoring methodologies into the media arena. But media providers need to know what real users are experiencing, rather than panels of expert assessors. So MOS evaluation became robotized and algorithms were developed to simulate those subjective reactions from imperceptible to very annoying. The robots watch the service and the data is fed into the algorithmic engine, which attempts to simulate the subjective reaction of the viewer, with scores for factors such as jerkiness, blurriness and blockiness.

This might appear sophisticated, but its really a tortuous evolution of an opinion-gathering methodology developed for an entirely different industry, and as the word opinion in the term MOS suggests based on the subjective reactions of human assessors. Subjectivity is complicated: a human viewer watching a top-quality 1080i transmission of the Superbowl, followed on the same channel by the 1940s B&W movie Casablanca might be fully satisfied. Yet in a robotized QoE subjective assessment based on MOS criteria, the Superbowl would score highly, while Casablanca would be marked way down for blurriness, scratches and other artifacts, for its lack of resolution and color. Distorted by results like this, the data from this kind of QoE system becomes less reliable to the provider.

Its time to take a more objective approach. And it can be done, because there is now the technology to monitor the quality of the service continuously and direct from the users device connected TV, tablet or smartphone. The Objective QoE solution launched by Bridge Technologies at IBC 2013 will collect data from the entire delivery chain, from the head end right up to and including the viewers devices. So, rather than have a robot watching the service and simulating the response of a mythical average viewer, media providers will be able to get real data direct from the viewing devices. And what makes this new approach objective is that the QoE evaluation is made from empirical factors that lower the quality of digital media services such as lost packets, timeouts, and buffering. If you can accurately and continuously detect that a viewer is experiencing any of these errors, you no longer need to confect an algorithmic opinion about it, because you have a completely trustworthy, objective set of data on which to assess in real time the quality of each users experience.


Tags: iss083 | bridge technologies | quality of service | connected tv | ibc 2013 | digital media | service | quality | Simen Frostad
Contributing Author Simen Frostad

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Bridge Technologies QoE Monitoring with Mobile Videowall Display at IBC 2013

    Bridge Technologies QoE Monitoring with Mobile Videowall Display at IBC 2013

  • Bridge Technologies PocketProbe App at NAB 2013

    Bridge Technologies PocketProbe App at NAB 2013

  • Bridge Technologies at IBC 2012

    Bridge Technologies at IBC 2012

  • Bridge Technologies at IBC2011

    Bridge Technologies at IBC2011

  • SMPTE on BroadcastShow LIVE at IBC 2013

    SMPTE on BroadcastShow LIVE at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Softron Media Services on BroadcastShow LIVE at IBC 2013

    Softron Media Services on BroadcastShow LIVE at IBC 2013

  • SIS LIVE and their products ManPak and DriveForce at IBC 2013

    SIS LIVE and their products ManPak and DriveForce at IBC 2013

  • Audio Network on BroadcastShow LIVE at IBC 2013

    Audio Network on BroadcastShow LIVE at IBC 2013

  • Dejero on BroadcastShow LIVE at IBC 2013

    Dejero on BroadcastShow LIVE at IBC 2013

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Canon on BroadcastShow LIVE at IBC 2013

    Canon on BroadcastShow LIVE at IBC 2013

  • Harmonics Peter Alexander on BroadcastShow LIVE at IBC 2013

    Harmonics Peter Alexander on BroadcastShow LIVE at IBC 2013

  • Haivision on BroadcastShow LIVE at IBC 2013

    Haivision on BroadcastShow LIVE at IBC 2013

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Atomos with the Samurai Blade on BroadcastShow LIVE at IBC 2013

    Atomos with the Samurai Blade on BroadcastShow LIVE at IBC 2013

  • ChyronHego on BroadcastShow LIVE at IBC 2013

    ChyronHego on BroadcastShow LIVE at IBC 2013

  • Cinegy on BroadcastShow LIVE at IBC 2013

    Cinegy on BroadcastShow LIVE at IBC 2013

  • Trilogy on BroadcastShow LIVE at IBC 2013

    Trilogy on BroadcastShow LIVE at IBC 2013

  • Blackmagic Design on BroadcastShow LIVE at IBC 2013

    Blackmagic Design on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Calrec on BroadcastShow LIVE at IBC 2013

    Calrec on BroadcastShow LIVE at IBC 2013

  • Glensound on BroadcastShow LIVE at IBC 2013

    Glensound on BroadcastShow LIVE at IBC 2013

  • Zylight on BroadcastShow LIVE at IBC 2013

    Zylight on BroadcastShow LIVE at IBC 2013

  • Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

    Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

  • Carl Zeiss on BroadcastShow LIVE at IBC 2013

    Carl Zeiss on BroadcastShow LIVE at IBC 2013

  • Miranda on BroadcastShow LIVE at IBC 2013

    Miranda on BroadcastShow LIVE at IBC 2013

  • Orad Hi Tec Systems on BroadcastShow LIVE at IBC 2013

    Orad Hi Tec Systems on BroadcastShow LIVE at IBC 2013

  • Photon Beard on BroadcastShow LIVE at IBC 2013

    Photon Beard on BroadcastShow LIVE at IBC 2013

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Guntermann and Drunck on BroadcastShow LIVE at IBC 2013

    Guntermann and Drunck on BroadcastShow LIVE at IBC 2013

  • Brainstorm on BroadcastShow LIVE at IBC 2013

    Brainstorm on BroadcastShow LIVE at IBC 2013

  • Pilat Media on BroadcastShow LIVE at IBC 2013

    Pilat Media on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Nucomm on BroadcastShow LIVE at IBC 2013

    Nucomm on BroadcastShow LIVE at IBC 2013

  • Wohler Technologies on BroadcastShow LIVE at IBC 2013

    Wohler Technologies on BroadcastShow LIVE at IBC 2013

  • Nexidia QC on BroadcastShow LIVE at IBC 2013

    Nexidia QC on BroadcastShow LIVE at IBC 2013

  • SIS LIVEs Martyn Hopkins on BroadcastShow LIVE at IBC 2013

    SIS LIVEs Martyn Hopkins on BroadcastShow LIVE at IBC 2013

  • Atomos with the Ronin on BroadcastShow LIVE at IBC 2013

    Atomos with the Ronin on BroadcastShow LIVE at IBC 2013

  • Snell on BroadcastShow LIVE at IBC 2013

    Snell on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Primestream on BroadcastShow LIVE at IBC 2013

    Primestream on BroadcastShow LIVE at IBC 2013

  • NewTek on BroadcastShow LIVE at IBC 2013

    NewTek on BroadcastShow LIVE at IBC 2013

  • Wyplay on BroadcastShow LIVE at IBC 2013

    Wyplay on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Forbidden Technologies on BroadcastShow LIVE at IBC 2013

    Forbidden Technologies on BroadcastShow LIVE at IBC 2013

  • Atomos with the Datacolor Spyder on BroadcastShow LIVE at IBC 2013

    Atomos with the Datacolor Spyder on BroadcastShow LIVE at IBC 2013

  • Sound Devices on BroadcastShow LIVE at IBC 2013

    Sound Devices on BroadcastShow LIVE at IBC 2013

  • Cooke Optics on BroadcastShow LIVE at IBC 2013

    Cooke Optics on BroadcastShow LIVE at IBC 2013

  • SIS LIVEs Mark Shadbolt on BroadcastShow LIVE at IBC 2013

    SIS LIVEs Mark Shadbolt on BroadcastShow LIVE at IBC 2013

  • Cobham on BroadcastShow LIVE at IBC 2013

    Cobham on BroadcastShow LIVE at IBC 2013

  • Riedel Communications on BroadcastShow LIVE at IBC 2013

    Riedel Communications on BroadcastShow LIVE at IBC 2013

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Behind the scenes of the Teradek booth on BroadcastShow LIVE at IBC 2013

    Behind the scenes of the Teradek booth on BroadcastShow LIVE at IBC 2013

  • Wowza Media Systems on BroadcastShow LIVE at IBC 2013

    Wowza Media Systems on BroadcastShow LIVE at IBC 2013

  • Stream UK on BroadcastShow LIVE at IBC 2013

    Stream UK on BroadcastShow LIVE at IBC 2013

  • Cambridge Imaging Systems on BroadcastShow LIVE at IBC 2013

    Cambridge Imaging Systems on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Polecam talk high speed on BroadcastShow LIVE at IBC 2013

    Polecam talk high speed on BroadcastShow LIVE at IBC 2013

  • TSL Products PAM1, SAM1 and PAM PiCo at IBC 2013

    TSL Products PAM1, SAM1 and PAM PiCo at IBC 2013

  • Calrec Callisto at IBC 2013

    Calrec Callisto at IBC 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • DPA Microphones at IBC 2013

    DPA Microphones at IBC 2013

  • Gearhouse Broadcast HD OB trailer at IBC 2013

    Gearhouse Broadcast HD OB trailer at IBC 2013

  • Glensound with the USB Commentary Mixer at IBC 2013

    Glensound with the USB Commentary Mixer at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • RTW at IBC 2013

    RTW at IBC 2013

  • Sound Devices PIX260i Recorder at IBC 2013

    Sound Devices PIX260i Recorder at IBC 2013

  • Thomson Broadcast at IBC 2013

    Thomson Broadcast at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • Atomos Ninja2 Recorder at IBC 2013

    Atomos Ninja2 Recorder at IBC 2013

  • Atomos Ronin at IBC 2013

    Atomos Ronin at IBC 2013

  • Atomos new Samurai Blade at IBC 2013

    Atomos new Samurai Blade at IBC 2013

  • Autocue DSLR Prompter using iPad at IBC 2013

    Autocue DSLR Prompter using iPad at IBC 2013

  • Blueshape Batteries at IBC 2013

    Blueshape Batteries at IBC 2013

  • Datacolor Spyder on the Atomos stand at IBC 2013

    Datacolor Spyder on the Atomos stand at IBC 2013

  • i-Movix Slow Motion and D-Flicker at IBC 2013

    i-Movix Slow Motion and D-Flicker at IBC 2013

  • JVC GY-HM600 series camera updates at IBC 2013

    JVC GY-HM600 series camera updates at IBC 2013

  • Marshall Monitors at IBC 2013

    Marshall Monitors at IBC 2013

  • Metus MAM and Ingest at IBC 2013

    Metus MAM and Ingest at IBC 2013

  • Polecams range of accessories and minicams at IBC 2013

    Polecams range of accessories and minicams at IBC 2013

  • Tektronix at IBC 2013

    Tektronix at IBC 2013

  • TSL Products at IBC 2013

    TSL Products at IBC 2013

  • TSL Systems at IBC 2013

    TSL Systems at IBC 2013

  • Zylight LED Fresnel at IBC 2013

    Zylight LED Fresnel at IBC 2013

  • Pixel Power at IBC 2013

    Pixel Power at IBC 2013

  • Albis Technologies celebrating ten years at IBC 2013

    Albis Technologies celebrating ten years at IBC 2013

  • ATEME at IBC 2013

    ATEME at IBC 2013

  • Brainstorm with Aston demonstration at IBC 2013

    Brainstorm with Aston demonstration at IBC 2013

  • Brainstorm at IBC 2013

    Brainstorm at IBC 2013

  • Broadpeak at IBC 2013

    Broadpeak at IBC 2013

  • Cobham transmitters at IBC 2013

    Cobham transmitters at IBC 2013

  • Guntermann and Drunck CrossDisplay switching and CCD at IBC 2013

    Guntermann and Drunck CrossDisplay switching and CCD at IBC 2013

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Haivision live encoding HEVC at IBC 2013

    Haivision live encoding HEVC at IBC 2013

  • Multidyne SilverBack, BullDog and Spider at IBC 2013

    Multidyne SilverBack, BullDog and Spider at IBC 2013

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • Lighting cameraman Jonathan Harrison shows the Celeb and Tegra at IBC 2013

    Lighting cameraman Jonathan Harrison shows the Celeb and Tegra at IBC 2013

  • Blue Lucy Media at IBC 2013

    Blue Lucy Media at IBC 2013

  • ChyronHego Paint at IBC 2013

    ChyronHego Paint at IBC 2013

  • Cinegy at IBC 2013

    Cinegy at IBC 2013

  • ChyronHego Virtual Placement at IBC 2013

    ChyronHego Virtual Placement at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • Blackmagic Design with the new ATEM Production Suite at IBC 2013

    Blackmagic Design with the new ATEM Production Suite at IBC 2013

  • Softron Media at IBC 2013

    Softron Media at IBC 2013

  • Viaccess-Orca at IBC 2013

    Viaccess-Orca at IBC 2013

  • Quantel Pablo Rio at IBC 2013

    Quantel Pablo Rio at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Matrox Maevex at IBC 2013

    Matrox Maevex at IBC 2013

  • Matrox Avio at IBC 2013

    Matrox Avio at IBC 2013

  • Matrox Video with Monarch HD at IBC 2013

    Matrox Video with Monarch HD at IBC 2013

  • NuGen Audio at IBC 2013

    NuGen Audio at IBC 2013

  • IBC 2013

    IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Verizon Digital Media Services at IBC 2017

    Verizon Digital Media Services at IBC 2017

  • Omnitek Consultancy and Design Services at IBC 2017

    Omnitek Consultancy and Design Services at IBC 2017

  • SIS Live Anylive and Gateway Service Platform at IBC 2017

    SIS Live Anylive and Gateway Service Platform at IBC 2017

  • Volicon Media Intelligence service at IBC 2016

    Volicon Media Intelligence service at IBC 2016

  • ERA - Cloud Services - at BVE 2015

    ERA - Cloud Services - at BVE 2015

  • Prime Focus Technologies at IBC 2016

    Prime Focus Technologies at IBC 2016

  • Elemental Technologies - AWS acquisition at NAB 2016

    Elemental Technologies - AWS acquisition at NAB 2016

  • Elemental Technologies - Amazon acquisition at IBC 2015

    Elemental Technologies - Amazon acquisition at IBC 2015

  • Viaccess-Orca at IBC 2014

    Viaccess-Orca at IBC 2014

  • Autodesk at NAB 2013

    Autodesk at NAB 2013

  • Thear Technology at BVE 2013

    Thear Technology at BVE 2013

  • True Lens at BVE 2013

    True Lens at BVE 2013

  • Lens Adaptor at BVE 2013

    Lens Adaptor at BVE 2013

  • Thear Technology Limited at BVE North 2012

    Thear Technology Limited at BVE North 2012

  • Broadpeak at NAB 2012

    Broadpeak at NAB 2012

  • Thear Technology Limited at BVE 2012

    Thear Technology Limited at BVE 2012

  • Thear Technology Limited at ProVideo2011

    Thear Technology Limited at ProVideo2011

  • MTF at IBC2011

    MTF at IBC2011

  • Prime Focus at IBC2011

    Prime Focus at IBC2011


Related Shows
  • HbbTV: Connected TV for broadcasters

    HbbTV: Connected TV for broadcasters


Articles
GoPro HERO 7 Review
Tim Bearder When I heard I was filming a nature restoration project in the pouring rain this week I was excited. WHY? No Cameraman enjoys the rain, surely but this time I was enthusiastic because I knew this would be the perfect opportunity to try out the brand new GoPro Hero 7 Black Edition.
Tags: iss134 | gopro | hero 7 | review | liberal media | Tim Bearder
Contributing Author Tim Bearder Click to read or download PDF
Sony HDC-4800 Review
Andy McKenzie First announced at NAB 2016, the Sony HDC-4800 is a studio camera system capable of shooting 4K/UHD at up to 8x or full HD at up to 16x. With a price point upwards of £250,000 it is a very high-end product with a wide feature set. In Sony's own words, "This is the future of live production, designed to satisfy the storytelling aspect of modern sports production.” Deliveries began in mid 2017 and, after careful preliminary evaluation, we invested in several systems for our hire fleet ahead of the FIFA World Cup in Russia.
Tags: iss134 | review | hdc-4800 | sony | finepoint | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF
University and Mental Health
Rhiannon Jenkins University study and mental health has been in the media quite a bit over the last year, and I’m sure there are many people wondering what is going on? The issues are complex, and I suppose the focus of employability off the back of a degree course has raised the stress stakes for a lot of young people. I’m only qualified to talk about this from my perspective, and my story began when I joined a course not knowing I had a mental health condition.
Tags: iss134 | portsmouth uni | mental health | student | tvfutures | Rhiannon Jenkins
Contributing Author Rhiannon Jenkins Click to read or download PDF
Protecting the continuity of transmission
Lorna Garrett Your viewers love you. You consistently bring them their preferred channels 24/7. They’ve come to rely on you for their viewing pleasure. They never miss cheering on their beloved sports teams. They’re the envy of their friends as they watch live concerts of their favourite bands. They gather the family around and catch up on their must-see shows. They don’t have a bad word to say about you.
Tags: iss134 | garland | gpl | streaming | artel | disaster recovery | Lorna Garrett
Contributing Author Lorna Garrett Click to read or download PDF
NEP Student Day
KitPlus The world of broadcasting is in constant flux, with undefined boundaries between broadcast, TV, film and interactive media. With a changing consumer demographic now a key influence on the industry, many companies no longer specialise in a single medium, but have had to become adaptable to deliver multiple services through a combination of media.
Tags: iss134 | students | tvfutures | NEP | KitPlus
Contributing Author KitPlus Click to read or download PDF