Metadata rules


Will Strauss# TV-Bay Magazine
Read ezine online
Regular readers of this column will know that I am frequently tasked with covering the sexiest of broadcast industry subjects. Indeed, just two months ago I got to discuss test and measurement (T&M). Well, if you thought that was intense, just wait for this month's topic. Not only is it potentially even less glamorous than T&M, it has the potential to be an instant cure for insomnia. Unfortunately, it's also a hugely important subject if you're talking about storage and archive (as this issue of TV-Bay is). What am I talking about? Is it a gizmo? Is it a widget? Nope, it’s metadata. Hold on to your hats. This is going to be wild.
Now, before I wade into the metadata stuff, let’s give this article some context. Whether owned by a broadcaster, a producer or a library, a content archive isn’t just a historical record of its acquisition efforts and its televisual or filmic output. In many cases it is its biggest asset.
The exploitation of archive content is a big thing. Rich media content is in demand and, if you can make it available quickly and easily, archive footage and programmes can be re-licensed for a myriad of uses from historical documentaries and primetime clip shows to one-off mobile downloads.
To make this happen, more and more archives are being digitized, turning film, tape and digital tape into files. This allows for instant retrieval and exploitation, micro payments, process automation and more.
Yet, while digitization is a must, I see two major inconveniences when it comes to long-term preservation of file-based media: the choice of format that the footage should be archived in; and the amount of metadata required for it to be at its most useful.
Currently, television is produced by anything from a smartphone to a 4k camcorder, in any format from mp4 to AVC-Intra. The commonly held belief is that you archive in the highest quality possible. Which would be AVC-Intra. But in several years time another format will be in vogue. So, if we archive in AVC-Intra now, in order to re-use that content in the future, we would need to match it up with the ‘new’ format. To this you would have to transcode the AVC-Intra. And when the next format comes round after that, you would have to transcode it again. Is that a problem? Well, um, yes, because as each transcode takes place the content gradually gains coding impairments. And if you’re compressing or re-compressing it, you get even more impairments. This is far from ideal.
What may be required (please don’t shoot me) is ANOTHER format. An agreed format that we stick to for archiving purposes. It must be one that is high quality, lossless, open, widely adopted and easy for future computer systems to recompile. The suggested format should be either lightly compressed (in order to keep storage requirements realistic), lossless compressed or completely uncompressed and it should be wrapped up in something like MXF.
John Zubrzycki, the Section Leader for Archives Research at BBC R&D has done a lot of good work on this subject. He urges archive owners to “work together to present common requirements to industry” and argues for what he calls a “light compression standard” that can be used for SD and HDTV archiving. This would avoid the need to recode footage every time production moves forward. Which makes a lot of sense.
So, that’s a potential solution to the format problem. What about this here metadata stuff then? In the new file-based world that TV now inhabits, metadata rules (pun intended) and the efficient implementation of metadata is key to content management and file-based workflows.
Technical metadata is used, for example, to drive entire MAM systems or playout operations and, without it, some files simply won’t work properly in certain devices. While descriptive metadata (shot length, content, music type etc) is what interests humans and is the information required for indexing and archiving, monetizing and more.
Unfortunately, as a result of its increasing importance, metadata requirements have got very complicated. To quote Niall Duffy, the managing director of the media technology consultancy Mediasmiths, “you’ve currently got people who for very correct reasons want to come up with very structured metadata models because from their point of view that is essential for building any sort of long-term archive. But from an archive user’s point-of-view the more fields there are, the less they’ll find as it becomes too confusing for them.”
Having lots of metadata fields to complete also makes data inputting nigh on impossible and unrealistic for human beings. And, in fact, when you look closely into how researchers actually use metadata and what they look for, what you find is that searches revert back to what we all know best: a Google-type search.
With that in mind, metadata should not be about increasing the number of fields to improve archive or asset search. Instead it should be concerned with thinking about how people actually seek out content.
The ideal scenario, it would seem, is for footage to be given a small set of structured metadata fields that allow for things like categorization and process automation and then a large unstructured data field that is automatically generated, user generated or based on tagging.
This approach would allow researchers (human or otherwise) to ‘discover’ based on their own requirements rather than the restrictions of the metadata fields. In short, less is definitely more when it comes to metadata.
To my mind, in order to make this work, metadata therefore needs to be dealt with further up the production chain. It cannot be left to the archivists or the data wranglers. Producers need to take responsibility for it and that doesn’t mean reminding a runner or a camera assistant as an afterthought. If that happens, and the importance of metadata is not asserted, you get poor metadata. And programmes and footage with poor metadata will have a limited archival afterlife.
So, there you have it: what we need is a new format for archive preservation and a different approach to metadata. It’s not sexy. But it is big. And it is clever. When it comes to archives and storage, metadata rules.
I wonder what subject is up next?

Tags: iss069 | metadata | object matrix | archive | storage | avc-intra | mam | Will Strauss#
Contributing Author Will Strauss#

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Primestream on BroadcastShow LIVE at IBC 2013

    Primestream on BroadcastShow LIVE at IBC 2013

  • SGL at NAB 2012

    SGL at NAB 2012

  • Prime Focus Technologies at IBC 2016

    Prime Focus Technologies at IBC 2016

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Root6 at BVE 2012

    Root6 at BVE 2012

  • MOG Technologies at BVE 2012

    MOG Technologies at BVE 2012

  • Cooke at IBC2011

    Cooke at IBC2011

  • Radiant Grid at IBC2011

    Radiant Grid at IBC2011

  • Object Based Storage Solutions from Object Matrix at NAB 2017

    Object Based Storage Solutions from Object Matrix at NAB 2017

  • Object based Storage Solution from Object Matrix at IBC 2017

    Object based Storage Solution from Object Matrix at IBC 2017

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • NOA Archive Solutions at IBC 2014

    NOA Archive Solutions at IBC 2014

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • SGL at NAB 2013

    SGL at NAB 2013

  • Facilis Technology at BVE 2013

    Facilis Technology at BVE 2013

  • Nexsan at NAB 2012

    Nexsan at NAB 2012

  • Arkivum at BVE 2012

    Arkivum at BVE 2012

  • ERA - Cloud Services - at BVE 2015

    ERA - Cloud Services - at BVE 2015

  • Cinegy: Multiviewer at NAB 2013

    Cinegy: Multiviewer at NAB 2013

  • Netia at IBC2011

    Netia at IBC2011

  • SGL at NAB 2014

    SGL at NAB 2014

  • Softron Media Services on BroadcastShow LIVE at IBC 2013

    Softron Media Services on BroadcastShow LIVE at IBC 2013

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Cinegy at IBC 2013

    Cinegy at IBC 2013

  • TMD at BVE 2012

    TMD at BVE 2012

  • FOR-A at IBC2011

    FOR-A at IBC2011

  • Facilis Technology Shared Storage at IBC 2018

    Facilis Technology Shared Storage at IBC 2018

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • Glyph External Storage including the Studio and Atom raid at IBC 2017

    Glyph External Storage including the Studio and Atom raid at IBC 2017

  • Storage DNA at BVE 2016

    Storage DNA at BVE 2016

  • Fibrenetix with StorageDNA at IBC 2014

    Fibrenetix with StorageDNA at IBC 2014

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • Fibrenetix with Quadrus at IBC 2014

    Fibrenetix with Quadrus at IBC 2014

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • GB Labs Space at NAB 2014

    GB Labs Space at NAB 2014

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • Facilis at BVE 2012

    Facilis at BVE 2012

  • Real Life Kit at ProVideo2011

    Real Life Kit at ProVideo2011

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Sonnet Technology at IBC2011

    Sonnet Technology at IBC2011

  • ATTO Technology at IBC2011

    ATTO Technology at IBC2011

  • Metus MAM and Ingest at IBC 2013

    Metus MAM and Ingest at IBC 2013

  • Cloud Media Management with Medway from Marquis Broadcast at IBC 2017

    Cloud Media Management with Medway from Marquis Broadcast at IBC 2017

  • Be More with Clear from Prime Focus Technologies at NAB 2017

    Be More with Clear from Prime Focus Technologies at NAB 2017

  • Hybrid Cloud Media Aggregation from Cantemo at NAB 2017

    Hybrid Cloud Media Aggregation from Cantemo at NAB 2017

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Media Asset Management Software from Blue Lucy at NAB 2017

    Media Asset Management Software from Blue Lucy at NAB 2017

  • Mediaflex UMS Platform from TMD at NAB 2017

    Mediaflex UMS Platform from TMD at NAB 2017

  • Blue Lucy at IBC 2015

    Blue Lucy at IBC 2015

  • EditShare at BVE 2015

    EditShare at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Tedial at IBC 2014

    Tedial at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • TMD summarises their work this year at IBC 2014

    TMD summarises their work this year at IBC 2014

  • Metus at IBC 2014

    Metus at IBC 2014

  • Metus at NAB 2014

    Metus at NAB 2014

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

    Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • Dalet at NAB 2013

    Dalet at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Dalet at NAB 2012

    Dalet at NAB 2012

  • Broadway Systems at NAB 2012

    Broadway Systems at NAB 2012

  • VIZRT at BVE 2012

    VIZRT at BVE 2012

  • PlayBox Technology at BVE North 2011

    PlayBox Technology at BVE North 2011

  • Globecast at IBC2011

    Globecast at IBC2011

  • IPV at IBC2011

    IPV at IBC2011


Related Shows
  • StorageDNA LIVE at BVE 2016

    StorageDNA LIVE at BVE 2016


Articles
Taking on a self employed placement year
Joshua Round The idea of being self-employed or freelancing has always been somewhat terrifying for me. There is a level of uncertainty and responsibility that comes with the freedom of being self-employed, the likes of which makes me wonder why I chose to give it a go for my placement year as part of my university course - (BSc) Television and Broadcasting.
Tags: iss133 | placement year | university | student | education | portsmouth | Joshua Round
Contributing Author Joshua Round Click to read or download PDF
Ride along on the Tour de Tech at IBC
Lorna Garrett Finding your way through the halls and aisles of the massive IBC can feel a bit like attempting to complete all 21 stages of the Tour de France — but in far fewer days. But have no fear; your team at Garland is all geared up and ready to show you the best on show at this year's exhibition.
Tags: iss133 | garland | liveu | lu600 | media excel | teracue | wisi | broadcast wireless systems | bws | artel | ibc | Lorna Garrett
Contributing Author Lorna Garrett Click to read or download PDF
Ruth Matos Interview - A Career Unfolds
Ruth Matos We first met Ruth when we employed her as a student to help with our live studio at BVE in 2013. We were struck by her determination, passion and ‘I’ll do anything to help’ attitude. We became friends of Facebook and have since seen her career in the industry develop and unfold. We thought it was about time to catch up and share her inspiring story of where she is now, how she got there and the challenges she has faced along the way.
Tags: iss133 | interview | filmming | career | education | Ruth Matos
Contributing Author Ruth Matos Click to read or download PDF
How todays technology platforms can make AMAZING seem mundane
Bruce Devlin - new Some things never change. It's September and it's time for another IBC. Maybe I'm getting old and maybe I have selective memory about the past, but recently there hasn't been the buzz of years gone by. Sure, as you wander around the vast halls there will be lots of enthusiastic sales people trying to give you the best demo possible, but something has changed and to survive, I believe that vendors and customers need to change too.
Tags: iss133 | ibc | class | vulture | imf user group | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF
Perimeter LED screens management
Nicolas Houel Opened in January 2016, Parc Olympique Lyonnais, also known as Groupama Stadium, is the new home of Olympique Lyonnais football club, one of the most popular clubs in France. Since its inauguration, the stadium was a host of UEFA Euro 2016, and was also chosen to stage, among other important events, the 2018 UEFA Europa League Final and football at the 2024 Summer Olympics.
Tags: iss133 | 3dstorm | graphics | groupama stadium | liveexpert | livecg | deltacast | Nicolas Houel
Contributing Author Nicolas Houel Click to read or download PDF