Metadata rules


Will Strauss# TV-Bay Magazine
Read ezine online
Regular readers of this column will know that I am frequently tasked with covering the sexiest of broadcast industry subjects. Indeed, just two months ago I got to discuss test and measurement (T&M). Well, if you thought that was intense, just wait for this month's topic. Not only is it potentially even less glamorous than T&M, it has the potential to be an instant cure for insomnia. Unfortunately, it's also a hugely important subject if you're talking about storage and archive (as this issue of TV-Bay is). What am I talking about? Is it a gizmo? Is it a widget? Nope, it’s metadata. Hold on to your hats. This is going to be wild.
Now, before I wade into the metadata stuff, let’s give this article some context. Whether owned by a broadcaster, a producer or a library, a content archive isn’t just a historical record of its acquisition efforts and its televisual or filmic output. In many cases it is its biggest asset.
The exploitation of archive content is a big thing. Rich media content is in demand and, if you can make it available quickly and easily, archive footage and programmes can be re-licensed for a myriad of uses from historical documentaries and primetime clip shows to one-off mobile downloads.
To make this happen, more and more archives are being digitized, turning film, tape and digital tape into files. This allows for instant retrieval and exploitation, micro payments, process automation and more.
Yet, while digitization is a must, I see two major inconveniences when it comes to long-term preservation of file-based media: the choice of format that the footage should be archived in; and the amount of metadata required for it to be at its most useful.
Currently, television is produced by anything from a smartphone to a 4k camcorder, in any format from mp4 to AVC-Intra. The commonly held belief is that you archive in the highest quality possible. Which would be AVC-Intra. But in several years time another format will be in vogue. So, if we archive in AVC-Intra now, in order to re-use that content in the future, we would need to match it up with the ‘new’ format. To this you would have to transcode the AVC-Intra. And when the next format comes round after that, you would have to transcode it again. Is that a problem? Well, um, yes, because as each transcode takes place the content gradually gains coding impairments. And if you’re compressing or re-compressing it, you get even more impairments. This is far from ideal.
What may be required (please don’t shoot me) is ANOTHER format. An agreed format that we stick to for archiving purposes. It must be one that is high quality, lossless, open, widely adopted and easy for future computer systems to recompile. The suggested format should be either lightly compressed (in order to keep storage requirements realistic), lossless compressed or completely uncompressed and it should be wrapped up in something like MXF.
John Zubrzycki, the Section Leader for Archives Research at BBC R&D has done a lot of good work on this subject. He urges archive owners to “work together to present common requirements to industry” and argues for what he calls a “light compression standard” that can be used for SD and HDTV archiving. This would avoid the need to recode footage every time production moves forward. Which makes a lot of sense.
So, that’s a potential solution to the format problem. What about this here metadata stuff then? In the new file-based world that TV now inhabits, metadata rules (pun intended) and the efficient implementation of metadata is key to content management and file-based workflows.
Technical metadata is used, for example, to drive entire MAM systems or playout operations and, without it, some files simply won’t work properly in certain devices. While descriptive metadata (shot length, content, music type etc) is what interests humans and is the information required for indexing and archiving, monetizing and more.
Unfortunately, as a result of its increasing importance, metadata requirements have got very complicated. To quote Niall Duffy, the managing director of the media technology consultancy Mediasmiths, “you’ve currently got people who for very correct reasons want to come up with very structured metadata models because from their point of view that is essential for building any sort of long-term archive. But from an archive user’s point-of-view the more fields there are, the less they’ll find as it becomes too confusing for them.”
Having lots of metadata fields to complete also makes data inputting nigh on impossible and unrealistic for human beings. And, in fact, when you look closely into how researchers actually use metadata and what they look for, what you find is that searches revert back to what we all know best: a Google-type search.
With that in mind, metadata should not be about increasing the number of fields to improve archive or asset search. Instead it should be concerned with thinking about how people actually seek out content.
The ideal scenario, it would seem, is for footage to be given a small set of structured metadata fields that allow for things like categorization and process automation and then a large unstructured data field that is automatically generated, user generated or based on tagging.
This approach would allow researchers (human or otherwise) to ‘discover’ based on their own requirements rather than the restrictions of the metadata fields. In short, less is definitely more when it comes to metadata.
To my mind, in order to make this work, metadata therefore needs to be dealt with further up the production chain. It cannot be left to the archivists or the data wranglers. Producers need to take responsibility for it and that doesn’t mean reminding a runner or a camera assistant as an afterthought. If that happens, and the importance of metadata is not asserted, you get poor metadata. And programmes and footage with poor metadata will have a limited archival afterlife.
So, there you have it: what we need is a new format for archive preservation and a different approach to metadata. It’s not sexy. But it is big. And it is clever. When it comes to archives and storage, metadata rules.
I wonder what subject is up next?

Tags: iss069 | metadata | object matrix | archive | storage | avc-intra | mam | Will Strauss#
Contributing Author Will Strauss#

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Primestream on BroadcastShow LIVE at IBC 2013

    Primestream on BroadcastShow LIVE at IBC 2013

  • SGL at NAB 2012

    SGL at NAB 2012

  • Prime Focus Technologies at IBC 2016

    Prime Focus Technologies at IBC 2016

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Root6 at BVE 2012

    Root6 at BVE 2012

  • MOG Technologies at BVE 2012

    MOG Technologies at BVE 2012

  • Cooke at IBC2011

    Cooke at IBC2011

  • Radiant Grid at IBC2011

    Radiant Grid at IBC2011

  • Object Based Storage Solutions from Object Matrix at NAB 2017

    Object Based Storage Solutions from Object Matrix at NAB 2017

  • Object based Storage Solution from Object Matrix at IBC 2017

    Object based Storage Solution from Object Matrix at IBC 2017

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • NOA Archive Solutions at IBC 2014

    NOA Archive Solutions at IBC 2014

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • SGL at NAB 2013

    SGL at NAB 2013

  • Facilis Technology at BVE 2013

    Facilis Technology at BVE 2013

  • Nexsan at NAB 2012

    Nexsan at NAB 2012

  • Arkivum at BVE 2012

    Arkivum at BVE 2012

  • ERA - Cloud Services - at BVE 2015

    ERA - Cloud Services - at BVE 2015

  • Cinegy: Multiviewer at NAB 2013

    Cinegy: Multiviewer at NAB 2013

  • Netia at IBC2011

    Netia at IBC2011

  • SGL at NAB 2014

    SGL at NAB 2014

  • Softron Media Services on BroadcastShow LIVE at IBC 2013

    Softron Media Services on BroadcastShow LIVE at IBC 2013

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Cinegy at IBC 2013

    Cinegy at IBC 2013

  • TMD at BVE 2012

    TMD at BVE 2012

  • FOR-A at IBC2011

    FOR-A at IBC2011

  • Facilis Technology Shared Storage at IBC 2018

    Facilis Technology Shared Storage at IBC 2018

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • Glyph External Storage including the Studio and Atom raid at IBC 2017

    Glyph External Storage including the Studio and Atom raid at IBC 2017

  • Storage DNA at BVE 2016

    Storage DNA at BVE 2016

  • Fibrenetix with StorageDNA at IBC 2014

    Fibrenetix with StorageDNA at IBC 2014

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • Fibrenetix with Quadrus at IBC 2014

    Fibrenetix with Quadrus at IBC 2014

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • GB Labs Space at NAB 2014

    GB Labs Space at NAB 2014

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • Facilis at BVE 2012

    Facilis at BVE 2012

  • Real Life Kit at ProVideo2011

    Real Life Kit at ProVideo2011

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Sonnet Technology at IBC2011

    Sonnet Technology at IBC2011

  • ATTO Technology at IBC2011

    ATTO Technology at IBC2011

  • Metus MAM and Ingest at IBC 2013

    Metus MAM and Ingest at IBC 2013

  • Cloud Media Management with Medway from Marquis Broadcast at IBC 2017

    Cloud Media Management with Medway from Marquis Broadcast at IBC 2017

  • Be More with Clear from Prime Focus Technologies at NAB 2017

    Be More with Clear from Prime Focus Technologies at NAB 2017

  • Hybrid Cloud Media Aggregation from Cantemo at NAB 2017

    Hybrid Cloud Media Aggregation from Cantemo at NAB 2017

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Media Asset Management Software from Blue Lucy at NAB 2017

    Media Asset Management Software from Blue Lucy at NAB 2017

  • Mediaflex UMS Platform from TMD at NAB 2017

    Mediaflex UMS Platform from TMD at NAB 2017

  • Blue Lucy at IBC 2015

    Blue Lucy at IBC 2015

  • EditShare at BVE 2015

    EditShare at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Tedial at IBC 2014

    Tedial at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • TMD summarises their work this year at IBC 2014

    TMD summarises their work this year at IBC 2014

  • Metus at IBC 2014

    Metus at IBC 2014

  • Metus at NAB 2014

    Metus at NAB 2014

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

    Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • Dalet at NAB 2013

    Dalet at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Dalet at NAB 2012

    Dalet at NAB 2012

  • Broadway Systems at NAB 2012

    Broadway Systems at NAB 2012

  • VIZRT at BVE 2012

    VIZRT at BVE 2012

  • PlayBox Technology at BVE North 2011

    PlayBox Technology at BVE North 2011

  • Globecast at IBC2011

    Globecast at IBC2011

  • IPV at IBC2011

    IPV at IBC2011


Related Shows
  • StorageDNA LIVE at BVE 2016

    StorageDNA LIVE at BVE 2016


Articles
Looking for the Silver Lining
Harry Grinling According to the World Meteorological Organisation, there are 10 different types of cloud, each of which can be divided further into sub-types. They range from the cirrus, the thin floaty clouds which generally serve only to make the sky look beautiful to the towering, all-embracing cumulonimbus which can deliver fearful quantities of rain – the biggest cumulonimbus clouds can contain 50 million tonnes of water.
Tags: iss136 | cloud | lto | archive | storage | Harry Grinling
Contributing Author Harry Grinling Click to read or download PDF
Keeping Your Post Prodction on Track with Subclips and Search Bins
Alex Macleod

For my 2nd Kit Plus article I thought I’d try and build on the theme of my first, and that’s one of making sure things are organised at all levels of your post production projects.

Last time I talked about trying as best as you can to stick to the ‘two week rule’, making sure that the names & locations of every asset you import, and every bin & sequence that you create in your project - will make sense to you regardless of how long it is you spend away from it.

Tags: iss136 | mediacity training | subclip | premiere pro | gvs | bve | bve2019 | Alex Macleod
Contributing Author Alex Macleod Click to read or download PDF
Remote Teams and Talent
Megan Cater If your studio works with non-local creative talent, you already know that there are opportunities and challenges associated with distributed production and post production. Bridging the distance not only allows you to find the best talent for the job anywhere in the world, it creates the potential for a diverse and globally-minded workforce that boosts the creativity and vision of your entire company.
Tags: iss136 | signiant | file acceleration | ftp | dropbox | sharepoint | slack | saas | media shuttle | Megan Cater
Contributing Author Megan Cater Click to read or download PDF
Painting Performance Analytics with ChyronHego
KitPlus By now, most people are familiar with the sport of mixed martial arts (MMA) and its leading organization – UFC (Ultimate Fighting Championship). And while the sport and its leading promotion are only 25 years old, a great deal has changed in those 25 years, including the training of UFC athletes.
Tags: iss136 | paint | telestrator | ufc | chyron | chyronhego | KitPlus
Contributing Author KitPlus Click to read or download PDF
Rotolight Anova Pro 2 User Review
Andy McKenzie The Anova PRO 2 is the fourth generation of Rotolight’s studio/location light, offering 70% more power output than its predecessor. It is claimed be one of the brightest LED lights ever launched in its class, delivering 10,700 lux at 3 feet yet consuming only 72 watts. Figure 1 shows the front with accessory mounting spigots (1), optional barn doors (2) and a gel frame holder.
Tags: iss136 | rotolight | anova pro 2 | led | lighting | flash light | dmx control | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF