Digital Archiving in Media Workflows


Why is there a growing need for digital archiving in the broadcast and post-production industry?
Across all segments of the media and entertainment industry, the past decade has seen a fundamental transformation, where virtually the entire industry moved from tape or film to digital media. This change required investing in digital equipment and processes: new cameras and new edit workstations, additional servers, storage and software, and hiring or training staff for these new workflows. While the transition to digital is complete for most, the work is far from over. The transition to HD and other higher definition formats is the next wave that has already hit many facilities.
Despite the advantages of digital workflows, there are significant challenges. The most significant is the expanding storage volumes. With each hour of HD footage requiring up to six times more storage capacity than standard definition, the transition from SD to HD is causing content creators to rethink their existing storage architecture. An inadequate storage infrastructure can leave editors scrambling to free up space before they can start editing a new project—or even turn down higher definition projects for lack of storage capacity. Sadly, in many cases the capacity exists, but is tied up in “storage waste,” content that is duplicated within the system as editors swap files and neglect to clean up.
Another challenge is protecting the data. With new high definition digital cameras using more expensive, reusable hard drives or flash as capture media, the natural “backup” that the original filmor videotape provided no longer exists. Without a proper data protection strategy, raw footage is left at risk. A disk crash or accidental deletion can mean a permanent loss of irreplaceable footage.
Also, as digital media workflow technologies emerged, a host of proprietary formats and devices were established with them. Proprietary technologies can hinder media companies from rolling out new technologies and new processes or worse, can create vendor lock-in.
For all these data management challenges, there is a solution—the digital archive.
How is the role of archiving changing with the introduction of digital archives?
Traditionally, archiving was the last step of the creative process. Just before broadcast or distribution, the content would be archived onto a master tape
or film would be stored in a media vault. These vaults were intended as repositories that could be manually searched to retrieve content for later re- use. The harsh reality is that these vaults have been more burial ground than archive.
A well-architected digital archive can be much more than a final resting place. Instead of a final step in the edit process, the archive should be an integral part of the digital media workflow. In a March 2008 seminar, Samuel Reisner of ESPN Sports underscored this concept, stating that: “Any piece of media must be available on any production platform whenever it is needed.”
What are the benefits of a central comprehensive archive in digital media workflows?
From ingest, to edit, to playout or distribution, offering digital archive access to all steps in the workflow can help achieve a range of benefits.
First of all an online digital archive, especially one with rich search and retrieval capabilities, allows for easier content re-use. Previously created content is kept at producers or editors fingertips, making re-use quick and easy. It also empowers content owners to archive and retrieve their projects themselves, minimizing the delays and inefficiency of having archivists acting
as the sole access point to the archive’s content.
Storage growth can also be better managed. Data migration or hierarchical storage management (HSM) features allow administrators to align the storage
performance (and cost) with the access needs and relative value of the content. Making content available doesn’t necessarily mean it needs to be available on high-performance storage. Moving less frequently used content to tape or lower- performance disk can significantly reduce infrastructure costs.
More importantly, the digital archive protects and preserves content throughout its lifespan, from the moment it enters the workflow until decades after the finished work is archived for long-term preservation. The archive not only backs up otherwise unprotected raw footage but also becomes the master vault for historical preservation projects.
With the new expanded role of the archive, what advice would you give to organisations wanting to maximise its potential?
There are several strategies that can help centre the archive in the workflow and maximise its potential.
The first rule is that anyone working with creative content should always archive all raw footage to make sure it will always be available. Many often don’t know what to keep and what to leave on the cutting room floor. However, even the most ordinary clip can become priceless overnight.
Content creators should also be given direct access to the archive. Once the archive is set up, there’s no reason why editors shouldn’t be able to archive and retrieve projects on their own.
Using automatic archiving can significantly reduce clutter in work areas. Despite users’ best intentions, work areas on edit workstations and shared storage areas tend to grow out of control.
Finally, set up multi-tier archives so that more expensive near-line storage can be reserved for content that requires higher performance. Less expensive, off-line “deep” storage can be used for long term preservation or lower priority content. After all, as your archive expands, the reality is that only a subset may be retrieved from the archive.
What should one be looking out for in an archiving solution?
In order to get the best out of your archiving solution, there are several things to look out for that can make a big difference to your workflow and archive management.
An archive solution should be capable of automatically archiving new footage as soon as it enters the workflow, using precise criteria, and attaching metadata to archived files to make it easier and search and locate them in the future.
Choosing a solution with a simple and easy-to-use end-user interface for archive and retrieval, will allow content creators to be able to archive and search themselves.
In order to avoid proprietary workflow components choose a solution that will; provide support for archiving in standard open file format, is integrated with open asset management solutions, offers easy integration with other management tools and supports a variety of archive media and platforms.
To reduce clutter and reclaim waster storage, look for an archive solution that will automatically archive outdated files and once archived, removes files from the work area. Additionally, choose a solution that offers de-duplication to optimise available storage space and does not rely on stubbing as the only retrieval method.
Lastly, pick a solution that supports both near-line and deep archives that can automatically remove files from near-line storage based on criteria such as file age or last access date.
What advice would you give to those undertaking a large-scale preservation?
Those undertaking a large-scale long term preservation project need a robust, reliable archiving solution that will enable long term retention as well as efficient management. The archive solution will need to support high performance configurations to allow massive numbers of files to be archived daily. It will also need to use non-proprietary formats and technologies that support a broad range of media and device types. Cataloguing, indexation and search functionalities should also be a must.

Tags: atempo | iss045 | archiving | workflow | backup | storage | digital archiving | post production | N/A
Contributing Author N/A

Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Global Distribution at IBC2011

    Global Distribution at IBC2011

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • ChyronHego show their PRIME Workflow at IBC 2018

    ChyronHego show their PRIME Workflow at IBC 2018

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Snell Advanced Media (SAM) 4k workflow at IBC 2015

    Snell Advanced Media (SAM) 4k workflow at IBC 2015

  • LIVEU Workflow solutions at NAB 2015

    LIVEU Workflow solutions at NAB 2015

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Quantel LiveTouch at IBC 2014

    Quantel LiveTouch at IBC 2014

  • Quantel deal with AFP at IBC 2014

    Quantel deal with AFP at IBC 2014

  • Snell Kahuna Production Switcher at IBC 2014

    Snell Kahuna Production Switcher at IBC 2014

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Streaming Media with Interra Systems at NAB 2017

    Streaming Media with Interra Systems at NAB 2017

  • ERA - Adobe Anywhere - at BVE 2015

    ERA - Adobe Anywhere - at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • Telestream Wirecast and Switch at IBC 2014

    Telestream Wirecast and Switch at IBC 2014

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Broadstream Solutions at NAB 2014

    Broadstream Solutions at NAB 2014

  • Telestream Enterprise at NAB 2014

    Telestream Enterprise at NAB 2014

  • Telestream Switch at NAB 2014

    Telestream Switch at NAB 2014

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Yospace: Advert Insertion at NAB 2013

    Yospace: Advert Insertion at NAB 2013

  • Vimond Control Center at NAB 2013

    Vimond Control Center at NAB 2013

  • Telestream Vantage v5 at NAB 2013

    Telestream Vantage v5 at NAB 2013

  • Telestream at BVE 2013

    Telestream at BVE 2013

  • Guntermann and Drunck GmbH at BVE 2013

    Guntermann and Drunck GmbH at BVE 2013

  • Tedial at NAB 2012

    Tedial at NAB 2012

  • Marquis Broadcast at NAB 2012

    Marquis Broadcast at NAB 2012

  • Root6 at NAB 2012

    Root6 at NAB 2012

  • TMD at NAB 2012

    TMD at NAB 2012

  • PlayBox Technology at BVE 2012

    PlayBox Technology at BVE 2012

  • Net Insight at IBC2011

    Net Insight at IBC2011

  • Tedial at IBC2011

    Tedial at IBC2011

  • IPV at IBC2011

    IPV at IBC2011

  • TSL at IBC2011

    TSL at IBC2011

  • Marquis Broadcast show Backup Versioning at NAB 2018

    Marquis Broadcast show Backup Versioning at NAB 2018

  • Facilis Technology at BVE 2013

    Facilis Technology at BVE 2013

  • Facilis Technology Shared Storage at IBC 2018

    Facilis Technology Shared Storage at IBC 2018

  • Object based Storage Solution from Object Matrix at IBC 2017

    Object based Storage Solution from Object Matrix at IBC 2017

  • Glyph External Storage including the Studio and Atom raid at IBC 2017

    Glyph External Storage including the Studio and Atom raid at IBC 2017

  • Object Based Storage Solutions from Object Matrix at NAB 2017

    Object Based Storage Solutions from Object Matrix at NAB 2017

  • Storage DNA at BVE 2016

    Storage DNA at BVE 2016

  • Fibrenetix with StorageDNA at IBC 2014

    Fibrenetix with StorageDNA at IBC 2014

  • Fibrenetix with Quadrus at IBC 2014

    Fibrenetix with Quadrus at IBC 2014

  • GB Labs Space at NAB 2014

    GB Labs Space at NAB 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • SGL at NAB 2013

    SGL at NAB 2013

  • Nexsan at NAB 2012

    Nexsan at NAB 2012

  • Arkivum at BVE 2012

    Arkivum at BVE 2012

  • Facilis at BVE 2012

    Facilis at BVE 2012

  • Real Life Kit at ProVideo2011

    Real Life Kit at ProVideo2011

  • Sonnet Technology at IBC2011

    Sonnet Technology at IBC2011

  • ATTO Technology at IBC2011

    ATTO Technology at IBC2011

  • Davinci Resolve 14 with Fairlight from Blackmagic Design at NAB 2017

    Davinci Resolve 14 with Fairlight from Blackmagic Design at NAB 2017

  • Forscene at IBC 2014

    Forscene at IBC 2014

  • Forbidden Technologies FORscene at BVE 2014

    Forbidden Technologies FORscene at BVE 2014

  • Forbidden Technologies FORscene App at BVE 2014

    Forbidden Technologies FORscene App at BVE 2014

  • Autodesk at NAB 2012

    Autodesk at NAB 2012

  • SGO at IBC2011

    SGO at IBC2011


Related Shows
  • StorageDNA LIVE at BVE 2016

    StorageDNA LIVE at BVE 2016


Articles
NEP Student Day
KitPlus The world of broadcasting is in constant flux, with undefined boundaries between broadcast, TV, film and interactive media. With a changing consumer demographic now a key influence on the industry, many companies no longer specialise in a single medium, but have had to become adaptable to deliver multiple services through a combination of media.
Tags: iss134 | students | tvfutures | NEP | KitPlus
Contributing Author KitPlus Click to read or download PDF
Keeping it remotely real
Reuben Such Everyone wants to do more with less. Always have, although it could be argued that doing more with more is something to aspire to, not many have that luxury. So let’s stick with the prevailing winds of doing more with less, and not just doing more, but doing it remotely, particularly in terms of production. Remote production, in particular, is getting a lot of attention in the field these days, but not so much in terms of the remote operation of fixed studios.
Tags: iss134 | remote control | IPE | IDS | Reuben Such
Contributing Author Reuben Such Click to read or download PDF
Protecting the continuity of transmission
Lorna Garrett Your viewers love you. You consistently bring them their preferred channels 24/7. They’ve come to rely on you for their viewing pleasure. They never miss cheering on their beloved sports teams. They’re the envy of their friends as they watch live concerts of their favourite bands. They gather the family around and catch up on their must-see shows. They don’t have a bad word to say about you.
Tags: iss134 | garland | gpl | streaming | artel | disaster recovery | Lorna Garrett
Contributing Author Lorna Garrett Click to read or download PDF
The Wireless Way to 4k
JP Delport DTC’s AEON group of products have been specifically designed for the 4K market. We encode with the more efficient HEVC algorithm, which means we are taking a 12G signal and compressing it to a bitrate that can be managed over an RF link. So what makes this a leading idea in the 4K revolution?
Tags: iss134 | wireless | 4k | transmission | JP Delport
Contributing Author JP Delport Click to read or download PDF
What content providers need to know about OTT
Hiren Hindocha As OTT (Over-The-Top) technology has gotten more mature and established robust standards over the years, the concept of OTT monitoring is gaining popularity. With customer expectations soaring, it’s vital for OTT providers to deliver superior quality content. To deliver Quality of Experience (QoE) on par with linear TV broadcast, the entire system, starting from ingest to multi-bitrate encoding to delivery to CDN must be monitored continuously.
Tags: iss134 | ott monitoring | qos | logging | compliance | dash | atsc | cloud | Hiren Hindocha
Contributing Author Hiren Hindocha Click to read or download PDF