Digital Archiving in Media Workflows


Why is there a growing need for digital archiving in the broadcast and post-production industry?
Across all segments of the media and entertainment industry, the past decade has seen a fundamental transformation, where virtually the entire industry moved from tape or film to digital media. This change required investing in digital equipment and processes: new cameras and new edit workstations, additional servers, storage and software, and hiring or training staff for these new workflows. While the transition to digital is complete for most, the work is far from over. The transition to HD and other higher definition formats is the next wave that has already hit many facilities.
Despite the advantages of digital workflows, there are significant challenges. The most significant is the expanding storage volumes. With each hour of HD footage requiring up to six times more storage capacity than standard definition, the transition from SD to HD is causing content creators to rethink their existing storage architecture. An inadequate storage infrastructure can leave editors scrambling to free up space before they can start editing a new project—or even turn down higher definition projects for lack of storage capacity. Sadly, in many cases the capacity exists, but is tied up in “storage waste,” content that is duplicated within the system as editors swap files and neglect to clean up.
Another challenge is protecting the data. With new high definition digital cameras using more expensive, reusable hard drives or flash as capture media, the natural “backup” that the original filmor videotape provided no longer exists. Without a proper data protection strategy, raw footage is left at risk. A disk crash or accidental deletion can mean a permanent loss of irreplaceable footage.
Also, as digital media workflow technologies emerged, a host of proprietary formats and devices were established with them. Proprietary technologies can hinder media companies from rolling out new technologies and new processes or worse, can create vendor lock-in.
For all these data management challenges, there is a solution—the digital archive.
How is the role of archiving changing with the introduction of digital archives?
Traditionally, archiving was the last step of the creative process. Just before broadcast or distribution, the content would be archived onto a master tape
or film would be stored in a media vault. These vaults were intended as repositories that could be manually searched to retrieve content for later re- use. The harsh reality is that these vaults have been more burial ground than archive.
A well-architected digital archive can be much more than a final resting place. Instead of a final step in the edit process, the archive should be an integral part of the digital media workflow. In a March 2008 seminar, Samuel Reisner of ESPN Sports underscored this concept, stating that: “Any piece of media must be available on any production platform whenever it is needed.”
What are the benefits of a central comprehensive archive in digital media workflows?
From ingest, to edit, to playout or distribution, offering digital archive access to all steps in the workflow can help achieve a range of benefits.
First of all an online digital archive, especially one with rich search and retrieval capabilities, allows for easier content re-use. Previously created content is kept at producers or editors fingertips, making re-use quick and easy. It also empowers content owners to archive and retrieve their projects themselves, minimizing the delays and inefficiency of having archivists acting
as the sole access point to the archive’s content.
Storage growth can also be better managed. Data migration or hierarchical storage management (HSM) features allow administrators to align the storage
performance (and cost) with the access needs and relative value of the content. Making content available doesn’t necessarily mean it needs to be available on high-performance storage. Moving less frequently used content to tape or lower- performance disk can significantly reduce infrastructure costs.
More importantly, the digital archive protects and preserves content throughout its lifespan, from the moment it enters the workflow until decades after the finished work is archived for long-term preservation. The archive not only backs up otherwise unprotected raw footage but also becomes the master vault for historical preservation projects.
With the new expanded role of the archive, what advice would you give to organisations wanting to maximise its potential?
There are several strategies that can help centre the archive in the workflow and maximise its potential.
The first rule is that anyone working with creative content should always archive all raw footage to make sure it will always be available. Many often don’t know what to keep and what to leave on the cutting room floor. However, even the most ordinary clip can become priceless overnight.
Content creators should also be given direct access to the archive. Once the archive is set up, there’s no reason why editors shouldn’t be able to archive and retrieve projects on their own.
Using automatic archiving can significantly reduce clutter in work areas. Despite users’ best intentions, work areas on edit workstations and shared storage areas tend to grow out of control.
Finally, set up multi-tier archives so that more expensive near-line storage can be reserved for content that requires higher performance. Less expensive, off-line “deep” storage can be used for long term preservation or lower priority content. After all, as your archive expands, the reality is that only a subset may be retrieved from the archive.
What should one be looking out for in an archiving solution?
In order to get the best out of your archiving solution, there are several things to look out for that can make a big difference to your workflow and archive management.
An archive solution should be capable of automatically archiving new footage as soon as it enters the workflow, using precise criteria, and attaching metadata to archived files to make it easier and search and locate them in the future.
Choosing a solution with a simple and easy-to-use end-user interface for archive and retrieval, will allow content creators to be able to archive and search themselves.
In order to avoid proprietary workflow components choose a solution that will; provide support for archiving in standard open file format, is integrated with open asset management solutions, offers easy integration with other management tools and supports a variety of archive media and platforms.
To reduce clutter and reclaim waster storage, look for an archive solution that will automatically archive outdated files and once archived, removes files from the work area. Additionally, choose a solution that offers de-duplication to optimise available storage space and does not rely on stubbing as the only retrieval method.
Lastly, pick a solution that supports both near-line and deep archives that can automatically remove files from near-line storage based on criteria such as file age or last access date.
What advice would you give to those undertaking a large-scale preservation?
Those undertaking a large-scale long term preservation project need a robust, reliable archiving solution that will enable long term retention as well as efficient management. The archive solution will need to support high performance configurations to allow massive numbers of files to be archived daily. It will also need to use non-proprietary formats and technologies that support a broad range of media and device types. Cataloguing, indexation and search functionalities should also be a must.

Tags: atempo | iss045 | archiving | workflow | backup | storage | digital archiving | post production | N/A
Contributing Author N/A

Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Global Distribution at IBC2011

    Global Distribution at IBC2011

  • Dalet at NAB 2014

    Dalet at NAB 2014

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • ChyronHego show their PRIME Workflow at IBC 2018

    ChyronHego show their PRIME Workflow at IBC 2018

  • Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

    Object Matrix Hybrid Workflow and Artifical Intelligence at NAB 2018

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Snell Advanced Media (SAM) 4k workflow at IBC 2015

    Snell Advanced Media (SAM) 4k workflow at IBC 2015

  • LIVEU Workflow solutions at NAB 2015

    LIVEU Workflow solutions at NAB 2015

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Quantel LiveTouch at IBC 2014

    Quantel LiveTouch at IBC 2014

  • Quantel deal with AFP at IBC 2014

    Quantel deal with AFP at IBC 2014

  • Snell Kahuna Production Switcher at IBC 2014

    Snell Kahuna Production Switcher at IBC 2014

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Streaming Media with Interra Systems at NAB 2017

    Streaming Media with Interra Systems at NAB 2017

  • ERA - Adobe Anywhere - at BVE 2015

    ERA - Adobe Anywhere - at BVE 2015

  • NETIA at BVE 2015

    NETIA at BVE 2015

  • Dalet at IBC 2014

    Dalet at IBC 2014

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • Telestream Wirecast and Switch at IBC 2014

    Telestream Wirecast and Switch at IBC 2014

  • Telestream Vantage support for DPP at IBC 2014

    Telestream Vantage support for DPP at IBC 2014

  • Broadstream Solutions at NAB 2014

    Broadstream Solutions at NAB 2014

  • Telestream Enterprise at NAB 2014

    Telestream Enterprise at NAB 2014

  • Telestream Switch at NAB 2014

    Telestream Switch at NAB 2014

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Yospace: Advert Insertion at NAB 2013

    Yospace: Advert Insertion at NAB 2013

  • Vimond Control Center at NAB 2013

    Vimond Control Center at NAB 2013

  • Telestream Vantage v5 at NAB 2013

    Telestream Vantage v5 at NAB 2013

  • Telestream at BVE 2013

    Telestream at BVE 2013

  • Guntermann and Drunck GmbH at BVE 2013

    Guntermann and Drunck GmbH at BVE 2013

  • Tedial at NAB 2012

    Tedial at NAB 2012

  • Marquis Broadcast at NAB 2012

    Marquis Broadcast at NAB 2012

  • Root6 at NAB 2012

    Root6 at NAB 2012

  • TMD at NAB 2012

    TMD at NAB 2012

  • PlayBox Technology at BVE 2012

    PlayBox Technology at BVE 2012

  • Net Insight at IBC2011

    Net Insight at IBC2011

  • Tedial at IBC2011

    Tedial at IBC2011

  • IPV at IBC2011

    IPV at IBC2011

  • TSL at IBC2011

    TSL at IBC2011

  • Marquis Broadcast show Backup Versioning at NAB 2018

    Marquis Broadcast show Backup Versioning at NAB 2018

  • Facilis Technology at BVE 2013

    Facilis Technology at BVE 2013

  • Facilis Technology Shared Storage at IBC 2018

    Facilis Technology Shared Storage at IBC 2018

  • Object based Storage Solution from Object Matrix at IBC 2017

    Object based Storage Solution from Object Matrix at IBC 2017

  • Glyph External Storage including the Studio and Atom raid at IBC 2017

    Glyph External Storage including the Studio and Atom raid at IBC 2017

  • Object Based Storage Solutions from Object Matrix at NAB 2017

    Object Based Storage Solutions from Object Matrix at NAB 2017

  • Storage DNA at BVE 2016

    Storage DNA at BVE 2016

  • Fibrenetix with StorageDNA at IBC 2014

    Fibrenetix with StorageDNA at IBC 2014

  • Fibrenetix with Quadrus at IBC 2014

    Fibrenetix with Quadrus at IBC 2014

  • GB Labs Space at NAB 2014

    GB Labs Space at NAB 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • SGL at NAB 2013

    SGL at NAB 2013

  • Nexsan at NAB 2012

    Nexsan at NAB 2012

  • Arkivum at BVE 2012

    Arkivum at BVE 2012

  • Facilis at BVE 2012

    Facilis at BVE 2012

  • Real Life Kit at ProVideo2011

    Real Life Kit at ProVideo2011

  • Sonnet Technology at IBC2011

    Sonnet Technology at IBC2011

  • ATTO Technology at IBC2011

    ATTO Technology at IBC2011

  • Davinci Resolve 14 with Fairlight from Blackmagic Design at NAB 2017

    Davinci Resolve 14 with Fairlight from Blackmagic Design at NAB 2017

  • Forscene at IBC 2014

    Forscene at IBC 2014

  • Forbidden Technologies FORscene at BVE 2014

    Forbidden Technologies FORscene at BVE 2014

  • Forbidden Technologies FORscene App at BVE 2014

    Forbidden Technologies FORscene App at BVE 2014

  • Autodesk at NAB 2012

    Autodesk at NAB 2012

  • SGO at IBC2011

    SGO at IBC2011


Related Shows
  • StorageDNA LIVE at BVE 2016

    StorageDNA LIVE at BVE 2016


Articles
Taking on a self employed placement year
Joshua Round The idea of being self-employed or freelancing has always been somewhat terrifying for me. There is a level of uncertainty and responsibility that comes with the freedom of being self-employed, the likes of which makes me wonder why I chose to give it a go for my placement year as part of my university course - (BSc) Television and Broadcasting.
Tags: iss133 | placement year | university | student | education | portsmouth | Joshua Round
Contributing Author Joshua Round Click to read or download PDF
Creating Authentic Content That Counts
Frank le Mair We’re deep into the 2010’s and the way in which we consume content has changed forever. Broadcasters and content owners are fighting for eyeballs in a saturated market where consumers are watching their favourite shows on different devices across a number of platforms. To target millennials and younger generations, who generally consume short bursts of video on YouTube and social media - particularly Snapchat, Instagram and now IGTV - media companies are creating more and more authentic stories and are using platforms that are compelling for this demographic. Unlike generations before them, they have totally new video viewing patterns and ideals.
Tags: iss133 | insight tv | monster energy | amazon | millennial | Frank le Mair
Contributing Author Frank le Mair Click to read or download PDF
Perimeter LED screens management
Nicolas Houel Opened in January 2016, Parc Olympique Lyonnais, also known as Groupama Stadium, is the new home of Olympique Lyonnais football club, one of the most popular clubs in France. Since its inauguration, the stadium was a host of UEFA Euro 2016, and was also chosen to stage, among other important events, the 2018 UEFA Europa League Final and football at the 2024 Summer Olympics.
Tags: iss133 | 3dstorm | graphics | groupama stadium | liveexpert | livecg | deltacast | Nicolas Houel
Contributing Author Nicolas Houel Click to read or download PDF
The Pace of Change
Dick Hobbs - new

The youngest human to stand on the moon (so far) was Charles Moss, the lunar module pilot of Apollo 16. Charlie had a wonderful claim: his father witnessed the Wright Brothers’ first flight at Kitty Hawk, and lived to see his son on the moon.

Does anything capture the speed of technological advance better than that? The whole of the history of powered flight in one lifetime.

Tags: iss133 | state of the nation | st2110 | st2110-10 | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF
Managing Technological Change
Alan Wheable Continual technological change in the broadcast and media industries can make it difficult to plan for the mid to long term. Typically, broadcasters and media organisation are still implementing the last set of changes to working practices when the next changes come along.
Tags: iss133 | omnitek | ip | waveform | vectorscope | ultra tq | Alan Wheable
Contributing Author Alan Wheable Click to read or download PDF