From archival master to usable video with Mark Gray


Mark Gray TV-Bay Magazine
Read ezine online
by Mark Gray
Issue 80 - August 2013
With about 4 million new video channels expected to hit the market in the next five years, its going to be more important than ever to make high-quality content readily available. Many media archives and libraries; studios; broadcasters; production and postproduction facilities; and mastering, quality control, and digital cinema facilities have volumes of pristine master files in their vaults just waiting to be repurposed. The challenge will be making use of their archival masters efficiently. Industry veteran Mark Gray touches on some of the key points.

Some might not grasp the complexity and enormity of archiving; can you share some insight into the process?

Video files represent big data, and in most cases, a lot of time, money, creativity, and expertise go into creating them. These video files can be massive, which requires media organizations to make a significant investment in storing the original, pristine master files, copies of which might go to a postproduction house for manipulation, a studio for previewing, a broadcaster for transcoding, an engineer for test and measurement, an editor for repurposing, or any number of other destinations in the life of a video asset. In any case, archiving the original master files ensures they are preserved and available for later use. In order to ensure high-quality, artifact-free copies, the files are archived in uncompressed or other lossless mastering formats such as JPEG 2000 or DPX.
When it comes to viewing and delivering archival master files, what are some of the challenges the industry faces?

Master files are typically stored in archive formats that cannot be played on a standard player or screen, so those who need to work with the files must have expensive, proprietary players for different vendors and file types. In addition, they need professional HD-SDI monitors that can cost tens of thousands of dollars each. Many facilities cannot afford expensive systems with DCP projectors, HD-SDI cards and monitors, and software from each of the manufacturers represented in their media collection for everyone in a group, so many archivists and production teams have to work with lower-quality mezzanine and proxy copies of their masters. This scenario means that those facilities also need storage for those lower-quality files, plus the asset management tools to keep track of them with their masters a system that can cost as much as or more than the storage of the master files themselves.

These challenges are particularly difficult for archives, which often operate without IT support and under incredibly tight budgets that dont allow for expensive, proprietary equipment and software from multiple manufacturers. The result? Many archivists are unable to view the master files in their care. When retrieving a requested file from the archive for delivery or digitizing a tape into a master file for storage, the archivist has no way of knowing whether the copy is faithful to the original or what it will look like on the screen.
Why is the process of viewing and delivering master files so complex?

There are several video file formats depending on which vendors hardware (e.g., a camera) or software (e.g., a nonlinear editor) created the video. In turn, those files can be bundled together with codecs, metadata, and other file-based elements to create a package, which is contained inside yet another format, a wrapper, for archiving or transporting. Wrappers including the most common MXF, MOV, and AVI formats hold files from a long list of video essences such as JPEG 2000; DPX (a wrapper/essence format in one); various MPEG-2 and MPEG-4 master profiles; and DCI or DCP for digital cinema, to name just a few. To complicate matters, the video file and wrapper formats arent interchangeable.

On top of the seemingly endless combination of file and wrapper formats, there are nearly as many pieces of third-party equipment on which to play them. Users might need to stream the masters to a host of hardware, including HDMI and SDI monitors, projectors, and multiviewers; editing software; encoders and capture cards; color correctors, restoration workstations, and standards converters; videotape and disk recorders; broadcast playout servers; test equipment such as vectorscopes and waveform monitors; and SDI routers. Theres no guarantee a particular file will play on a particular piece of equipment.

Based on the sheer number of formats and all the possible places to play them, its safe to say that no single format will rise above the rest to become an industry standard anytime soon.

So complexity will continue to present challenges. How do we resolve them?

Master files are the ideal format for converting to other formats, so the industry must adapt solutions that can accommodate those formats. There are no universal players or management tools to support such large and cumbersome files, so most facilities simply store the masters and use other lossy formats when editing or converting files for delivery. Whats needed is an affordable, universal way to work with, interchange, and deliver copies of master files directly from their originals while eliminating the need to store multiple formats. (And by affordable, I mean able to be purchased online using a credit card, with prices and buying experiences like on Amazon.com.) The solution must work for video engineers, archivists, and manufacturers, and allow all of them to see the same things on their standard HDMI HDTV screens without proprietary software or expensive hardware.
The ideal solution would support multiple vendors formats and mastering formats in the most popular wrappers from all major manufacturers, thus eliminating the need to work with mezzanine and proxy copies. Instead, users could see their masters with internal technical metadata on their choice of third-party equipment. This revolutionary capability ultimately would make content available more quickly at higher quality, and would make it unnecessary to create and store the content in other formats. As a bonus, it would eliminate distortions by playing files in their native frame rate as opposed to the frame rate in the desktop setting.

Rather than move the people to where the master files can be viewed, or create and store mezzanine and proxy files, or supply everyone with proprietary workstations just for viewing, facilities could simply install an affordable, vendor-neutral product so that users could view master files wherever they work with confidence that theyre seeing the pristine quality thats intended.

Archimedia Technology is new to us. Who are you?

Simply put, we are a new company with ground-breaking technology and applications that bridge the gap between content producers and their archives without loss of quality. My partners and I co-founded Archimedia and created our flagship product, Archimedia Reference Player, to ensure video engineers and archivists can access, play, manage, and test files in mastering formats including JPEG 2000 and DPX. We are explicitly committed to supporting all current and future master formats regardless of origin.

At NAB this year, we saw 4K cameras everywhere, but no way to deliver and read the files. There are no tools to satisfy both the people who create the master files and the people who archive them, so we decided to solve that problem. Weve all spent many years developing video archiving, digitization, and test technologies for some of the worlds leading archive companies, broadcasters, and digital cinema studios. Now weve channeled that expertise into technology that gives users innovative tools theyve never had before but have long needed streamlining an incredibly complex process.


Tags: iss080 | Archimedia Technology | mark gray | usable video | archiving | process | guide | ask the experts | Mark Gray
Contributing Author Mark Gray

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Dalet at NAB 2014

    Dalet at NAB 2014

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Nugen Audio at IBC 2016

    Nugen Audio at IBC 2016

  • Elemental Technologies at IBC 2015

    Elemental Technologies at IBC 2015

  • Cobalt Digital at NAB 2014

    Cobalt Digital at NAB 2014

  • Elemental Technologies Software-Defined Video at NAB 2014

    Elemental Technologies Software-Defined Video at NAB 2014

  • Elemental Technologies HEVC solutions at NAB 2014

    Elemental Technologies HEVC solutions at NAB 2014

  • Elemental Technologies Multiscreen Solutions at NAB 2014

    Elemental Technologies Multiscreen Solutions at NAB 2014

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Blue Lucy Media at IBC 2013

    Blue Lucy Media at IBC 2013

  • TC Electronic at IBC2011

    TC Electronic at IBC2011

  • Guided Tour of the Polecam Stand at BVE 2016

    Guided Tour of the Polecam Stand at BVE 2016

  • Guided tour of the Global Distribution Stand at BVE 2016

    Guided tour of the Global Distribution Stand at BVE 2016

  • Guided tour of the Holdan Stand at BVE 2016

    Guided tour of the Holdan Stand at BVE 2016

  • PAG talk about flying with batteries at IBC 2014

    PAG talk about flying with batteries at IBC 2014

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Triveni Digital at NAB 2012

    Triveni Digital at NAB 2012


Related Shows
  • Talking Streaming with Stream UK: BVE Day 3

    Talking Streaming with Stream UK: BVE Day 3


Articles
Ruth Matos Interview - A Career Unfolds
Ruth Matos We first met Ruth when we employed her as a student to help with our live studio at BVE in 2013. We were struck by her determination, passion and ‘I’ll do anything to help’ attitude. We became friends of Facebook and have since seen her career in the industry develop and unfold. We thought it was about time to catch up and share her inspiring story of where she is now, how she got there and the challenges she has faced along the way.
Tags: iss133 | interview | filmming | career | education | Ruth Matos
Contributing Author Ruth Matos Click to read or download PDF
Perimeter LED screens management
Nicolas Houel Opened in January 2016, Parc Olympique Lyonnais, also known as Groupama Stadium, is the new home of Olympique Lyonnais football club, one of the most popular clubs in France. Since its inauguration, the stadium was a host of UEFA Euro 2016, and was also chosen to stage, among other important events, the 2018 UEFA Europa League Final and football at the 2024 Summer Olympics.
Tags: iss133 | 3dstorm | graphics | groupama stadium | liveexpert | livecg | deltacast | Nicolas Houel
Contributing Author Nicolas Houel Click to read or download PDF
Peli Air 1507 Review
Phil Vinter Originally named after a bird that carries its precious cargo through the skies, it was, perhaps, only a matter of time before Peli released its Air range.
Tags: iss133 | peli | peli air 1507 | trekpak | Phil Vinter
Contributing Author Phil Vinter Click to read or download PDF
The Biggest Toy Shop in the World
Emma Morrison When Nigel Woodford started his career at BBC Wood Norton in 1962, television was black and white and BBC Two had not yet been launched. In 2018 Nigel will retire, and Richmond Film Services, the pro-audio equipment rental company set up by Nigel in 1973, can count numerous contributions to iconic moments in British cultural, sporting and film history over this time.
Tags: iss133 | richmond film services | audio rental | auction | liquidity | go-dove | Emma Morrison
Contributing Author Emma Morrison Click to read or download PDF
How todays technology platforms can make AMAZING seem mundane
Bruce Devlin - new Some things never change. It's September and it's time for another IBC. Maybe I'm getting old and maybe I have selective memory about the past, but recently there hasn't been the buzz of years gone by. Sure, as you wander around the vast halls there will be lots of enthusiastic sales people trying to give you the best demo possible, but something has changed and to survive, I believe that vendors and customers need to change too.
Tags: iss133 | ibc | class | vulture | imf user group | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF