From archival master to usable video with Mark Gray


Mark Gray TV-Bay Magazine
Read ezine online
by Mark Gray
Issue 80 - August 2013
With about 4 million new video channels expected to hit the market in the next five years, its going to be more important than ever to make high-quality content readily available. Many media archives and libraries; studios; broadcasters; production and postproduction facilities; and mastering, quality control, and digital cinema facilities have volumes of pristine master files in their vaults just waiting to be repurposed. The challenge will be making use of their archival masters efficiently. Industry veteran Mark Gray touches on some of the key points.

Some might not grasp the complexity and enormity of archiving; can you share some insight into the process?

Video files represent big data, and in most cases, a lot of time, money, creativity, and expertise go into creating them. These video files can be massive, which requires media organizations to make a significant investment in storing the original, pristine master files, copies of which might go to a postproduction house for manipulation, a studio for previewing, a broadcaster for transcoding, an engineer for test and measurement, an editor for repurposing, or any number of other destinations in the life of a video asset. In any case, archiving the original master files ensures they are preserved and available for later use. In order to ensure high-quality, artifact-free copies, the files are archived in uncompressed or other lossless mastering formats such as JPEG 2000 or DPX.
When it comes to viewing and delivering archival master files, what are some of the challenges the industry faces?

Master files are typically stored in archive formats that cannot be played on a standard player or screen, so those who need to work with the files must have expensive, proprietary players for different vendors and file types. In addition, they need professional HD-SDI monitors that can cost tens of thousands of dollars each. Many facilities cannot afford expensive systems with DCP projectors, HD-SDI cards and monitors, and software from each of the manufacturers represented in their media collection for everyone in a group, so many archivists and production teams have to work with lower-quality mezzanine and proxy copies of their masters. This scenario means that those facilities also need storage for those lower-quality files, plus the asset management tools to keep track of them with their masters a system that can cost as much as or more than the storage of the master files themselves.

These challenges are particularly difficult for archives, which often operate without IT support and under incredibly tight budgets that dont allow for expensive, proprietary equipment and software from multiple manufacturers. The result? Many archivists are unable to view the master files in their care. When retrieving a requested file from the archive for delivery or digitizing a tape into a master file for storage, the archivist has no way of knowing whether the copy is faithful to the original or what it will look like on the screen.
Why is the process of viewing and delivering master files so complex?

There are several video file formats depending on which vendors hardware (e.g., a camera) or software (e.g., a nonlinear editor) created the video. In turn, those files can be bundled together with codecs, metadata, and other file-based elements to create a package, which is contained inside yet another format, a wrapper, for archiving or transporting. Wrappers including the most common MXF, MOV, and AVI formats hold files from a long list of video essences such as JPEG 2000; DPX (a wrapper/essence format in one); various MPEG-2 and MPEG-4 master profiles; and DCI or DCP for digital cinema, to name just a few. To complicate matters, the video file and wrapper formats arent interchangeable.

On top of the seemingly endless combination of file and wrapper formats, there are nearly as many pieces of third-party equipment on which to play them. Users might need to stream the masters to a host of hardware, including HDMI and SDI monitors, projectors, and multiviewers; editing software; encoders and capture cards; color correctors, restoration workstations, and standards converters; videotape and disk recorders; broadcast playout servers; test equipment such as vectorscopes and waveform monitors; and SDI routers. Theres no guarantee a particular file will play on a particular piece of equipment.

Based on the sheer number of formats and all the possible places to play them, its safe to say that no single format will rise above the rest to become an industry standard anytime soon.

So complexity will continue to present challenges. How do we resolve them?

Master files are the ideal format for converting to other formats, so the industry must adapt solutions that can accommodate those formats. There are no universal players or management tools to support such large and cumbersome files, so most facilities simply store the masters and use other lossy formats when editing or converting files for delivery. Whats needed is an affordable, universal way to work with, interchange, and deliver copies of master files directly from their originals while eliminating the need to store multiple formats. (And by affordable, I mean able to be purchased online using a credit card, with prices and buying experiences like on Amazon.com.) The solution must work for video engineers, archivists, and manufacturers, and allow all of them to see the same things on their standard HDMI HDTV screens without proprietary software or expensive hardware.
The ideal solution would support multiple vendors formats and mastering formats in the most popular wrappers from all major manufacturers, thus eliminating the need to work with mezzanine and proxy copies. Instead, users could see their masters with internal technical metadata on their choice of third-party equipment. This revolutionary capability ultimately would make content available more quickly at higher quality, and would make it unnecessary to create and store the content in other formats. As a bonus, it would eliminate distortions by playing files in their native frame rate as opposed to the frame rate in the desktop setting.

Rather than move the people to where the master files can be viewed, or create and store mezzanine and proxy files, or supply everyone with proprietary workstations just for viewing, facilities could simply install an affordable, vendor-neutral product so that users could view master files wherever they work with confidence that theyre seeing the pristine quality thats intended.

Archimedia Technology is new to us. Who are you?

Simply put, we are a new company with ground-breaking technology and applications that bridge the gap between content producers and their archives without loss of quality. My partners and I co-founded Archimedia and created our flagship product, Archimedia Reference Player, to ensure video engineers and archivists can access, play, manage, and test files in mastering formats including JPEG 2000 and DPX. We are explicitly committed to supporting all current and future master formats regardless of origin.

At NAB this year, we saw 4K cameras everywhere, but no way to deliver and read the files. There are no tools to satisfy both the people who create the master files and the people who archive them, so we decided to solve that problem. Weve all spent many years developing video archiving, digitization, and test technologies for some of the worlds leading archive companies, broadcasters, and digital cinema studios. Now weve channeled that expertise into technology that gives users innovative tools theyve never had before but have long needed streamlining an incredibly complex process.


Tags: iss080 | Archimedia Technology | mark gray | usable video | archiving | process | guide | ask the experts | Mark Gray
Contributing Author Mark Gray

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Dalet at NAB 2014

    Dalet at NAB 2014

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Telestream Vantage Powerful, Scalable, Software-Enabled Media Processing Platform at BVE 2019

    Telestream Vantage Powerful, Scalable, Software-Enabled Media Processing Platform at BVE 2019

  • Nugen Audio at IBC 2016

    Nugen Audio at IBC 2016

  • Elemental Technologies at IBC 2015

    Elemental Technologies at IBC 2015

  • Cobalt Digital at NAB 2014

    Cobalt Digital at NAB 2014

  • Elemental Technologies Software-Defined Video at NAB 2014

    Elemental Technologies Software-Defined Video at NAB 2014

  • Elemental Technologies HEVC solutions at NAB 2014

    Elemental Technologies HEVC solutions at NAB 2014

  • Elemental Technologies Multiscreen Solutions at NAB 2014

    Elemental Technologies Multiscreen Solutions at NAB 2014

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Blue Lucy Media at IBC 2013

    Blue Lucy Media at IBC 2013

  • TC Electronic at IBC2011

    TC Electronic at IBC2011

  • Guided Tour of the Polecam Stand at BVE 2016

    Guided Tour of the Polecam Stand at BVE 2016

  • Guided tour of the Global Distribution Stand at BVE 2016

    Guided tour of the Global Distribution Stand at BVE 2016

  • Guided tour of the Holdan Stand at BVE 2016

    Guided tour of the Holdan Stand at BVE 2016

  • PAG talk about flying with batteries at IBC 2014

    PAG talk about flying with batteries at IBC 2014

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Triveni Digital at NAB 2012

    Triveni Digital at NAB 2012


Related Shows
  • Talking Streaming with Stream UK: BVE Day 3

    Talking Streaming with Stream UK: BVE Day 3


Articles
Painting Performance Analytics with ChyronHego
KitPlus By now, most people are familiar with the sport of mixed martial arts (MMA) and its leading organization – UFC (Ultimate Fighting Championship). And while the sport and its leading promotion are only 25 years old, a great deal has changed in those 25 years, including the training of UFC athletes.
Tags: iss136 | paint | telestrator | ufc | chyron | chyronhego | KitPlus
Contributing Author KitPlus Click to read or download PDF
Keeping Your Post Prodction on Track with Subclips and Search Bins
Alex Macleod

For my 2nd Kit Plus article I thought I’d try and build on the theme of my first, and that’s one of making sure things are organised at all levels of your post production projects.

Last time I talked about trying as best as you can to stick to the ‘two week rule’, making sure that the names & locations of every asset you import, and every bin & sequence that you create in your project - will make sense to you regardless of how long it is you spend away from it.

Tags: iss136 | mediacity training | subclip | premiere pro | gvs | bve | bve2019 | Alex Macleod
Contributing Author Alex Macleod Click to read or download PDF
NAB Intelligence
Bruce Devlin - new It's that time of year again. 06:30 on road bikes on the Las Vegas Strip with a merry bunch of folks who believe they look good in lycra. As we pedal West up the hill towards Red Rock there will be stories of the year just gone and questions about what to look for at the show. I, for one, will be attending the Devoncroft event to check up on who is spending what and where. I will then look forward to the topics that might be buzzing around the halls. I predict there will be many displays of Machine Learning hiding under the banner of Artificial Intelligence. Some of these will show better picture quality, others will show personalisation engines, some will be improved search tools and yet more will be synthesising pictures and motions to replace reality with and Artificial reality that's more compelling for the viewer.
Tags: iss136 | nab | class | ai | smpte | st2110 | imf | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF
Rotolight Anova Pro 2 User Review
Andy McKenzie The Anova PRO 2 is the fourth generation of Rotolight’s studio/location light, offering 70% more power output than its predecessor. It is claimed be one of the brightest LED lights ever launched in its class, delivering 10,700 lux at 3 feet yet consuming only 72 watts. Figure 1 shows the front with accessory mounting spigots (1), optional barn doors (2) and a gel frame holder.
Tags: iss136 | rotolight | anova pro 2 | led | lighting | flash light | dmx control | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF
State of the Nation - Drama
Dick Hobbs - new If you can remember as far back as the 1980s, then you probably remember a Saturday night drama called Hill Street Blues, written and developed by Steven Bochco. Made for NBC in the States, it was widely shown around the world, including in the UK.
Tags: iss136 | drama holby city | night manager | netflix | bbc | ai | streaming | podcast | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF