Introducing Front Porch Digitals Archive eXchange Format


Ever since the Tower of Babel fell, human beings have sought a common way to communicate with one another and with future generations. Technological innovations — writing, for example — did a lot to preserve communication but at the same time posed new problems. Today the diversity of discrete formats for processing and storing digital media makes communication among different production and storage systems complicated, expensive, and sometimes downright impossible. The goal of establishing a common language among all the world’s people has proved elusive, but the goal of establishing a common format for digital media doesn’t have to be. In fact, over the past several years, we at Front Porch Digital have developed such a specification in the Archive eXchange Format, AXF. What’s required now is general agreement that this “common language” is a good idea.
AXF Defined
At the most basic level, AXF is a file container that can encapsulate any number and any type of files in a fully self-describing package. AXF’s defining attribute is its innovative embedded file system approach whereby the encapsulated package actually contains its own file system abstracting the underlying operating system, storage technology, and file system. This approach allows AXF to be both content and storage agnostic. In other words, because the AXF object itself contains the file system, it can exist on any generation of data tape, spinning disk, flash, optical media, or other storage technology that exists today or may exist tomorrow.
The AXF specification has been architected so that each AXF Object originates with an AXF Object Header a structure containing descriptive XML metadata such as the AXF Object’s unique identifier (UMID/GUID), creation date, object provenance, and file-tree information including file permissions. Following the AXF Object Header are any number of optional AXF Generic Metadata packages, which are self-contained, open metadata containers in which applications can include AXF Object-specific metadata. This metadata can be structured or unstructured, open or vendor-specific, binary, or XML.
The next part of the AXF Object construct is any number of File Data + File Padding + File Footer triplets or the AXF File Payload, the actual byte data of the files encapsulated in the object. File padding, which ensures alignment of all AXF Object elements on storage medium block boundaries, is key to the AXF specification. The File Footer structure contains the exact size of the preceding file, along with an optional file level checksum designed to be processed on-the-fly by the application during restore operations.
The final portion of an AXF Object is the AXF Object Footer, which repeats the information contained in the AXF Object Header and adds information captured during the AXF Object’s creation, including per-file checksums and structure block positions. The AXF Object Footer is important to the resiliency of the AXF specification because it allows efficient re-indexing by foreign systems when the media content is not previously known. This adds self-describing characteristics to the AXF medium itself, ensuring full recoverability in the case of application data loss or movement to a foreign system, regardless of the storage technology selected.
Supporting High-Volume Storage
For us at Front Porch Digital, a prime motivator for developing an advanced open file specification like AXF was to meet the needs of users attempting to leverage linear data tape storage in demanding environments. To meet this need, AXF is not only tuned for performance, it can scale from simple IT-centric disk-based, single file applications through to the most demanding media operations, those with petabytes of content stored in long-term archives on data tape. In all cases, AXF offers an abstraction layer that hides the complexities of the storage technology from the higher-level applications, while it also offers fundamental encapsulation, provenance, fixity, portability, and preservation characteristics. And the same self-describing AXF format can be used on disk storage as well.
Because AXF is media and storage agnostic, it can support the modern generation of data tape formats such as LTO5 and T10000C, which can exist with or without a file system — namely the Linear Tape File System, or LTFS. AXF brings to LTFS all of the benefits described previously, which are not part of the LTFS design itself. AXF and LTFS are certainly not mutually exclusive but users should understand the limitations and transport-centric nature of LTFS prior to selecting it as their archive or preservation format. In fact, AXF provides all of the features offered by LTFS with many additional benefits. Further, because AXF does not rely on features of modern data tape technologies, users with existing infrastructures do not have to upgrade, saving significant cost.
Advantages of an Embedded File System
The AXF specification’s embedded file system enables AXF to translate between any generic set of files and logical block positions on any storage medium, whether or not the medium has its own file system. This gives AXF an advantage over formats that rely on storage technology elements — file marks on data tape, for example — because that reliance hinders both storage capabilities as well as performance. Likewise, these alternative formats are ineffective for complex file collections with millions of related elements and for extremely large individual files, which may span more than one storage medium — some of the limitations of LTFS.
A further benefit of AXF over legacy container formats, as well as some modern operating and file systems, is its ability to scale without limits, encapsulating any number of related component files of any size. AXF thus supports large-scale archive and preservation systems as well as simple, standalone applications. The benefit to users is easier encapsulation or wrapping of content, longer term protection, and the ability to transport content between systems from different vendors without regard for the underlying storage technology, file system, or operating system.
Conclusion
According to the Biblical account of the Tower of Babel, a people with one language are capable of accomplishing anything they can imagine. While an open standard for storage and preservation of media assets may not be quite the panacea that implies, its ability to support interoperability among systems helps ensure long-term accessibility to valued assets. And to keep up with evolving storage technologies offers profound present and future benefits for any enterprise that uses media — from heritage institutions, to schools, to broadcasters. Front Porch Digital has incorporated AXF into the latest release of its DIVArchive Content Storage Management (CSM) system, which was launched at the 2011 NAB Show, and today the company is committed to working closely with standards bodies such as SMPTE for ratification and standardization of AXF.
More information on AXF and SMTPE standards body activities is available at OpenAXF.org and smpte.org.

Tags: fpd | front porch digital | axf | lto5 | t10000c | iss057 | N/A
Contributing Author N/A

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Front Porch Digital at IBC 2014

    Front Porch Digital at IBC 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Front Porch Digital at IBC2011

    Front Porch Digital at IBC2011


Related Shows
  • Day 1 of BroadcastShow at IBC

    Day 1 of BroadcastShow at IBC


Articles
The way forward in a changing world
Alan Wheable The broadcast and media industries are evolving in all areas to adapt to the challenges of competition from OTT, changing user viewing habits, technological advances in image resolution, dynamic range and colour rendition as well as embracing new video over IP infrastructures.
Tags: iss136 | omnitek | aims | ip | smpte | 2110 | 4k qc | ultra xr | nmos | ultra tq | Alan Wheable
Contributing Author Alan Wheable Click to read or download PDF
BSC Expo 2019 Report
Paul MacKenzie BSC Expo returned to the Battersea Evolution on Friday February 1st and Saturday 2nd. It is a busy and friendly event though this year in need of some temperature control: exhibitors around the entrance area were uncomfortably aware of the wintry conditions outside and the main hall was in need of cooling.
Tags: iss136 | bsc | cinematography | canon | c700 | c200 | cartoni | holdan | blackmagic | peli | panasonic | sennheiser | sony | fs5 | teradek | viten | flowtech | Paul MacKenzie
Contributing Author Paul MacKenzie Click to read or download PDF
Flexible Connectivity Hits Europe
Yvonne Monterroso The low-latency Dejero CellSat network was launched in Europe in January of 2019, which is excellent news for European broadcasters who can now benefit from the same flexibility CellSat provides North American Dejero customers, such as World Racing Group (WRG).
Tags: iss136 | dejero | wrg | cellsat | ku sat | is-35e | cellular | Yvonne Monterroso
Contributing Author Yvonne Monterroso Click to read or download PDF
Keeping Your Post Prodction on Track with Subclips and Search Bins
Alex Macleod

For my 2nd Kit Plus article I thought I’d try and build on the theme of my first, and that’s one of making sure things are organised at all levels of your post production projects.

Last time I talked about trying as best as you can to stick to the ‘two week rule’, making sure that the names & locations of every asset you import, and every bin & sequence that you create in your project - will make sense to you regardless of how long it is you spend away from it.

Tags: iss136 | mediacity training | subclip | premiere pro | gvs | bve | bve2019 | Alex Macleod
Contributing Author Alex Macleod Click to read or download PDF
Sennheiser Memory Mic User Review
Dr Anthony Willman The Sennheiser Memory mic, adding to the new generation of semi-professional equipment that is helping the need for high quality audio in parallel markets that until now, did not have the budgets to achieve ‘great’ results.
Tags: iss136 | sennheiser | memory mic | doctor | audio recorder | Dr Anthony Willman
Contributing Author Dr Anthony Willman Click to read or download PDF