Introducing Front Porch Digitals Archive eXchange Format


Ever since the Tower of Babel fell, human beings have sought a common way to communicate with one another and with future generations. Technological innovations — writing, for example — did a lot to preserve communication but at the same time posed new problems. Today the diversity of discrete formats for processing and storing digital media makes communication among different production and storage systems complicated, expensive, and sometimes downright impossible. The goal of establishing a common language among all the world’s people has proved elusive, but the goal of establishing a common format for digital media doesn’t have to be. In fact, over the past several years, we at Front Porch Digital have developed such a specification in the Archive eXchange Format, AXF. What’s required now is general agreement that this “common language” is a good idea.
AXF Defined
At the most basic level, AXF is a file container that can encapsulate any number and any type of files in a fully self-describing package. AXF’s defining attribute is its innovative embedded file system approach whereby the encapsulated package actually contains its own file system abstracting the underlying operating system, storage technology, and file system. This approach allows AXF to be both content and storage agnostic. In other words, because the AXF object itself contains the file system, it can exist on any generation of data tape, spinning disk, flash, optical media, or other storage technology that exists today or may exist tomorrow.
The AXF specification has been architected so that each AXF Object originates with an AXF Object Header a structure containing descriptive XML metadata such as the AXF Object’s unique identifier (UMID/GUID), creation date, object provenance, and file-tree information including file permissions. Following the AXF Object Header are any number of optional AXF Generic Metadata packages, which are self-contained, open metadata containers in which applications can include AXF Object-specific metadata. This metadata can be structured or unstructured, open or vendor-specific, binary, or XML.
The next part of the AXF Object construct is any number of File Data + File Padding + File Footer triplets or the AXF File Payload, the actual byte data of the files encapsulated in the object. File padding, which ensures alignment of all AXF Object elements on storage medium block boundaries, is key to the AXF specification. The File Footer structure contains the exact size of the preceding file, along with an optional file level checksum designed to be processed on-the-fly by the application during restore operations.
The final portion of an AXF Object is the AXF Object Footer, which repeats the information contained in the AXF Object Header and adds information captured during the AXF Object’s creation, including per-file checksums and structure block positions. The AXF Object Footer is important to the resiliency of the AXF specification because it allows efficient re-indexing by foreign systems when the media content is not previously known. This adds self-describing characteristics to the AXF medium itself, ensuring full recoverability in the case of application data loss or movement to a foreign system, regardless of the storage technology selected.
Supporting High-Volume Storage
For us at Front Porch Digital, a prime motivator for developing an advanced open file specification like AXF was to meet the needs of users attempting to leverage linear data tape storage in demanding environments. To meet this need, AXF is not only tuned for performance, it can scale from simple IT-centric disk-based, single file applications through to the most demanding media operations, those with petabytes of content stored in long-term archives on data tape. In all cases, AXF offers an abstraction layer that hides the complexities of the storage technology from the higher-level applications, while it also offers fundamental encapsulation, provenance, fixity, portability, and preservation characteristics. And the same self-describing AXF format can be used on disk storage as well.
Because AXF is media and storage agnostic, it can support the modern generation of data tape formats such as LTO5 and T10000C, which can exist with or without a file system — namely the Linear Tape File System, or LTFS. AXF brings to LTFS all of the benefits described previously, which are not part of the LTFS design itself. AXF and LTFS are certainly not mutually exclusive but users should understand the limitations and transport-centric nature of LTFS prior to selecting it as their archive or preservation format. In fact, AXF provides all of the features offered by LTFS with many additional benefits. Further, because AXF does not rely on features of modern data tape technologies, users with existing infrastructures do not have to upgrade, saving significant cost.
Advantages of an Embedded File System
The AXF specification’s embedded file system enables AXF to translate between any generic set of files and logical block positions on any storage medium, whether or not the medium has its own file system. This gives AXF an advantage over formats that rely on storage technology elements — file marks on data tape, for example — because that reliance hinders both storage capabilities as well as performance. Likewise, these alternative formats are ineffective for complex file collections with millions of related elements and for extremely large individual files, which may span more than one storage medium — some of the limitations of LTFS.
A further benefit of AXF over legacy container formats, as well as some modern operating and file systems, is its ability to scale without limits, encapsulating any number of related component files of any size. AXF thus supports large-scale archive and preservation systems as well as simple, standalone applications. The benefit to users is easier encapsulation or wrapping of content, longer term protection, and the ability to transport content between systems from different vendors without regard for the underlying storage technology, file system, or operating system.
Conclusion
According to the Biblical account of the Tower of Babel, a people with one language are capable of accomplishing anything they can imagine. While an open standard for storage and preservation of media assets may not be quite the panacea that implies, its ability to support interoperability among systems helps ensure long-term accessibility to valued assets. And to keep up with evolving storage technologies offers profound present and future benefits for any enterprise that uses media — from heritage institutions, to schools, to broadcasters. Front Porch Digital has incorporated AXF into the latest release of its DIVArchive Content Storage Management (CSM) system, which was launched at the 2011 NAB Show, and today the company is committed to working closely with standards bodies such as SMPTE for ratification and standardization of AXF.
More information on AXF and SMTPE standards body activities is available at OpenAXF.org and smpte.org.

Tags: fpd | front porch digital | axf | lto5 | t10000c | iss057 | N/A
Contributing Author N/A

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Front Porch Digital at IBC 2014

    Front Porch Digital at IBC 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Front Porch Digital at IBC2011

    Front Porch Digital at IBC2011


Related Shows
  • Day 1 of BroadcastShow at IBC

    Day 1 of BroadcastShow at IBC


Articles
Using Wireless Transmission
Jeremy Benning Wireless acquisition is a staple of live sports, entertainment and reality shows where cable free capture permits shots not previously possible, for health and safety reasons, and gives the camera-operator greater artistic licence to roam. The same is increasingly true of narrative drama where cinematographers are keen to work handheld or Steadicam where that helps tell the story. Any equipment which frees their movement and time by being lighter, easier to use and reliable in performance is going to tick a lot of boxes.
Tags: iss134 | wireless | 4k | transmission | Jeremy Benning
Contributing Author Jeremy Benning Click to read or download PDF
Giving Welsh sport a global audience
Adam Amor From the Ospreys Rugby Union team, to the Football Association of Wales, as well as national cycling, swimming and boxing coverage, Port Talbot based Buffoon Film and Media has been heavily involved in putting Welsh sports on the world stage.
Tags: iss134 | blackmagic | atem | buffoon | micro studio camera | Adam Amor
Contributing Author Adam Amor Click to read or download PDF
Shedding Light on the Blackmagic Pocket Cinema Camera 4k BMCPP4K
Garth de Bruno Austin “What is it about light that has us craving it?” Is the question asked in the opening seconds of Garth de Bruno Austin’s latest short, The Colour of Light. Exploring this natural, human need as well as our innate desire to control it, Garth’s film showcases everyday people going about their lives in differing degrees of luminance, whether that be an artificial streetlight or a natural morning sunrise.
Tags: iss134 | blackmagic | cinema camera | 4k | cpp4k | Garth de Bruno Austin
Contributing Author Garth de Bruno Austin Click to read or download PDF
The brave new world of software based production
Boromy Ung In today’s rapidly evolving broadcast industry, the only constant media organizations can truly count on is change — and the need to adapt as rapidly and cost-effectively as possible. One of the biggest agents of change is the IP revolution, driving broadcasters to migrate their operations to all-software solutions running on commodity, IT-based technologies.
Tags: iss134 | chyronhego | graphics | sports | ott | Boromy Ung
Contributing Author Boromy Ung Click to read or download PDF
Protecting the continuity of transmission
Lorna Garrett Your viewers love you. You consistently bring them their preferred channels 24/7. They’ve come to rely on you for their viewing pleasure. They never miss cheering on their beloved sports teams. They’re the envy of their friends as they watch live concerts of their favourite bands. They gather the family around and catch up on their must-see shows. They don’t have a bad word to say about you.
Tags: iss134 | garland | gpl | streaming | artel | disaster recovery | Lorna Garrett
Contributing Author Lorna Garrett Click to read or download PDF