Introducing Front Porch Digitals Archive eXchange Format


Ever since the Tower of Babel fell, human beings have sought a common way to communicate with one another and with future generations. Technological innovations — writing, for example — did a lot to preserve communication but at the same time posed new problems. Today the diversity of discrete formats for processing and storing digital media makes communication among different production and storage systems complicated, expensive, and sometimes downright impossible. The goal of establishing a common language among all the world’s people has proved elusive, but the goal of establishing a common format for digital media doesn’t have to be. In fact, over the past several years, we at Front Porch Digital have developed such a specification in the Archive eXchange Format, AXF. What’s required now is general agreement that this “common language” is a good idea.
AXF Defined
At the most basic level, AXF is a file container that can encapsulate any number and any type of files in a fully self-describing package. AXF’s defining attribute is its innovative embedded file system approach whereby the encapsulated package actually contains its own file system abstracting the underlying operating system, storage technology, and file system. This approach allows AXF to be both content and storage agnostic. In other words, because the AXF object itself contains the file system, it can exist on any generation of data tape, spinning disk, flash, optical media, or other storage technology that exists today or may exist tomorrow.
The AXF specification has been architected so that each AXF Object originates with an AXF Object Header a structure containing descriptive XML metadata such as the AXF Object’s unique identifier (UMID/GUID), creation date, object provenance, and file-tree information including file permissions. Following the AXF Object Header are any number of optional AXF Generic Metadata packages, which are self-contained, open metadata containers in which applications can include AXF Object-specific metadata. This metadata can be structured or unstructured, open or vendor-specific, binary, or XML.
The next part of the AXF Object construct is any number of File Data + File Padding + File Footer triplets or the AXF File Payload, the actual byte data of the files encapsulated in the object. File padding, which ensures alignment of all AXF Object elements on storage medium block boundaries, is key to the AXF specification. The File Footer structure contains the exact size of the preceding file, along with an optional file level checksum designed to be processed on-the-fly by the application during restore operations.
The final portion of an AXF Object is the AXF Object Footer, which repeats the information contained in the AXF Object Header and adds information captured during the AXF Object’s creation, including per-file checksums and structure block positions. The AXF Object Footer is important to the resiliency of the AXF specification because it allows efficient re-indexing by foreign systems when the media content is not previously known. This adds self-describing characteristics to the AXF medium itself, ensuring full recoverability in the case of application data loss or movement to a foreign system, regardless of the storage technology selected.
Supporting High-Volume Storage
For us at Front Porch Digital, a prime motivator for developing an advanced open file specification like AXF was to meet the needs of users attempting to leverage linear data tape storage in demanding environments. To meet this need, AXF is not only tuned for performance, it can scale from simple IT-centric disk-based, single file applications through to the most demanding media operations, those with petabytes of content stored in long-term archives on data tape. In all cases, AXF offers an abstraction layer that hides the complexities of the storage technology from the higher-level applications, while it also offers fundamental encapsulation, provenance, fixity, portability, and preservation characteristics. And the same self-describing AXF format can be used on disk storage as well.
Because AXF is media and storage agnostic, it can support the modern generation of data tape formats such as LTO5 and T10000C, which can exist with or without a file system — namely the Linear Tape File System, or LTFS. AXF brings to LTFS all of the benefits described previously, which are not part of the LTFS design itself. AXF and LTFS are certainly not mutually exclusive but users should understand the limitations and transport-centric nature of LTFS prior to selecting it as their archive or preservation format. In fact, AXF provides all of the features offered by LTFS with many additional benefits. Further, because AXF does not rely on features of modern data tape technologies, users with existing infrastructures do not have to upgrade, saving significant cost.
Advantages of an Embedded File System
The AXF specification’s embedded file system enables AXF to translate between any generic set of files and logical block positions on any storage medium, whether or not the medium has its own file system. This gives AXF an advantage over formats that rely on storage technology elements — file marks on data tape, for example — because that reliance hinders both storage capabilities as well as performance. Likewise, these alternative formats are ineffective for complex file collections with millions of related elements and for extremely large individual files, which may span more than one storage medium — some of the limitations of LTFS.
A further benefit of AXF over legacy container formats, as well as some modern operating and file systems, is its ability to scale without limits, encapsulating any number of related component files of any size. AXF thus supports large-scale archive and preservation systems as well as simple, standalone applications. The benefit to users is easier encapsulation or wrapping of content, longer term protection, and the ability to transport content between systems from different vendors without regard for the underlying storage technology, file system, or operating system.
Conclusion
According to the Biblical account of the Tower of Babel, a people with one language are capable of accomplishing anything they can imagine. While an open standard for storage and preservation of media assets may not be quite the panacea that implies, its ability to support interoperability among systems helps ensure long-term accessibility to valued assets. And to keep up with evolving storage technologies offers profound present and future benefits for any enterprise that uses media — from heritage institutions, to schools, to broadcasters. Front Porch Digital has incorporated AXF into the latest release of its DIVArchive Content Storage Management (CSM) system, which was launched at the 2011 NAB Show, and today the company is committed to working closely with standards bodies such as SMPTE for ratification and standardization of AXF.
More information on AXF and SMTPE standards body activities is available at OpenAXF.org and smpte.org.

Tags: fpd | front porch digital | axf | lto5 | t10000c | iss057 | N/A
Contributing Author N/A

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Front Porch Digital at IBC 2014

    Front Porch Digital at IBC 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Front Porch Digital at IBC2011

    Front Porch Digital at IBC2011


Related Shows
  • Day 1 of BroadcastShow at IBC

    Day 1 of BroadcastShow at IBC


Articles
Industry Ready Broadcasting Students
KitPlus Cardiff Metropolitan University traces its roots back to 1865, when they opened their first school in an old library. Now Cardiff Met is a global, practice-focused and professionally oriented institution, committed to ensure every student fulfills their potential to make outstanding contributions in their future industries.
Tags: iss138 | newtek | live production | cardiff metropolitain | sports broadcasting | tricaster | cardiff devils | KitPlus
Contributing Author KitPlus Click to read or download PDF
How LiveU Has Changed The Way We Work At ITV Daytime
Tim Guilder As my job title suggests, I’m responsible for planning and executing technology projects across ITV’s Daytime, including outside broadcasts, which are central to what we do. We broadcast live for more than six hours every weekday with four shows: Good Morning Britain, This Morning, Lorraine and Loose Women. With a history as one of the main UK terrestrial broadcasters – of course, we have also evolved alongside the industry in terms of viewing options – we have high standards. We constantly generate new ideas to keep audiences engaged and we also need to be responsive to the latest breaking news.
Tags: iss138 | liveu | itv | daytime | cobham | explorer 8100 | satellite | lu600 hevc | ka-band | lu300 | garland | Tim Guilder
Contributing Author Tim Guilder Click to read or download PDF
How the Womens World Cup was Boosted by Broadcast
John Griffiths As we write this article, England’s Lionesses have just beaten Norway 3-0 and are smashing viewing figure records, with 6.9 million viewers tuning in to watch them play Cameroon on BBC One. Rewind to the previous World Cup in 2015, and England’s group and early stage knockout games tempted up to 2.5 million viewers for each match. It’s safe to say that women’s football is finally having its moment; what was perhaps once considered a niche sport is finally gaining momentum in the mainstream space with broadcasters giving it the attention it deserves.
Tags: iss138 | wwc | world cup | spicymango | diversity | John Griffiths
Contributing Author John Griffiths Click to read or download PDF
Making the Workflow Flow
Bruce Devlin - new The toughest things about being the Standards Vice President (SVP) is that everyone expects standards to be the most important thing. In all the systems that I’ve designed and deployed over the years, I've yet to find any production workflow that is 100% standards based. True, the core technologies may well be standards based, but the overall workflow is made up of many technology pieces from open source code, through de-facto delivery specifications based upon SMPTE or Trade Association Specifications that in turn depend on full, International Standards to work. I can already hear some folks saying "In the good old days, everything used standards", but I beg to disagree.
Tags: iss138 | pye museum | pye-philips | smpte | ietf | ieee | w3c | aes | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF
State of the Nation: Wings, Fine Coffee and Fake Nudes
Dick Hobbs - new One of the many attractions of this time of year is that I get to meet with my fellow judges for the IBC Innovation Awards, and see what the industry sees as the most important, most forward-looking projects of the day. For me, this says what the key talking points are going to be in September, far more eloquently than the endless press releases from vendors which are already tumbling into my inbox.
Tags: iss138 | ibc | smpte 2110 | deepnude | cisco | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF