As we’ve already demonstrated on this blog, the Digital Preservation Unit loves file formats! But we’re not the only ones – we’re lucky to be joined by colleagues across NARA who share our format fixation and produce public resources on file formats. For Valentine’s Day, we wanted to share the love and talk a little bit more about these resources.
In previous posts, we’ve discussed the NARA Digital Preservation Framework and how we use it to document our risk analyses and preservation action plans for formats in NARA’s holdings. This post will focus on two other public resources at NARA that include information about file formats: the NARA Transfer Guidance and the Internal Products and Services webpages.
We realize that there are colleagues in cultural heritage organizations and interested members of the general public who may share our love and reference these resources as they are performing research or putting together their own file format guidance. Our hope is that by laying out the landscape of file format resources at NARA, it will be easier for this audience to navigate through these resources, understand how they relate to each other, and find the one best suited for their needs.
NARA Transfer Guidance
The “Transfer Guidance” is the short name for NARA Bulletin 2014-04: Format Guidance for the Transfer of Permanent Electronic Records; NARA Bulletin 2015-04: Metadata Guidance for the Transfer of Permanent Electronic Records; and NARA Bulletin 2025-01: Metadata Guidance for the Transfer of Classified Electronic Records. A NARA Bulletin provides formal records management guidance for Federal agencies. As such, the Transfer Guidance only applies to permanent records created and maintained by Federal agencies; it does not apply to Presidential or Legislative records or NARA’s internal digitization for access purposes.
According to John Martinez, Supervisory Electronic Records Management Policy Analyst, “NARA does not – and can not – mandate which file formats and metadata schemas are used by Federal agencies in their day-to-day business; however, in the Transfer Guidance, we are able to specify which file formats and elements NARA identifies as acceptable when transferring the subset of federal electronic records identified as permanently valuable into NARA’s holdings.”
Therefore, in terms of the OAIS Reference Model, the Transfer Guidance can be seen as Submission Information Package (SIP) guidance for Federal agencies. This stands in contrast to the Digital Preservation Framework, which conveys Archival Information Package (AIP) recommendations through the preservation action plans that specify target formats and tools for format conversions for the long-term preservation of electronic records in NARA’s holdings.
NARA Bulletin 2014-04 Appendix A contains file format tables, organized by record category (Email, Geospatial Formats, etc.). For each category, there may be formats listed as Preferred, Acceptable, or Acceptable for Imminent Transfer.
It’s important to note that Preferred or Acceptable designations per NARA’s Transfer Guidance do not mean preferred or acceptable for long-term preservation. Formats that are appropriate for transfer are not necessarily appropriate for preservation.
A good example is the Microsoft Personal Folders Format (PST): the Transfer Guidance lists PST as a Preferred Format for Aggregations of Email, because it uses a hierarchical structure and properties to maintain contextual information about message objects. This does not mean, however, that NARA deems PST a good format for long-term preservation. It is not an open email standard, and it requires special tools to access its contents. But if given the choice between receiving a full inbox export in PST format or a series of decontextualized, unordered individual EML files, NARA would greatly prefer the PST.
The Transfer Guidance also does not recommend file format migrations or conversions, and the file formats listed in the Transfer Guidance are not necessarily target formats for preservation migrations. Indeed, NARA prefers that Federal agencies that maintain records in formats not listed in the Transfer Guidance contact the appropriate NARA office before migrating, converting, or altering records to meet our guidance. This means that, in practice, the Transfer Guidance is often the starting point for an individualized conversation about what NARA would prefer to receive, balanced with what a given agency is able to transfer.
The Transfer Guidance is maintained and published by the Office of the Chief Records Officer for the U.S. Government (CRO). The Digital Preservation Unit often collaborates with CRO to research and provide recommendations for additions and updates to the Transfer Guidance; however, we still consider the Digital Preservation Framework to be our primary resource for documenting and disseminating information about risks and preservation planning for file formats.
You can read more about the excellent work done by our colleagues in CRO on their blog, Records Express.
NARA Internal Products and Services
The Internal Products and Services webpages provide information about the reformatting approaches used by NARA’s Moving Image and Sound Preservation Labs for audio, video, and motion picture records. The Labs create copies of these records in response to reference requests and for internal preservation projects – in other words, these specifications cover both Dissemination Information Packages (DIPs) meant for public access and SIPs that are transferred to the NARA custodian after preservation reformatting.
Similar to the Digital Preservation Framework, the Labs make information available through Internal Products and Services to promote transparency and provide more context about the work that NARA does. Neither Internal Products and Services nor the Digital Preservation Framework are meant to provide guidance or prescribe specific processes outside of NARA. This stands in contrast to the Transfer Guidance, which is meant to provide specific guidance for Federal agencies transferring electronic records to NARA.
Whereas both the Transfer Guidance and the Digital Preservation Framework are organized by broad record categories of electronic records, the reformatting approaches on the Internal Products and Services webpages are organized by the original media type of physical archival records that will be digitized for preservation and access purposes:
- Digital Output from Audio Source
- Digital Output from Video Source
- Digital Output from Motion Picture Film Source
- Film Output from Motion Picture Film Source
Each of these pages describe various product types (Preservation Master, Reproduction Master, or Distribution Copies), customer needs, and desired outcomes. From there, you can get to information about each product, including the preferred container format and file specifications (such as the codec, sampling frequency, and bit depth). In this way, although the scope of record categories is smaller than the Transfer Guidance and the Digital Preservation Framework, the Internal Products and Services webpages provide deeper and more specific information about each format and the context of its creation at NARA.
Martin Jacobson, Director of the Moving Image and Sound Preservation Labs, states, “The Internal Products and Services pages provide choices and clarity about the creation of audio, video and motion picture files and film output. Preservation, reproduction and access formats are covered. Dynamic media records are among those with the highest preservation demands and degrees of difficulty. The pages are actively utilized by NARA’s Audio-Video and Motion Picture labs and our customers and colleagues. They also showcase NARA’s ability to safeguard our moving image and sound records and they allow community debate. The pages are reviewed and adjusted regularly as demands and options evolve and abilities change.”
In addition to CRO, the Digital Preservation Unit collaborates with Moving Image and Sound Preservation Labs on file format research, especially for the Digital Preservation Framework. The staff from the Labs provide essential subject matter expertise when we are adding or updating preservation action plans for file formats, and they helped us make substantial revisions to the significant properties for the Digital Audio, Digital Video, and Digital Cinema record category plans last year.
You can read more about the work done by the Labs and the Special Media Division at NARA in The Unwritten Record.
Sometimes, there are bigger questions and projects that require the Digital Preservation Unit to collaborate with not just one other group at NARA, but across multiple groups. For example, CRO is currently working on updated digitization regulations for special media, and the Labs have been brought in to provide subject matter expertise about digitizing moving images, while the Digital Preservation Unit was consulted about the long-term sustainability of digital file formats. And in a future post, we’ll discuss a case study where we used a file format decision tree process to work with colleagues in CRO and the Electronic Records Division to deal with questions around a format for storing seismic data.
In the meantime, we hope this has provided some insight into the labor of love and collaboration that goes into making these file format resources available to the public!