.TAR File Format

Introduction

This is the archive format of GNU tar, the tar of Linux. GNU tar began in the late 1980s from John Gilmore’s public-domain pdtar, and its format is older than the POSIX ustar standard. GNU tar still writes it by default: its manual says the default is set when GNU tar is built, is usually gnu, and that the POSIX format will be the default in a future version.

The manual names two flavours: oldgnu, the “format used by GNU tar of versions prior to 1.12”, and gnu, the “format used by GNU tar versions starting with 1.13”. Both write the same magic and header layout, and the explorer does not tell them apart.

The header starts like a ustar header, but bytes 257 to 264 hold ustar, two spaces and a NUL, and bytes 345 to 511, where ustar keeps a path prefix, hold access and change times, a multi-volume offset and a sparse map. A ustar reader would take those times as part of the path, so this is a separate format. Go’s documentation puts it plainly: “The GNU header format is older than the USTAR and PAX standards and is not compatible with them.”

GNU tar solved the problems pax later solved in its own way: long names and link targets in extra entries before the member, sparse files as maps of the parts that hold data, and incremental and multi-volume backups with type flags of their own. Large numbers are written in base-256.

A tar file dropped on any page of this site opens in the explorer, which shows its structure: each member’s header field by field, its long names and sparse maps, its data, and the blocks that end the archive.

Structure

An archive is a sequence of 512-byte blocks. Each member is a header block, maybe led by long-name entries and followed by sparse extension blocks, then its contents padded with zeros to a whole number of blocks. Two blocks of zeros end the archive, and GNU tar fills the last record with zeros after them.

Telling it apart

The explorer’s parser for this format checks the first header: its checksum must be right, and bytes 257 to 264 must be 75 73 74 61 72 20 20 00, ustar, two spaces and a NUL. A first header with the POSIX magic ustar\0 00, or no magic at all, is turned down, and the explorer tries the parsers for the other tar formats.

According to libarchive’s tar(5), some writers from before POSIX also wrote ustar, a space and a version of a space and a NUL: the same bytes. Their archives are read as GNU archives.

Variants

• POSIX ustar and pax archive – the POSIX header with the “ustar” magic, owner names and a 155-byte prefix; pax adds records
• GNU tar archive – GNU tar’s own header, with times and sparse maps where ustar has the prefix
• Schily tar archive (star) – star’s header: a 131-byte prefix, with access and change times after it

Links