.TAR File Format – End of archive

Padding file data

A member’s data starts in the block after its header and fills as many blocks as it needs. The last block is padded, so a member of n bytes takes n rounded up to the next multiple of 512. The next header always starts on a block boundary, and a reader finds it by rounding up the size in the header: the next header is at the header’s offset + 512 + ((n + 511) & ~511).

Writers pad with zeros, but readers must not depend on it. A Sun Java documentation archive from 2003, examined for this site, has padding that holds bytes of earlier files, left over in the writer’s buffer.

The two zero blocks

POSIX ends the archive with two 512-byte blocks of zeros. A reader stops at the first block that is all zeros, since a real header never is: its checksum would fail. Readers differ over the second block. libarchive tries to read it but carries on if it is missing; Go returns an error if a zero block is followed by one that is not zero.

For a 13-byte file, hello.txt, the archive written by bsdtar is four blocks long:

Some archives have no zero blocks at all and simply stop after the last member’s data; among the files examined for this site these were mostly test files from Go and tar-stream. The explorer reads them, and reports an archive that ends with a single zero block, or with a zero block followed by one that is not zero.

Records and the blocking factor

Tape drives work best with large writes, so tar writes its blocks in groups called records. The number of blocks in a record is the blocking factor. tar(5): early implementations wrote at most 10,240 bytes at a time, 20 blocks, and “this is still the default for most implementations”. POSIX says “the last physical block is always full size”, so the last record is filled up after the two zero blocks; those blocks “may contain undefined data”, and in practice they are zeros.

Whether a writer pads depends on the writer and where it writes. GNU tar, writing with its default blocking factor, pads to a multiple of 10,240 bytes, so the one-file archive above would be 10,240 bytes long. bsdtar writing to a disk file stops after the two zero blocks, as in the bsdtar archives examined for this site. That libarchive pads only when it writes to a device or a pipe is from memory of its source and not confirmed.

The explorer shows zeros after the end blocks as record padding, and any other bytes there as extra bytes.

Appending and joining

To add members to an archive, tar overwrites the two zero blocks with the new headers and writes new zero blocks after them. Joining two archives with a plain file copy leaves the first archive’s zero blocks in the middle, so most readers stop there. GNU tar’s --ignore-zeros option reads past them to the members beyond. The explorer shows a tar header found after the end of the archive as a further archive.

A compressed tar file such as .tar.gz compresses the whole stream, padding and end blocks included. Runs of zeros compress to almost nothing, so the padding costs very little.