.7Z File Format – Special cases and damage

Self-extracting archives

A self-extracting archive is a program (a Windows .exe with 7-Zip’s 7z.sfx module, or the equivalent for Linux or macOS) with an ordinary 7z archive appended. Nothing about the archive changes, and every offset in it, including NextHeaderOffset, still counts from the end of its own signature header, not from the start of the file. A reader finds the archive by looking for the six signature bytes in the first megabytes of the file and accepting the first place where the 20 bytes after the version have the CRC-32 the signature header states. Checking the CRC is what tells a real archive from the same six bytes in the program’s own code or data. 7-Zip looks through the first 8 MiB (kMaxCheckStartPosition in OpenArchive.cpp); the explorer does the same.

A stub can be any format: MZ for a Windows program, 7F 45 4C 46 for ELF, the Mach-O magics for macOS. The archive is not aligned to anything.

Split archives (volumes)

7-Zip can cut an archive into pieces of a fixed size: name.7z.001, name.7z.002, and so on. The pieces are the archive, cut into equal parts and nothing more: concatenated they give an ordinary 7z file. The signature header is in the first piece, and, since the header is written last, the file list is at the end of the last piece. So the first volume alone has a signature header whose NextHeaderOffset and NextHeaderSize reach past the end of the file, and the first part of the packed data, and it cannot say what the packed data is. 7-Zip’s header file says its own multi-volume layout for the 7z format is not finished and keeps it disabled, so the .001 naming is a convention of the program, not of the format.

The explorer opens a .7z.001 file for what it is: it shows the signature header and the packed bytes that are there, says that the file list lives in the last volume, and does not guess names or sizes.

A damaged or unfinished start header

A 7z archive is written in one pass and the signature header is filled in last, once the header’s position is known. An archive whose writer was interrupted has a signature header of 32 bytes in which everything after the version is zero (the CRC, NextHeaderOffset, NextHeaderSize and NextHeaderCRC), and, if the writer got that far, a header at the end of the file.

• All zero. 7-Zip’s reader (FORMAT_7Z_RECOVERY in 7zIn.cpp) looks at the last 512 bytes of the file for a header: the last byte must be 00 (kEnd), and the bytes found must start with 17 06 (kEncodedHeader then kPackInfo) or 01 04 (kHeader then kMainStreamsInfo), searching backwards from the end. The header is taken to run from there to the end of the file.
• Not zero but wrong. If the start header’s CRC does not match, 7-Zip does not take the file for a 7z archive at all. The explorer reads it anyway, because the signature is there: it reports the mismatch, and if the offsets point at a header that reads, it uses them.
• A header cut off or beyond the end of the file (the archive was truncated). 7-Zip stops with an “unexpected end”. The explorer searches the last 4 MiB for the same patterns, tries each candidate from the end back, and takes the first that reads as a whole header (and, for an encoded one, whose unpacked bytes have the CRC-32 the record states), since a pattern can occur by chance.

A header found this way is marked in the explorer as recovered, and the problem is listed. The header is not trusted for anything the data does not check: the files’ CRC-32 values are what show whether the packed data is whole.

Unix archives

7-Zip on Linux and macOS, and p7zip before it, store Unix permissions in the attribute word: bit 15 (0x8000) says the high 16 bits are the st_mode. A regular file with mode 0644 has the attribute 81A48020: 81A4 is octal 100644 (a regular file, rw-r--r--), 8000 is the flag, and 20 is the Windows “archive” bit. A folder has 41ED8010 (octal 40755 and the Windows directory bit 10).

A symbolic link is stored as an ordinary file whose data is the target path and whose mode says it is a link: a link link to a.txt was seen as a 5-byte file with attribute A1ED8020 (mode octal 120755, a symbolic link), its data the bytes a.txt. 7-Zip stores the link itself only when asked (-snl); otherwise it follows the link and stores the target’s data under the link’s name. Names use / as the separator. Device files, sockets and FIFOs have a mode type but no data.

Anti items, empty files and folders

Items with no data stream are marked by the kEmptyStream bit. Of those, the kEmptyFile bit separates empty files from folders (a bit set: file; clear: folder). The kAnti bit marks an anti item: in an archive used to update another, it says that the path is to be deleted. An anti item has no data and keeps its name; whether it stands for a file or a folder is the kEmptyFile bit. Most archives have none.

An archive with no items at all can be only the 32-byte signature header with NextHeaderSize 0, or one with an empty header.

Encrypted archives

7zAES (the properties) encrypts the data of a folder: it is one of the coders, almost always the last, so a folder reads LZMA2 then 7zAES. Without a password the sizes and the names are still listed, as they are in the header, and the explorer marks those files as encrypted and does not offer them as entries. When the option encrypt file names is on (-mhe=on), the next header itself is packed, first compressed and then encrypted, so even the file list is hidden: the file shows as encrypted with only the signature header, the encoded-header record that describes it and its packed bytes visible, with the salt and initialisation vector from the coder’s properties. The explorer does not ask for a password and does not try one.

External data and additional streams

The format lets the names, the times, the attributes and the folders be stored in additional streams (the kAdditionalStreamsInfo record) and referred to by an index (the External byte). 7-Zip’s writer never does this, but its reader accepts it, and so does the explorer: the additional streams are unpacked first and each External record is read from the stream it names.