.PDF File Format – Linearization

The linearization parameter dictionary

The first object in the file, straight after the header, is a dictionary that says the file is linearized. It must lie wholly within the first 1,024 bytes, hold only direct values, and not be referred to by anything else.

43 0 obj
<< /Linearized 1.0 /L 54567 /H [475 598] /O 45 /E 5437 /N 11 /T 52786 >>
endobj

The order of the parts

1. The header.
2. The linearization parameter dictionary.
3. The first-page cross-reference section and its trailer: a table, or a cross-reference stream, for the objects of parts 2 to 6, whose Prev points at the main section at the end.
4. The document catalog and the document-level objects a viewer needs first: the ViewerPreferences, the PageMode, the outline if it opens shown, AcroForm fields, the OCProperties and so on.
5. The primary hint stream.
6. The first page: its page object, then everything only it uses, then the shared objects it uses.
7. The other pages, each with the objects only it uses, in page order.
8. The shared objects not used by the first page, in groups.
9. Other objects: the page tree, thumbnails, the outline when it is not in part 4, threads, named destinations, the information dictionary, the rest of the form, the structure tree, renditions, embedded files.
10. The overflow hint stream, if any.
11. The main cross-reference section and trailer, for everything else.

Parts 5 and 6 may be swapped. The first page ends at E. A reader that has the bytes up to E can show page one.

Hint streams

A hint stream is an ordinary stream, often Flate compressed, whose dictionary says where each hint table starts in the decoded data. The page offset hint table is always first, at 0; the others are named by these keys:

The tables are bit streams, read most significant bit first, and each table starts on a byte boundary. Positions in them are offsets in the file, except that a position past the primary hint stream must have the hint stream’s length added: the tables are written as if the hint stream were not there. Most tables store each value as a difference from a least value in the header, in as few bits as the header says.

The page offset hint table

A header, then one entry for each page. In the header:

Each page entry, its items in the bits given by the header:

The items are not stored page by page but item by item: item 1 of every page, then item 2 of every page, then item 3, then items 4 and 5 for the pages after the first, then item 6 of every page and item 7 of every page.

The shared object hint table

Shared objects, those used by more than one page, are listed in groups of objects with consecutive numbers, of which only the first may be referred to from outside. The first page’s objects come first in the table, then the groups of part 8.

In a one-page document every object of the page is treated as shared.

The other hint tables

With object streams and cross-reference streams

• The first-page section, part 3, may be a cross-reference stream, and so may the main section at the end. T then gives the offset of the main cross-reference stream object.
• The linearization dictionary, the catalog and the page objects must stand alone, not in object streams.
• Other objects may be in object streams. A hint table cannot give the exact offset of an object inside an object stream; its positions then cover the byte range of the object stream that holds it, and a reader finds the object itself through the cross-reference stream.

Telling whether a file is linearized

1. Read the first object after the header. It must be a dictionary with Linearized.
2. Check that L equals the length of the file. An incremental update adds bytes at the end, so a linearized file that has been updated is no longer linearized; its first-page structures are then out of date and must be ignored.
3. Check that H points at a hint stream.

A viewer that reads a whole file from disk gains nothing from linearization, and most readers simply ignore it. Writers such as qpdf can linearize a file and check that the hints are right.