A PDF 1.5 to 2.0 file has the same four parts as every PDF: a header line, a body of objects, a cross-reference section that says where each object is, and a trailer that names the document’s root. What sets these versions apart is that the cross-reference section can be a compressed cross-reference stream, and that most objects can be packed, many at a time, into compressed object streams. A reader can then find such an object only through the cross-reference stream, never by its own offset. A reader starts at the end of the file, not the beginning.
0Header%PDF-1.7or%PDF-2.0, then a comment of binary bytestwo lines…Body indirect objects, some of them object streams that hold more objectsmost of the file…Cross-reference section anxreftable, or a cross-reference stream objectvaries…Trailer atrailerdictionary, or the stream’s own dictionary;startxref, an offset,%%EOFa few lines
A real file
A small PDF 1.7 file of 3,527 bytes, a one-page drawing. It starts with the header and a binary comment, and object 3 follows at once:
00000000 25 50 44 46 2D 31 2E 37 0A 25 E4 E3 CF D2 0A 33 %PDF-1.7.%.....3
00000010 20 30 20 6F 62 6A 0A 3C 3C 2F 4C 65 6E 67 74 68 0 obj.<</Length
00000020 20 32 20 30 20 52 2F 46 69 6C 74 65 72 2F 46 6C 2 0 R/Filter/FlIts body holds ordinary objects and one object stream, object 10, which packs
five more objects into compressed data. There is no xref keyword and
no trailer keyword. The file ends with a cross-reference stream,
object 16, whose dictionary does the trailer’s job, then startxref
and the offset of that object:
10 0 obj
<</Length 15 0 R/Filter/FlateDecode/Type/ObjStm/N 5/First 27>>
stream
…
16 0 obj
<</Filter/FlateDecode/Type/XRef/W[1 2 2]/Size 17/Root 14 0 R/Info 11 0 R
/ID[<57A984D8D8017E7021C18DB41CC7EE07><57A984D8D8017E7021C18DB41CC7EE07>]
/DecodeParms<</Columns 5/Predictor 12>>/Length 87 >>
stream
… 87 bytes of compressed entries …
endstream
endobj
startxref
3180
%%EOFW [1 2 2] says each entry is 5 bytes: a 1-byte type, a 2-byte offset or
object stream number, and a 2-byte generation or index. Size 17 means
entries for objects 0 to 16. The entries are compressed with Flate after a PNG Up
predictor on rows of 5 columns. How to decode them is on
Cross-reference
and trailer.
Header
The first line is %PDF- and the version the file was written to:
%PDF-1.5, %PDF-1.6, %PDF-1.7 or
%PDF-2.0. It is a comment, so a reader that looks for objects skips
it.
The document catalog may hold a Version name, such as
/1.7 or /2.0. If it is later than the header’s version, it
wins. This lets an incremental update raise the version without rewriting the first
line. A reader should not refuse a file only because the version is newer than it
knows; it reads what it can.
The header version does not say which cross-reference form the file uses. Many
files with a 1.5, 1.6 or 1.7 header still have a plain xref table, and
some files with an older header number have a cross-reference stream. A reader
looks at what is at the startxref offset instead.
The header is meant to be at byte 0. Readers such as Acrobat also accept it within the first 1,024 bytes, which happens when something has been put in front of the file. The offsets in the cross-reference section are then out by the length of what was put in front, and readers that allow for this correct them.
Binary comment
Streams hold raw binary bytes. A file with binary data should have, on the line
after the header, a comment of at least four bytes with values of 128 or more, such
as %âãÏÓ (25 E2 E3 CF D3). Programs that move files about
look at the first bytes to decide whether a file is text, and these bytes tell them
it is not. A PDF reader skips the comment like any other. Writers pick their own
bytes; the example above uses E4 E3 CF D2.
Body
The body is a run of indirect objects, each written
12 0 obj
<< /Type /Page /Parent 3 0 R /MediaBox [0 0 612 792] /Contents 13 0 R >>
endobjThe first number is the object number and the second the generation, 0 for every
object of a new file. Objects can come in any order, with gaps in the numbering, and
point at each other with references such as 3 0 R. See
Objects and syntax for
every kind of object.
From PDF 1.5 an object can also sit inside an object
stream instead: a stream of /Type /ObjStm whose decoded data is a
list of object numbers and offsets followed by the objects themselves, written
without obj and endobj. Dictionaries, arrays, numbers and
strings can go there; streams cannot, and neither can anything with a generation
other than 0. Writers put most small objects into object streams, because the
whole stream is compressed at once, which saves much space in files with many
pages, fonts and annotations.
Cross-reference section
After the body comes the cross-reference section, in one of two forms.
- Cross-reference table – the keyword
xref, subsections of 20-byte text entries, then thetrailerkeyword and dictionary. It can only point at objects that stand alone in the file. - Cross-reference stream – an indirect stream object of
/Type /XRef, whose data is binary entries of fixed widths and whose dictionary also holds the trailer’s keys. Its entries can point at objects in object streams.
A file may use one form or the other in each section. A file that keeps a table for old readers and adds a stream for the objects only newer readers can find is a hybrid file.
Trailer
With a table, the keyword trailer and a dictionary follow it. With a
cross-reference stream there is no trailer keyword: the stream’s
dictionary holds the same keys. They are:
| Key | Type | Since | Meaning |
|---|---|---|---|
Size | integer | 1.0 | Required. One more than the highest object number in the file |
Prev | integer | 1.0 | Only after an incremental update: the offset of the previous cross-reference section |
Root | reference | 1.0 | Required. The document catalog |
Encrypt | dictionary | 1.1 | Only in an encrypted file: the encryption dictionary |
Info | reference | 1.0 | The document information dictionary; in PDF 2.0 only its dates are still current |
ID | array | 1.1 | Two byte strings that identify the file; required when it is encrypted, and in every PDF 2.0 file |
XRefStm | integer | 1.5 | Only in a table’s trailer, in a hybrid file: the offset of a cross-reference stream |
startxref and %%EOF
The file ends with three lines:
startxref
3180
%%EOFThe number is the byte offset of the last cross-reference section. For a table it is
the offset of the x of xref. For a cross-reference stream it
is the offset of the object itself, the first digit of 16 0 obj.
%%EOF is a comment that marks the end. Readers look for it within the
last 1,024 bytes, since some writers add bytes after it.
How a reader opens a file
- Check the
%PDF-header and read the version. - Find
%%EOFnear the end, then the laststartxrefbefore it, and read the offset. - Look at what is at that offset. If it is
xref, read the table and then the trailer after it. If it isn g obj, read that object: it must be a stream of/Type /XRef. Decode its data and read the entries. - In a table’s trailer, if there is
XRefStm, read the cross-reference stream at that offset too, and add its entries to those of the table. - If the trailer or the stream has
Prev, read the older section at that offset in the same way, keeping the entries already read: the newest entry for an object wins. Repeat until there is noPrev. - Take
Rootfrom the newest trailer and read the catalog, then the page tree. IfEncryptis there, set up decryption first. The cross-reference streams themselves are never encrypted. - For an object of type 1, go to its offset. For an object of type 2, read the object stream named in its entry, decode it, and take the object at the index given.
When the offsets are wrong, many readers rebuild the cross-reference by scanning the
file for n g obj. In these versions that is harder than before: objects
in object streams have no obj line, so a reader must also find and
decode every /Type /ObjStm stream to find them.
Incremental updates
A PDF can be changed without rewriting it. The changes go at the end: the new and
changed objects, a new cross-reference section that lists only those objects, and a
new trailer whose Prev points at the previous section.
0Original file header, body, cross-reference stream,startxref,%%EOFas written…Update changed objects, a new cross-reference stream withPrev,startxref,%%EOFappended
- A changed object appears again with the same number; the newest copy counts. An object that was in an object stream can be written in the update as a normal object, or into a new object stream.
- A new object stream can say, with
Extends, that it continues an earlier one; this links the collection but does not replace the old stream. - A deleted object gets a free entry with its generation raised by one.
- Writers keep to the kind of section the file already has: a file whose last
section is a cross-reference stream is updated with another stream, and a hybrid
file keeps its table and adds a new
XRefStm. Otherwise a reader that could open the file before might no longer open it. - The new trailer repeats
Root, the whole file’sSize, andInfo,EncryptandIDwhere present. The first string ofIDstays the same; the second usually changes.
Updates keep the earlier versions in the file. This is how a document can carry several signatures, each covering the bytes as they were when it was made, and how the document security store of PDF 2.0 is added after signing.
Hybrid files
A hybrid-reference file is written for both old and new readers. Its last section is
a normal xref table with a trailer, so a PDF 1.4 reader can open it, and
the trailer has an extra key, XRefStm, the offset of a cross-reference
stream. The table lists the objects that stand alone. The stream lists the objects
that are in object streams, which an old reader does not know how to find.
- A PDF 1.4 reader ignores
XRefStm, so it sees only the table. The objects in object streams are missing for it, and the file is written so that those objects are ones it can do without: the objects for features it does not know, such as optional content or structure added for newer readers. - A PDF 1.5 or later reader reads the table, then the stream at
XRefStm, and lets the stream’s entries fill in the objects that the table lists as free. Only after that does it followPrev.
The details are on Cross-reference and trailer.
Line endings
A line ends with a carriage return (0D), a line feed (0A)
or both (0D 0A), and readers accept all three. The entries of a
cross-reference table are an exception: each ends with exactly two bytes, a space
and a CR or LF, or CR LF, so that every entry is 20 bytes. The keyword
stream must be followed by CR LF or LF, never by CR alone, so that a
reader knows where the data starts.