Every large piece of data in a PDF is a stream: page contents, images, fonts, embedded files, metadata, and in PDF 1.5 to 2.0 also object streams and cross-reference streams. A stream is a dictionary followed by bytes. The dictionary says how long the bytes are and which filters were used to encode them; a reader applies the matching decoders in order to get the data back.
The stream dictionary
13 0 obj
<< /Length 1234 /Filter [/ASCII85Decode /FlateDecode] >>
stream
… 1,234 bytes …
endstream
endobjThese keys can appear in any stream’s dictionary, besides the keys of what the stream is, such as an image or a font:
| Key | Type | Since | Meaning |
|---|---|---|---|
Length | integer | 1.0 | Required. The number of bytes between the line end after stream and the line end before endstream, as stored. May be an indirect reference |
Filter | name or array | 1.0 | The filter, or the filters in the order to apply them when decoding |
DecodeParms | dictionary or array | 1.0 | The parameters of the filter, or an array of them matching Filter (with null for a filter that takes none) |
F | file specification | 1.2 | The stream’s data is in this external file instead; see External streams |
FFilter | name or array | 1.2 | The filters of the external file |
FDecodeParms | dictionary or array | 1.2 | Their parameters |
DL | integer | 1.5 | The length of the decoded data, a hint only, so a reader can set aside memory or tell how big an image is before decoding it |
When Length is wrong, which is common in damaged and hand-edited files,
readers look for endstream instead. When a file is encrypted, the stream
data is encrypted after the filters are applied, so it is decrypted before any
decoding; see Encryption.
The standard filters
PDF 1.5 to 2.0 have ten filters. A reader must support all of them, though it may leave the image formats to the code that draws images. Filters of the general kind can be used on any stream; those of the image kind are for image data only.
| Filter | Abbreviation | Since | Kind | Parameters | What it does |
|---|---|---|---|---|---|
ASCIIHexDecode | AHx | 1.0 | general | none | Two hexadecimal digits a byte |
ASCII85Decode | A85 | 1.0 | general | none | Five printable characters for four bytes |
LZWDecode | LZW | 1.0 | general | predictors, EarlyChange | LZW compression, 9 to 12-bit codes |
FlateDecode | Fl | 1.2 | general | predictors | zlib and deflate compression |
RunLengthDecode | RL | 1.0 | general | none | A simple byte-oriented run-length code |
CCITTFaxDecode | CCF | 1.0 | image | eight, below | Group 3 and Group 4 fax coding of one-bit images |
JBIG2Decode | – | 1.4 | image | JBIG2Globals | JBIG2 coding of one-bit images |
DCTDecode | DCT | 1.0 | image | ColorTransform | JPEG baseline and progressive |
JPXDecode | – | 1.5 | image | none | JPEG 2000 |
Crypt | – | 1.5 | general | Type, Name | Decrypts with a named crypt filter |
The abbreviations are used only in inline
images; in a stream dictionary the full name must be written. Many readers
accept abbreviations in stream dictionaries too, since some writers use them. A
Filter array may chain filters, such as
[/ASCII85Decode /DCTDecode] for a JPEG written as text. An image filter
should come last, since its output is image samples.
Only the general filters may be used on a cross-reference stream, and the
Crypt filter may not, since cross-reference streams are never
encrypted.
ASCIIHexDecode
Each pair of hexadecimal digits, 0–9 and
A–F or a–f, gives one byte.
White space is ignored. The character > ends the data. If it follows
an odd number of digits, the last digit is taken as followed by 0. Any other
character is an error.
48 65 6C 6C 6F 2> → 48 65 6C 6C 6F 20 ("Hello ")2 becomes 20ASCII85Decode
Four bytes are written as five characters from ! (33) to
u (117). Read the four bytes as a big-endian 32-bit number
b; write b in base 85 as five digits c1 to
c5, most significant first, and add 33 to each:
b = c1 × 85⁴ + c2 × 85³ + c3 × 85² + c4 × 85 + c5- A group of four zero bytes may be written as the single character
z, but only at the start of a group. ~>ends the data.- A final group of n bytes (1 to 3) is padded with zero bytes to four,
encoded, and only the first n + 1 characters written. To decode it, pad
the characters with
uto five, decode, and keep the first n bytes. - White space is ignored. Any character outside
!–u,zand~is an error.
An example: the bytes 4D 61 6E 20, “Man ”, are the number
1,298,230,816. In base 85 that is 24, 73, 80, 78, 61; adding 33 gives 57, 106, 113,
111, 94, which are 9jqo^. The data grows by a quarter, against double
for ASCIIHexDecode.
LZWDecode
The Lempel-Ziv-Welch method, as in TIFF. The data is a run of codes, read most significant bit first. Codes start 9 bits wide.
- Codes 0 to 255 stand for the single bytes 0 to 255.
- Code 256 is Clear: empty the table and go back to 9-bit codes. Writers start with it.
- Code 257 is the end of the data.
- Codes from 258 on are added to the table as decoding goes: each new entry is the previous code’s string plus the first byte of the current one.
- The width grows to 10 bits when the table reaches entry 511, to 11 at 1,023 and to 12 at 2,047. It never goes past 12 bits; a writer must send Clear before the table passes 4,095 entries.
EarlyChange (default 1) says when the width grows. With 1 the width
changes one code early, when the next entry to be added is 511, 1,023 or 2,047, as
most LZW coders do. With 0 it changes when that entry has been used. The same
predictor parameters as for Flate may follow the decoding.
LZW was little used after Flate came in, because of the patents on it at the time.
FlateDecode
The zlib data format (RFC 1950) holding deflate-compressed data (RFC 1951): a
two-byte header, usually 78 9C, 78 DA or
78 01, the deflate blocks, and a four-byte Adler-32 checksum. This is
the filter most streams use. Readers usually decode a stream whose checksum is
missing or wrong, and many also accept raw deflate data with no zlib header, which
some writers produce. Its DecodeParms may name a
predictor, applied after inflating.
| Parameter | Default | Meaning |
|---|---|---|
Predictor | 1 | 1 none; 2 TIFF predictor 2; 10 to 15 PNG predictors |
Colors | 1 | Colour components a sample, 1 or more |
BitsPerComponent | 8 | Bits a component: 1, 2, 4, 8 or 16 |
Columns | 1 | Samples a row |
RunLengthDecode
A run of records, each a length byte n followed by data:
| Length byte | Meaning |
|---|---|
| 0 to 127 | Copy the next n + 1 bytes as they are (1 to 128 bytes) |
| 128 | End of the data |
| 129 to 255 | Repeat the next single byte 257 − n times (2 to 128 times) |
FD 00 02 41 42 43 80 decodes to four zero bytes, then
41 42 43, then stops.
CCITTFaxDecode
One-bit image data coded as by fax machines: ITU-T T.4 (Group 3, one- or two-dimensional) or T.6 (Group 4). The output is one bit a pixel, rows packed from the most significant bit and each row padded to a whole byte. By default 0 is black and 1 white in the output, the reverse of what the coding calls them.
| Parameter | Default | Meaning |
|---|---|---|
K | 0 | Below 0: pure two-dimensional, Group 4. 0: pure one-dimensional, Group 3 (Modified Huffman). Above 0: mixed Group 3, where each one-dimensional line may be followed by up to K − 1 two-dimensional ones |
EndOfLine | false | Whether each line starts with an end-of-line code (always allowed, required when true) |
EncodedByteAlign | false | Whether each coded line (for K ≥ 0) or each line of the image (for K < 0) starts on a byte boundary |
Columns | 1728 | Pixels a row |
Rows | 0 | Rows in the image; 0 means not given, so the data or the image’s Height decides |
EndOfBlock | true | Whether the data ends with an end-of-block pattern (RTC for Group 3, EOFB for Group 4) |
BlackIs1 | false | When true, 1 bits mean black in the output |
DamagedRowsBeforeError | 0 | How many damaged rows to accept before an error, when EndOfLine is true |
A CCITT image is usually an image XObject with BitsPerComponent 1 and
DeviceGray, or a stencil mask.
JBIG2Decode
One-bit images coded with JBIG2 (ITU-T T.88, ISO/IEC 14492), from PDF 1.4. The
stream holds the JBIG2 data in the embedded organisation, without the JBIG2
file header, and with the segments of one page, which is page 1. Segments shared by
several images, such as a symbol dictionary of letter shapes used on many pages, go
in a separate stream named by the parameter JBIG2Globals. The output
is one bit a pixel. JBIG2 can be lossless or lossy; in lossy symbol coding one
similar character can be stored for another, so a scanned page can read differently
from the paper.
DCTDecode
A JPEG image (ISO/IEC 10918-1), the whole JFIF or Adobe file from
FF D8 to FF D9. Baseline and progressive coding are both
allowed. The output is samples of 8 bits a component, with 1, 3 or 4 components.
The one parameter, ColorTransform, says whether the data is stored as
YCbCr (or YCCK) and must be turned back into RGB (or CMYK): 1 yes, 0 no. The default
is 1 for three components and 0 otherwise. An Adobe APP14 marker in the
JPEG data overrides the parameter. Many CMYK JPEGs from Adobe programs are stored
inverted, which an image’s Decode [1 0 1 0 1 0 1 0] undoes.
JPXDecode
A JPEG 2000 image (ISO/IEC 15444-1), from PDF 1.5: either a JP2 file, which starts
with the box 00 00 00 0C 6A 50 20 20, or a bare codestream, which starts
FF 4F FF 51. It takes no parameters. The image’s colour space,
bit depth and transparency may come from the JPEG 2000 data instead of the image
dictionary: ColorSpace may be left out, BitsPerComponent is
ignored, and SMaskInData says whether to use the data’s own alpha. The
details are on Images.
Crypt
From PDF 1.5, in an encrypted file, a stream can name its own crypt filter instead of
using the file’s default. Crypt must then be the first filter in the
array, and its parameters are:
| Parameter | Default | Meaning |
|---|---|---|
Type | – | CryptFilterDecodeParms, if present |
Name | Identity | The crypt filter to use, a key in the encryption dictionary’s CF, or Identity |
Identity means the stream is not encrypted at all, which is how a file
can leave its metadata stream readable while the rest is encrypted. Embedded files can
likewise be given their own filter. See
Crypt filters.
Predictors
Flate and LZW compress image-like data better when each byte is stored as a
difference from its neighbours. DecodeParms then names a predictor,
which the reader undoes after decompressing. Cross-reference streams use one too,
since one entry is much like the next.
A row holds Columns samples of Colors components of
BitsPerComponent bits, rounded up to whole bytes:
rowBytes = ⌈Colors × BitsPerComponent ×
Columns / 8⌉. The bytes a pixel, bpp, is
⌈Colors × BitsPerComponent / 8⌉, at least 1.
TIFF predictor 2
Each row holds exactly rowBytes bytes. Each component of each sample is
stored as the difference from the same component of the sample to its left, modulo
2BitsPerComponent; the first sample of a row is stored as it
is. To decode, add the component on the left, row by row. This works on the
component values, so with 1, 2 or 4 bits the bits must be unpacked first.
PNG predictors 10 to 15
Each row is rowBytes + 1 bytes: a filter-type byte, then the row. The
type byte says how this row was coded, so rows can differ even when
Predictor is 10, 11, 12, 13 or 14; the value says only what the writer
chose, and 15 says the writer chose for each row. Readers go by each row’s type
byte.
| Predictor | Type byte | Name | To decode byte x |
|---|---|---|---|
| 10 | 0 | None | x |
| 11 | 1 | Sub | x + a |
| 12 | 2 | Up | x + b |
| 13 | 3 | Average | x + ⌊(a + b) / 2⌋ |
| 14 | 4 | Paeth | x + Paeth(a, b, c) |
| 15 | any | Optimum | as each row’s type byte says |
All sums are modulo 256. a is the decoded byte bpp bytes to the left in the same row, b the decoded byte above in the previous row, c the decoded byte above and bpp to the left; any of them is 0 where it falls outside the image, and the row above the first row is all zeros. Paeth: let p = a + b − c, and pa, pb, pc the distances |p − a|, |p − b|, |p − c|. Take a if pa ≤ pb and pa ≤ pc, else b if pb ≤ pc, else c.
An example, a cross-reference stream with W [1 2 2] and
DecodeParms <</Columns 5/Predictor 12>>: each row is 6
bytes, a type byte and one 5-byte entry. If the decoded row above is
01 00 0F 00 00 and the stored row is 02 00 00 1A 00 00,
the type is 2, Up, so each byte is added to the one above: the entry is
01 00 29 00 00, an object at offset 0x29, generation 0.
External streams
From PDF 1.2 a stream’s data may be kept in another file. The dictionary’s
F names the file with a
file
specification, and FFilter and FDecodeParms say how
that file is encoded. The bytes between stream and
endstream are then ignored. External streams are rare, are not allowed
in PDF/A, and many readers do not follow them. Cross-reference streams may not use them; PDF 2.0 deprecates the
keys there.