CWS compression: zlib and Deflate

In a CWS movie everything after the first 8 bytes is one zlib stream, as RFC 1950 defines it, holding Deflate data, as RFC 1951 defines it. Inflated, it gives the rest of the movie, FileLength − 8 bytes, starting with the frame size. This page describes the stream byte by byte and the Deflate decoder step by step, using the stream of a 645-byte SWF 9 test movie as the example.

The zlib stream

Offset	Size	Type	Name	Description
8	1	UI8	`CMF`	The low 4 bits are the method, 8 for Deflate. The high 4 bits are the window size as log₂(size) − 8: 7 means a 32 KB window.
9	1	UI8	`FLG`	Bits 0 to 4 are check bits, chosen so that `CMF` × 256 + `FLG` is a multiple of 31. Bit 5 says a preset dictionary follows; it is 0 in SWF. Bits 6 and 7 are the level the compressor used: 0 fastest, 1 fast, 2 default, 3 maximum.
10	…	bits	Deflate data	The compressed blocks, described below.
end − 4	4	UI32, big-endian	`ADLER32`	The Adler-32 checksum of the decompressed bytes. The one big-endian number in a SWF.

The usual header bytes are 78 9C: method 8, a 32 KB window, level 2, and 0x789C = 30,876 = 31 × 996. 78 DA (level 3, 30,938 = 31 × 998) also occurs. The level is only a note; a decoder ignores it. The explorer shows the two bytes as “Deflate, 32768-byte window” and “level 2”, refuses a stream that needs a preset dictionary, and shows the Deflate data as one range of the file, followed by the Adler-32.

00000000  43 57 53 09 93 03 00 00 78 9C 7D 53 4D 4E DB 40
00000010  14 7E CF 33 F1 78 9C 84 24 24 76 12 07 FA 23 05
…
00000280  00 6E D7 DF F7

The zlib header at offset 8, 631 bytes of Deflate data from offset 10, and the Adler-32 at offset 0x281

Deflate

Deflate data is a sequence of blocks. Unlike the rest of a SWF, it is read as a stream of bits starting at the lowest bit of each byte. Numbers made of several bits, such as lengths and extra bits, are read lowest bit first too. Huffman codes are the exception: they are packed starting with their most significant bit, so a decoder takes one bit at a time and follows the code down its tree.

Each block starts with 3 bits:

Bits	Name	Meaning
1	`BFINAL`	1 for the last block of the stream.
2	`BTYPE`	0 stored, 1 compressed with the fixed Huffman codes, 2 compressed with dynamic Huffman codes, 3 not used (an error).

The output of every block goes on the end of one buffer. Matches can reach back up to 32,768 bytes into it, across block boundaries. After the block with BFINAL set, the decoder skips to the next byte boundary, where the Adler-32 starts.

Stored blocks

Skip the rest of the current byte.
Read LEN, a 16-bit little-endian number, and NLEN, its one’s complement. If LEN XOR NLEN is not 0xFFFF, the data is damaged.
Copy the next LEN bytes to the output as they are.

Fixed Huffman blocks

The codes are fixed by the standard, so the block’s data starts straight after its 3 header bits:

Literal/length symbols	Code length	Codes
0 to 143	8 bits	`00110000` to `10111111`
144 to 255	9 bits	`110010000` to `111111111`
256 to 279	7 bits	`0000000` to `0010111`
280 to 287	8 bits	`11000000` to `11000111`

Distance codes 0 to 29 are 5 bits each, the code being the number itself. Symbols 286, 287, 30 and 31 have codes but are never used.

Dynamic Huffman blocks

The block first describes its own two code tables, in a compact form, then holds the data:

Read HLIT (5 bits): the number of literal/length codes is HLIT + 257.
Read HDIST (5 bits): the number of distance codes is HDIST + 1.
Read HCLEN (4 bits): HCLEN + 4 code-length codes follow.
Read that many 3-bit lengths, which belong to the code-length symbols in this order: 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15. Symbols not reached have length 0. Build a Huffman table from them (below).
With that table, read HLIT + 257 + HDIST + 1 code lengths as one run. Symbols 0 to 15 are a length. 16 repeats the previous length 3 to 6 times (2 extra bits, plus 3). 17 repeats a length of 0 3 to 10 times (3 extra bits, plus 3). 18 repeats 0 11 to 138 times (7 extra bits, plus 11). A repeat may cross from the literal lengths into the distance lengths.
Build the literal/length table from the first HLIT + 257 lengths and the distance table from the rest.
Decode the data as for any compressed block (below).

Building a Huffman table

Deflate’s codes are canonical: the lengths alone decide them.

Count how many symbols have each length from 1 to 15. Length 0 means the symbol is not used.
The first code of length 1 is 0. The first code of each longer length is (the first code of the length before + the count of that length) × 2.
Go through the symbols in order. Each used symbol takes the next code of its length.

A complete set of lengths leaves no code unused: the sum of 2^−length over the used symbols is exactly 1.

Decoding the data

Repeat until the end-of-block symbol:

Decode a symbol with the literal/length table.
0 to 255: a literal byte. Append it to the output.
256: the end of the block.
257 to 285: a match. Its length is the base length in the table below plus the given number of extra bits, read as a number. Then decode a distance symbol with the distance table; the distance is its base plus its extra bits. Copy length bytes, one at a time, from distance bytes back in the output. The copy may overlap what it is writing: a distance of 1 and a length of 10 repeat the last byte ten times.

Symbol	Extra bits	Lengths	Symbol	Extra bits	Lengths
257	0	3	272	2	31–34
258	0	4	273	3	35–42
259	0	5	274	3	43–50
260	0	6	275	3	51–58
261	0	7	276	3	59–66
262	0	8	277	4	67–82
263	0	9	278	4	83–98
264	0	10	279	4	99–114
265	1	11–12	280	4	115–130
266	1	13–14	281	5	131–162
267	1	15–16	282	5	163–194
268	1	17–18	283	5	195–226
269	2	19–22	284	5	227–257
270	2	23–26	285	0	258
271	2	27–30

Code	Extra bits	Distances	Code	Extra bits	Distances
0	0	1	15	6	193–256
1	0	2	16	7	257–384
2	0	3	17	7	385–512
3	0	4	18	8	513–768
4	1	5–6	19	8	769–1024
5	1	7–8	20	9	1025–1536
6	2	9–12	21	9	1537–2048
7	2	13–16	22	10	2049–3072
8	3	17–24	23	10	3073–4096
9	3	25–32	24	11	4097–6144
10	4	33–48	25	11	6145–8192
11	4	49–64	26	12	8193–12288
12	5	65–96	27	12	12289–16384
13	5	97–128	28	13	16385–24576
14	6	129–192	29	13	24577–32768

The example, bit by bit

The test movie’s Deflate data starts at offset 10 with 7D 53 4D 4E DB 40 14 7E. Reading each byte from its lowest bit:

0x7D is 01111101. Bit 0 is 1: BFINAL, this is the only block. Bits 1 and 2 are 0 and 1: BTYPE = 2, dynamic Huffman codes. Bits 3 to 7 are 1, 1, 1, 1, 0: HLIT = 15, so 272 literal/length codes.
0x53 is 01010011. Bits 0 to 4, 1, 1, 0, 0, 1, give HDIST = 19: 20 distance codes. Bits 5 to 7 (0, 1, 0) and bit 0 of 0x4D (1) give HCLEN = 10: 14 code-length codes.
The next 42 bits are the fourteen 3-bit lengths, for the symbols 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12 and 3: 6, 4, 4, 3, 2, 3, 3, 3, 0, 4, 0, 5, 0, 6. Symbols 13, 2, 14, 1 and 15 are not reached, so their length is 0.

Those lengths make a complete code (one of 2 bits, four of 3, three of 4, one of 5 and two of 6: ¼ + ½ + 3⁄16 + 1⁄32 + 1⁄32 = 1). Built as above:

Code-length symbol	Length	Code	Means
8	2	`00`	length 8
0	3	`010`	length 0 (symbol not used)
6	3	`011`	length 6
7	3	`100`	length 7
9	3	`101`	length 9
5	4	`1100`	length 5
17	4	`1101`	3 to 10 zeros
18	4	`1110`	11 to 138 zeros
4	5	`11110`	length 4
3	6	`111110`	length 3
16	6	`111111`	repeat the previous length

59 bits into the stream, the 292 code lengths of the two real tables begin, written with these codes; then the movie’s bytes follow as literals and matches, until the end-of-block symbol. The whole block is 631 bytes and inflates to 907.

The Adler-32 checksum

After the last block, on a byte boundary, come 4 bytes: the Adler-32 of the decompressed data, big-endian. To compute it:

Start with a = 1 and b = 0.
For each decompressed byte: a = (a + byte) mod 65,521, then b = (b + a) mod 65,521.
The checksum is b × 65,536 + a.

The test movie ends 6E D7 DF F7: b is 0x6ED7 and a is 0xDFF7. The checksum covers only the inflated bytes, not the 8-byte header in front of them.

Seven CWS movies examined for this site end straight after the Deflate data, without the Adler-32. The explorer reads them, shows no Adler-32 field, and notes that the file ends before the checksum. The same seven hold a DefineShape2 with character ID 65,535 whose records are not a valid shape (fill style type 0x02), which suggests a protection tool rewrote them (an inference). The explorer also notes a checksum that does not match.

Offsets in the decompressed data

Every structure after the header, from the frame size to the last tag, is inside the compressed bytes and cannot be pointed at in the file. The reference pages, and the explorer, count its offsets in the decompressed movie: the 8 header bytes followed by the inflated bytes. So inflated byte n is at offset 8 + n, the frame size is at offset 8, and every offset is the one the same movie would have as an uncompressed FWS file.

Deflate blocks and matches pay no attention to the movie’s structure: a tag can start in the middle of a match, and one block can hold the end of one tag and most of the next. So there is no way to find a tag in the compressed bytes without inflating everything before it.

In the explorer the file’s own tree shows the 8 header bytes and the zlib stream: CMF, FLG, the Deflate data as one range, and the Adler-32. The decompressed movie is parsed as a separate tree, whose header fields and tags are listed in the model as “offsets in the decompressed data”. In the test movie, that list starts with FileAttributes at 0x15 and has a DoABC tag at 0x31.

Compressing and decompressing a movie

To decompress: inflate everything from offset 8, check that it gave FileLength − 8 bytes and that the Adler-32 matches, then write FWS, the version byte, the FileLength and the inflated bytes. The version stays as it was.
To compress: take an uncompressed SWF 6 or later movie, compress everything after its first 8 bytes with zlib, and write CWS, the version, the unchanged FileLength and the zlib stream.

Bitmaps inside a movie may be compressed already: lossless bitmaps and JPEG alpha planes are zlib streams of their own, and JPEG and MP3 data hardly compress, so the outer zlib stream mostly shrinks shapes, text, fonts and code.

.SWF File Format – Compression