.LZ4

Introduction

LZ4 is a compression method built for speed rather than size. Yann Collet released it in 2011. It compresses less than gzip but decompresses many times faster, often at several gigabytes a second, so it is used where data must be packed and unpacked on the fly: in file systems, databases, game assets, the Linux kernel and network protocols. An .lz4 file uses the LZ4 frame format, which wraps the compressed blocks with a header and checks.

Structure

Numbers are little-endian. A file can hold several frames one after another.

The top bit of a block’s size means the block is stored uncompressed. Inside a compressed block, data is a series of sequences, each some literal bytes followed by a copy of earlier data, with no entropy coding, which is why decoding is so fast. Old files from the first lz4 tool use a legacy frame with the magic 02 21 4C 18.

Links