JPEG XL: frames

After the headers, a codestream is a series of frames. Each frame is a frame header, a table of contents, and sections of coded data that can be decoded in parallel. Frames can be layers blended together, the frames of an animation, or hidden frames kept only as references. The last frame has is_last set. Fields are coded as described under How fields are coded; the lists follow libjxl, because the ISO text is not freely available.

FrameHeader

Field	Coding	Condition	Meaning
all_default	Bool		1: a single regular VarDCT frame covering the image, replace blending, last frame.
frame_type	U32(Val(0), Val(1), Val(2), Val(3))		Table below.
encoding	Bool		0 VarDCT, 1 modular.
flags	U64		Bit flags, table below.
do_YCbCr	Bool	not xyb_encoded	1 when the samples are YCbCr (as in recompressed JPEGs).
chroma_subsampling	3 × Bits(2)	do_YCbCr and not use_dc_frame	Per channel: 0 full size, 1 halved both ways, 2 halved horizontally, 3 halved vertically.
upsampling	U32(Val(1), Val(2), Val(4), Val(8))	not use_dc_frame	The frame is coded at 1/upsampling of its size and upsampled after decoding.
ec_upsampling	same, per extra channel	extra channels, not use_dc_frame	At least upsampling, at most 8.
group_size_shift	Bits(2), default 1	modular	Group size = 128 << shift: 128, 256, 512 or 1024. VarDCT groups are 256.
x_qm_scale, b_qm_scale	Bits(3), defaults 3 and 2	VarDCT and XYB	Scaling of the X and B quantisation matrices.
passes	Passes	not reference-only	Progressive passes, below.
lf_level	U32(Val(1), Val(2), Val(3), Val(4))	LF frame	This frame holds the LF image of a later frame at 1 : 8^level.
have_crop	Bool	not LF frame	1 when the frame has its own position and size.
x0, y0	signed U32(Bits(8), BitsOffset(11, 256), BitsOffset(14, 2304), BitsOffset(30, 18688))	have_crop, regular or skip-progressive	Position of the frame; may be negative.
width, height	same U32, unsigned	have_crop	Size of the frame; not 0.
blending_info	BlendingInfo	regular or skip-progressive	How the frame combines with what is beneath.
ec_blending_info	BlendingInfo per extra channel	regular or skip-progressive
duration	U32(Val(0), Val(1), Bits(8), Bits(32))	animation	In ticks. A frame of duration 0 is a layer of the next shown frame.
timecode	Bits(32)	have_timecodes
is_last	Bool, default 1	regular or skip-progressive	1 on the final frame of the codestream.
save_as_reference	U32(Val(0), Val(1), Val(2), Val(3))	not LF frame and not last	Which of four reference slots to keep this frame in, for patches and blending.
save_before_ct	Bool	see note	1 to save the frame before the colour transform (XYB or YCbCr to RGB) is applied.
name	name		Layer name.
restoration_filter	LoopFilter		Below.
extensions	extensions

save_before_ct is read for reference-only frames (default 1) and for regular frames that can be referenced, use replace blending and cover the whole image; LF frames always save before the colour transform. The order of the three subsampling modes within the field is not confirmed here.

frame_type	Meaning
0	Regular frame: shown, or blended into the next shown frame.
1	LF frame: the low-frequency (1 : 8) image of a following frame, itself coded as a frame.
2	Reference only: never shown; kept for patches.
3	Skip progressive: a regular frame that progressive decoders need not show early.

flags bit	Value	Meaning when set
0	1	Noise: synthetic, luminance-modulated noise is added.
1	2	Patches: rectangles copied from reference frames are blended on.
4	16	Splines: centripetal Catmull-Rom curves of varying colour and width are drawn.
5	32	Use LF frame: the LF image comes from an earlier LF frame.
7	128	Skip adaptive LF smoothing.

BlendingInfo

Field	Coding	Condition	Meaning
mode	U32(Val(0), Val(1), Val(2), BitsOffset(2, 3))		0 replace, 1 add, 2 blend (alpha over), 3 alpha-weighted add, 4 multiply. Above 4 is invalid.
alpha_channel	U32(Val(0), Val(1), Val(2), BitsOffset(3, 3))	blend or alpha-weighted add, with extra channels	Which extra channel is the alpha.
clamp	Bool	those modes, or multiply	1 to clamp alpha to 0 to 1.
source	U32(Val(0), Val(1), Val(2), Val(3))	not replace, or a cropped frame	Which saved reference slot is underneath.

Passes

Field	Coding	Condition	Meaning
num_passes	U32(Val(1), Val(2), Val(3), BitsOffset(3, 4))		Progressive passes, at most 11.
num_ds	U32(Val(0), Val(1), Val(2), BitsOffset(1, 3))	more than one pass	How many downsampling marks follow, at most 4.
shift[i]	Bits(2) each	for passes 0 to num_passes − 2	Coefficient shift of each pass; the last pass has 0.
downsample[i]	U32(Val(1), Val(2), Val(4), Val(8)) each	num_ds entries	Strictly decreasing.
last_pass[i]	U32(Val(0), Val(1), Val(2), Bits(3)) each	num_ds entries	After this pass the image is complete at that downsampling. Strictly increasing.

LoopFilter (restoration filters)

Field	Coding	Default	Meaning
all_default	Bool
gab	Bool	1	Gaborish: a 3 × 3 smoothing applied after decoding.
gab_custom	Bool	0	With gab: six F16 weights follow, two each for X, Y and B (defaults 0.115169525 and 0.061248592).
epf_iters	Bits(2)	2	Edge-preserving filter iterations, 0 to 3.
epf_sharp_custom	Bool	0	VarDCT with EPF: a table of 8 F16 sharpness values follows.
epf_weight_custom	Bool	0	With EPF: channel scales (40, 5, 3.5) and two zero-flush thresholds (0.45, 0.6), all F16.
epf_sigma_custom	Bool	0	With EPF: quant multiplier (0.46, VarDCT only), pass 0 and pass 2 sigma scales (0.9, 6.5) and border SAD multiplier (2/3), all F16.
epf_sigma_for_modular	F16	1.0	Modular with EPF.
extensions	extensions

Table of contents and sections

The frame is cut into groups of group_dim × group_dim pixels (256 for VarDCT; 128 << group_size_shift for modular) and LF groups of 8 × group_dim. The coded data is a list of sections:

LfGlobal: patches, splines and noise parameters when flagged, LF dequantisation, and (VarDCT) the quantiser, block context map and colour correlation; the global modular tree and the small modular channels.
LfGroup, one per LF group: the 1 : 8 LF coefficients and, for VarDCT, the block types and quantisation field.
HfGlobal (VarDCT): dequantisation matrices and the coefficient orders and histograms for each pass.
PassGroup, for each pass and each group: the high-frequency coefficients, or the modular data of that group.

When there is one pass and one group, the whole frame is a single section. The TOC before the sections lists their sizes:

Field	Coding	Meaning
permuted	Bool	1 when the sections are stored in a different order; a permutation follows, entropy coded as a Lehmer code.
(padding)	0 bits	To the next byte.
size[i]	U32(Bits(10), BitsOffset(14, 1024), BitsOffset(22, 17408), BitsOffset(30, 4211712))	Bytes in each section; there are 1 + LF groups + 1 + passes × groups entries, or 1.
(padding)	0 bits	To the next byte. The sections follow, each starting on a byte.

Because every group has its own section, a decoder can read a crop, decode groups in parallel, or show a progressive image from the first passes.

VarDCT and modular coding

VarDCT is the lossy mode. The image, normally in XYB, is split into blocks transformed with variable-size DCTs, quantised and entropy coded. The block types (AC strategies) as libjxl numbers them:

Code	Transform	Code	Transform
0	DCT 8 × 8	14 to 17	AFV 0 to 3 (corner-adaptive 8 × 8)
1	Identity (Hornuss)	18	DCT 64 × 64
2	DCT 2 × 2	19	DCT 64 × 32
3	DCT 4 × 4	20	DCT 32 × 64
4	DCT 16 × 16	21	DCT 128 × 128
5	DCT 32 × 32	22	DCT 128 × 64
6	DCT 16 × 8	23	DCT 64 × 128
7	DCT 8 × 16	24	DCT 256 × 256
8	DCT 32 × 8	25	DCT 256 × 128
9	DCT 8 × 32	26	DCT 128 × 256
10	DCT 32 × 16
11	DCT 16 × 32
12	DCT 4 × 8
13	DCT 8 × 4

A recompressed JPEG uses only DCT 8 × 8 with the JPEG’s own quantisation, which is why it can be turned back into the identical JPEG.

Modular is the integer mode, used for lossless images, extra channels and the LF image. Channels can be transformed first (0 reversible colour transform, 1 palette, 2 squeeze, a reversible Haar-like pyramid), then each sample is predicted and the residual entropy coded, with a decision tree choosing the context. The predictors:

Code	Prediction	Code	Prediction
0	0	7	NE
1	W (left)	8	NW
2	N (above)	9	WW
3	(W + N) ÷ 2	10	(W + NW) ÷ 2
4	Select (as in lossless WebP)	11	(N + NW) ÷ 2
5	Gradient: W + N − NW, clamped to the range of W and N	12	(N + NE) ÷ 2
6	Self-correcting weighted predictor	13	(6N − 2NN + 7W + WW + NEE + 3NE + 8) ÷ 16

Not described on this site: the entropy coder (ANS or prefix codes, with LZ77 and hybrid integer coding and clustered contexts), the modular tree syntax, the transform parameters, the quantisation and dequantisation tables, the XYB conversion and the patch, spline and noise syntax. The AC strategy and predictor numbers are as libjxl uses them and have not been checked against the standard.