GGUF file format: dequantisation

From the dequantize_row_* functions of ggml/src/ggml-quants.c, the tables of ggml/src/ggml-common.h and the helper functions of ggml/src/ggml-impl.h in llama.cpp (master, read 2026-11-13). The block layouts (field order and sizes) are on the tensor types page. Each function turns k values stored as blocks into k floats, y, block after block; d and m are binary16 values converted to 32-bit floats, qs means the block's quant array, and the bit tests are on bytes read in file order (little-endian where a larger integer is read, as in the memcpy of qh). Output is written in the order the values are listed here.

Unquantised types

F32, F16 (IEEE binary16), BF16 (bfloat16: the upper 16 bits of a binary32), F64, I8, I16, I32, I64 are plain little-endian values, one per element; F8_E4M3 is decoded by ggml_f8_e4m3_to_fp32 (below).

Legacy types (32-value blocks)

Type	Value `j` of the block
Q4_0	for `j` = 0 to 15: `y[j] = d × ((qs[j] & 0xF) − 8)` and `y[j+16] = d × ((qs[j] >> 4) − 8)`.
Q4_1	`y[j] = d × (qs[j] & 0xF) + m`, `y[j+16] = d × (qs[j] >> 4) + m`.
Q5_0	`qh` is read as a little-endian 32-bit integer; for `j` = 0 to 15: bit 4 of value `j` is bit `j` of `qh` (`((qh >> j) << 4) & 0x10`), and bit 4 of value `j`+16 is bit `j`+16 of `qh` (`(qh >> (j+12)) & 0x10`); `y[j] = d × (((qs[j] & 0xF) \| xh0) − 16)`, `y[j+16] = d × (((qs[j] >> 4) \| xh1) − 16)`.
Q5_1	the same bits, `y[j] = d × ((qs[j] & 0xF) \| xh0) + m`, `y[j+16] = d × ((qs[j] >> 4) \| xh1) + m`.
Q8_0	`y[j] = d × qs[j]` with `qs` signed bytes.
Q8_1	no `dequantize_row` function exists for it in the file; the layout (`d`, `s`, 32 signed bytes) is that of Q8_0 plus the stored sum `s`, and a reader can use `y[j] = d × qs[j]` (an inference from the layout, not stated in the source).
Q1_0 (128 values)	bit `j` of the block (bit `j` mod 8 of byte `j` div 8, low bit first) gives `y[j] = +d` if 1 and `−d` if 0.
Q2_0 (64 values)	value `j` is the 2 bits at bit offset 2·(`j` mod 4) of byte `j` div 4, `q` = 0 to 3: `y[j] = (q − 1) × d` (codes 00, 01, 10, 11 mean −1, 0, +1, +2).
Q8_K (256 values)	`y[j] = d × qs[j]` with `d` a 32-bit float; `bsums` are not used.

k-quants (256-value super-blocks)

Q2_K. d and dmin binary16. Four passes j = 0 to 3 over each 128 values, with shift = 2j; two 16-value groups per pass, group 0 from q[0..15], group 1 from q[16..31] (q advances by 32 after every four passes). For a group with scale byte sc (the 16 scale bytes in order): dl = d × (sc & 0xF), ml = dmin × (sc >> 4); y = dl × ((q[l] >> shift) & 3) − ml.
Q3_K. d binary16. The 12 scales bytes are copied into the first three of four 32-bit words aux[0..3] and unpacked into 16 signed bytes (the 16 bytes of aux, in memory order, are then the scales, each used as a signed byte) with kmask1 = 0x03030303, kmask2 = 0x0F0F0F0F: tmp = aux[2]; aux[2] = ((aux[0] >> 4) & kmask2) | (((tmp >> 4) & kmask1) << 4); aux[3] = ((aux[1] >> 4) & kmask2) | (((tmp >> 6) & kmask1) << 4); aux[0] = (aux[0] & kmask2) | (((tmp >> 0) & kmask1) << 4); aux[1] = (aux[1] & kmask2) | (((tmp >> 2) & kmask1) << 4). Then, with the same loop shape as Q2_K (passes of 128 values, shift 0, 2, 4, 6, and a mask m starting at 1 and doubled each pass): dl = d × (scale − 32) and y = dl × (((q[l] >> shift) & 3) − (hmask[l] & m ? 0 : 4)), where hmask is indexed with the same l (0 to 15 or 16 to 31) as q.
Q4_K. d, dmin binary16. For j = 0, 64, 128, 192 (index is = 0, 2, 4, 6): get_scale_min_k4(is) gives (sc, m) and get_scale_min_k4(is+1) gives the second pair; the first 32 values are d × sc1 × (q[l] & 0xF) − dmin × m1 and the next 32 d × sc2 × (q[l] >> 4) − dmin × m2, with q advancing by 32 per step. get_scale_min_k4(j, q): for j < 4, sc = q[j] & 63 and m = q[j+4] & 63; otherwise sc = (q[j+4] & 0xF) | ((q[j−4] >> 6) << 4) and m = (q[j+4] >> 4) | ((q[j] >> 6) << 4).
Q5_K. As Q4_K, but each value adds 16 when a bit of qh[l] is set: the low-nibble values test bit mask u1 and the high-nibble values u2, which start at 1 and 2 and are shifted left by 2 after every 64 values; value = d × sc × ((nibble) + (qh[l] & u ? 16 : 0)) − dmin × m.
Q6_K. d binary16, scales signed bytes. In two halves of 128 values, for l = 0 to 31 and is = l/16: q1 = ((ql[l] & 0xF) | (((qh[l] >> 0) & 3) << 4)) − 32, q2 = ((ql[l+32] & 0xF) | (((qh[l] >> 2) & 3) << 4)) − 32, q3 = ((ql[l] >> 4) | (((qh[l] >> 4) & 3) << 4)) − 32, q4 = ((ql[l+32] >> 4) | (((qh[l] >> 6) & 3) << 4)) − 32; y[l] = d × sc[is] × q1, y[l+32] = d × sc[is+2] × q2, y[l+64] = d × sc[is+4] × q3, y[l+96] = d × sc[is+6] × q4; then y += 128, ql += 64, qh += 32, sc += 8.

Ternary types (256 values)

TQ1_0. d binary16. The 48 bytes of qs hold 5 ternary digits each (base 3, as multiples of the powers 1, 3, 9, 27, 81). For each byte take q = (byte × pow3[n]) mod 256 (as an 8-bit product) and the digit is xi = (q × 3) >> 8 (0, 1 or 2); the value is (xi − 1) × d. Order: for the first 32 bytes (the part of qs up to the largest multiple of 32, 32 here), for each n = 0 to 4, all 32 bytes in order; then for the remaining 16 bytes n = 0 to 4 over the 16; then the 4 bytes of qh for n = 0 to 3.
TQ2_0. d binary16; for each 32-byte group of qs, for l = 0 to 3 (bit offset 2l), all 32 bytes: y = (((qs >> 2l) & 3) − 1) × d.

i-quants

These look the values up in tables of ggml-common.h. kmask_iq2xs is {1, 2, 4, 8, 16, 32, 64, 128}. ksigns_iq2xs has 128 entries and entry i is the 7-bit number i with bit 7 set so that the whole byte has an even number of set bits (the values begin 0, 129, 130, 3, 132, 5, 6, 135, ... and end 126, 255). The grid tables hold the values packed in integers: iq2xxs_grid 256 entries of 64 bits, iq2xs_grid 512 and iq2s_grid 1024 entries of 64 bits (eight unsigned bytes each, read byte by byte in memory order, little-endian); iq3xxs_grid 256 and iq3s_grid 512 entries of 32 bits (four unsigned bytes); iq1s_grid 2048 entries of 64 bits (eight signed bytes). The tables' contents are fixed numbers in ggml-common.h; all 4,608 values are listed on the quantisation grids page.

IQ2_XXS: d binary16; per 32-value group ib32, read two 32-bit words aux32[0..1] from qs + 4·ib32 (qs viewed as bytes; the layout is 4 index bytes then a 32-bit word). db = d × (0.5 + (aux32[1] >> 28)) × 0.25. For l = 0 to 3: the grid row is iq2xxs_grid[aux8[l]], the sign byte is ksigns_iq2xs[(aux32[1] >> 7l) & 127], and value j (0 to 7) is db × grid[j] × (signs & kmask_iq2xs[j] ? −1 : +1).
IQ2_XS: qs is 16-bit words. Per group: db[0] = d × (0.5 + (scales[ib32] & 0xF)) × 0.25, db[1] = d × (0.5 + (scales[ib32] >> 4)) × 0.25; for l = 0 to 3, word w = qs[4·ib32 + l]: grid row iq2xs_grid[w & 511], signs ksigns_iq2xs[w >> 9], scale db[l/2]; value as IQ2_XXS.
IQ2_S: qs bytes (the first 32 are grid low bytes), qh, and the sign bytes are the next 32 bytes of qs (signs = qs + 32 in a 64-byte qs). Scales as IQ2_XS. For l = 0 to 3 in each group: grid row iq2s_grid[qs[l] | ((qh[ib32] << (8 − 2l)) & 0x300)]; sign byte signs[l] (not looked up); value dl × grid[j] × (signs[l] & kmask_iq2xs[j] ? −1 : +1); after each group qs and signs advance by 4.
IQ3_XXS: d binary16; qs (64 bytes of grid indices, 8 per 32-value group) followed by 32 bytes of scales and signs (a 32-bit word aux32 per group). db = d × (0.5 + (aux32 >> 28)) × 0.5; for l = 0 to 3: sign byte ksigns_iq2xs[(aux32 >> 7l) & 127], grid rows iq3xxs_grid[qs[2l]] and iq3xxs_grid[qs[2l+1]] giving values 0 to 3 and 4 to 7; value db × grid × sign with the sign bits kmask_iq2xs[j] (j 0 to 7 across the two rows).
IQ3_S: d binary16; qs, qh, signs, scales. Two 32-value groups per step: db1 = d × (1 + 2 × (scales[ib32/2] & 0xF)) and db2 = d × (1 + 2 × (scales[ib32/2] >> 4)). For the first group l = 0 to 3: grid rows iq3s_grid[qs[2l] | ((qh[0] << (8 − 2l)) & 256)] and iq3s_grid[qs[2l+1] | ((qh[0] << (7 − 2l)) & 256)], sign byte signs[l]; value db1 × grid[j] × sign (rows give j 0 to 3 and 4 to 7). Then qs += 8, signs += 4; the second group uses qh[1] and db2; then qh += 2, qs += 8, signs += 4.
IQ1_S: d binary16, qs bytes, qh 16-bit words, IQ1S_DELTA = 0.125. Per 32-value group ib: dl = d × (2 × ((qh[ib] >> 12) & 7) + 1); delta = −0.125 if bit 15 of qh[ib] is set, else +0.125; for l = 0 to 3: grid row iq1s_grid[qs[l] | (((qh[ib] >> 3l) & 7) << 8)] and value dl × (grid[j] + delta), j 0 to 7; qs += 4 per group.
IQ1_M: no separate scale field. The 16-bit scale d (binary16) is assembled from the four 16-bit words sc[0..3] of scales: u16 = (sc[0] >> 12) | ((sc[1] >> 8) & 0x00F0) | ((sc[2] >> 4) & 0x0F00) | (sc[3] & 0xF000). Per 32-value group ib: dl1 = d × (2 × ((sc[ib/2] >> (6·(ib mod 2) + 0)) & 7) + 1) and dl2 = d × (2 × ((sc[ib/2] >> (6·(ib mod 2) + 3)) & 7) + 1); grid indices idx[0] = qs[0] | ((qh[0] << 8) & 0x700), idx[1] = qs[1] | ((qh[0] << 4) & 0x700), idx[2] = qs[2] | ((qh[1] << 8) & 0x700), idx[3] = qs[3] | ((qh[1] << 4) & 0x700); deltas ±0.125 from bits qh[0] & 0x08, qh[0] & 0x80, qh[1] & 0x08, qh[1] & 0x80 (set means negative); values dl1 × (grid + delta) for the first two rows and dl2 × (grid + delta) for the last two; then qs += 4, qh += 2.
IQ4_NL (32 values): kvalues_iq4nl = {−127, −104, −83, −65, −49, −35, −22, −10, 1, 13, 25, 38, 53, 69, 89, 113}; y[j] = d × kvalues_iq4nl[qs[j] & 0xF] and y[j+16] = d × kvalues_iq4nl[qs[j] >> 4].
IQ4_XS (256 values): per 32-value group ib: 6-bit scale ls = ((scales_l[ib/2] >> 4·(ib mod 2)) & 0xF) | (((scales_h >> 2ib) & 3) << 4), dl = d × (ls − 32); y[j] = dl × kvalues_iq4nl[qs[j] & 0xF] and y[j+16] = dl × kvalues_iq4nl[qs[j] >> 4] with qs advancing 16 bytes per group.

FP4 and FP8 types

kvalues_mxfp4 (the same table is kvalues_fp4) is {0, 1, 2, 3, 4, 6, 8, 12, 0, −1, −2, −3, −4, −6, −8, −12}: the E2M1 values doubled (index bit 3 is the sign).
MXFP4 (32 values): d = ggml_e8m0_to_fp32_half(e): for e < 2 the 32-bit pattern 0x00200000 << e, otherwise the pattern (e − 1) << 23, read as a float (that is 2^e−128, so it includes the factor 1/2 that undoes the doubled table). y[j] = d × kvalues_mxfp4[qs[j] & 0xF] and y[j+16] = d × kvalues_mxfp4[qs[j] >> 4].
NVFP4 (64 values): four 16-value sub-blocks s, each with scale d[s] decoded by ggml_ue4m3_to_fp32; y[j] = d × kvalues_mxfp4[qs[8s+j] & 0xF] and y[j+8] = d × kvalues_mxfp4[qs[8s+j] >> 4] within the sub-block, j 0 to 7. ggml_ue4m3_to_fp32(x): 0 if x is 0 or 0x7F; exp = (x >> 3) & 0xF, man = x & 7; if exp is 0 the value is man × 2⁻⁹, otherwise (1 + man/8) × 2^exp−7; the result is multiplied by 0.5 (to match the doubled table).
F8_E4M3 (one byte per value): ggml_f8_e4m3_to_fp32(x): NaN if x & 0x7F is 0x7F; exp = (x >> 3) & 0xF, man = x & 7; magnitude man × 2⁻⁹ if exp is 0, else (1 + man/8) × 2^exp−7; negative if bit 7 is set (OCP FP8 revision 1.1).