After the metadata come
tensor_count tensor infos, zero padding to the alignment, and the tensor
data. This page follows
docs/gguf.md;
the types and block sizes come from ggml’s
include/ggml.h,
src/ggml-common.h, src/ggml.c and
src/ggml-quants.c, all read 2026-11-12.
Tensor info
| Field | Type | Meaning |
|---|---|---|
name | string | At most 64 bytes, such as blk.0.attn_q.weight. |
n_dimensions | uint32 | At most 4. |
dimensions | uint64 each | Sizes, as ggml’s ne: the first is the fastest-changing (its stride is the element size), so the order is the reverse of NumPy’s default. |
type | uint32 | ggml type. |
offset | uint64 | Where the tensor’s data starts, counted from the start of the tensor data; a multiple of the alignment. |
The tensor data starts at the first multiple of general.alignment (32
by default) after the tensor infos, and tensors are padded with zeros to the
alignment. A file can therefore be memory-mapped and used in place.
Tensor types
| Value | Type | Block (values, bytes) |
|---|---|---|
| 0 | F32 | 1, 4 |
| 1 | F16 | 1, 2 |
| 2 | Q4_0: 4-bit, one F16 scale | 32, 18 |
| 3 | Q4_1: 4-bit, F16 scale and minimum | 32, 20 |
| 4, 5 | Q4_2, Q4_3: support removed | – |
| 6 | Q5_0 | 32, 22 |
| 7 | Q5_1 | 32, 24 |
| 8 | Q8_0 | 32, 34 |
| 9 | Q8_1 | 32, 36 |
| 10 | Q2_K: “k-quant”, a super-block of 16 blocks of 16 values | 256, 84 |
| 11 | Q3_K | 256, 110 |
| 12 | Q4_K | 256, 144 |
| 13 | Q5_K | 256, 176 |
| 14 | Q6_K | 256, 210 |
| 15 | Q8_K (only used for intermediate quantisation and dot products) | 256, 292 |
| 16 | IQ2_XXS: “i-quant”, indexes into fixed grid tables | 256, 66 |
| 17 | IQ2_XS | 256, 74 |
| 18 | IQ3_XXS | 256, 98 |
| 19 | IQ1_S | 256, 50 |
| 20 | IQ4_NL | 32, 18 |
| 21 | IQ3_S | 256, 110 |
| 22 | IQ2_S | 256, 82 |
| 23 | IQ4_XS | 256, 136 |
| 24–27 | I8, I16, I32, I64 | 1, 1/2/4/8 |
| 28 | F64 | 1, 8 |
| 29 | IQ1_M (no separate scale field) | 256, 56 |
| 30 | BF16 | 1, 2 |
| 31–33 | Q4_0_4_4, Q4_0_4_8, Q4_0_8_8: removed from GGUF files | – |
| 34, 35 | TQ1_0, TQ2_0: ternary quantisation | 256, 54; 256, 66 |
| 36–38 | IQ4_NL_4_4, IQ4_NL_4_8, IQ4_NL_8_8: removed | – |
| 39 | MXFP4: an E8M0 scale byte and 4-bit values | 32, 17 |
| 40 | NVFP4: four sub-block scales and 4-bit values | 64, 36 |
| 41 | Q1_0 | 128, 18 |
| 42 | Q2_0 | 64, 18 |
| 43 | F8_E4M3 | 1, 1 |
A Q4_0 block is an F16 scale d and 16 bytes of 4-bit values; value
i is d × (qi − 8), with the low nibbles holding
values 0–15 and the high nibbles 16–31. The other formats are defined by their C
structures and dequantisation code in ggml; the GGUF document does not describe
them.