ONNX tensor data types

A tensor’s data_type is a number from the enum TensorProto.DataType. It decides how many bits each element takes in raw_data and which typed field holds the values otherwise. The parser for this format accepts 1 to 28; 0 is UNDEFINED and is not allowed in a tensor.

The data types

Value	Name	Bits	Typed field	Meaning
0	UNDEFINED	–	–	Not set.
1	FLOAT	32	`float_data`	IEEE 754 single.
2	UINT8	8	`int32_data`	Unsigned byte.
3	INT8	8	`int32_data`	Signed byte.
4	UINT16	16	`int32_data`	Unsigned 16-bit.
5	INT16	16	`int32_data`	Signed 16-bit.
6	INT32	32	`int32_data`	Signed 32-bit.
7	INT64	64	`int64_data`	Signed 64-bit.
8	STRING	–	`string_data`	Strings; never in `raw_data`.
9	BOOL	8	`int32_data`	One byte in `raw_data`, 0 or 1.
10	FLOAT16	16	`int32_data`	IEEE half.
11	DOUBLE	64	`double_data`	IEEE 754 double.
12	UINT32	32	`uint64_data`	Unsigned 32-bit.
13	UINT64	64	`uint64_data`	Unsigned 64-bit.
14	COMPLEX64	64	`float_data`	Two floats.
15	COMPLEX128	128	`double_data`	Two doubles.
16	BFLOAT16	16	`int32_data`	The top half of a float. IR 4.
17	FLOAT8E4M3FN	8	`int32_data`	8-bit float, 4 exponent bits. IR 9.
18	FLOAT8E4M3FNUZ	8	`int32_data`	The same, with no negative zero. IR 9.
19	FLOAT8E5M2	8	`int32_data`	8-bit float, 5 exponent bits. IR 9.
20	FLOAT8E5M2FNUZ	8	`int32_data`	The same, with no negative zero. IR 9.
21	UINT4	4	`int32_data`	Unsigned 4-bit. IR 10.
22	INT4	4	`int32_data`	Signed 4-bit. IR 10.
23	FLOAT4E2M1	4	`int32_data`	4-bit float. IR 11.
24	FLOAT8E8M0	8	`int32_data`	A power of two, for scales. IR 12.
25	UINT2	2	`int32_data`	Unsigned 2-bit. IR 13.
26	INT2	2	`int32_data`	Signed 2-bit. IR 13.
27	FLOAT6E2M3	6	`int32_data`	6-bit float, 2 exponent bits. IR 14.
28	FLOAT6E3M2	6	`int32_data`	6-bit float, 3 exponent bits. IR 14.

“IR 4” and so on is the ONNX IR version that added the type. Types are only ever added, so an older tensor file reads the same in every later version.

FLOAT16

An IEEE 754 half: 1 sign bit, 5 exponent bits and 10 mantissa bits, with a bias of 15. In raw_data each value is two bytes, little-endian; in int32_data each is its 16-bit pattern as an integer. 64 38 in raw_data is 0x3864, 0.548828.

BFLOAT16

The top 16 bits of a 32-bit float: 1 sign bit, 8 exponent bits and 7 mantissa bits. To read one, shift its pattern left 16 bits and read the result as a float. Stored like FLOAT16.

The float8 types

Each is one byte: a sign bit, then exponent and mantissa bits. When the exponent is 0 the value is subnormal: m ÷ 2^{mantissa bits} × 2^{1 − bias}.

Type	Sign, exponent, mantissa	Bias	Special values
FLOAT8E4M3FN	1, 4, 3	7	No infinity; only S.1111.111 is NaN. Largest 448.
FLOAT8E4M3FNUZ	1, 4, 3	8	No infinity, no negative zero; `0x80` is NaN.
FLOAT8E5M2	1, 5, 2	15	Like IEEE: exponent 31 is infinity (mantissa 0) or NaN.
FLOAT8E5M2FNUZ	1, 5, 2	16	No infinity, no negative zero; `0x80` is NaN.
FLOAT8E8M0	0, 8, 0	127	No sign: the value is 2^{e − 127}; `0xFF` is NaN. The scale type of the OCP Microscaling formats.

onnx.proto cites “FP8 Formats for Deep Learning” (arXiv 2209.05433) and “8-bit Numerical Formats For Deep Neural Networks” (arXiv 2206.02915) for these types. In raw_data each value is one byte; in int32_data each is its byte as an integer.

The 6-bit floats

FLOAT6E2M3 has 1 sign, 2 exponent and 3 mantissa bits; FLOAT6E3M2 has 1 sign, 3 exponent and 2 mantissa bits. Neither has infinity or NaN. They are the 6-bit formats of the OCP Microscaling specification. onnx.proto gives only the split of the bits; the biases the explorer uses, 1 for FLOAT6E2M3 and 3 for FLOAT6E3M2, are taken from the OCP specification and are an inference, not part of ONNX’s schema.

In raw_data, four values share three bytes as one stream of bits, lowest bit first, padded at the end: n values take ⌈6 × n ÷ 8⌉ bytes. In int32_data each value takes one integer, in bits 0 to 5.

The 4-bit types

UINT4 is 0 to 15, INT4 is −8 to 7 in two’s complement, and FLOAT4E2M1 is a float with 1 sign, 2 exponent and 1 mantissa bit and a bias of 1, whose magnitudes are 0, 0.5, 1, 1.5, 2, 3, 4 and 6. Two values share a byte in raw_data and an integer in int32_data, the first in the low four bits. An odd count leaves the last high nibble unused.

In ONNX’s test data, 88 01 in an INT4 tensor’s int32_data is 136, 0x88: two elements of 8, both −8.

The 2-bit types

UINT2 is 0 to 3 and INT2 is −2 to 1. Four values share a byte in raw_data as x0 | x1<<2 | x2<<4 | x3<<6, and share an integer in int32_data the same way.

Strings

A STRING tensor keeps each element as one string_data field: UTF-8, no terminator, no byte-order mark. STRING cannot use raw_data, and the explorer turns down a tensor that tries.

Complex numbers

COMPLEX64 is two floats, the real part then the imaginary part; COMPLEX128 is two doubles. In the typed fields that makes two values per element, in float_data and double_data; in raw_data, 8 or 16 bytes per element.

.PB File Format – Data types