The elements of a SequenceProto,
and the value of an OptionalProto, are most often tensors: TensorProto
messages of ONNX’s schema onnx.proto (onnx 1.23,
syntax = "proto2", package onnx), each nested in a LEN field
of the sequence. Sparse tensors are SparseTensorProto messages. This page
lists every field of both and of the messages they use. For the bytes of tags, varints
and lengths, see the Protocol Buffers
reference.
TensorProto
A tensor: its shape, its data type and its values, stored in
raw_data or in the typed field for its type. The table is in the order
of the schema, which is not the order of the numbers.
| Field | Name | Type | Wire type | Meaning |
|---|---|---|---|---|
| 1 | dims | repeated int64 | VARINT, one field per dimension | The size of each dimension, outermost first. None at all is a scalar, with one element. A dimension of 0 makes an empty tensor. |
| 2 | data_type | int32 | VARINT | The element type, a DataType. Every tensor must have one. |
| 3 | segment | Segment | LEN | For a large tensor stored in chunks: which elements this one holds. |
| 4 | float_data | repeated float, packed | LEN | Values of FLOAT tensors; COMPLEX64 as real, imaginary pairs. |
| 5 | int32_data | repeated int32, packed | LEN | Values of INT32, INT16, INT8, UINT16, UINT8, BOOL, FLOAT16, BFLOAT16, the float8, 4-bit, 2-bit and 6-bit types (see below). |
| 6 | string_data | repeated bytes | LEN, one field per string | Values of STRING tensors, each UTF-8, with no terminator and no byte-order mark. |
| 7 | int64_data | repeated int64, packed | LEN | Values of INT64 tensors. |
| 8 | name | string | LEN | The tensor’s name. Inside a sequence it may be left out; the sequence has a name of its own. |
| 12 | doc_string | string | LEN | Documentation, Markdown allowed. |
| 9 | raw_data | bytes | LEN | The values as fixed-width little-endian bytes, row-major, used instead of the typed fields (see below). Not allowed for STRING. |
| 13 | external_data | repeated StringStringEntryProto | LEN, one field per entry | Where the values are when stored in another file: keys location (required), offset, length and checksum (SHA-1). |
| 14 | data_location | DataLocation | VARINT | DEFAULT (0): the values are in this message; EXTERNAL (1): in external_data. |
| 10 | double_data | repeated double, packed | LEN | Values of DOUBLE tensors; COMPLEX128 as pairs. |
| 11 | uint64_data | repeated uint64, packed | LEN | Values of UINT32 and UINT64 tensors. |
| 16 | metadata_props | repeated StringStringEntryProto | LEN, one field per entry | Named metadata, added in IR version 10. |
Field 15 is not used. Fields in proto2 are written even when they hold their default,
and dims is not marked packed, so ONNX’s writers give each dimension its
own field. A reader must also accept dims as one packed LEN field, and
the typed fields one value at a time; the explorer reads both forms.
Where the values are
Exactly one place holds the values: raw_data, the typed field for the
data type, or external data. A tensor with no elements has none.
| Data type | Typed field | Values in the typed field |
|---|---|---|
| FLOAT | float_data | one float per element |
| COMPLEX64 | float_data | two floats per element: real, imaginary |
| DOUBLE | double_data | one double per element |
| COMPLEX128 | double_data | two doubles per element |
| INT64 | int64_data | one per element |
| UINT32, UINT64 | uint64_data | one per element |
| INT32, INT16, INT8, UINT16, UINT8, BOOL | int32_data | one per element |
| FLOAT16, BFLOAT16, the float8 types, the 6-bit floats | int32_data | one per element: the bit pattern as an integer |
| INT4, UINT4, FLOAT4E2M1 | int32_data | one per two elements |
| INT2, UINT2 | int32_data | one per four elements |
| STRING | string_data | one string per element |
Small types in int32_data
FLOAT16, BFLOAT16 and the float8 types are stored as their bit patterns, so
FLOAT16 0.5 (0x3800) is the integer 14336. Two 4-bit values share one
int32, the first in the low four bits. Four 2-bit values share one int32, the first
in the lowest two bits. A 6-bit float takes one int32 of its own, in bits 0 to 5.
raw_data
The elements one after another in row-major order (the last dimension varies fastest), each little-endian and fixed-width. BOOL takes one byte per element. Smaller types are packed into bytes:
- 4-bit types: two to a byte, the first element in the low nibble;
- 2-bit types: four to a byte, as
x0 | x1<<2 | x2<<4 | x3<<6; - 6-bit floats: four in three bytes, as a stream of bits starting at the lowest bit, padded at the end; n elements take ⌈6 × n ÷ 8⌉ bytes.
So raw_data is always ⌈elements × bits ÷ 8⌉ bytes long. The explorer
checks this length, and turns the file down if it does not match.
Examples from ONNX’s test data
A tensor inside a sequence
0A 08 73 65 71 75 65 6E 63 65 10 01 1A 26 08 04 10 07 4A 20 01 00 00 00 00 00 00 00 …sequence, elem_type TENSOR, and its first element: field 3, 38 bytesThe 38 bytes after 1A 26 are a TensorProto: dims [4]
(08 04), data_type 7, INT64 (10 07), and raw_data of 32
bytes (4A 20) holding 1, 2, 3 and 4 as little-endian 64-bit integers.
The tensor has no name.
FLOAT16 in raw_data
08 03 08 04 10 0A 42 06 6F 75 74 70 75 74 4A 18 64 38 …output; raw_data of 24 bytesThe first value, 64 38, is 0x3864: sign 0, exponent 14,
mantissa 100, so (1 + 100 ÷ 1024) × 214 − 15 = 0.548828. The float the
test started from was 0.5488135.
STRING in string_data
08 02 10 08 32 03 61 62 63 32 03 64 65 66 42 01 78abc and def; name xINT4 in int32_data
08 05 08 05 10 16 2A 11 88 01 A9 01 CB 01 …25 elements need 13 int32 values. The first, 88 01, is 136
(0x88): the low nibble 8 and the high nibble 8, both −8 as signed
4-bit numbers. The input was −9, saturated to the smallest INT4.
TensorProto.Segment
The range of elements that one chunk of a large tensor holds, for tensors split
across several messages. It is the type of TensorProto’s field 3,
segment, and this section is where both the field and the message are
described.
| Field | Name | Type | Wire type | Meaning |
|---|---|---|---|---|
| 1 | begin | int64 | VARINT | The first element this chunk holds. |
| 2 | end | int64 | VARINT | The element after the last. |
The explorer does not check the size of a tensor that has a segment.
StringStringEntryProto
A key and a value, both strings: ONNX’s stand-in for a map, used for
external_data and metadata_props.
| Field | Name | Type | Wire type | Meaning |
|---|---|---|---|---|
| 1 | key | string | LEN | The key. |
| 2 | value | string | LEN | The value. |
For external data the keys are:
| Key | Value |
|---|---|
location | Required. The path of the file holding the values, relative to the folder of the file that refers to it. |
offset | Where the values start in that file, in bytes, written as a decimal string. |
length | How many bytes they take, as a decimal string. |
checksum | The SHA-1 digest of the bytes. |
The bytes in the other file are laid out like raw_data. External data
belongs mostly to models, whose weights would otherwise pass the 2 GiB limit of a
message.
SparseTensorProto
A sparse tensor: only the values that are not zero, their positions, and the shape
of the whole tensor. It is the element type of a sequence whose
elem_type is SPARSE_TENSOR (2).
| Field | Name | Type | Wire type | Meaning |
|---|---|---|---|---|
| 1 | values | TensorProto | LEN | The values that are not the default, as a 1-D tensor of NNZ elements. It must have a name when the sparse tensor is a model’s initialiser. |
| 2 | indices | TensorProto | LEN | Their positions: a tensor of shape [NNZ, rank], or linear indices of shape [NNZ], in ascending order. |
| 3 | dims | repeated int64 | VARINT, or one packed LEN | The shape of the dense tensor. |