.PB File Format – Packed repeated fields

One field per element

In the plain form each element is a field of its own, with its own tag. Strings, bytes and messages are always written this way. ONNX writes a tensor’s dimensions like this; a 3 × 4 × 5 tensor’s dims (field 1) are:

08 03 08 04 08 05

An empty repeated field is simply absent.

Packed

Repeated numbers (any type stored as VARINT, I32 or I64) may instead be packed: one LEN field whose bytes are the values one after another, with no tags between them. Varints are written back to back; fixed-width values take 4 or 8 bytes each. The documentation’s example is field 4 holding 3, 270 and 86942:

Parsers must accept both forms for any repeated number field, whichever the schema says: “Protobuf parsers must be able to parse repeated fields that were compiled as packed as if they were not packed, and vice versa.” Several packed runs of the same field are joined into one list, in order.

Without the schema, a packed run looks like any other LEN value. The explorer reads it as packed varints only when it is neither text nor a message and the bytes read as whole varints to the end; see how the explorer guesses.

When fields are packed

ONNX’s schema is proto2 and marks its tensor data fields [packed = true], but not dims. So ONNX’s Python writes each dimension as its own field, as above, in every test file examined for this site, while the values in, say, int32_data are one packed run.