TensorFlow TensorProto, field by field

Tensors in a TensorFlow graph are TensorProto messages from tensor.proto, with shapes from tensor_shape.proto (syntax = "proto3", package tensorflow). They hold the values of Const nodes and any other attribute of type tensor. In a frozen graph every weight of the model is such a tensor, and their values make up most of the file. TensorFlow’s TensorProto is a different message from ONNX’s, with different numbers: here field 1 is the data type, in ONNX it is the dimensions.

TensorProto

A tensor: its data type, shape and values, either as raw bytes in tensor_content or in the typed field for its type.

Field	Name	Type	Wire type	Meaning
1	`dtype`	DataType	VARINT	The element type. The explorer requires one that is set and known.
2	`tensor_shape`	TensorShapeProto	LEN	The shape. Absent is a scalar.
3	`version_number`	int32	VARINT	The tensor encoding’s version; 0.
4	`tensor_content`	bytes	LEN	The values as raw bytes, little-endian, row-major: element size × element count bytes. Used instead of the typed fields.
13	`half_val`	repeated int32, packed	LEN	DT_HALF and DT_BFLOAT16 values, each 16-bit pattern in an int32.
5	`float_val`	repeated float, packed	LEN	DT_FLOAT values.
6	`double_val`	repeated double, packed	LEN	DT_DOUBLE values.
7	`int_val`	repeated int32, packed	LEN	DT_INT32, DT_INT16, DT_UINT16, DT_INT8 and DT_UINT8 values.
8	`string_val`	repeated bytes	LEN, one per value	DT_STRING values.
9	`scomplex_val`	repeated float, packed	LEN	DT_COMPLEX64 values: real and imaginary parts in turn.
10	`int64_val`	repeated int64, packed	LEN	DT_INT64 values.
11	`bool_val`	repeated bool, packed	LEN	DT_BOOL values.
12	`dcomplex_val`	repeated double, packed	LEN	DT_COMPLEX128 values, in pairs.
14	`resource_handle_val`	repeated ResourceHandleProto	LEN, one per value	DT_RESOURCE values.
15	`variant_val`	repeated VariantTensorDataProto	LEN, one per value	DT_VARIANT values.
16	`uint32_val`	repeated uint32, packed	LEN	DT_UINT32 values.
17	`uint64_val`	repeated uint64, packed	LEN	DT_UINT64 values.
18	`float8_val`	bytes	LEN	The float8 types’ values, one byte each.

When tensor_content is used with a fixed-width type, the explorer checks that its length is the element count times the element size, and shows the values. It shows the 4-bit, 2-bit and FLOAT4 types as plain bytes, because how TensorFlow packs them in tensor_content was not confirmed for this page.

Fewer values than the shape

A typed field may hold fewer values than the shape needs. TensorFlow then repeats the last value given to fill the rest, and with no values at all it fills with zeros (FromProtoField in TensorFlow’s tensor.cc). So a tensor of shape [1000] with float_val holding the one value 0 is a thousand zeros, stored in a few bytes. Constant tensors in frozen graphs use this.

08 01 12 05 12 03 08 E8 07 2A 04 00 00 00 00

dtype DT_FLOAT, shape [1000], float_val [0.0]: a thousand zeros (built for this page)

TensorShapeProto

A shape: its dimensions, outermost first, or unknown rank.

Field	Name	Type	Wire type	Meaning
2	`dim`	repeated Dim	LEN, one per dimension	The dimensions. None at all is a scalar.
3	`unknown_rank`	bool	VARINT	True when even the number of dimensions is not known; `dim` must then be empty.

Field 1 is not used.

TensorShapeProto.Dim

One dimension. It is the type of TensorShapeProto’s field 2, dim, and this section is where both the field and the message are described.

Field	Name	Type	Wire type	Meaning
1	`size`	int64	VARINT	The size; −1 when not known, which takes ten bytes.
2	`name`	string	LEN	An optional name.

In proto3 a size of 0 is not written, so a dimension of size 0 is an empty Dim message: 12 00.

ResourceHandleProto

A handle to a resource, such as a variable, held by a DT_RESOURCE tensor.

Field	Name	Type	Wire type	Meaning
1	`device`	string	LEN	The device holding it.
2	`container`	string	LEN	Its container.
3	`name`	string	LEN	Its name.
4	`hash_code`	uint64	VARINT	A hash of its type.
5	`maybe_type_name`	string	LEN	Its type’s name, for debugging.
6	`dtypes_and_shapes`	repeated DtypeAndShape	LEN, one per entry	The types and shapes it holds.

ResourceHandleProto.DtypeAndShape

Field	Name	Type	Wire type	Meaning
1	`dtype`	DataType	VARINT	A data type.
2	`shape`	TensorShapeProto	LEN	A shape.

VariantTensorDataProto

One DT_VARIANT value: an object of some other type, with its tensors.

Field	Name	Type	Wire type	Meaning
1	`type_name`	string	LEN	The type of the object.
2	`metadata`	bytes	LEN	Its parts that are not tensors.
3	`tensors`	repeated TensorProto	LEN, one per tensor	Its tensors.

.PB File Format – TensorProto