Most numbers in a Protocol Buffers message are varints: integers stored in as few bytes as they need. Every tag is a varint, so is every length, and so is the value of every field with wire type 0 (VARINT).
Varints
A varint stores an unsigned integer seven bits to a byte, the lowest seven bits
first. The top bit of each byte (0x80) is set when another byte follows
and clear on the last byte. A varint takes 1 to 10 bytes: 64 bits need
⌈64 ÷ 7⌉ = 10 groups of seven, and the tenth byte can only be 0 or 1.
| Byte | Bits | Meaning |
|---|---|---|
| bit 7 | 0x80 | Continuation: 1 when another byte follows, 0 on the last byte. |
| bits 0 to 6 | 0x7F | Seven bits of the number. The first byte holds bits 0 to 6, the second bits 7 to 13, and so on. |
The Protocol Buffers documentation’s own example is 150, stored as
96 01. 0x96 is 1 0010110: the continuation bit
is set and the low seven bits are 0x16, 22. 0x01 has no
continuation bit and adds 1 × 128. 22 + 128 = 150.
To decode:
- Set the value and the shift to 0.
- Read a byte b.
- Add
(b & 0x7F) << shiftto the value. - If
b & 0x80is set, add 7 to the shift and go back to step 2; otherwise the varint ends here.
More than 10 bytes, or a tenth byte above 1, is an error, and the explorer reports it. The explorer keeps values that need more than 53 bits exact, as big integers.
Examples
| Value | Varint | Bytes |
|---|---|---|
| 0 | 00 | 1 |
| 1 | 01 | 1 |
| 127 | 7F | 1 |
| 128 | 80 01 | 2 |
| 150 | 96 01 | 2 |
| 240 | F0 01 | 2 |
| 1000 | E8 07 | 2 |
| 16,383 | FF 7F | 2 |
| 16,384 | 80 80 01 | 3 |
| 1,068,886,759 | E7 D5 D7 FD 03 | 5 |
| 264 − 1 | FF FF FF FF FF FF FF FF FF 01 | 10 |
The five-byte example is the length of the graph in a 1 GB ONNX model examined for
this site: E7 gives 103, D5 85 × 128, D7
87 × 1282, FD 125 × 1283 and 03
3 × 1284, which add up to 1,068,886,759.
Negative numbers: int32 and int64
Fields declared int32 or int64 store a negative number as
its 64-bit two’s complement. Its top bit is set, so it always takes all ten bytes. A
negative int32 is sign-extended to 64 bits first, so it takes ten bytes
too. Field 2 holding −2 is:
10 FE FF FF FF FF FF FF FF FF 0110 (field 2, VARINT), then −2 in ten bytes; read as an unsigned 64-bit number it is 18446744073709551614This example was built for this page. Numbers that are often negative should use
sint32 or sint64 instead.
Zigzag: sint32 and sint64
Fields declared sint32 or sint64 use zigzag encoding, which
interleaves negative and positive numbers so that small numbers of either sign stay
short. The result is stored as an ordinary varint.
| Number | 0 | −1 | 1 | −2 | 2 | −500 |
|---|---|---|---|---|---|---|
| Zigzag value | 0 | 1 | 2 | 3 | 4 | 999 (E7 07) |
- Encode a
sint32n as(n << 1) ^ (n >> 31), and asint64as(n << 1) ^ (n >> 63), where the shift on the right is arithmetic (it copies the sign bit). - Decode a zigzag value z as
(z >>> 1) ^ −(z & 1): half of it, with every bit flipped when it is odd.
−500 encodes as 999, which Protoscope writes as -500z.
Other varint types
| Type | Stored as |
|---|---|
uint32, uint64 | A plain varint. |
int32, int64 | A varint of the two’s complement; negative numbers take ten bytes. |
sint32, sint64 | A varint of the zigzag value. |
bool | A varint, 0 for false and 1 for true. |
enum | A varint of the enum’s int32 value; a negative value takes ten bytes. |
Without the schema
A varint does not say which of these types it is. The explorer shows a varint with no
schema as an unsigned 64-bit number, and its note gives the value read as
int64 and as sint64 too, and as a bool when it is 0 or 1.
With a schema, it shows the declared type.