.PB File Format – File conventions

No signature

A file holding one message is just the message’s bytes. There is no signature, no version and no type name, so a reader must know the schema from the file’s name or where it was found: a TensorFlow program knows that saved_model.pb is a SavedModel, and the files in ONNX’s test folders are tensors for the model beside them. A program given an unknown .pb file can only read it without a schema, as this site’s explorer does.

Extensions

Descriptor sets

protoc --descriptor_set_out=FILE (or -o FILE) writes a FileDescriptorSet: a Protocol Buffers message describing one or more .proto files, their messages, fields and types. Such files are often named .pb, .desc or .binpb. Sending a descriptor set with the data is how a schema can travel with its messages, which the documentation calls self-describing messages. The explorer reads a descriptor set as any other message, without a schema.

Several messages in one file

Messages cannot simply be written one after another, because “The Protobuf wire format is not self-delimiting”: a reader would see one message, the merge of them all. The usual way is to “write the size of each message before you write the message itself”, as a varint:

Java’s writeDelimitedTo and parseDelimitedFrom, and C++’s SerializeDelimitedToOstream (in delimited_message_util.h), read and write this form. Such a stream is not itself a message, and the explorer, reading it without a schema, takes the first size for a tag. Other formats, such as OpenStreetMap PBF, have framings of their own.

Text format

Text format is a readable form of a message: fields by name rather than number, one per line, nested messages in braces, and # comments.

name: "John Smith"
pet {
  kind: DOG
}

Optional header comments # proto-file: and # proto-message: name the schema. The specification calls itself a draft and warns that implementations differ. A text-format file needs its schema to be read at all, because the names must be turned back into numbers. The explorer turns down a text-format file named .pb, because it is printable text.

JSON

The ProtoJSON mapping writes a message as a JSON object:

• field names in lowerCamelCase, or the schema’s json_name;
• int64, uint64 and fixed64 values as decimal strings;
• bytes as base64;
• enums by name;
• fields at their default left out;
• maps as JSON objects;
• Any with an "@type" member;
• Timestamp as an RFC 3339 date and time, and Duration as seconds followed by s.