CheatPress

Protocol Buffers Schema compiler

Version 36.2 · source

Protocol Buffers, Postcard, light preview Light · Postcard 102 × 152 mm · PDF Protocol Buffers, Postcard, dark preview Dark · Postcard 102 × 152 mm · PDF Protocol Buffers, Postcard, ink saver preview Ink saver · Postcard 102 × 152 mm · PDF
Protocol Buffers, A5, light preview Light · A5 148 × 210 mm · PDF Protocol Buffers, A5, dark preview Dark · A5 148 × 210 mm · PDF Protocol Buffers, A5, ink saver preview Ink saver · A5 148 × 210 mm · PDF
Protocol Buffers, A4, light preview Light · A4 210 × 297 mm · PDF Protocol Buffers, A4, dark preview Dark · A4 210 × 297 mm · PDF Protocol Buffers, A4, ink saver preview Ink saver · A4 210 × 297 mm · PDF
Protocol Buffers, A4 landscape, light preview Light · A4 landscape 297 × 210 mm · PDF Protocol Buffers, A4 landscape, dark preview Dark · A4 landscape 297 × 210 mm · PDF Protocol Buffers, A4 landscape, ink saver preview Ink saver · A4 landscape 297 × 210 mm · PDF
Protocol Buffers, A3, light preview Light · A3 297 × 420 mm · PDF Protocol Buffers, A3, dark preview Dark · A3 297 × 420 mm · PDF Protocol Buffers, A3, ink saver preview Ink saver · A3 297 × 420 mm · PDF
Protocol Buffers, A3 landscape, light preview Light · A3 landscape 420 × 297 mm · PDF Protocol Buffers, A3 landscape, dark preview Dark · A3 landscape 420 × 297 mm · PDF Protocol Buffers, A3 landscape, ink saver preview Ink saver · A3 landscape 420 × 297 mm · PDF
Protocol Buffers, Letter, light preview Light · Letter 8.5 × 11 in · PDF Protocol Buffers, Letter, dark preview Dark · Letter 8.5 × 11 in · PDF Protocol Buffers, Letter, ink saver preview Ink saver · Letter 8.5 × 11 in · PDF
Protocol Buffers, Tabloid, light preview Light · Tabloid 11 × 17 in · PDF Protocol Buffers, Tabloid, dark preview Dark · Tabloid 11 × 17 in · PDF Protocol Buffers, Tabloid, ink saver preview Ink saver · Tabloid 11 × 17 in · PDF

Start here

protoc --version
Check the compiler
protoc -I. --python_out=. <file>.proto
Generate Python code
protoc -I. --go_out=. <file>.proto
Generate Go code
protoc -I. --cpp_out=. <file>.proto
Generate C++ code
protoc -I. --descriptor_set_out=<f>.pb <file>.proto
Write a descriptor set
protoc -I. --encode=<Msg> <file>.proto
Text in, binary out
protoc -I. --decode=<Msg> <file>.proto
Binary in, text out
protoc --decode_raw
Dump binary with no schema

Concepts

  1. .proto file schema
  2. Generated code classes, stubs
  3. Binary wire data compact, tagged by field number

Invoke protoc

protoc --version
Print the compiler version
protoc -I <dir> <file>.proto
Add an import search path
protoc --proto_path=<dir> <file>.proto
Long form of -I
protoc --python_out=<dir> <file>.proto
Generate Python modules
protoc --cpp_out=<dir> <file>.proto
Generate C++ sources
protoc --java_out=<dir> <file>.proto
Generate Java classes
protoc --go_out=<dir> --go_opt=paths=source_relative <file>.proto
Generate Go (needs protoc-gen-go)
protoc --pyi_out=<dir> <file>.proto
Generate Python type stubs
protoc --<lang>_out=<opts>:<dir> <file>.proto
Pass options to a generator
Built-in generators take a comma-separated option list before the colon.
protoc --plugin=protoc-gen-<name>=<path> --<name>_out=<dir> <file>.proto
Use a plugin not on PATH
protoc @<argsfile>
Read arguments from a file
protoc --fatal_warnings <file>.proto
Fail on warnings
protoc --help
List all flags

Descriptors

protoc --descriptor_set_out=<f>.pb <file>.proto
Write a FileDescriptorSet
protoc --descriptor_set_out=<f>.pb --include_imports <file>.proto
Bundle imported files too
Tools such as grpcurl need imports included to load the set.
protoc --descriptor_set_out=<f>.pb --include_source_info <file>.proto
Keep comments and locations
protoc --descriptor_set_in=<f>.pb <Msg>
Load schemas from a descriptor set
protoc --dependency_out=<f>.d <file>.proto
Write a make dependency file

Encode and decode

protoc --decode_raw < <f>.bin
Print wire data without a schema
protoc -I. --decode=<pkg.Msg> <file>.proto < <f>.bin
Decode binary to text
protoc -I. --encode=<pkg.Msg> <file>.proto < <f>.txt > <f>.bin
Encode text format to binary
protoc --decode=<pkg.Msg> --descriptor_set_in=<f>.pb < <f>.bin
Decode using a descriptor set
protoc --deterministic_output --encode=<pkg.Msg> <file>.proto
Stable map ordering on encode

Messages and fields

syntax = "proto3";
First line of a proto3 file
package <name>;
Namespace for the file's types
import "<other>.proto";
Use types from another file
message <Name> { <type> <field> = <n>; }
Declare a message
optional <type> <field> = <n>;
Field with presence tracking
repeated <type> <field> = <n>;
A list of values
map<<key>, <value>> <field> = <n>;
Key-value map field
oneof <name> { <type> <a> = 1; <type> <b> = 2; }
At most one of these fields set
enum <Name> { <NAME>_UNSPECIFIED = 0; }
Declare an enum; first value must be 0
reserved 2, 15 to 20;
Block retired field numbers
reserved "<old_name>";
Block retired field names
message <Outer> { message <Inner> {} }
Nested message type
import public "<other>.proto";
Re-export an import
option allow_alias = true;
Allow duplicate enum numbers
option go_package = "<path>";
Set the Go import path
<type> <field> = <n> [deprecated = true];
Mark a field deprecated

Scalar types

Common scalars
Type Notes
int32 / int64 varint; poor for negatives
sint32 / sint64 zigzag; negatives are cheap
uint32 / uint64 unsigned varint
fixed32 / fixed64 fixed width, large values
float / double IEEE 754
bool true or false
string UTF-8 or ASCII text
bytes arbitrary byte data

Services

service <Name> { rpc <Method>(<Req>) returns (<Resp>); }
Declare a unary RPC
rpc <Method>(stream <Req>) returns (<Resp>);
Client streaming RPC
rpc <Method>(<Req>) returns (stream <Resp>);
Server streaming RPC
rpc <Method>(stream <Req>) returns (stream <Resp>);
Bidirectional streaming RPC

Compatibility

Safe or not
Change Safe?
Add a new field yes
Remove a field and reserve it yes
Rename a field wire yes, JSON no
Reuse a field number never
Change a field number never
int32 / int64 / uint / bool interchangeable, may truncate
string and bytes if valid UTF-8
Singular to repeated strings, bytes, messages only
Move a field into an existing oneof no

Recipes

Compile a tree into one directory
mkdir -p genprotoc -I proto --python_out=gen --pyi_out=gen proto/*.proto
Descriptor set for grpcurl
protoc -I proto --include_imports \  --descriptor_set_out=api.pb proto/api.protogrpcurl -protoset api.pb list
Inspect an unknown payload
# no schemaprotoc --decode_raw < payload.bin# with a schemaprotoc -I proto --decode=pkg.Msg proto/api.proto < payload.bin
Round trip text to binary and back
echo 'name: "Ada"' \  | protoc -I proto --encode=pkg.Msg proto/api.proto > m.binprotoc -I proto --decode=pkg.Msg proto/api.proto < m.bin