gRPC and Protocol Buffers

REST over JSON is the right default between a browser and a backend and between teams who negotiate a contract in a document. Between two services owned by the same organisation, calling each other thousands of times a second, its costs start to show: a JSON body that is mostly field names, a client written by hand per language, a contract that lives in a wiki. gRPC is the answer to those three: a binary encoding with a schema, generated clients and servers, and the schema as the contract — with a set of rules for changing it that you must actually follow.

#Protocol Buffers: the schema and the wire

A .proto file declares messages and their fields, each with a number that is the field's identity on the wire:

syntax = "proto3";
package orders.v1;
option java_multiple_files = true;
option java_package = "in.code10x.orders.v1";
 
message PlaceOrderRequest {
  string customer_id = 1;
  repeated Line lines = 2;
  optional string coupon = 3;           // proto3 optional: presence is tracked
}
message Line { string sku = 1; int32 qty = 2; }
message Order { int64 id = 1; int64 total_paise = 2; Status status = 3; }
enum Status { STATUS_UNSPECIFIED = 0; PAID = 1; SHIPPED = 2; }

On the wire a message is a sequence of (field number, wire type, value) triples — no names, no whitespace, varint-packed integers — which is why a protobuf payload is typically a third to a tenth of the equivalent JSON and parses without a tokenizer. The rules that follow from "the number is the identity" are the compatibility rules at the end of this lesson. Two more things the file says: every field has a default (empty string, zero, the first enum value) and proto3 does not distinguish "unset" from "default" unless the field is optional; and the zero enum value should be UNSPECIFIED, so that an old client sending nothing does not accidentally mean PAID.

protoc with the Java plugin (the protobuf-maven-plugin runs it in the build) generates immutable message classes with builders, a parser, and — for the services below — the stubs.

#Services and the four kinds of call

service Orders {
  rpc Place(PlaceOrderRequest) returns (Order);                         // unary
  rpc Watch(WatchRequest) returns (stream OrderEvent);                  // server streaming
  rpc Import(stream Order) returns (ImportSummary);                     // client streaming
  rpc Chat(stream Message) returns (stream Message);                    // bidirectional
}

Unary is a request and a response, the REST call you already know. Server streaming is one request and many responses over one HTTP/2 stream — a subscription to order events, a large export delivered in pages the server chooses. Client streaming is the reverse — upload in chunks, get a summary. Bidirectional is both, independently — rare in backends, common in anything interactive. gRPC runs over HTTP/2, so a stream is a multiplexed stream on one connection, and a client can hold one connection to a service for its lifetime.

In Java the server is a class extending the generated OrdersGrpc.OrdersImplBase, registered with a Server (or with Spring through grpc-spring-boot-starter); the client is a stub — blocking, async, or future — over a ManagedChannel:

ManagedChannel channel = ManagedChannelBuilder.forTarget("dns:///orders.internal:9090").build();   // one, shared, long-lived
OrdersGrpc.OrdersBlockingStub orders = OrdersGrpc.newBlockingStub(channel);
Order order = orders.withDeadlineAfter(2, SECONDS).place(request);

The channel is expensive and pooled internally; create one per target for the life of the application, as with an HTTP client. It also does client-side load balancing across the addresses the target resolves to — which is why dns:/// matters, and why, in Kubernetes, a headless service (one A record per pod) load-balances and a ClusterIP service (one virtual IP, one long-lived HTTP/2 connection) sends everything to one pod.

#Deadlines and errors

A gRPC call has a deadline, not a timeout: an absolute point in time, set by the caller, propagated to every downstream call the server makes on its behalf, so that a request with two seconds left arrives at the third service with whatever remains. Set one on every call — the default is none, which is the retry-storm lesson's "a call with no timeout is a thread you have lost" — and check Context.current().isCancelled() in long server work, because a cancelled client stops nothing on its own.

Errors are a status code from a fixed set (INVALID_ARGUMENT, NOT_FOUND, ALREADY_EXISTS, FAILED_PRECONDITION, UNAVAILABLE, DEADLINE_EXCEEDED, UNAUTHENTICATED, PERMISSION_DENIED, INTERNAL, a few more) with a message, and optionally typed error details (google.rpc.BadRequest with field violations) attached as trailers. The mapping to think in: INVALID_ARGUMENT is 400, NOT_FOUND 404, FAILED_PRECONDITION 409-ish, UNAVAILABLE 503 and the one a client may retry, INTERNAL 500 and the one it should not. Throw Status.NOT_FOUND.withDescription("order 42").asRuntimeException() on the server; catch StatusRuntimeException and switch on getStatus().getCode() on the client. Never put the stack trace in the description; it crosses the wire.

Interceptors are the filter chain: authentication (a token in metadata, validated once per call), deadline enforcement, logging, metrics, tracing propagation — grpc-java ships an OpenTelemetry interceptor, and the observability course's trace context crosses a gRPC hop the same way it crosses HTTP.

#Evolving a contract

The rules are few and absolute, and a linter (buf breaking) enforces them in the pipeline:

• Never reuse or renumber a field. The number is the identity; a reused number silently decodes old data as the new field. Mark removed fields reserved 4, 7; so nobody can.
• Adding a field is safe. Old readers skip unknown numbers; new readers see the default from old writers. Design the default to mean "absent" (the UNSPECIFIED zero enum).
• Renaming is free on the wire and breaks only generated code — and JSON mapping, if you use it, so treat names as fixed too.
• Changing a type is breaking except within a small set of compatible pairs (int32/int64/uint32/bool share a wire type); do not rely on it.
• required does not exist in proto3, on purpose: a required field can never be removed. Validate in the handler.
• Version the package (orders.v1) and make a v2 for the changes the rules forbid, running both while clients migrate — the REST versioning lesson, with the version in the package name.

For everything else — the schema registry, compatibility modes and the Avro comparison — the Kafka course's schema-evolution lesson applies unchanged: protobuf on a topic follows the same rules as protobuf on an RPC.