What I learned about APIs in my first year at Google

What I learned about APIs in my first year at Google
(Protocol Buffers and gRPC)
timburks@google.com

10 billion+
API callsevery second

I was an “indie” iOS developer

What I knew about APIs before I joined Google:
APIs are essential.
APIs must be tested to be sure they work.
APIs must be designed appropriately for their
applications.

Google Cloud Platform
The Fallacies of Distributed Computing
The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
https://blogs.oracle.com/jag/resource/Fallacies.html
Topology doesn't change
There is one administrator
Transport cost is zero
The network is homogeneous

There’s this wall.
Conway’s Law: “organizations which design systems... are constrained to produce
designs which are copies of the communication structures of these organizations.”

Cloud platforms can help
us tear down that wall.

When you do a lot of something, you want to pay
attention to it.
You develop a language for describing it.
You optimize it.
You standardize it.
You make it flexible, so it can adapt to changes.

Protocol buffers are a language-neutral, platform-neutral, extensible mechanism
for serializing structured data.
Implemented in many languages, sometimes many times...

“Protocol Buffers” means several things
1. A serialization mechanism
2. An interface description language
3. A methodology

Protocol Buffer Serialization
It’s just a stream of bytes
[field_number<<3 + wire_type] [length if necessary] [data]...
$ hexdump /tmp/request.pb
0000000 0a 05 68 65 6c 6c 6f
0a is “0000 1010”, so
field_number = 1
wire_type = 2

echo.proto
syntax = "proto3";
package echo;
service Echo {
rpc Get(EchoRequest) returns (EchoResponse) {}
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}
message EchoRequest {
string text = 1;
}
message EchoResponse {
string text = 1;
}

The Protocol Buffer methodology
$ protoc echo.proto --go_out=.
$ more echo.pb.go
// Code generated by protoc-gen-go.
// source: echo.proto
// DO NOT EDIT!
...

A remote procedure call (RPC) framework layered on HTTP/2
Enables client and server applications to communicate transparently
Client “calls” methods on a Server (which may be on a different machine) as if it
were a local service
Makes it easy to build heterogeneous distributed systems
Separate full implementations in C, Java, and Go.

Interoperability
Java
Service
Python
Service
GoLang
Service
C++
Service
gRPC
Service
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Service
gRPC
Service
gRPC
Service
gRPC
Stub

Unary (nonstreaming) API
syntax = “proto3”;
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
service HelloService {
rpc SayHello(HelloRequest) returns (HelloReply);
}
Unary API: Client sends a
request, server sends a
response

Streaming APIs - 1 (of 3) Client Streaming
message Latency {
string name = 1;
double val = 2;
}
message Histogram {
string name = 1
double p99 = 2;
double p999 = 3;
double avg = 4;
}
service MetricsService {
rpc ReportLateny(stream Latency) returns Histogram;
}
Client Streaming API: Client
sends multiple messages;
Server sends one response
(Note: Server may choose to send the response
before all the client messages are received)

Streaming APIs - 2 (of 3) Server Streaming
message StockRequest {
string stocksymbol = 1;
}
message StockResponse {
double price = 1;
}
service StockTickerService {
rpc GetTicker(StockRequest) returns (stream StockResponse);
}
Server Streaming API: Client
sends one message; Server
sends multiple messages

Streaming APIs - 3 (of 3) Bidirectional Streaming
message ChatMessage {
string msg = 1;
}
service ChatService {
rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}
Bidi Streaming API: Client and
Server can send multiple
messages to each other
(Note: Client and server can independently send
messages to each other i.e they do not have to wait
for receiving a client message before sending a
message)

Some gRPC Adopters

Netflix deprecated its RPC framework in favor of
gRPC:
“Our team has instead building an RPC solution on top
of gRPC. We are doing this transition for two main
reasons: multi-language support and better
extensibility/composability through request
interceptors. That’s our current plan moving forward.”
https://github.com/Netflix/ribbon
gRPC Adopters

We’re building Swift support for gRPC.

We are working to bring Protocol Buffers
and gRPC support into OpenAPI.

What I learned about APIs in my first year at Google

Recommended

More Related Content

What's hot (20)

Similar to What I learned about APIs in my first year at Google (20)

More from Tim Burks (9)

Recently uploaded (20)

What I learned about APIs in my first year at Google