apidays LIVE Helsinki - Implementing OpenAPI and GraphQL Services with gRPC by Tim Burks

Implementing OpenAPI and
GraphQL Services with gRPC
Tim Burks (timburks@google.com, @timburks)

1. API technologies: OpenAPI, GraphQL, gRPC
2. Why implement your API with gRPC?
3. Implementing OpenAPI services with gRPC
4. Implementing GraphQL services with gRPC
Topics

1
API Technologies:
OpenAPI, GraphQL,
gRPC

Five Questions about API Systems
1. What problem is this API system addressing?
2. How are APIs called?
3. How are APIs built?
4. What’s great about this?
5. What’s challenging about this?

What problem is OpenAPI addressing?
“The OpenAPI Specification, originally known
as the Swagger Specification, is a specification
for machine-readable interface files for
describing, producing, consuming, and
visualizing RESTful web services.”
Wikipedia

How an OpenAPI service is called:
You might call it from your browser.
In your applications, you call it from code.
openapi-generator
autorest
APIMatic
SwagGen (Swift)
Paperclip (Rust)

How an OpenAPI service is built:
🤞 Code-first
Handwritten implementations.
Handwritten API descriptions.
⚙ Code-first (better)
Handwritten implementations. API
descriptions generated from code.
📐 Spec-first
Handwritten API descriptions.
Implementations generated from API descriptions.

What’s great about OpenAPI:
● Standards and tools make it easy to get
started.
○ HTTP
○ JSON/YAML
● Very popular - large community.

What’s challenging about OpenAPI:
“Unfortunately, many of the APIs that claim to be
RESTful layer a lot of proprietary concepts on top
of HTTP. Essentially, they invent their own API and
use HTTP as a lower-level transport layer, rather
than using HTTP directly as it was designed. In fact,
there’s so much variability in the way that
people use the term REST in the context of APIs
that it’s difficult to know what they mean by it
unless you know them well.”
Martin Nally

What problem is GraphQL addressing?
“GraphQL is a query language for APIs and a
runtime for fulfilling those queries with your
existing data. GraphQL provides a complete
and understandable description of the data in
your API, gives clients the power to ask for
exactly what they need and nothing more,
makes it easier to evolve APIs over time, and
enables powerful developer tools.”
graphql.org

How a GraphQL service is called:
“With express-graphql, you can just send an
HTTP POST request to the endpoint you
mounted your GraphQL server on, passing the
GraphQL query as the query field in a JSON
payload.”
https://graphql.org/graphql-js/graphql-clients/
Also GraphiQL!

How a GraphQL service is built:
● Code-first, schema extracted from code
● Schema-first - code generated from SDL
● Generated from underlying APIs
○ Federated from other GraphQL APIs
○ OpenAPI-to-GraphQL
○ rejoiner

What’s great about GraphQL:
● Simple client-side interface
● Optimizes for client performance
● Layered on familiar technology
● Sharable BFF

What’s challenging about GraphQL:
● Complex implementation
● Performance challenges
● Versioning
● Schema design
Public GraphQL APIs are rare.

What problem is gRPC addressing?
“gRPC is a modern open source high
performance RPC framework that can run in
any environment. It can efficiently connect
services in and across data centers with
pluggable support for load balancing, tracing,
health checking and authentication. It is also
applicable in last mile of distributed computing
to connect devices, mobile applications and
browsers to backend services.”
grpc.io

How a gRPC service is called:
ALWAYS specification-first.
Clients are generated from API
descriptions written in the Protocol Buffer
language.

How a gRPC service is built:
ALWAYS specification-first.
Service code is generated. Typically this
generated code declares an interface that is
implemented in language-native data
structures and patterns.

What’s great about gRPC:
So much is built-in.
High performance.
Cross-language compatibility.
Potential for very nice client integration.

What’s challenging about gRPC:
Some learning and tooling is required!

2
Why implement
your API with
gRPC?

gRPC APIs have a native language.
Protocol Buffers

Protocol Buffers is a language-neutral, platform-neutral, extensible
mechanism for serializing structured data.

“Protocol Buffers” means several things
1. A serialization mechanism
2. An interface description language
3. A methodology

Protocol Buffer Serialization
A message is just a stream of bytes
[field_number<<3 + wire_type] [length if necessary][data]...
$ hexdump /tmp/request.bin
0000000 0a 05 68 65 6c 6c 6f
0a is “0000 1010”, so
field_number = 1 and wire_type = 2

Protocol Buffer IDL
message EchoMessage {
string text = 1;
// other fields...
}

Protocol Buffer Methodology
% protoc echo.proto --go_out=.
# This runs the Protocol Buffer Compiler
# (https://github.com/protocolbuffers/protobuf/releases)
#
# with a plugin called protoc-gen-go
#
# The plugin generates a Go source file that implements
# the data structures defined in echo.proto and code
# for reading and writing them as serialized bytes.

gRPC scales to many programming languages.
Java
Service
Python
Service
Go
Service
C++
Service
gRPC
Server
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Server
gRPC
Server
gRPC
Server
gRPC
Stub

apidays LIVE Helsinki - Implementing OpenAPI and GraphQL Services with gRPC by Tim Burks

gRPC scales to distributed systems.

gRPC supports Generated API Clients.

GAPICs: Trivial Getting-Started Experience
GAPIC client:
Channel channel =
NettyChannelBuilder.forAddress(API_ENTRY, API_PORT).build();
List<ClientInterceptor> interceptors = new ArrayList<>();
GoogleCredentials credentials =
GoogleCredentials.getApplicationDefault();
interceptors.add(
ChannelFactory.credentialInterceptor(credentials));
LoggingService stub =
LoggingServiceGrpc.newBlockingStub(channel, interceptors);
gRPC:
LoggingClient client = LoggingClient.create();

GAPICs: Flattening, Automatic Retry
GAPIC client:
LoggingService stub = ...;
// Note: this only does a simple retry, exponential backoff
// is more complex
DeleteLogRequest request = DeleteLogRequest.newBuilder()
.setLogName(LOG_NAME)
.build();
for (int i = 0; i < MAX_RETRY; i++) {
try {
stub.deleteLog(request);
} catch (RpcException e) {
if (i == MAX_RETRY) throw e;
}
}
gRPC:
client.deleteLog(LOG_NAME);

GAPICs: Natural Pagination
GAPIC client:
LoggingService stub = ...;
ListLogsRequest request = ListLogsRequest.newBuilder()
.setParentName("projects/" + PROJECT_ID)
.build();
while (true) {
ListLogsResponse response = stub.listLogs(request);
for (Log log : response.getLogsList()) {
System.out.printf("Log: %s%n", log.getDisplayName());
}
String nextPageToken = response.getNextPageToken();
if (nextPageToken != null) {
request = ListLogsRequest.newBuilder()
.setPageToken(nextPageToken).build();
} else {
break;
}
}
gRPC:
ParentNameOneOf parent =
ParentNameOneOf.from(ProjectName.create(PROJECT_ID));
for (Log log : client.listLogs(PARENT).iterateAll()) {
System.out.printf("Log: %s%n", log.getDisplayName());
}

GAPICs: Long-Running Operations
SpeechClient speechClient = SpeechClient.create();
OperationFuture<LongRunningRecognizeResponse> recognizeOperation =
speechClient.longRunningRecognizeAsync(config, audio);
…
LongRunningRecognizeResponse response = recognizeOperation.get();
● Java: OperationFuture
○ Polls the service until the LRO is done
○ Provides metadata as the LRO is in progress
○ Provides the final result

GAPICs: Resource Name Types
GAPIC client:
PublisherService stub = ...;
CreateTopicRequest request = CreateTopicRequest.newBuilder()
.setTopic("projects/my-project/topics/my-topic")
.build();
Topic response = stub.createTopic(request);
gRPC:
TopicAdminClient topicAdminClient = TopicAdminClient.create();
TopicName name = TopicName.create("my-project", "my-topic");
Topic response = topicAdminClient.createTopic(name);

GAPIC Project Architecture (at github.com/googleapis)
gapic-generator-python
Wanted: Swift, Rust, Dart...
gapic-generator
(java, php, python, go,
csharp, node, ruby)
gapic-generator-ruby
gapic-generator-go
gapic-generator-typescript
gapic-generator-csharp
One API
description
(Annotated
.proto ﬁles)
Many
language-
speciﬁc
client
libraries

“Showcase” API verification

protoc-gen-go_cli
A protoc plugin that generates
CLIs in Go for APIs described
with protocol buffers.
gRPC supports Generated Command-Line Interfaces.

gRPC-Web Supports Browser Apps.
● Call gRPC from Browsers (Javascript, Dart)
● Proxy with Envoy
● Third party implementations
(github.com/improbable-eng/grpc-web)
gRPC Supports Reflection.
● Protocol Discussion
● Go implementation (example)
● Specification (protos)

gRPC works with API Management.

3
Implementing
OpenAPI Services
with gRPC

Build a gRPC service, get a free REST API!
Originally defined in google/api/http.proto
Now documented in AIP-127: HTTP and gRPC Transcoding

service Display {
rpc CreateKiosk(Kiosk) returns (Kiosk) {
option (google.api.http) = {post:"/v1/kiosks"};
}
rpc GetKiosk(GetKioskRequest) returns (Kiosk) {
option (google.api.http) =
{get: "/v1/kiosks/{id}" };
}
rpc DeleteKiosk(DeleteKioskRequest)
returns (google.protobuf.Empty) {
option (google.api.http) =
{ delete: "/v1/signs/{id}" };
}
[...]
}
Add REST annotations to your service
gRPC definitions with HTTP annotations
service Display {
rpc CreateKiosk(Kiosk) returns (Kiosk) {
}
rpc GetKiosk(GetKioskRequest) returns(Kiosk) {
}
rpc DeleteKiosk(DeleteKioskRequest)
returns (google.protobuf.Empty) {
}
[...]
}
gRPC definitions

# Call the CreateKiosk method
$ curl -d '{"name":"HTTP Kiosk", "size": { width: 1080, height: 720 } }'
localhost:8082/v1/kiosks?key=AIzaSy[...]bBo
# Call the GetKiosk method
$ curl localhost:8082/v1/kiosks/1?key=AIzaSy[...]bBo
Now you can use HTTP+JSON!

gnostic-grpc
● 6 useful things I learned from GSoC
● How to build a REST API with gRPC and get
the best of two worlds
● gnostic-grpc (end-to-end example)
● Envoy gRPC-JSON transcoding

4
Implementing
GraphQL Services
with gRPC

Code-first or Schema-first?
● Evolving the Graph (Jon Wong, Coursera)
● Hard-Learned GraphQL Lessons: Based on
a True Story (Natalie Qabazard & Aditi
Garg, Trulia)
● The Problems of "Schema-First" GraphQL
Server Development (Nikolas Burk, Prisma)

Code-first GraphQL with Rejoiner

Medallion (Rejoiner demo by Danny Baggett)

Sample query (Rejoiner + Medallion)
{
events {
id
name
olympics {
year
season
}
olympians {
id
gender
last
first
}
}
}

Sample query (Rejoiner + Medallion)
{
getEvents(input: {year: 2018, season: "WINTER"}) {
id
name
}
}

Let’s hand-wrap a gRPC service (graphql-showcase)

Impressions / Project Ideas
gRPC has a lot of API information in the .proto files and we
can extend that with annotations.
There are some gaps between GraphQL assumptions and
gRPC practices that we would want to smooth with
configuration or conventions.
We could generate a GraphQL interface directly from gRPC
reflection.
Can we define a standard for gRPC-GraphQL transcoding?

apidays LIVE Helsinki - Implementing OpenAPI and GraphQL Services with gRPC by Tim Burks

More Related Content

What's hot (20)

Similar to apidays LIVE Helsinki - Implementing OpenAPI and GraphQL Services with gRPC by Tim Burks (20)

More from apidays (20)

Recently uploaded (20)

apidays LIVE Helsinki - Implementing OpenAPI and GraphQL Services with gRPC by Tim Burks