Build Great Networked APIs with Swift, OpenAPI, and gRPC

Build Great Networked APIs
with Swift, OpenAPI, and gRPC
Tim Burks, Google
Swift Cloud Workshop No. 2
September 30, 2017

Protocol Buffers
a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

Interface Builder for Data
message Person {
string name = 1;
int32 id = 2;
string email = 3;
message PhoneNumber {
string number = 1;
}
repeated PhoneNumber phone = 4;
}
Interface Builder: Developers specify
their interfaces using a special tool,
tooling compiles and integrates that into
their apps.
Protocol Buffers: Developers specify
their data structures using a special
language, tooling compiles and
integrates that into their apps.

Discovery Format Code Generators

github.com/google/google-api-go-client

github.com/google/google-api-python-client

Toolkit and the next generation of Google APIs

So far, so good.
REST (Discovery Format)
● 232 API descriptions at https://www.googleapis.com/discovery/v1/apis
● 10 generators listed at https://developers.google.com/api-client-library/
gRPC (Toolkit)
● 30+ gRPC-based APIs
● 7 target languages

OpenAPI 3.0
Industry standard format for describing for REST APIs
Originally designed for documentation, now with many other applications: API
Authoring, Validation, Documentation, Analysis, Search, Testing, Mocking,
Management, Code Generation

OpenAPI-based code generators
● swagger-codegen (Open source, Smartbear + community)
○ 70+ targets
○ First commit: July 6, 2011
○ Used by Lyft and Square to generate SDKs
● AutoRest (Open source, Microsoft)
● oas-nodegen (Open source, Capital One)
● APIMatic (Proprietary, APIMatic)
more?

Problems we’ve seen so far in open source code generators
● Missing or weak build system integration.
● Invalid service addresses (error in OpenAPI description).
● No auth.
● No documentation (not even a README!).
● No samples.
● No test harness.
● No assurance that the called service even works.
● Ugly generated code.
○ Machine-generated operation names.
○ Machine-generated struct names.
● Unwanted dependencies.

swagger-codegen often isn’t used “out of the box”:
“Generating client libraries involves customizing the provided language-specific
templates…
The amount of modification each template needs varies by language and we’re
looking forward to working with the Swagger Codegen community to share our
refinements.” Val Polouchkine, Lyft
“...Swagger Codegen is a pretty active project. If you don’t check in your
templates, things are gonna break unexpectedly because Swagger Codegen just
uses the latest and greatest templates that are out there. So if you don’t sort of
manually make sure that those things work, you’re gonna have an issue there.”
Tristan Sokol, Square

Code generation pipeline
API Description
Generatable
API Description
Language-
specific model
Generated
API support
code
render idiomatic API
support code according
to user preferences
(re. build systems,
dependencies)
verify model,
define structures and
entry points,
name everything
filter language-
specific reserved words,
(optionally) define file
structure for generated code

typical monolithic
code generation pipeline
API Description
(OpenAPI)
Generatable
API Description
(internal data
structures)
Language-
specific model
(internal data
structures)
Generated
API support
code
All-in-one repo and package

Problems with monolithic code generators
● Long build times: changing one target requires rebuilding everything.
● Long test times: new builds must be tested for every target language.
● For stability, teams may prefer to archive their own generator builds.
● Forks will abound.
● Quality is uneven.
● Versioning is hard.
● Complexity and potentially unfamiliar build systems deter contributors.

protoc
API Description
(.proto)
Generatable
API Description
(binary
FileDescriptors)
Language-
specific model
(internal data
structures)
Generated
API support
code
language target-
specific plugins
protoc

Why does protoc have a plug-in architecture?
● Fast build times: changing one target only requires rebuilding its plugin.
● Fast test times: new builds need only be tested for the affected targets.
● For stability, teams can archive their own protoc and plugin builds.
● New plugins can abound.
● Separately-maintained plugins can offer different maturity levels.
● Separately-maintained plugins can be appropriately versioned.
● Separately-maintained plugins can be in languages that contributors
prefer.

What’s the catch?
Plugins require a well-defined interchange format.
Fortunately, we have two great tools for that.

gnostic
gnostic processed and
verified protobuf
representation of
OpenAPI description
protoc +
pluginsOpenAPI
.proto
reusable data structures
and reader for protobuf
OpenAPI descriptions
gnostic apps
and plugins
OpenAPI
description
gnostic-
generator
OpenAPI
.proto and
compiler
code
OpenAPI
JSON
schema

Kubernetes OpenAPI: .json vs .pb
Format Size Deserialization time Download time
(at 80 Mbps)
Json 1653 KB >500 ms 165.3 ms
Proto binary 914 KB 9.3 ms 91.4 ms
Proto binary compressed 96 KB 13.5 ms 1.3 ms
Source: mehdy@google.com

gnostic
API Description
(OpenAPI v3)
Generatable
API Description
(Normalized/
Annotated
OpenAPI v3?)
Language-
specific model
(Normalized/
Annotated
OpenAPI v3?)
Generated
API support
code
target-specific
plugins
gnostic +
linter

let anyone write code generation plugins...
● in their own repositories
● with their own versioning
● in whatever implementation language they choose
API code generation is a community problem that needs community-based
solutions...

Code generators should
belong to their communities.

@grpcio
RPC: Use Cases
Direct RPCs:
Microservices
RPCs to
access APIs
Google APIs
OSS APIs
Mobile
Web
Desktop
RPCs
Datacenters Cloud
Service 1
Service 2
Service 3
Service 4
Containers

@grpcio
What is gRPC?
● HTTP/2 based RPC framework developed by Google
● Open, Multiplatform, Secure, Performant
Multiplatform
● Idiomatic APIs in popular languages (C++, Go, Java, Python, Node.js, C#, Ruby, PHP)
● Supports mobile devices (Android Java, iOS Obj-C, Swift)
● Linux, Windows, Mac OS X
● (web browser support in development)
OpenSource
● developed fully in open on GitHub: https://github.com/grpc/

@grpcio
● Builds on Apple’s swift-protobuf and grpc-core.
● Includes:
○ gRPC framework (C and Swift components)
○ generated code surface
○ protoc plugin for code generation
● Full-service gRPC:
○ All four gRPC API styles are supported.
○ gRPC framework supports both clients and servers.
○ Plugin generates client and server code in separate files.
○ Testing on MacOS and Ubuntu.
● Audiences: client and server developers, inside/outside Google
gRPC for Swift (https://github.com/grpc/grpc-swift)

@grpcio
package echo;
service Echo {
// Immediately returns an echo of a request.
rpc Get(EchoRequest) returns (EchoResponse) {}
// Splits a request into words and returns each word in a stream of messages.
rpc Expand(EchoRequest) returns (stream EchoResponse) {}
// Collects a stream of messages and returns them concatenated when the caller closes.
rpc Collect(stream EchoRequest) returns (EchoResponse) {}
// Streams back messages as they are received in an input stream.
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}
message EchoRequest {
// The text of a message to be echoed.
string text = 1;
}
message EchoResponse {
// The text of an echo response.
string text = 1;
}
gRPC Swift sample service

@grpcio
/// To build a server, implement a class that conforms to this protocol.
public protocol Echo_EchoProvider {
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse
func expand(request : Echo_EchoRequest, session : Echo_EchoExpandSession) throws
func collect(session : Echo_EchoCollectSession) throws
func update(session : Echo_EchoUpdateSession) throws
}
gRPC Swift server protocol

@grpcio
// get returns requests as they were received.
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse {
return Echo_EchoResponse(text:"Swift echo get: " + request.text)
}
...
// update streams back messages as they are received in an input stream.
func update(session : Echo_EchoUpdateSession) throws -> Void {
while true {
do {
let request = try session.Receive()
try session.Send(Echo_EchoResponse(text:"Swift echo update: (request.text)"))
} catch Echo_EchoServerError.endOfStream {
break
}
}
try session.Close()
}
}
gRPC Swift server sample

@grpcio
gRPC Swift unary client sample
// Unary
if client == "get" {
var requestMessage = Echo_EchoRequest(text:message)
let responseMessage = try service.get(requestMessage) // blocking
print("get received: " + responseMessage.text)
}

@grpcio
gRPC Swift bidirectional streaming client sample (1/2)
// Bidirectional streaming
if client == "update" {
let sem = DispatchSemaphore(value: 0)
let updateCall = try service.update() // blocking
DispatchQueue.global().async {
while true {
do {
let responseMessage = try updateCall.Receive() // blocking
print("Received: (responseMessage.text)")
} catch Echo_EchoClientError.endOfStream {
sem.signal()
break
}
}
}
...

@grpcio
gRPC Swift bidirectional streaming client sample (2/2)
...
let parts = message.components(separatedBy:" ")
for part in parts {
let requestMessage = Echo_EchoRequest(text:part)
try updateCall.Send(requestMessage)
sleep(1)
}
try updateCall.CloseSend()
// Wait for the call to complete.
sem.wait()
}

gRPC-Swift TODO
● Build system integration
○ Package Manager
○ Cocoapods?
○ Carthage?
● gRPC interoperability tests
● Samples that wrap Google APIs
○ Google Cloud Speech API
○ Google Datastore API
github.com/grpc/grpc-swift/issues

More gRPC Information
Website: http://grpc.io
Sources: https://github.com/grpc/grpc
Mailing list: https://groups.google.com/d/forum/grpc-io
Ecosystem: https://github.com/grpc-ecosystem

github.com/googleapis/gnostic
plugins for OpenAPI-based code generation
github.com/grpc/grpc-swift
Fast streaming APIs in Swift
github.com/google/auth-library-swift
OAuth support for Google Cloud
http://twitter.com/timburks
timburks@google.com

Build Great Networked APIs with Swift, OpenAPI, and gRPC

More Related Content

What's hot (20)

Similar to Build Great Networked APIs with Swift, OpenAPI, and gRPC (20)

More from Tim Burks (6)

Recently uploaded (20)

Build Great Networked APIs with Swift, OpenAPI, and gRPC