Managing your microservices

with Kubernetes and Istio

Craig Box
Agenda
What is a Service Mesh?
How we got here: a story
Architecture and details
Q&A

2
What is a service mesh?
A network for services, not bytes

● Traffic Control
● Visibility
● Resiliency & Efficiency
● Security
● Policy Enforcement
Remember the monolith?

Monolithic App

Module Module

File System

main() Module Module

Database

Module Module
Microservices

main() Service Service File System

Service Service Database

Service

Service

Database

File System

5
Success!
Gained development velocity!
Easy testing because of abstractions!
Scale services independently!

6
At what cost?
● Replaced a reliable in-process call with
an unreliable RPC.
● Secure in-process communication is
replaced by insecure network.
● Access control within process was a
no-op
● Latency went up
● Trivial single-stepping replaced by … ?

7
Can we fix it?
● Add retry logic to the application code
● Add entry-exit traces
● Secure inter-service connections with
strong authentication

8
While we're adding code...
Choose the RPC endpoint intelligently

● Endpoints with low latency

● Endpoints with warm caches

9
Service mesh
Istio
A complete framework for
connecting, securing, managing
and monitoring services

12
Kubernetes provides service abstraction

web web

auth auth

payments payments

logging logging

13
Weaving the mesh

Users
frontend auth Cloud SQL

External
Payment
pictures payments Processor

14
Weaving the mesh

frontend auth Users

Cloud SQL

proxy proxy
ingress

proxy proxy
External
Payment
pictures payments Processor

15
The sidecar proxy: Envoy
● A C++ based L4/L7 proxy
● Low memory footprint
● Battle-tested @ Lyft
○ 100+ services
○ 10,000+ VMs
○ 2M req/s
● An awesome team willing to work with the
community!

16
The sidecar proxy: Envoy
Goodies:
● HTTP/2 & gRPC
● Zone-aware load balancing w/ failover
● Health checks, circuit breakers, timeouts, retry budgets
● No hot reloads - API driven config updates

Istio's contributions
● Transparent proxying w/ SO_ORIGINAL_DST
● Traffic routing and splitting
● Request tracing using Zipkin
● Fault injection

17
Injection

spec: spec:
containers: containers:
- image: frontend:latest - image: frontend:latest
- image: istio/proxy

frontend frontend

proxy

Initializer policy

initImage: docker.io/istio/proxy_init
proxyImage: docker.io/istio/proxy

18
Architecture

frontend payments
Traffic transparently proxied —
unaware of Envoy sidecars

proxy proxy

Policy checks,
Discovery & config telemetry TLS certs
data to Envoy sidecars to Envoy sidecars

Pilot Mixer Istio-Auth

Control Plane

19
● Traffic Control
● Visibility
● Resiliency & Efficiency
● Security
● Policy Enforcement
Application rollout
// A simple traffic splitting rule
destination: pictures.example.local frontend
match:
source: frontend.example.local proxy
route:
- tags: 99% 1%

version: v1.5
env: prod pictures

weight: 99
- tags: proxy proxy
version: v2.0-alpha
env: staging pictures pictures
weight: 1

version: 1.5 version: 2.0-alpha

env: prod env: staging
21
Traffic steering
// Content-based traffic steering rule
destination: pictures.example.local frontend
match:
httpHeaders: proxy
user-agent:
regex: ^(.*?;)?(iPhone)(;.*)?$
precedence: 2
route: pictures

- tags:
version: 2.0-alpha proxy proxy
env: staging
pictures pictures

version: 1.5 version: 2.0-alpha

env: prod env: staging
22
● Traffic Control
● Visibility
● Resiliency & Efficiency
● Security
● Policy Enforcement
Visibility
Monitoring & tracing should not be an
afterthought in the infrastructure

Goals:
● Metrics without instrumenting apps
● Consistent metrics across fleet
● Trace flow of requests across services
● Portable across metric backend
providers

24
Metrics flow
frontend pictures

● Mixer collects metrics emitted by proxy proxy

Envoys
API: /pictures
Latency: 10ms
Status Code: 503
● Adapters in the Mixer normalize and src: 10.0.0.1
dst: 10.0.0.2
forward to monitoring backends
Mixer

● Metrics backend can be swapped at

Custom
Prometheus

InfluxDB
runtime

27
Tracing
frontend pictures

● Applications do not have to deal with proxy proxy

generating spans or correlating
causality X-B3-TraceId
X-B3-SpanId
X-B3-ParentSpanId
● Envoys generate spans X-B3-Sampled
X-B3-Flags
○ Applications need to forward
context headers on outbound Mixer

calls

Zipkin

Custom
Stackdriver
● Envoys send traces to Mixer
○ Adapters at Mixer send traces to
respective backends
28
● Traffic Control
● Visibility
● Resiliency & Efficiency
● Security
● Policy Enforcement
Resiliency
// Circuit breakers
destination: auth.cluster.local
policy:
- tags:
version: v1
circuitBreaker:
simpleCb:
httpConsecutiveErrors: 7
sleepWindow: 5m
httpDetectionInterval: 1m

30
Resiliency features
● Timeouts
● Retries with timeout budget
● Circuit breakers
● Health checks
● AZ-aware load balancing w/ automatic failover
● Control connection pool size and request load
● Systematic fault injection

31
Resiliency testing
Systematic fault injection to identify weaknesses in failure
recovery policies
● HTTP/gRPC error codes
● Delay injection

300ms 400ms

frontend movies stars

proxy proxy proxy

timeout: 100ms timeout: 200ms

retries: 3 retries: 2
32
Efficiency
● L7 load balancing
○ Passive/Active health checks, circuit breaks
○ Backend subsets
○ Affinity
● TLS offload
○ No more JSSE or stale SSL versions
● HTTP/2 and gRPC proxying

33
● Traffic Control
● Visibility
● Resiliency & Efficiency
● Security
● Policy Enforcement
Security
● Verifiable identity
● Secure naming / addressing
● Traffic encryption
● Revocation

35
Security at scale

frontend payments

Envoy Envoy

SAN: "spiffe://example.local/ns/prod/sa/foo" SAN: "spiffe://example.local/ns/prod/sa/bar"

Namespace: prod Namespace: prod
Service account: foo Service account: bar

Istio-Auth

36
● Traffic Control
● Visibility
● Resiliency & Efficiency
● Security
● Policy Enforcement
What does Mixer do?
● Check()
○ Precondition checking
○ Quotas & Rate Limiting
● Report()
○ telemetry reporting
● Primary point of extensibility
● Enabler for platform mobility
● Operator-focused configuration model

38
Putting it all together

frontend payments

proxy proxy

Policy checks,
Discovery & config telemetry TLS certs
data to Envoy sidecars to Envoy sidecars

Pilot Mixer Istio-Auth

Control Plane

39
Ultimately, it's just this

Users
frontend auth Cloud SQL

External
Payment
pictures payments Processor

40
Getting started
● Install Kubernetes (v1.7+ for Initializers)
○ Google Container Engine Alpha clusters
● istio.io quickstart
● Helm chart
○ helm install incubator/istio
● Bookinfo service example

41
Roadmap
● 0.1: a single Kubernetes namespace
● 0.2 (just launched): a single Kubernetes
cluster and external VMs
● 0.3 (by end of year): production readiness
within a single cluster
● 1.0: (2018): complete mesh across all
environments
Release roadmap

42
Thank you
Craig Box

43