Infrastructure-as-code: bridging the gap between Devs and Ops

Infrastructure-as-Code
bridging the gap between Devs and Ops
April 6th, 2019 - DevOps Fest 2019 - Kyiv, Ukraine

Who am I?
Mykyta Protsenko
Software developer @ Netflix
(Edge Developer Productivity)
Twitter: @mykyta_p

How long does it take
you to provision
infrastructure?

How many resources
do you need for one
microservice?

...and how many do you
need for all of them?

Too much stuff for OPS
to handle

Bridging the gap
Empowering devs
Ensuring safety

12 factor apps
Codebase
Dependencies
Configuration
Backing services
Build, release, run
Processes
Port binding
Concurrency
Disposability
Dev/prod parity
Logs
Admin processes

12 factor apps
Codebase
Dependencies
Configuration
Backing services
Build, release, run
Processes
Port binding
Concurrency
Disposability
Logs
Dev/prod parity
Admin processes

12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

What tools do we need?
aws elb create-load-balancer
--load-balancer-name myELB
--listeners
"Protocol=HTTP,
LoadBalancerPort=80,
InstanceProtocol=HTTP,
InstancePort=80"
--subnets subnet-15aaab61
--security-groups sg-a61988c3

Imperative tools?
aws elb create-load-balancer
--load-balancer-name myELB
--listeners
"Protocol=HTTP,
LoadBalancerPort=80,
InstanceProtocol=HTTP,
InstancePort=80"
--subnets subnet-15aaab61
--security-groups sg-a61988c3

Cloudformation?
"MyDNSRecord": {
"Type": "AWS::Route53::RecordSet",
"Properties": { "HostedZoneName":
{"Fn::Join":
["", [{"Ref":"HostedZone"},
"."]]},
"Comment" : "DNS for inst.",
"Name" : {"Fn::Join":
["",[{"Ref":"EC2Instance"},
".",{"Ref": "AWS::Region"},
".", {"Ref:"HostedZone"},"."]]},
"Type" : "A",
"TTL" : "300",
"ResourceRecords":
[{"Fn::GetAtt" :
["EC2Instance", PublicIp"]}]}
}

Infrastructure-as-code: bridging the gap between Devs and Ops

resource "aws_route53_record" "www"
{
zone_id = "${...}"
name = "www.example.com"
type = "A"
ttl = "300"
records =
["${aws_eip.lb.public_ip}"]
}
Terraform

Terraform
Simple
Human-friendly

What else?
@RestController
public class HelloWorld {
@GetMapping("/")
public String hello() {
return "Hello World!n";
}
}
https://github.com/iac-demo

Terraform FTW!
resource "aws_ecs_task_definition"
"hello-world" {
...
family = "hello-world"
cpu = "256" // 0.25 vCPU
memory = "512" // 512 MB
container_definitions = <<DEF
[
{
...
"image": "helloworld:latest",
...
}
]
DEF
}

Terraform FTW!
resource "aws_ecs_service" "hello-world" {
cluster = "${aws_ecs_cluster.main.id}"
task_definition =
"${aws_ecs_task_definition.hello-world.arn}"
desired_count = "2"
network_configuration {
subnets = ["${aws_subnet.sb_a.id}",...]
security_groups =
["${aws_security_group.web_ecs.id}"]
}
load_balancer {
target_group_arn =
"${aws_alb_target_group.hello-world.id}"
container_name = "helloworld"
container_port = "8080"
}
...

Terraform FTW!
resource "aws_alb" "hello-world" {
name = "hello-world"
subnets = [...]
security_groups = [...]
vpc_id = "${aws_vpc.iacdemo_vpc.id}"
}

Core Infrastructure
vs
Project Infrastructure

Core Infrastructure
resource "aws_vpc" "iacdemo_vpc" {
cidr_block = "10.0.0.0/16"
enable_dns_hostnames = true
}
resource "aws_internet_gateway"
"default" {
vpc_id = "${aws_vpc.iacdemo_vpc.id}"
}
...

Core Infrastructure
output "vpc_id" {
value = "${aws_vpc.iacdemo_vpc.id}"
}
terraform {
backend "s3" {
key = "iacdemo.tfstate"
region = "us-west-2"
bucket = "demobucket"
}
}

Project Infrastructuredata "terraform_remote_state" "core" {
backend = "s3"
config {
}
}
name = "hello-world"
...
vpc_id =
"${data.
terraform_remote_state.core.vpc_id}"
}

State of the world
local terraform.tfstate

State of the world
remote S3
local terraform.tfstate

State of the world
terraform {
backend "s3" {
}

State of the world
terraform init

Automated Pipeline
git clone git@github.com:my/repo.git

Automated Pipeline
terraform init

Automated Pipeline
terraform init
export TF_VAR_foo="bar"
terraform plan

Automated Pipeline
terraform init
export TF_VAR_foo="bar"
terraform plan
terraform apply

Encapsulation
Hiding Complexity
Reusing Code

Terraform modules
variable "service_name" {}
variable "docker_image" {}

Terraform modules
variable "service_name" {}
variable "docker_image" {}
resource "aws_ecs_task_definition"
"service" {
family = "${var.service_name}"
container_definitions = <<DEF
[{
...
"image": "${var.docker_image}",
"name": "${var.service_name}"
...
}]DEF
}

Terraform modulesmodule "hello_world" {
source = "./microservice_module"
service_name = "helloworld1"
docker_image= "helloworld:latest"
}

}
module "another_hello_world" {
}

source = ...
service_name = "helloworld"
}

Create Before Destroy
...
lifecycle {
create_before_destroy = "true"
}
}

Create Before Destroy
...
lifecycle {
}
}
resource "aws_ecs_service" "hello-world" {
...
lifecycle {
}
}

Declarative K8S
k8s_template.yaml.sh
#!/bin/bash
cat <<YAML
apiVersion: apps/v1beta1
kind: Deployment
...
spec:
replicas: 1
template:
spec:
containers:
- name: $SERVICE_NAME
image: $DOCKER_IMAGE
imagePullPolicy: Always
ports:
- containerPort: 8090
...
YAML

Declarative K8S
$ export DOCKER_IMAGE=hello:latest
$ export SERVICE_NAME=helloworld
$ k8s_template.yaml.sh |
kubectl apply -f -

Still Need Core Infra
provider "google" {
project = "breakme-europe"
region = "europe-west1"
}
resource "google_container_cluster"
"main" {
name = "k8s-cluster"
zone = "europe-west1-d"
initial_node_count = 4
node_config {
machine_type = "n1-standard-2"
...
}
...
}

30-40 mins vs 3-4 mins
10x better!

Centralize?
Vagrant?
Package Repository?

Leverage Build System
build.gradle
buildscript {
dependencies {
classpath "com.roku:henka:1.0.0-RELEASE"
}
}
task terraformPlan(type: TerraformTask) {
description "Runs a terraform script"
tfDir = "${projectDir}/terraform"
tfAction = "plan -input=false"
terraformBaseDir = "/opt/terraform"
terraformVersion = "0.11.11"
}
// ...

Leverage Build System
$ ./gradlew build
$ ./gradlew terraformPlan

Best Practices
Unified Build Logic

Best Practices
Unified Build Logic
Unified Monitoring

Best Practices
Unified Build Logic
Unified Monitoring
Code Reviews

Best Practices
Unified Build Logic
Unified Monitoring
Code Reviews
CI/CD

Best Practices
Unified Build Logic
Unified Monitoring
Code Reviews
CI/CD
No Documentation

Terraform https://www.terraform.io
12 factor app https://12factor.net/
Kubernetes https://kubernetes.io/
Gradle https://gradle.org/
Henka https://github.com/roku-oss/henka

Twitter @mykyta_p
Slides http://devopsfest2019.protsenko.com
Sources https://github.com/iac-demo

Infrastructure-as-code: bridging the gap between Devs and Ops

Recommended

More Related Content

What's hot (18)

Similar to Infrastructure-as-code: bridging the gap between Devs and Ops (20)

Recently uploaded (20)

Infrastructure-as-code: bridging the gap between Devs and Ops