Gitops And Kubernetes Continuous Deployment With Argo Cd Jenkins X And Flux 1st Edition Billy Yuen

Gitops And Kubernetes Continuous Deployment With
Argo Cd Jenkins X And Flux 1st Edition Billy
Yuen download
https://ebookbell.com/product/gitops-and-kubernetes-continuous-
deployment-with-argo-cd-jenkins-x-and-flux-1st-edition-billy-
yuen-23459512
Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.
Implementing Gitops With Kubernetes Automate Manage Scale And Secure
Infrastructure And Cloudnative Applications On Aws And Azure Pietro
Libro And Artem Lajko
https://ebookbell.com/product/implementing-gitops-with-kubernetes-
automate-manage-scale-and-secure-infrastructure-and-cloudnative-
applications-on-aws-and-azure-pietro-libro-and-artem-lajko-59184412
Infrastructure And Cloudnative Applications Pietro Libro
applications-pietro-libro-59113188
Infrastructure And Cloudnative Applications On Aws And Azure 1st
Edition Pietro Libro Artem Lajko
applications-on-aws-and-azure-1st-edition-pietro-libro-artem-
lajko-232416938
Gitops And Kuberentes Meap V06 Epub Billy Yuen Alexander Matyushentsev
https://ebookbell.com/product/gitops-and-kuberentes-meap-v06-epub-
billy-yuen-alexander-matyushentsev-232063090

Practical Gitops Infrastructure Management Using Terraform Aws And
Github Actions 1st Edition Rohit Salecha
https://ebookbell.com/product/practical-gitops-infrastructure-
management-using-terraform-aws-and-github-actions-1st-edition-rohit-
salecha-47554278
Practical Gitops Infrastructure Management Using Terraform Aws And
Github Actions 1st Edition Rohit Salecha
https://ebookbell.com/product/practical-gitops-infrastructure-
management-using-terraform-aws-and-github-actions-1st-edition-rohit-
salecha-47554280
Gitops Cookbook 1st Edition Natale Vinto Alex Bueno
https://ebookbell.com/product/gitops-cookbook-1st-edition-natale-
vinto-alex-bueno-47425218
Gitops Cookbook Second Early Release Natale Vinto Alex Soto Bueno
https://ebookbell.com/product/gitops-cookbook-second-early-release-
natale-vinto-alex-soto-bueno-43167624
Implementing Gitops With Kubernetes Pietro Libro
pietro-libro-74164218

M A N N I N G
Billy Yuen
Alexander Matyushentsev
Todd Ekenstam
Jesse Suen
Continuous Deployment with Argo CD, Jenkins X, and Flux

302
Chapter 3 Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 10
Chapter 11
Chapter 9
Environment
management
Access control
and security
Secrets
Observability
Pipelines
Deployment
strategies
Argo CD
Jenkins X
Flux
Chapter 2
Kubernetes and
GitOps
Chapter 1
Why GitOps?
Part 1: Background Part 2: Patterns and processes Part 3: Tools

GitOps and Kubernetes
Continuous Deployment with Argo CD, Jenkins X, and Flux

GitOps and
Kubernetes
CONTINUOUS DEPLOYMENT WITH
ARGO CD, JENKINS X, AND FLUX
BILLY YUEN
ALEXANDER MATYUSHENTSEV
TODD EKENSTAM
AND JESSE SUEN
M A N N I N G
SHELTER ISLAND

For online information and ordering of this and other Manning books, please visit
www.manning.com. The publisher offers discounts on this book when ordered in quantity.
For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road
PO Box 761
Shelter Island, NY 11964
Email: orders@manning.com
©2021 by Manning Publications Co. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in
any form or by means electronic, mechanical, photocopying, or otherwise, without prior written
permission of the publisher.
Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in the book, and Manning
Publications was aware of a trademark claim, the designations have been printed in initial caps
or all caps.
Recognizing the importance of preserving what has been written, it is Manning’s policy to have
the books we publish printed on acid-free paper, and we exert our best efforts to that end.
Recognizing also our responsibility to conserve the resources of our planet, Manning books
are printed on paper that is at least 15 percent recycled and processed without the use of
elemental chlorine.
Manning Publications Co. Development editor: Dustin Archibald
20 Baldwin Road Technical development editor: Al Krinker
PO Box 761 Review editor: Aleks Dragosavljević
Shelter Island, NY 11964 Productioneditor: Deirdre S. Hiam
Proofreader: Katie Tennant
Technical proofreader: Sam Brown
Typesetter and cover designer: Marija Tudor
ISBN 9781617297977
Printed in the United States of America

v
contents
preface xi
acknowledgments xii
about this book xiii
about the authors xviii
about the cover illustration xx
PART 1 BACKGROUND ................................................. 1
1 Why GitOps? 3
1.1 Evolution to GitOps 4
Traditional Ops 4 ■
DevOps 6 ■
GitOps 7
1.2 Developer benefits of GitOps 9
Infrastructure as code 9 ■
Self-service 10 ■
Code reviews 11
Git pull requests 12
1.3 Operational benefits of GitOps 13
Declarative 13 ■
Observability 15 ■
Auditability and
compliance 16 ■
Disaster recovery 19
2 Kubernetes and GitOps 20
2.1 Kubernetes introduction 20
What is Kubernetes? 21 ■
Other container orchestrators 22
Kubernetes architecture 22 ■
Deploying to Kubernetes 25
2.2 Declarative vs. imperative object management 30
How declarative configuration works 34

CONTENTS
vi
2.3 Controller architecture 37
Controller delegation 37 ■
Controller pattern 38
NGINX operator 40
2.4 Kubernetes + GitOps 44
2.5 Getting started with CI/CD 44
Basic GitOps operator 45 ■
Continuous integration pipeline 47
PART 2 PATTERNS AND PROCESSES ............................. 53
3 Environment management 55
3.1 Introduction to environment management 56
Components of an environment 57 ■
Namespace
management 59 ■
Network isolation 64 ■
Preprod and prod
clusters 67
3.2 Git strategies 68
Single branch (multiple directories) 69 ■
Multiple branches 70
Multirepo vs. monorepo 70
3.3 Configuration management 71
Helm 72 ■
Kustomize 76 ■
Jsonnet 79 ■
Configuration
management summary 83
3.4 Durable vs. ephemeral environments 83
4 Pipelines 86
4.1 Stages in CI/CD pipelines 86
GitOps continuous integration 88 ■
GitOps continuous
delivery 94
4.2 Driving promotions 98
Code vs. manifest vs. app config 98 ■
Code and image
promotion 99 ■
Environment promotion 101 ■
Putting it all
together 102
4.3 Other pipelines 102
Rollback 103 ■
Compliance pipeline 106
5 Deployment strategies 109
5.1 Deployment basics 110
Why ReplicaSet is not a good fit for GitOps 111 ■
How Deployment
works with ReplicaSets 114 ■
Traffic routing 120
Configuring minikube for other strategies 122

CONTENTS vii
5.2 Blue-green 123
Blue-green with Deployment 125 ■
Blue-green with Argo
Rollouts 130
5.3 Canary 133
Canary with Deployment 134 ■
Canary with Argo
Rollouts 138
5.4 Progressive delivery 140
Progressive delivery with Argo Rollouts 140
6 Access control and security 148
6.1 Introduction to access control 149
What is access control? 149 ■
What to secure 150 ■
Access
control in GitOps 153
6.2 Access limitations 155
Git repository access 155 ■
Kubernetes RBAC 163 ■
Image
registry access 168
6.3 Patterns 171
Full access 171 ■
Deployment repo access 172 ■
Code access
only 172
6.4 Security concerns 173
Preventing image pull from untrusted registries 173
Cluster-level resources in a Git repository 174
7 Secrets 176
7.1 Kubernetes Secrets 177
Why use Secrets? 177 ■
How to use Secrets 178
7.2 GitOps and Secrets 181
No encryption 181 ■
Distributed Git repos 181 ■
No granular
(file-level) access control 181 ■
Insecure storage 181
Full commit history 182
7.3 Secrets management strategies 182
Storing Secrets in Git 182 ■
Baking Secrets into the container
image 182 ■
Out-of-band management 183 ■
External Secrets
management systems 184 ■
Encrypting Secrets in Git 185
Comparison of strategies 186
7.4 Tooling 187
HashiCorp Vault 187 ■
Vault Agent Sidecar Injector 190
Sealed Secrets 194 ■
Kustomize Secret generator plugin 198

CONTENTS
viii
8 Observability 203
8.1 What is observability? 204
Event logging 205 ■
Metrics 209 ■
Tracing 212
Visualization 217 ■
Importance of observability in GitOps 219
8.2 Application health 219
Resource status 220 ■
Readiness and liveness 224
Application monitoring and alerting 225
8.3 GitOps observability 227
GitOps metrics 227 ■
Application sync status 228
Configuration drift 232 ■
GitOps change log 234
PART 3 TOOLS ........................................................ 239
9 Argo CD 241
9.1 What is Argo CD? 241
Main use cases 242 ■
Core concepts 243 ■
Sync and health
statuses 244 ■
Architecture 246
9.2 Deploy your first application 249
Deploying the first application 249 ■
Inspect the application
using the user interface 251
9.3 Deep dive into Argo CD features 253
GitOps-driven deployment 253 ■
Resource hooks 254
Postdeployment verification 257
9.4 Enterprise features 257
Single sign-on 258 ■
Access control 261 ■
Declarative
management 263
10 Jenkins X 267
10.1 What is Jenkins X? 267
10.2 Exploring Prow, Jenkins X pipeline operator,
and Tekton 269
10.3 Importing projects into Jenkins X 273
Importing a project 274 ■
Promoting a release to the production
environment 281
11 Flux 284
11.1 What is Flux? 284
What Flux does 285 ■
Docker registry scanning 286
Architecture 288

CONTENTS ix
11.2 Simple application deployment 289
Deploying the first application 289 ■
Observing application
state 290 ■
Upgrading the deployment image 291 ■
Using
Kustomize for manifest generation 292 ■
Securing deployment
using GPG 294
11.3 Multitenancy with Flux 296
appendix A Setting up a test Kubernetes cluster 299
appendix B Setting up GitOps tools 302
appendix C Configuring GPG key 313
index 315

xi
preface
As Intuit embarked on the journey from on-premises to cloud-native, the journey itself
presented an opportunity to reinvent our build and deployment process. Similar to
many large enterprises, our old deployment process was data-center-centric with sepa-
rate QA, Ops, and Infrastructure teams. Code could take weeks to get deployed, and
developers had no access to infrastructure when there were production issues. Infra-
structure issues could take a long time to resolve and required many groups’ collabo-
ration.
As Marianna Tessel (Intuit CTO) and Jeff Brewer (Intuit SBSEG chief architect)
decided to bet big on Kubernetes and Docker, we were fortunate to be the first team
to fully migrate one of our production applications with Kubernetes and Docker.
Along the way, we got to reinvent our CI/CD pipeline and adopt the GitOps process.
Jesse and Alex created Argo CD (CNCF incubator project) to address enterprise
needs for GitOps. Todd and his team created world-class cluster management tools so
we can scale out to hundreds of clusters with ease.
Having a standard like Kubernetes and Docker enables all engineers to speak a
common language in terms of infrastructure and deployment. Engineers can easily
contribute to other projects and deploy as soon as the development process is com-
plete. GitOps also allows us to know exactly who and what gets changed in our environ-
ments, which is especially important if you are subject to compliance requirements. We
cannot imagine going back to the old way we did deployment, and we hope that this
book can help accelerate your journey to embrace GitOps!

xii
acknowledgments
This book turned out to be an 18-month journey that required a lot of work and addi-
tional research to tell the complete story. We believe that we have delivered what we
set out to do, and it is a great book for anyone who wants to adopt GitOps and Kuber-
netes.
There are quite a few people we’d like to thank for helping us along the way. At Man-
ning, we would like to thank our development editor, Dustin Archibald, project editor,
Deirdre Hiam, proofreader, Katie Tennant, and reviewing editor, Aleks Dragosavljevic.
We want to thank Marianna Tessel and Jeff Brewer, who provided us the opportu-
nity and freedom to transform and experiment with GitOps and Kubernetes. We
would also like to thank Pratik Wadher, Saradhi Sreegiriaju, Mukulika Kupas, and
Edward Lee for their guidance throughout the process. We want to call out Viktor Far-
cic and Oscar Medina for their insightful contributions to the Jenkins X chapter.
To all the reviewers: Andres Damian Sacco, Angelo Simone Scotto, Björn Neuhaus,
Chris Viner, Clifford Thurber, Conor Redmond, Diego Casella, James Liu, Jaume
López, Jeremy Bryan, Jerome Meyer, John Guthrie, Marco Massenzio, Matthieu Evrin,
Mike Ensor, Mike Jensen, Roman Zhuzha, Samuel Brown, Satej Kumar Sahu, Sean T.
Booker, Wendell Beckwith, and Zorodzayi Mukuya, we say thank you. Your suggestions
helped make this a better book.
For Jeff Brewer, who inspired us all for this awesome transformation journey!

xiii
about this book
Who this book is for
This book is intended for both Kubernetes infrastructure and operation engineers
and software developers who want to deploy applications to Kubernetes through a
declarative model using the GitOps process. It will benefit anyone looking to improve
the stability, reliability, security, and auditability of their Kubernetes clusters while at
the same time reducing operational costs through automated continuous software
deployments.
Readers are expected to have a working knowledge of Kubernetes (Deployment,
Pod, Service, and Ingress resources, for example) as well as an understanding of mod-
ern software development practices including continuous integration/continuous
deployment (CI/CD), revision control systems (such as Git), and deployment/infra-
structure automation.
Who this book is not for
Advanced users who have successfully implemented a mature GitOps system may be
better off reading a more advanced book on their chosen tool.
This book is not intended to cover all aspects of Kubernetes in depth. While we
cover many Kubernetes concepts that are relevant to GitOps, readers looking for a
comprehensive guide to Kubernetes should look at the other great books and online
resources available on the topic.
How this book is organized: A roadmap
This book describes the benefits of GitOps on Kubernetes, including flexible configu-
ration management, monitoring, robustness, multienvironment support, and security.

ABOUT THIS BOOK
xiv
You will learn the best practices, techniques, and tools to achieve these benefits, which
enable enterprises to use Kubernetes to accelerate application development without
compromising on stability, reliability, or security.
You will also gain in-depth understanding of the following topics:
 Multiple-environment management with branching, namespace, and configu-
ration
 Access control with Git, Kubernetes, and pipelines
 Pipeline considerations with CI/CD, promotion, push/pull, and release/roll-
back
 Observability and drift detection
 Managing Secrets
 Deployment strategy selection among rolling update, blue/green, canary, and
progressive delivery
This book takes a hands-on approach with tutorials and exercises to develop the skills
you need to embrace GitOps using Kubernetes. After reading this book, you will know
how to implement a declarative continuous delivery system for your applications run-
ning on Kubernetes. This book contains hands-on tutorials on
 Getting started with managing Kubernetes application deployments
 Configuration and environment management using Kustomize
 Writing your own basic Kubernetes continuous delivery (CD) operator
 Implementing CI/CD using Argo CD,1
Jenkins X,2
and Flux3
IMPERATIVE VS. DECLARATIVE There are two basic ways to deploy Kubernetes:
imperatively using many kubectl commands or declaratively by writing mani-
fests and using kubectl apply. The former is useful for learning and interac-
tive experimentation. The latter is best for reproducible deployments and
tracking changes.
This book is intended for you to follow along, running the hands-on portion of the
tutorials, using your own test Kubernetes cluster. Appendix A describes several
options for creating a test cluster.
There are many code listings contained in the book. All code listings and addi-
tional supporting material can be found in the publicly accessible GitHub repository
for this book:
https://github.com/gitopsbook/resources
We encourage you to clone or fork this repository and use it as you work through the
tutorials and exercises in the book.
1
https://argoproj.github.io/argo-cd.
2
https://jenkins-x.io.
3
https://github.com/fluxcd/flux.

ABOUT THIS BOOK xv
The following tools and utilities should be installed on your workstation:
 Kubectl (v1.16 or later)
 Minikube (v1.4 or later)
 Bash or the Windows Subsystem for Linux (WSL)
Most tutorials and exercises can be completed using a minikube running on your
workstation. If not, we will mention if the cluster running on a cloud provider is
needed, and you can refer to appendix A for details on creating the cluster.
NOTE You may incur additional costs for running a test Kubernetes cluster
on a cloud provider. While we have attempted to reduce the cost of the rec-
ommended test configuration as much as possible, remember you are respon-
sible for these costs. We recommend you delete your test cluster after
completing each tutorial or exercise.
This book has 3 parts that cover 11 chapters. Part 1 covers the background and intro-
duces GitOps and Kubernetes:
 Chapter 1 walks you through the journey of software deployment evolution and
how GitOps became the latest practice. It also covers the many key concepts
and benefits of GitOps.
 Chapter 2 provides key concepts on Kubernetes and why its declarative nature
is perfect for GitOps. It also covers the core operator concept and how to imple-
ment a simple GitOps operator.
Part 2 goes over the patterns and processes to adopt the GitOps process:
 Chapter 3 discusses the definition of an environment and how Kubernetes
Namespaces nicely map as environments. It also covers branching strategy and
config management to your environment implementation.
 Chapter 4 goes deep into the GitOps CI/CD pipeline with comprehensive
descriptions of all stages necessary for a complete pipeline. It also covers code,
image, and environment promotion as well as the rollback mechanism.
 Chapter 5 describes various deployment strategies, including rolling update,
blue/green, canary, and progressive delivery. It also covers how to implement
each strategy by using native Kubernetes resources and other open source tools.
 Chapter 6 discusses GitOps-driven deployment’s attack surfaces and how to mit-
igate each area. It also reviews Jsonnet, Kustomize, and Helm and how to
choose the right configuration management pattern for your use cases.
 Chapter 7 discusses various strategies for managing Secrets for GitOps. It also
covers several Secret management tools as well as native Kubernetes Secrets.
 Chapter 8 explains the core concepts of observability and why it is important to
GitOps. It also describes various methods to implement observability with
GitOps and Kubernetes.

ABOUT THIS BOOK
xvi
Part 3 goes over several enterprise-grade GitOps tools:
 Chapter 9 discusses the intent and architecture for Argo CD. It also covers con-
figuring application deployment using Argo CD and how to secure Argo CD in
production.
 Chapter 10 discusses the intent and architecture for Jenkins X. It also covers
configuring application deployment and promotion to various environments.
 Chapter 11 discusses the intent and motivation for Flux. It also covers configur-
ing application deployment using Flux and multitenancy.
The book was organized to read all the chapters in sequential order. However, if there is a particular area
of interest you’d like to jump into, we recommend you read the prerequisite chapters. For example, if you
would like to jump right into learning to use Argo CD, we recommend you read chapters 1, 2, 3, and 5
before reading chapter 9.
About the code
This book contains many examples of source code both in numbered listings and
inline with normal text. In both cases, source code is formatted in a fixed-width
font to separate it from ordinary text. Sometimes, code is also in bold to highlight
Chapter 3 Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 10
Chapter 11
Chapter 9
Environment
management
Access control
and security
Secrets
Observability
Pipelines
Deployment
strategies
Argo CD
Jenkins X
Flux
Chapter 2
Kubernetes and
GitOps
Chapter 1
Why GitOps?
Part 1: Background Part 2: Patterns and processes Part 3: Tools
Appendix A
Set up a test
Kubernetes cluster
Appendix B
Set up
GitOps tools
Appendix C
Configure
GPG key
Appendixes: optional information

ABOUT THIS BOOK xvii
code that has changed from previous steps in the chapter, such as when a new feature
adds to an existing line of code.
In many cases, the original source code has been reformatted; we’ve added line
breaks and reworked indentation to accommodate the available page space in the
book. Additionally, comments in the source code have often been removed from the
listings when the code is described in the text. Code annotations accompany many of
the listings, highlighting important concepts. Source code for the examples in this
book is available for download from https://github.com/gitopsbook /resources.
liveBook discussion forum
Purchase of GitOps and Kubernetes includes free access to a private web forum run by
Manning Publications where you can make comments about the book, ask technical
questions, and receive help from the authors and from other users. To access the forum,
go to https://livebook.manning.com/book/GitOps-and-Kubernetes/discussion. You
can also learn more about Manning’s forums and the rules of conduct at https://
livebook.manning.com/#!/discussion.
Manning’s commitment to our readers is to provide a venue where a meaningful
dialogue between individual readers and between readers and authors can take place.
It is not a commitment to any specific amount of participation on the part of the
authors, whose contribution to the forum remains voluntary (and unpaid). We sug-
gest you try asking them some challenging questions lest their interest stray! The
forum and the archives of previous discussions will be accessible from the publisher’s
website as long as the book is in print.

xviii
about the authors
BILLY YUEN is a principal engineer with Intuit’s Platform team, focusing on AWS and
Kubernetes adoption, system resiliency, and monitoring. Previously, Billy worked on
Netflix’s Edge Services team to build the next generation of edge-service infrastruc-
ture to support millions of customers (more than 3 billion requests per day) with high
scalability, resilience to failure, and rapid innovation. Billy was a speaker at Java One
2016 and Velocity NY 2016 on “Operational Excellence with Netflix Hystrix,” “CI/CD
at Lightspeed” at KubeCon 2018, and “Automated Canary Release” at Container
World 2019.
ALEXANDER MATYUSHENTSEV is a principal engineer on the Intuit Platform team,
focusing on building tools that make it easier to use Kubernetes. Alexander is passion-
ate about open source, cloud-native infrastructure, and tools that increase developers’
productivity. He is one of the core contributors to the Argo Workflows and Argo CD
projects. Alexander was a speaker at KubeCon 2019 on “How Intuit Does Canary and
Blue-Green Deployments with a K8s Controller.”
TODD EKENSTAM is a principal engineer at Intuit, building a platform for secure,
multitenant Kubernetes infrastructure supporting applications serving Intuit’s
approximately 50 million customers. Todd has worked on a variety of large-scale dis-
tributed systems projects during his career of more than 25 years, including hierarchi-
cal storage management, peer-to-peer database replication, enterprise storage
virtualization, and two-factor authentication SaaS. Todd has presented at academic,
government, and industry conferences, most recently as a guest speaker on “Introduc-
tion to Open Policy Agent” at KubeCon 2018.
JESSE SUEN is a principal engineer on the Intuit Platform team, developing
microservices-based, distributed applications for Kubernetes. He was an early

ABOUT THE AUTHORS xix
engineer at Applatix (acquired by Intuit), building a platform to help users run con-
tainerized workloads in the public cloud. Before that, he was part of the engineering
team at Tintri and Data Domain, working on virtualized infrastructure, storage, tool-
ing, and automation. Jesse is one of the core contributors to the open source Argo
Workflows and Argo CD projects.

xx
about the cover illustration
The figure on the cover of GitOps and Kubernetes is captioned “Habitant de Styrie,” or
resident of Styria. The illustration is taken from a collection of dress costumes from
various countries by Jacques Grasset de Saint-Sauveur (1757-1810), titled Costumes de
Différents Pays, published in France in 1797. Each illustration is finely drawn and col-
ored by hand. The rich variety of Grasset de Saint-Sauveur’s collection reminds us viv-
idly of how culturally apart the world’s towns and regions were just 200 years ago.
Isolated from each other, people spoke different dialects and languages. In the streets
or in the countryside, it was easy to identify where they lived and what their trade or
station in life was just by their dress.
The way we dress has changed since then, and the diversity by region, so rich at the
time, has faded away. It is now hard to tell apart the inhabitants of different conti-
nents, let alone different towns, regions, or countries. Perhaps we have traded cultural
diversity for a more varied personal life—certainly for a more varied and fast-paced
technological life.
At a time when it is hard to tell one computer book from another, Manning cele-
brates the inventiveness and initiative of the computer business with book covers
based on the rich diversity of regional life of two centuries ago, brought back to life by
Grasset de Saint-Sauveur’s pictures.

Part 1
Background
This part of the book covers background and gives you an introduction to
GitOps and Kubernetes.
Chapter 1 walks you through the journey of software deployment evolution
and how GitOps became the latest practice. It also covers the many key concepts
and benefits of GitOps.
Chapter 2 provides key concepts of Kubernetes and why its declarative nature
is perfect for GitOps. It also covers the core operator concept and how to imple-
ment a simple GitOps operator.
After you grasp the core concepts of GitOps and Kubernetes, you will be
ready to dive into the patterns and processes required to adopt GitOps in your
deployments. Part 2 covers the GitOps CI/CD pipeline along with environment
setup and promotion as well as different deployment strategies. It also covers
how you can secure your deployment process and reviews several configuration
management tools and various techniques to manage Secrets in GitOps. There is
also a chapter devoted to observability as it is related to GitOps.

3
Why GitOps?
Kubernetes is a massively popular open source platform that orchestrates and auto-
mates operations. Although it improves the management and scaling of infrastruc-
ture and applications, Kubernetes frequently has challenges managing the
complexity of releasing applications.
Git is the most widely used version-control system in the software industry today.
GitOps is a set of procedures that uses the power of Git to provide both revision and
change control within the Kubernetes platform. A GitOps strategy can play a big
part in how quickly and easily teams manage their services’ environment creation,
promotion, and operation.
Using GitOps with Kubernetes is a natural fit, with the deployment of declara-
tive Kubernetes manifest files being controlled by common Git operations. GitOps
brings the core benefits of Infrastructure as Code and immutable infrastructure to
This chapter covers
 What is GitOps?
 Why GitOps is important
 GitOps compared with other approaches
 Benefits of GitOps

4 CHAPTER 1 Why GitOps?
the deployment, monitoring, and life-cycle management of Kubernetes applications
in an intuitive, accessible way.
1.1 Evolution to GitOps
Two everyday tasks in managing and operating computer systems are infrastructure
configuration and software deployment. Infrastructure configuration prepares comput-
ing resources (such as servers, storage, and load balancers) that enable the software
application to operate correctly. Software deployment is the process of taking a particular
version of a software application and making it ready to run on the computing infra-
structure. Managing these two processes is the core of GitOps. Before we dig into how
this management is done in GitOps, however, it is useful to understand the challenges
that have led the industry toward DevOps and the immutable, declarative infrastruc-
ture of GitOps.
1.1.1 Traditional Ops
In a traditional information technology operations model, development teams are
responsible for periodically delivering new versions of a software application to a
quality-assurance (QA) team that tests the new version and then delivers it to an oper-
ations team for deployment. New versions of software may be released once a year,
once a quarter, or at shorter intervals. It becomes increasingly difficult for a tradi-
tional operations model to support increasingly compressed release cycles.
The operations team is responsible for the infrastructure configuration and deploy-
ment of the new software application versions to that infrastructure. The operations
team’s primary focus is to ensure the reliability, resilience, and security of the system
running the software. Without sophisticated management frameworks, infrastructure
management can be a difficult task that requires a lot of specialized knowledge.
Figure 1.1 Traditional IT teams are typically composed of separate development, QA, and operations
teams. Each team specializes in a different aspect of the application development process.
The development team is
responsible for writing
code that implements
business value.
The quality assurance
team is responsible for
verifying the code works
as intended and delivers
the business value.
The operations team is responsible
for the deployment and operation
of the system’s end users’ access to
receive the business value.
Development team
Quality assurance team
Operations team

5
Evolution to GitOps
IT OPERATIONS IT operations is the set of all processes and services that are
both provisioned by an IT staff to internal or external clients and used by the
staff to provide a business’s technology needs. Operations work can include
responding to tickets generated for maintenance work or customer issues.1
Because three teams are involved, often with different management-reporting struc-
tures, a detailed handoff process and thorough documentation of the application
changes are needed to ensure that the application is adequately tested, appropriate
changes are made to infrastructure, and the application is installed correctly. These
requirements, however, cause deployments to take a long time and reduce the fre-
quency at which deployments can be made. Also, with each transition between teams,
the possibility that essential details will not being communicated increases, possibly
leading to gaps in testing or incorrect deployment.
Figure 1.2 In the traditional deployment flow, the development team opens a ticket for the QA team
to test a new product version. When the testing is successful, the QA team opens a ticket for the
operations team to deploy the latest version to production.
Fortunately, most development teams compile, test, and produce their deployable
artifacts by using automated build systems and a process called continuous integration
(CI). But the new code’s deployment is often a manual process performed by the oper-
ations team, involving lengthy manual procedures or partial automation through
deployment scripts. In a worst-case scenario, the operations engineer manually copies
the executable binary file to the needed location on multiple servers and manually
restarts the application to make the new binary version take effect. This process is
1
https://en.wikipedia.org/wiki/Data_center_management#Operations.
Issue queue
Ticket 1 Ticket 2 Ticket 3
Development team Quality assurance team Operations team
Development team opens a
ticket for the QA team to
test the new build.
QA team opens a ticket for
the operations team to
deploy the new build.
Operations team deploys
the new build to the
production environment.

error prone and offers few options for controls such as review, approval, auditability,
and rollback.
CONTINUOUS INTEGRATION (CI) CI involves automated building, testing, and
packaging of software applications. In a typical development workflow, soft-
ware engineers make code changes that are checked into the central code
repository. These changes must be tested and integrated with the main code
branch intended to be deployed to production. A CI system facilitates the
review, building, and testing of code to ensure its quality before merging to
the main branch.
With the rise of cloud computing infrastructure, the interfaces that manage compute
and network resources have become increasingly based on application programming
interfaces (APIs), allowing for more automation but requiring more programming
skills to implement. This fact, coupled with many organizations’ search for ways to
optimize operations, reduce deployment times, increase deployment frequency, and
improve their computing systems’ reliability, stability, and performance, led to a new
industry trend: DevOps.
1.1.2 DevOps
DevOps is both an organizational structure and a mindset change with an emphasis on
automation. An operations team is no longer responsible for deployment and opera-
tion; the application’s development team takes on these responsibilities.
DEVOPS DevOps is a set of software development practices that combine soft-
ware development (Dev) and IT operations (Ops) to shorten the system
development life cycle while delivering features, fixes, and updates frequently
in close alignment with business objectives.2
Figure 1.3 shows how, in a traditional operations model, the organization is divided by
functional boundaries, with different teams for development, quality, and operations.
In the DevOps model, teams are divided by products or components and are interdis-
ciplinary, containing team members who have skill sets across all functions. Although
figure 1.3 indicates team members with a specific role, all members of a high-
functioning team practicing DevOps contribute across functions; each member is able
to code, test, deploy, and operate their product or component.
The benefits of DevOps include
 Better collaboration between development and operations
 Improved product quality
 More frequent releases
 Reduced time to market for new features
 Decreased costs of design, development, and operations
2
https://en.wikipedia.org/wiki/DevOps.

7
Evolution to GitOps
1.1.3 GitOps
The term GitOps was coined in August 2017 in a series of blogs by Alexis Richardson,
cofounder and CEO of Weaveworks.3
Since then, the term has developed significant
mindshare in the cloud-native community in general and the Kubernetes community
in particular. GitOps is a DevOps process characterized by
 Best practices for deployment, management, and monitoring of containerized
applications
 A developer-centric experience for managing applications, with fully auto-
mated pipelines/workflows using Git for development and operations
 Use of the Git revision control system to track and approve changes to the infra-
structure and run-time environment of applications
Case study: Netflix
Netflix was one of the early adopters of the DevOps process, with every engineer
being responsible for coding, testing, deployment, and support of their features. Net-
flix’s culture promotes “Freedom and Responsibility,” which means that every engi-
neer can push releases independently but must ensure the proper operation of that
release. All deployment processes are fully automated, so engineers can deploy and
roll back with the press of a button. All new features are in end users’ hands as soon
as the functionality is complete.
3
https://www.weave.works/blog/gitops-operations-by-pull-request.
Traditional organizational model
IT
Development Quality Operations
DevOps organizational model
IT
Team 1 Team 2 Team 3
Figure 1.3 The traditional organizational model has separate teams for development, quality, and
operations. A DevOps organizational model allows interdisciplinary teams centered on a specific
product or component. Each DevOps team is self-sufficient and contains members who have the
skills to develop, test, and deploy their application.

Figure 1.4 The GitOps release workflow starts with creating a branch of the repository containing
changes to the definition of the system’s desired state.
GitHub (along with GitLab, Bitbucket, and so on) is central to the modern software
development life cycle, so it seems natural that it is also used for systems operation
and management.
In a GitOps model, the system’s desired configuration is stored in a revision con-
trol system, such as Git. Instead of making changes directly to the system via a UI or
CLI, an engineer makes changes to the configuration files that represent the desired
state. A difference between the desired state stored in Git and the system’s actual state
indicates that not all changes have been deployed. These changes can be reviewed
and approved through standard revision control processes such as pull requests, code
reviews, and merges to master. When changes have been approved and merged to the
main branch, an operator software process is responsible for changing the system’s
current state to the desired state based on the configuration stored in Git.
In an ideal implementation of GitOps, manual changes to the system are not per-
mitted, and all changes to the configuration must be made to files stored in Git. In an
extreme case, permission to change the system is granted only to the operator soft-
ware process. The infrastructure and operations engineers’ role in a GitOps model
shifts from performing the infrastructure changes and application deployments to
developing and maintaining the GitOps automation and helping teams review and
approve changes by using Git.
Git has many features and technical capabilities that make it an ideal choice for
use with GitOps:
 Git stores each commit. With proper access control and security configuration
(covered in chapter 6), all changes are auditable and tamperproof.
 Each commit in Git represents a complete configuration of the system up to
that point in time.
Create a branch.
Code review
comments
Approve changes and
merge the branch.
Deploy the branch.
GitOps team

9
Developer benefits of GitOps
 Each commit object in Git is associated with its parent commit so that as
branches are created and merged, the commit history is available when needed.
NOTE GitOps is important because it enables traceability of changes made to
an environment and enables easy rollback, recoverability, and self-healing
with Git, a tool with which most developers are already familiar.
Git provides the basis to validate and audit deployments. Although it may be possible
to implement GitOps by using a version-control system other than Git, Git’s distrib-
uted nature, branching and merging strategy, and widespread adoption make it an
ideal choice.
GitOps doesn’t require a particular set of tools, but the tools must offer this stan-
dard functionality:
 Operate on the desired state of the system that is stored in Git
 Detect differences between the desired state and the actual state
 Perform the required operations on the infrastructure to synchronize the
actual state with the desired state
Although this book focuses on GitOps in relation to Kubernetes, many of the princi-
ples of GitOps could be implemented independently of Kubernetes.
1.2 Developer benefits of GitOps
GitOps provides many benefits to developers because it allows them to treat the con-
figuration of infrastructure and deployment of code in much the same way that they
manage their software development process, and with a familiar tool: Git.
1.2.1 Infrastructure as code
Infrastructure as code (IaC) is a foundational paradigm for GitOps. The configuration
of the infrastructure that runs your applications is accomplished by executing an auto-
mated process rather than manual steps.4
In practice, IaC means that infrastructure
changes are codified and the source code for the infrastructure is stored in a version-
control system. Let’s go through the most notable benefits:
 Repeatability—Everyone who has experience provisioning infrastructure manu-
ally agrees that this process is time consuming and error prone. Don’t forget
that the same process has to be repeated multiple times, because applications
are typically deployed into multiple environments. If a problem is discovered, it
is easier to roll back to an earlier working configuration with a repeatable pro-
cess, allowing quicker recoveries.
 Reliability—The automated process significantly reduces the chance of inevita-
ble human errors, thereby reducing the possibility of outages. When the pro-
cess is codified, infrastructure quality no longer depends on the knowledge and
4
https://www.hashicorp.com/resources/what-is-infrastructure-as-code.

skill of the particular engineer who is performing the deployment. The automa-
tion of the infrastructure configuration can be steadily improved.
 Efficiency—IaC increases the productivity of the team. With IaC, engineers are
more productive because they use familiar tools, such as APIs, software develop-
ment kits (SDKs), version-control systems, and text editors. Engineers can use
familiar processes and take advantage of code review and automated testing.
 Savings—The initial implementation of IaC requires significant investment of
effort and time. Despite the initial cost, however, it is more cost effective in the
long run. The provisioning of infrastructure for the next environment does not
require wasting valuable engineer time for manual configuration. Because pro-
visioning is quick and cheap, there is no need to keep unused environments
running. Instead, each environment might be created on demand and
destroyed when it is no longer needed.
 Visibility—When you define IaC, the code itself documents how the infrastruc-
ture should look.
IaC enables developers to produce higher-quality software while saving time and
money. It might be easier to configure the infrastructure manually for one environ-
ment, but it will become increasingly challenging to maintain that environment,
along with dozens of other environments for your application. Using automated infra-
structure provisioning and following IaC principles enables repeatable deployments
and prevents run-time issues caused by configuration drift or missing dependencies.
1.2.2 Self-service
As mentioned previously, in a traditional operations model, infrastructure manage-
ment is performed by a dedicated team or even a separate organization within the
company.
There is a problem, however: this approach does not scale. The dedicated team
will quickly become a bottleneck, no matter how many members it has. Instead of
making an infrastructure change themselves, application developers have to file a
ticket, send an email, schedule a meeting, and wait. Regardless of the process, a bar-
rier exists, introducing many delays and discouraging the team from proactively pro-
posing infrastructure changes. GitOps aims to break the barrier by automating the
process and making it self-service.
Instead of sending a ticket, when using a GitOps model, the developer works on a
solution independently and commits a change to the infrastructure’s declarative con-
figuration in the repository. The infrastructure change does not require cross-team
communication anymore, allowing the application development team to move for-
ward much more quickly and have more freedom to experiment. The ability to make
infrastructure changes rapidly and independently encourages developers to take own-
ership of their application infrastructure. Instead of asking a central operations team
for a solution, developers can experiment and develop a design that efficiently solves
the business requirements.

11
Developer benefits of GitOps
Figure 1.5 The development team can change the system’s desired state by updating files stored in the
Git repository. These changes are code-reviewed by other team members and merged to the main branch
upon approval. The main branch is processed by a GitOps operator that deploys the cluster’s desired
configuration.
Developers don’t get full control to do whatever they want, however, possibly compro-
mising security or reliability. Every change requires creating a pull request that can be
reviewed by another member of the application development team, as described in
the following sections.
The advantage of GitOps is that it allows self-service infrastructure changes and
provides the right balance between control and development speed.
1.2.3 Code reviews
Code review is a software development practice in which code changes are proactively
examined for errors or omissions by a second pair of eyes, leading to fewer prevent-
able outages. Performing code reviews is a natural process in the software develop-
ment life cycle with which software engineers doing DevOps/GitOps should be
familiar. When the DevOps engineer can treat infrastructure as code, the logical next
step is to perform code reviews on the infrastructure changes before deployment.
When GitOps is used with Kubernetes, the “code” being reviewed may be primarily
Kubernetes YAML manifests or other declarative configuration files, not traditional
code written in a programming language.
Besides error prevention, code reviews provide the following benefits:
 Teaching and sharing knowledge—While reviewing the changes, the reviewer has a
chance not only to give feedback, but also to learn something.
 Consistency in design and implementation—During the review, the team can ensure
that changes are aligned with the overall code structure and follow the com-
pany’s code style guidelines.
Development team
Infrastructure team GitOps operator
Ensures that
platform and
infrastructure
tools are running
Uses tools
provided by the
infrastructure
team

 Team cohesion—Code review is not only for criticizing and requesting changes.
This process is also an excellent way for team members to give kudos to one
another, get closer, and make sure that everyone is fully engaged.
In a proper code review process, only verified and approved infrastructure changes
are committed to the main branch, preventing errors and incorrect modifications to
the operating environment. Code review doesn’t necessarily need to be done entirely
by humans. The code review process also can run automated tools such as code lint-
ers,5
static code analysis, and security tools.
NOTE Other automated tools for code and vulnerability analysis are covered
in chapter 4.
Code reviews have long been accepted as a critical part of software development
best practices. The key premise of GitOps is that the same rigor of code reviews used
in the application code should be applied to changes in the application operational
environment.
1.2.4 Git pull requests
The Git version-control system provides a mechanism in which proposed changes can
be committed to a branch or fork and then merged with the main branch through a
pull request. In 2005, Git introduced a request-pull command. This command
generates a human-readable summary of all the changes, which can be mailed to the
project maintainer manually. The pull request collects all the changes to the reposi-
tory files and presents the differences for code review and approval.
Pull requests can be used to enforce premerge code reviews. Controls can be put
in place to require specific testing or approval before a pull request is merged to the
main branch. Like code reviews, pull requests are a familiar process in the software
development life cycle that software engineers likely already use.
Figure 1.6 demonstrates the typical pull request life cycle:
1 The developer creates a new branch and starts working on the changes.
2 When changes are ready, the developer sends a pull request for code review.
3 Team members review the pull request and request more changes (if needed).
4 The developer keeps making changes in a branch until the pull request is
approved.
5 The project maintainer merges the pull request into the main branch.
6 After the merge, the branch used for the pull request may be deleted.
5
https://en.wikipedia.org/wiki/Lint_(software).

13
Operational benefits of GitOps
Figure 1.6 The pull request life cycle allows multiple rounds of code review and revisions until the
changes are approved. Then the changes may be merged to the main branch and the pull request branch
deleted.
The review step is especially interesting when applied to an infrastructure change
review. After the pull request is created, the project maintainers receive a notification
and review the proposed changes. As a result, reviewers ask questions, receive answers,
and possibly request more changes. That information is typically stored and available
for future reference, so now the pull request is a live documentation of an infrastruc-
ture change. In case of an incident, it is straightforward to find out who made the
change and why it was applied.
1.3 Operational benefits of GitOps
Combining a GitOps methodology with Kubernetes’ declarative configuration and
active reconciliation model provides many operational benefits that provide a more
predictable and reliable system.
1.3.1 Declarative
One of the most prominent paradigms to emerge from the DevOps movement is the
model of declarative systems and configuration. Simply put, with declarative models, you
describe what you want to achieve as opposed to how to get there. By contrast, in an
imperative model, you describe a sequence of instructions for manipulating the sys-
tem to reach your desired state.
To illustrate this difference, imagine two styles of a television remote control: an
imperative style and a declarative style. Both remotes can control the TV’s power, vol-
ume, and channel. For the sake of discussion, assume that the TV has only three vol-
ume settings (loud, soft, mute) and only three channels (1, 2, 3).
1. The developer creates
a new branch and
starts working on the
changes.
2. Developer sends a pull
request and requests
code review.
3. Team members review
pull request and
request more changes.
4. Developer keeps
making changes.
5. The project
maintainer
merges pull
request.

.
Figure 1.7 This figure illustrates the differences between imperative and declarative
remote controls. The imperative remote lets you perform operations such as
“Increment the channel by 1” and “Toggle the state of the power.” By contrast, the
declarative remote lets you perform operations such as “Tune to channel 2” or “Set
the state of the power to off.”
IMPERATIVE REMOTE EXAMPLE
Suppose that you had the simple task of changing to channel 3 with both remotes. To
accomplish this task with the imperative remote, you would use the channel-up but-
ton, which signals the TV to increment the current channel by 1. To reach channel 3,
you would keep pressing the channel-up button some number of times until the TV
reached the desired channel.
DECLARATIVE REMOTE EXAMPLE
By contrast, the declarative remote provides individual buttons that jump directly to the
specific numbered channel. In this case, to switch to channel 3, you would press the
channel- 3 button once, and the TV would be on the correct channel. You are declaring
your intended end state (I want the TV to be tuned to channel 3). With the imperative
remote, you describe the actions that you need to be performed to achieve your desired
state (keep pressing the channel-up button until the TV is tuned to channel 3).
You may have noticed that in the imperative approach to changing channels, the
user must consider whether to continue pushing the channel-up button, depending
on the channel to which the TV is currently tuned. In the declarative approach, how-
ever, you can press the channel-3 button without a second thought because that but-
ton on the declarative remote is considered to be idempotent (and the channel-up
button on the imperative remote is not).
Imperative remote control
OFF ON
VOL
2
3
1
CHAN
Declarative remote control
VOL CHAN

15
IDEMPOTENCY Idempotency is a property of an operation whereby the opera-
tion can be performed any number of times and produce the same result. In
other words, an operation is said to be idempotent if you can perform the
operation an arbitrary number of times, and the system is in the same state as
it would be if you had performed the operation only once. Idempotency is
one of the properties that distinguish declarative systems from imperative sys-
tems. Declarative systems are idempotent; imperative systems are not.
1.3.2 Observability
Observability is the ability to examine and describe a system’s current running state and
be alerted when unexpected conditions occur. Deployed environments are expected
to be observable. In other words, you should always be able to inspect an environment
to see what is currently running and how things are configured. To that end, service
and cloud providers provide a fair number of methods to promote observability
(including CLIs, APIs, GUIs, dashboards, alerts, and notifications), making it as con-
venient as possible for users to understand the current state of the environment.
Although these observability mechanisms can help answer the question, “What’s cur-
rently running in my environment?” they cannot answer the question, “For the
resources currently configured and running in my environment, are they supposed to be
configured and running in that way?” If you have ever held duties as a systems admin-
istrator or operator, you’re likely to be all too familiar with this problem. At one point
or another—typically, when troubleshooting an environment—you come across a sus-
picious configuration setting and think that it doesn’t seem right. Did someone (possi-
bly you) accidentally or mistakenly change this setting, or is the setting intentional?
In all likelihood, you may already be practicing a fundamental principle of GitOps:
storing a copy of your application configuration in source control and using it as a
source of truth for the desired state of your application. You may not be storing this
configuration in Git to drive continuous deployment—only to have a copy duplicated
somewhere so that the environment can be reproduced, such as in a disaster recovery
scenario. This copy can be thought of as the desired application state, and aside from
the disaster recovery use case, it serves another useful purpose: it enables operators to
Observe
Operator
Environment
Running state
Figure 1.8 Observability is the operator’s
ability (perhaps human or automated) to
determine an environment’s running state.
The operator can make informed decisions
about what changes to the environment
are needed only once if the environment’s
current running state is known. Proper
control of an environment requires
observability of that environment.

compare the actual running state
with the desired state held in
source control at any point in time
to verify that the states match.
The ability to verify your envi-
ronment is a core tenet of GitOps
that has been formalized as a
practice. By storing your desired
state in one system (such as Git)
and regularly comparing that
desired state with the running
state, you unlock a new dimen-
sion of observability. Not only do
you have the standard observabil-
ity mechanisms provided by your
provider, but also, you are able to
detect divergence from the
desired state.
Divergence from your desired
state, also called configuration
drift, can happen for any number of reasons. Common examples include mistakes
made by operators, unintended side effects due to automation, and error scenarios.
Configuration drift could even be expected, such as a temporary state caused by a
transition period (maintenance mode, for example).
But the most significant reason for a divergence in configuration could be mali-
cious. In the worst case, a bad actor could have compromised the environment and
reconfigured the system to run a malicious image. For this reason, observability and
verification are crucial for the security of the system. Unless you have a source of truth
of your desired state in place, and without a mechanism to verify convergence to that
source of truth, it is impossible to know that your environment is truly secure.
1.3.3 Auditability and compliance
Allowing for compliance and auditability is a must for organizations that do business
in countries whose laws and regulations affect information management and frame-
works for assessing compliance—which is most countries in this day and age. Some
industries are more regulated than others, but almost all companies need to comply
with basic privacy and data security laws. Many organizations have to invest substan-
tially in their processes and systems to be compliant and auditable. With GitOps and
Kubernetes, most of the compliance and auditability requirements can be satisfied
with minimal effort.
Compliance refers to verifying that an organization’s information system meets a
particular set of industry standards, typically focused on customer data security and
adherence to the organization’s documented policies on the people and systems that
Verify
Observe
Operator
Environment
Running state
Source code
repository
Desired state
Figure 1.9 If the environment’s running state can be
observed, and the desired state of the environment is
defined in Git, the environment can be verified by
comparing the two states.

17
have access to that customer data. Chapter 6 covers access control in depth, and chap-
ter 4 covers pipelines to define and enforce your deployment process for compliance.
Auditability is a system’s capability to be verified as being compliant with a set of stan-
dards. If a system can’t be shown to an internal or external auditor to be compliant, no
statement about the system’s compliance can be made. Chapter 8 covers observability,
including using the Git commit history and Kubernetes events for auditability.
Auditability also refers to an auditor’s ability to achieve a comprehensive examination
of an organization’s internal controls. In a typical audit, the auditor requests evidence
to ensure that rules and policies are enforced accordingly. Evidence could include the
process of restricting access to user data, the handling of personally identifiable infor-
mation (PII), and the integrity of the software release process.
Figure 1.10 In a traditional audit process, it is often difficult to determine the
system’s desired state. Auditors may need to look at various sources of this
information, including documentation, change requests, and deployment scripts.
Case study: Facebook and Cambridge Analytica
Cambridge Analytica, a political data firm hired by President Trump’s 2016 election
campaign, gained improper access to the private information of more than 50 million
Facebook users. The data was used to generate a personality score for each user
and match that user with US voter records. Cambridge Analytica used this information
for its voter profiling and targeted advertising services. Facebook was found not to
have implemented the proper controls required to enforce data privacy and was even-
tually fined $5 billion by the Federal Trade Commission due to the breach.a
a
https://www.ftc.gov/news-events/press-releases/2019/07/ftc-imposes-5-billion-penalty-sweeping-new-privacy-
restrictions
Auditor
Hosting service
Scripts and configs
Documentation and
ticketing systems
Developers and
operations engineers Source code
repository

What does all that have to do with GitOps?
Git is version-control software that helps organizations manage changes and access
control to their code. Git keeps track of every modification to the code in a special
kind of database designed to preserve the managed source code’s integrity. The files’
content and the true relationships among files and directories, versions, tags, and
commits in the Git repository are secured with the Secure Hash Algorithm (SHA)
checksum hashing algorithm. This algorithm protects the code and the change his-
tory from both accidental and malicious change and ensures that the history is fully
traceable.
Git’s history tracking also includes the author, date, and written notes on each
change’s purpose. With well-written commit comments, you know why a particular
commit was made. Git can also integrate with project management and bug-tracking
software, allowing full traceability of all changes and enabling root-cause analysis and
other forensics.
As mentioned earlier, Git supports the pull request mechanism, which prevents
any single person from altering the system without approval by a second person.
When the pull request is approved, changes are recorded in the secured Git change
history. Git’s strength in change control, traceability, and change history authenticity,
along with Kubernetes’ declarative configuration, naturally satisfy the security, avail-
ability, and processing integrity principles needed for auditability and compliance.
Case study: Payment Card Industry Data Security Standard
The Payment Card Industry Data Security Standard (PCI DSS) is an information secu-
rity standard for organizations that handle branded credit cards from the major card
networks. Violation of the PCI DSS could result in steep fines and, in the worst case,
suspension from credit card processing. PCI DSS dictates that “access control sys-
tems are configured to enforce privileges assigned to individuals based on job clas-
sification and function.” During an audit, organizations need to provide evidence that
access control systems are in place for PCI compliance.a
a
https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard
Auditor
Source code
repository
Hosting service
Figure 1.11 With GitOps,
the audit process can be
simplified because the
auditor can determine the
system’s desired state by
examining the source code
repository. The current
state of the system can be
determined by reviewing
the hosting service and
Kubernetes objects.

19
Summary
1.3.4 Disaster recovery
Disasters happen for many reasons and take many forms. A disaster may be naturally
occurring (an earthquake hitting a data center), caused by an equipment failure (loss
of hard drives in a storage array), accidental (a software bug corrupting a critical data-
base table), or even malicious (a cyberattack causing data loss).
GitOps helps in the recovery of infrastructure environments by storing declarative
specifications of the environment under source control as a source of truth. Having a
complete definition of what the environment should be facilitates the re-creation of
the environment in the event of a disaster. Disaster recovery becomes a simple exer-
cise of (re)applying all the configuration stored in the Git repository. You might
observe that there is not much difference between the procedures followed during a
disaster versus those used in routine day-to-day upgrades and deployments. With
GitOps, you are in effect practicing disaster recovery procedures on a regular basis,
making you well prepared if a real disaster strikes.
IMPORTANCE OF DATA BACKUPS Although GitOps helps simplify disaster
recovery for computing and networking infrastructure, recovery of persistent
and stateful applications needs to be handled differently. There is no substi-
tute for traditional disaster recovery solutions for storage-related infrastruc-
ture: backups, snapshotting, and replication.
Summary
 GitOps is a DevOps deployment process that uses Git as the system of record to
manage deployment in complex systems.
 Traditional Ops requires a separate team for deployment, and a new version
can take days (if not weeks) to be deployed.
 DevOps enables engineers to deploy a new version as soon as the code is com-
plete without waiting for a centralized operations team.
 GitOps provides full traceability and release control.
 Declarative models describe what you want to achieve instead of the steps neces-
sary to achieve it.
 Idempotency is a property of an operation whereby the operation can be per-
formed any number of times and produce the same result.
 Additional GitOps benefits include
– Pull requests for code quality and release control
– Observable running state and desired state
– Simplified compliance and auditability process with historical authenticity
and traceability
– Straightforward disaster recovery and rollback procedures that are consistent
with the familiar deployment experience

20
Kubernetes and GitOps
In chapter 1, you learned about Kubernetes and why its declarative model makes it
an excellent match to be managed using GitOps. This chapter will briefly introduce
Kubernetes architecture and objects and the differences between declarative and
imperative object management. By the end of this chapter, you will implement a
basic GitOps Kubernetes deployment operator.
2.1 Kubernetes introduction
Before diving into why Kubernetes and GitOps work so well together, let’s talk about
Kubernetes itself. This section provides a high-level overview of Kubernetes, how it
compares to other container orchestration systems, and its architecture. We will also
have an exercise that demonstrates how to run Kubernetes locally, which will be used
This chapter covers
 Solving problems with Kubernetes
 Running and managing Kubernetes locally
 Understanding the basics of GitOps
 Implementing a simple Kubernetes GitOps
operator

21
Kubernetes introduction
for the other exercises in this book. This section is only a brief introduction and
refresher on Kubernetes. For a fun but informative overview of Kubernetes, check out
“The Illustrated Children’s Guide to Kubernetes” and “Phippy Goes to the Zoo” by the
Cloud Native Computing Foundation.1
If you are completely new to Kubernetes, we
recommend reading Kubernetes in Action, Second Edition, by Marko Lukša (Manning,
2020) and then returning to this book. If you are already familiar with Kubernetes and
running minikube, you may skip to the exercise at the end of section 2.1.
2.1.1 What is Kubernetes?
Kubernetes is an open source container orchestration system released in 2014. OK,
but what are containers, and why do you need to orchestrate them?
Containers provide a standard way to package your application’s code, configura-
tion, and dependencies into a single resource. This enables developers to ensure that
the application will run properly on any other machine regardless of any customized
settings that machine may have that could differ from the machine used for writing
and testing the code. Docker simplified and popularized containerization, which is
now recognized as a fundamental technology used to build distributed systems.
CHROOT An operation available in UNIX operating systems, which changes
the apparent root directory for the current running process and its children.
Chroot provides a way to isolate a process and its children from the rest of the
system. It was a precursor to containerization and Docker.2
While Docker solved the packaging and isolation problem of individual applications,
there were still many questions about how to orchestrate the operation of multiple
applications into a working distributed system:
 How do containers communicate?
 How is traffic routed between containers?
 How are containers scaled up to handle additional application load?
 How is the underlying infrastructure of the cluster scaled up to run the
required containers?
All these operations are the responsibility of a container orchestration system and are
provided by Kubernetes. Kubernetes helps to automate the deployment, scaling, and
management of applications using containers.
NOTE Borg is Google’s internal container cluster management system used
to power online services like Google search, Gmail, and YouTube. Kubernetes
leverages Borg’s innovations and lessons learned, explaining why it is more
stable and moves so much more quickly than its competitors.3
1
https://www.cncf.io/phippy.
2
https://en.wikipedia.org/wiki/Chroot.
3
https://kubernetes.io/blog/2015/04/borg-predecessor-to-kubernetes.

22 CHAPTER 2 Kubernetes and GitOps
Kubernetes was initially developed and open-sourced by Google based on a decade of
experience with container orchestration using Borg, Google’s proprietary cluster
management system. Because of this, Kubernetes is relatively stable and mature for a
system so complex. Because of its open API and extendable architecture, Kubernetes
has developed an extensive community around it, which has further fueled its success.
It is one of the top GitHub projects (as measured by stars), provides excellent docu-
mentation, and has a significant Slack and Stack Overflow community. An endless
number of blogs and presentations from community members share their knowledge
of using Kubernetes. Despite being started by Google, Kubernetes is not influenced by
a single vendor. This makes the community open, collaborative, and innovative.
2.1.2 Other container orchestrators
Since late 2016, Kubernetes has become recognized as the dominant de facto industry-
standard container orchestration system in much the same way that Docker has become
the standard for containers. However, several Kubernetes alternatives address the same
container orchestration problem as Kubernetes. Docker Swarm is Docker’s native con-
tainer orchestration engine that was released in 2015. It is tightly integrated with the
Docker API and uses a YAML-based deployment model called Docker Compose.
Apache Mesos was officially released in 2016 (although it has a history well before then)
and supports large clusters, scaling to thousands of nodes.
While it may be possible to apply a GitOps approach to deploying applications
using other container orchestration systems, this book focuses on Kubernetes.
2.1.3 Kubernetes architecture
By the end of this chapter, you will complete an exercise that implements a basic
GitOps continuous deployment operator for Kubernetes. But to understand how a
GitOps operator functions, it is essential that you first understand a few Kubernetes
core concepts and learn how it is organized at a high level.
Kubernetes is an extensive and robust system with many different types of
resources and operations that can be performed on those resources. Kubernetes pro-
vides a layer of abstraction over the infrastructure and introduces the following set of
basic objects that represent the desired cluster state:
 Pod—A group of containers deployed together on the same host. The Pod is
the smallest deployable unit on a node and provides a way to mount storage, set
environment variables, and provide other container configuration information.
When all the containers of a Pod exit, the Pod dies also.
 Service—An abstraction that defines a logical set of Pods and a policy to access
them.
 Volume—A directory accessible to containers running in a Pod.
Kubernetes architecture uses primary resources as a foundational layer for a set of
higher-level resources. The higher-level resources implement features needed for real

23
production use cases that leverage/extend the primary resources’ functionality. In fig-
ure 2.1, you see that the ReplicaSet resource controls the creation of one or more Pod
resources. Some other examples of high-level resources include
 ReplicaSet—Defines that a desired number of identically configured Pods are
running. If a Pod in the ReplicaSet terminates, a new Pod will be started to
bring the number of running Pods back to the desired number.
 Deployment—Enables declarative updates for Pods and ReplicaSets.
 Job—Creates one or more Pods that run to completion.
 CronJob—Creates Jobs on a time-based schedule.
Another important Kubernetes resource is the Namespace. Most kinds of Kubernetes
resources belong to one (and only one) Namespace. A Namespace defines a naming
scope where resources within a particular Namespace must be uniquely named. Name-
spaces also provide a way to isolate users and applications from each other through role-
based access controls (RBACs), network policies, and resource quotas. These controls
allow creating a multitenant Kubernetes cluster where multiple users share the same
cluster and avoid impacting each other (for example, the “noisy neighbor” problem).
Service
ReplicaSet
Namespace
Persistent Volume claim
Manages
Pod
creation
Routes
internal
traffic
Provides
persistent
storage
Persistent Volume
Container
Pod
Container
Volume
Container
Pod
Container
Volume
Persistent Volume claim
Persistent Volume
External traffic
Figure 2.1 This diagram illustrates a typical Kubernetes environment deployed
in a Namespace. A ReplicaSet is an example of a higher-level resource that
manages the life cycle of Pods, which are lower-level, primary resources.

As we will see in chapter 3, Namespaces are also essential in GitOps for defining appli-
cation environments.
Kubernetes objects are stored in a control plane,4
which monitors the cluster state,
makes changes, schedules work, and responds to events. To perform these duties,
each Kubernetes control plane runs the following three processes:
 kube-apiserver—An entry point to the cluster providing a REST API to eval-
uate and update the desired cluster state
Figure 2.2 A Kubernetes cluster consists of several Services that run on the master nodes of the control
plane and several other Services that run on the cluster’s worker nodes. Together, these Services provide
the essential Services that make up a Kubernetes cluster.
4
https://kubernetes.io/docs/concepts/overview/components/#control-plane-components.
etcd
etcd
api
kube-apiserver
sched
kube-scheduler
c-m
kube-controller-manager
Master
The cluster control plane
Key-value database used
as backing store for all
cluster configuration data
Decides which
node should be
used for each Pod
Runs Kubernetes
controllers
Allows interacting
with the control plane
Manages containers on the node
kubelet
pod k-proxy
kubelet
pod k-proxy
kubelet
pod k-proxy
Maintains network rules on nodes
Node
Linux
Node
Linux
Node
Linux

25
 kube-controller-manager—Daemon continuously monitoring the shared
state of the cluster through the API server to make changes attempting to move
the current state toward the desired state
 kube-scheduler—A component that is responsible for scheduling the work-
loads across the available nodes in the cluster
 etcd—A highly available key-value database typically used as Kubernetes’ back-
ing store for all cluster configuration data
The actual cluster workloads run using the compute resources of Kubernetes nodes. A
node is a worker machine (either a VM or physical machine) that runs the necessary
software to allow it to be managed by the cluster. Similar to the masters, each node
runs a predefined set of processes:
 kubelet—The primary “node agent” that manages the actual containers on
the node
 kube-proxy—A network proxy that reflects Services as defined in the Kuberne-
tes API on each node and can do simple TCP, UDP, and SCTP stream forwarding
2.1.4 Deploying to Kubernetes
In this exercise, you will deploy a website using NGINX on Kubernetes. You will review
some basic Kubernetes operations and become familiar with minikube, the single-
node Kubernetes environment you will use for most exercises in this book.
KUBERNETES TEST ENVIRONMENT: MINIKUBE Refer to appendix A to set up a
Kubernetes test environment using minikube to complete this exercise.
CREATING A POD
As was mentioned earlier in the chapter, a Pod is the smallest object in Kubernetes
and represents a particular application workload. A Pod represents a group of related
containers running on the same host and having the same operating requirements.
All containers of a single Pod share the same network address, port space, and
(optionally) file system using Kubernetes Volumes.
NGINX NGINX is an open source software web server used by many organiza-
tions and enterprises to host their websites because of its performance and
stability.
In this exercise, you will create a Pod that hosts a website using NGINX. In Kuberne-
tes, objects can be defined by a YAML text file “manifest” that provides all the informa-
tion needed for Kubernetes to create and manage the object. Here is the listing for
our NGINX Pod manifest.

kind: Pod
apiVersion: v1
metadata:
name: nginx
spec:
restartPolicy: Always
volumes:
- name: data
emptyDir: {}
initContainers:
- name: nginx-init5
image: docker/whalesay
command: [sh, -c]
args: [echo "<pre>$(cowsay -b 'Hello Kubernetes')</pre>" > /data/
index.html]
volumeMounts:
- name: data
mountPath: /data
containers:
- name: nginx
image: nginx:1.11
volumeMounts:
- name: data
mountPath: /usr/share/nginx/html
You are welcome to type in this listing and save it with a filename of nginx-Pod.yaml.
However, since this book’s object isn’t to improve your typing skills, we recommend
cloning our public Git repository mentioned in chapter 1 that contains all the listings
in this book and using those files directly:
https://github.com/gitopsbook/resources
Let’s go ahead and start a minikube cluster and create the NGINX Pod using the fol-
lowing commands:
$ minikube start
(minikube/default)
minikube v1.1.1 on darwin (amd64)
Creating virtualbox VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
Configuring environment for Kubernetes v1.14.3 on Docker 18.09.6
Pulling images ...
Launching Kubernetes ...
Verifying: apiserver proxy etcd scheduler controller dns
Done! kubectl is now configured to use "minikube"
$ kubectl create -f nginx-Pod.yaml
Pod/nginx created
Listing 2.1 NGINX Pod manifest (http://mng.bz/e5JJ)
5
https://en.wikipedia.org/wiki/Cowsay.
The field kind and apiVersion are present
in every Kubernetes resource and
determine what type of object should be
created and how it should be handled.
In this example, metadata has a
name field that helps identify each
Kubernetes resource. Metadata may
also contain UID, labels, and other
fields that will be covered later.
The spec section contains configuration that
is specific to a particular kind of object. In
the Pod example, spec includes a list of
containers, the Volume shared between
containers, and the Pod’s restartPolicy.
The Volume that is
used to share data
between containers
The init section contains HTML
generated using the cowsay5
command.
The main container that
serves the generated HTML
file using the NGINX server

27
Figure 2.3 shows what the Pod looks like running inside the minikube.
Figure 2.3 The nginx-init container writes the desired index.html file to the mounted
Volume. The main NGINX container also mounts the Volume and displays the generated
index.html when receiving HTTP requests.
GETTING POD STATUS
As soon as the Pod is created, Kubernetes inspects the spec field and attempts to run
the configured set of containers on an appropriate node in the cluster. The informa-
tion about progress is available in the Pod manifest in the status field. The kubectl
utility provides several commands to access it. Let’s try to get the Pod status using the
kubectl get Pods command:
$ kubectl get Pods
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 36s
The get Pods command provides a list of all the Pods running in a particular Name-
space. In this case, we didn’t specify a Namespace, so it gives the list of Pods running
in the default Namespace. Assuming all goes well, the NGINX Pod should be in the
Running state.
To learn even more about a Pod’s status or debug why the Pod is not in the
Running state, the kubectl describe Pod command outputs detailed information,
including related Kubernetes events:
$ kubectl describe Pod nginx
Name: nginx
Namespace: default
Priority: 0
Node: minikube/192.168.99.101
Pod
nginx-init container
nginx container
Volume
Volume
mount
Volume
mount
The Volume that is used to share
data between containers
The init container that
generates the HTML file
The main container that
serves the HTML file

Start Time: Sat, 26 Oct 2019 21:58:43 -0700
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"nginx",
"Namespace":"default"},"spec":{"containers":[{"image":"nginx:1...
Status: Running
IP: 172.17.0.4
Init Containers:
nginx-init:
Container ID: docker://
128c98e40bd6b840313f05435c7590df0eacfc6ce989ec15cb7b484dc60d9bca
Image: docker/whalesay
Image ID: docker-pullable://docker/
whalesay@sha256:178598e51a26abbc958b8a2e48825c90bc22e641de3d31e18aaf55f325
8ba93b
Port: <none>
Host Port: <none>
Command:
sh
-c
Args:
echo "<pre>$(cowsay -b 'Hello Kubernetes')</pre>" > /data/index.html
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sat, 26 Oct 2019 21:58:45 -0700
Finished: Sat, 26 Oct 2019 21:58:45 -0700
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/data from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-vbhsd
(ro)
Containers:
nginx:
Container ID: docker://
071dd946709580003b728cef12a5d185660d929ebfeb84816dd060167853e245
Image: nginx:1.11
Image ID: docker-pullable://
nginx@sha256:e6693c20186f837fc393390135d8a598a96a833917917789d63766cab6c59
582
Port: <none>
Host Port: <none>
State: Running
Started: Sat, 26 Oct 2019 21:58:46 -0700
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/usr/share/nginx/html from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-vbhsd (ro)
Conditions:
Type Status
Initialized True

Exploring the Variety of Random
Documents with Different Content

"Good day, uncle," she said, kissing him.
"Good day, my daughter," the general replied. He was accustomed to
call her so. "Eh! eh! my child, you are very gay this morning."
And he returned with interest the caresses she had lavished upon
him.
"Why should I not be gay, uncle? Thanks to God? we have just
escaped a great peril; everything in nature seems to smile, the birds
are singing upon every branch, the sun inundates us with warm
rays; we should be ungrateful towards the Creator if we remained
insensible to these manifestations of His goodness."
"Then the perils of last night have left no distressing impression
upon your mind, my dear child?"
"None at all, uncle, except a deep sense of gratitude for the benefits
God has favoured us with."
"That is well, my daughter," the general replied joyfully, "I am happy
to hear you speak thus."
"All the better, if it please you, uncle."
"Then," the general continued, following up the idea of his
preoccupation, "the life we are now leading is not fatiguing to you?"
"Oh, not at all; on the contrary, I find it very agreeable, and, above
all, full of incidents," she said with a smile.
"Yes," the general continued, partaking her gaiety; "but," he added,
becoming serious again, "I think we are too forgetful of our
liberators."
"They are gone," Doña Luz replied.
"Gone?" the general said, with great surprise.
"Full an hour ago."
"How do you know that, my child?"
"Very simply, uncle, they bade me adieu before they left us."

"That is not right," the general murmured in a tone of vexation; "a
service is as binding upon those who bestow it as upon those who
receive it; they should not have left us thus without bidding me
farewell, without telling us whether we should ever see them again,
and leaving us even unacquainted with their names."
"I know them."
"You know them, my daughter?" the general said, with
astonishment.
"Yes, uncle; before they went, they told me."
"And—what are they?" the general asked, eagerly.
"The younger is named Belhumeur."
"And the elder?"
"Loyal Heart."
"Oh! I must find these two men again," the general said, with an
emotion he could not account for.
"Who knows," the young girl replied, thoughtfully, "perhaps in the
very first danger that threatens us they will make their appearance
as our benevolent genii."
"God grant we may not owe their return among us to a similar
cause."
The captain came up to pay the compliments of the morning.
"Well, captain," said the general, with a smile, "have you recovered
from the effects of their alarm?"
"Perfectly, general," the young man replied, "and are quite ready to
proceed, whenever you please to give the order."
"After breakfast we will strike tents; have the goodness to give the
necessary orders to the lancers, and send the Babbler to me."
The captain bowed and retired.

"On your part, niece," the general continued, addressing Doña Luz,
"superintend the preparations for breakfast, if you please, whilst I
talk to our guide."
The young lady tripped away, and the Babbler almost immediately
entered.
His air was dull, and his manner more reserved than usual.
The general took no notice of this.
"You remember," he said, "that you yesterday manifested an
intention of finding a spot where we might conveniently encamp for
a few days?"
"Yes, general."
"You told me you were acquainted with a situation that would
perfectly suit our purpose?"
"Yes, general."
"Are you prepared to conduct us thither?"
"When you please."
"What time will it require to gain this spot?"
"Two days."
"Very well. We will set out, then, immediately after breakfast."
The Babbler bowed without reply.
"By the way," the general said, with feigned indifference, "one of
your men seems to be missing."
"Yes."
"What is become of him?"
"I do not know."
"How! you do not know?" said the general, with a scrutinizing
glance.

"No: as soon as he saw the fire, terror seized him, and he escaped."
"Very well!"
"He is most probably the victim of his cowardice."
"What do you mean by that?"
"The fire, most likely, has devoured him."
"Poor devil!"
A sardonic smile curled the lips of the guide.
"Have you anything more to say to me, general?"
"No;—but stop."
"I attend your orders."
"Do you know the two hunters who rendered us such timely
service?"
"We all know each other in the prairie."
"What are those men?"
"Hunters and trappers."
"That is not what I ask you."
"What then?"
"I mean as to their character."
"Oh!" said the guide, with an appearance of displeasure.
"Yes, their moral character."
"I don't know anything much about them."
"What are their names?"
"Belhumeur and Loyal Heart."
"And you know nothing of their lives?"
"Nothing."

"That will do—you may retire."
The guide bowed, and with tardy steps rejoined his companions,
who were preparing for departure.
"Hum!" the general murmured, as he looked after him, "I must keep
a watch upon that fellow; there is something sinister in his manner."
After this aside, the general entered his tent, where the doctor, the
captain, and Doña Luz were waiting breakfast for him.
Half an hour later, at most, the tent was folded up again, the
packages were placed upon the mules, and the caravan was
pursuing its journey under the direction of the Babbler, who rode
about twenty paces in advance of the troop.
The aspect of the prairie was much changed since the preceding
evening.
The black, burnt earth, was covered in places with heaps of smoking
ashes; here and there charred trees, still standing, displayed their
saddening skeletons; the fire still roared at a distance, and clouds of
coppery smoke obscured the horizon.
The horses advanced with precaution over this uneven ground,
where they constantly stumbled over the bones of animals that had
fallen victims to the terrible embraces of the flames.
A melancholy sadness, much increased by the sight of the prospect
unfolded before them, had taken possession of the travellers; they
journeyed on, close to each other, without speaking, buried in their
own reflections.
The road the caravan was pursuing wound along a narrow ravine,
the dried bed of some torrent, deeply enclosed between two hills.
The ground trodden by the horses was composed of round pebbles,
which slipped from under their hoofs, and augmented the difficulties
of the march, which was rendered still more toilsome by the burning
rays of the sun, that fell directly down upon the travellers, leaving no
chance of escaping them, for the country over which they were

travelling had completely assumed the appearance of one of those
vast deserts which are met with in the interior of Africa.
The day passed away thus, and excepting the fatigue which
oppressed them, the monotony of the journey was not broken by
any incident.
In the evening they encamped in a plain absolutely bare, but in the
horizon they could perceive an appearance of verdure, which
afforded them great consolation;—they were about, at last, to enter
a zone spared by the conflagration.
The next morning, two hours before sunrise, the Babbler gave
orders to prepare for departure.
The day proved more fatiguing than the last; the travellers were
literally worn out when they encamped.
The Babbler had not deceived the general. The site was admirably
chosen to repel an attack of the Indians. We need not describe it;
the reader is already acquainted with it. It was the spot on which we
met with the hunters, when they appeared on the scene for the first
time.
The general, after casting around him the infallible glance of the
experienced soldier, could not help manifesting his satisfaction.
"Bravo!" he said to the guide; "if we have had almost
insurmountable difficulties to encounter in getting here, we could at
least, if things should so fall out, sustain a siege on this spot."
The guide made no reply; he bowed with an equivocal smile, and
retired.
"It is surprising," the general murmured to himself, "that although
that man's conduct may be in appearance loyal, and however
impossible it may be to approach him with the least thing,—in spite
of all that, I cannot divest myself of the presentiment that he is
deceiving us, and that he is contriving some diabolical project
against us."

The general was an old soldier of considerable experience, who
would never leave anything to chance, that deus ex machinâ, which
in a second destroys the best contrived plans.
Notwithstanding the fatigue of his people, he would not lose a
moment; aided by the captain, he had an enormous number of trees
cut down, to form a solid intrenchment, protected by chevaux de
frise. Behind this intrenchment the lancers dug a wide ditch, of
which they threw out the earth on the side of the camp; and then,
behind this second intrenchment, the baggage was piled up, to
make a third and last enclosure.
The tent was pitched in the centre of the camp, the sentinels were
posted, and everyone else went to seek that repose of which they
stood so much in need.
The general, who intended sojourning on this spot for some time,
wished, as far as it could be possible, to assure the safety of his
companions, and, thanks to his minute precautions, he believed he
had succeeded.
For two days the travellers had been marching along execrable
roads, almost without sleep, only stopping to snatch a morsel of
food; as we have said, they were quite worn out with fatigue.
Notwithstanding, then, their desire to keep awake, the sentinels
could not resist the sleep which overpowered them and they were
not long in sinking into as complete a forgetfulness as their
companions.
Towards midnight, at the moment when everyone in the camp was
plunged in sleep, a man rose softly, and creeping along in the shade,
with the quickness of a reptile, but with extreme precaution, he
glided out of the barricades and intrenchments.
He then went down upon the ground, and by degrees, in a manner
almost insensibly, directed his course, upon his hands and knees,
through the high grass towards a forest which covered the first
ascent of the hill, and extended some way into the prairie. When he

had gone a certain distance, and was safe from discovery, he rose
up.
A moonbeam, passing between two clouds, threw a light upon his
countenance.
That man was the Babbler.
He looked round anxiously, listened attentively, and then with
incredible perfection imitated the cry of the prairie dog.
Almost instantly the same cry was repeated, and a man rose up,
within at most ten paces of the Babbler.
This man was the guide who, three days before, had escaped from
the camp on the first appearance of the conflagration.
CHAPTER XI.
THE BARGAIN.
Indians and wood rangers have two languages, of which they make
use by turns, according to circumstances—spoken language, and the
language of gestures.
Like the spoken language, the language of signs has, in America,
infinite fluctuations; everyone, so to say, invents his own. It is a
compound of strange and mysterious gestures, a kind of masonic
telegraph, the signs of which, varying at will, are only
comprehensible to a small number of adepts.
The Babbler and his companion were conversing in signs.
This singular conversation lasted nearly an hour; it appeared to
interest the speakers warmly; so warmly, indeed, that they did not
remark, in spite of all the precautions they had taken not to be

surprised, two fiery eyes that, from the middle of a tuft of
underwood, were fixed upon them with strange intenseness.
At length the Babbler, risking the utterance of a few words, said, "I
await your good pleasure."
"And you shall not wait it long," the other replied.
"I depend upon you, Kennedy; for my part, I have fulfilled my
promise."
"That's well! that's well! We don't require many words to come to an
understanding," said Kennedy, shrugging his shoulders; "only you
need not have conducted them to so strong a position—it will not be
very easy to surprise them."
"That's your concern," said the Babbler, with an evil smile.
His companion looked at him for a moment with great attention.
"Hum," said he; "beware, compadre, it is almost always awkward to
play a double game with men like us."
"I am playing no double game; but I think you and I have known
each other a pretty considerable time, Kennedy, have we not?"
"What follows?"
"What follows? Well! I am not disposed that a thing should happen
to me again that has happened before, that's all."
"Do you draw back, or are you thinking about betraying us?"
"I do not draw back, and I have not the least intention of betraying
you, only——"
"Only?" the other repeated.
"This time I will not give up to you what I have promised till my
conditions have been agreed to pretty plainly; if not, no——"
"Well, at least that's frank."
"People should speak plainly in business affairs," the Babbler
observed, shaking his head.

"That's true! Well, come, repeat the conditions; I will see if we can
accept them."
"What's the good of that? You are not the principal chief, are you?"
"No:—but—yet——"
"You could pledge yourself to nothing—so it's of no use. If Waktehno
—he who kills—were here now, it would be quite another thing. He
and I should soon understand one another."
"Speak then, he is listening to you," said a strong, sonorous voice.
There was a movement in the bushes, and the personage who, up
to that moment, had remained an invisible hearer of the
conversation of the two men, judged, without doubt, that the time
to take a part in it was arrived, for, with a bound, he sprang out of
the bushes that had concealed him, and placed himself between the
speakers.
"Oh! oh! you were listening to us, Captain Waktehno, were you?"
said the Babbler without being the least discomposed.
"Is that unpleasant to you?" the newcomer asked, with an ironical
smile.
"Oh! not the least in the world."
"Continue, then, my worthy friend—I am all ears."
"Well," said the guide, "it will, perhaps, be better so."
"Go on, then—speak; I attend to you."
The personage to whom the Babbler gave the terrible Indian name
of Waktehno was a man of pure white race, thirty years of age, of
lofty stature, and well proportioned, handsome in appearance, and
wearing with a certain dashing carelessness the picturesque costume
of the wood rangers. His features were noble, strongly marked, and
impressed with that loyal and haughty expression so often met with
among men accustomed to the rude, free life of the prairies.

He fixed his large, black, brilliant eyes upon the Babbler, a
mysterious smile curled his lips, and he leant carelessly upon his rifle
whilst listening to the guide.
"If I cause the people I am paid to escort and conduct to fall into
your hands, you may depend upon it I will not do so unless I am
amply recompensed," said the bandit.
"That is but fair," Kennedy remarked; "and the captain is ready to
assure your being so recompensed."
"Yes," said the other, nodding his head in sign of agreement.
"Very well," the guide resumed. "But what will be my recompense?"
"What do you ask?" the captain said. "We must know what your
conditions are before we agree to satisfy them."
"Oh! my terms are very moderate."
"Well, but what are they?"
The guide hesitated, or, rather, he calculated mentally the chances of
gain and loss the affair offered; then in an instant, he replied:
"These Mexicans are very rich."
"Probably," said the captain.
"Therefore it appears to me——"
"Speak without tergiversation, Babbler; we have not time to listen to
your circumlocutions. Like all half-bloods, the Indian nature always
prevails in you, and you never come frankly to the purpose."
"Well, then," the guide bluntly replied, "I will have five thousand
duros, or nothing shall be done."
"For once you speak out; now we know what we have to trust to;
you demand five thousand dollars?"
"I do."
"And for that sum you agree to deliver up to us, the general, his
niece, and all the individuals who accompany them."

"At your first signal."
"Very well! Now listen to what I am going to say to you."
"I listen."
"You know me, do you not?"
"Perfectly."
"You know dependence is to be placed upon my word?"
"It is as good as gold."
"That's well. If you loyally fulfil the engagements you freely make
with me, that is to say, deliver up to me, not all the Mexicans who
comprise your caravan, very respectable people no doubt, but for
whom I care very little, but only the girl, called, I think, Doña Luz, I
will not give you five thousand dollars as you ask, but eight
thousand—you understand me, do you not?"
The eyes of the guide sparkled with greediness and cupidity.
"Yes!" he said emphatically.
"That's well."
"But it will be a difficult matter to draw her out of the camp alone."
"That's your affair."
"I should prefer giving them all up in a lump."
"Go to the devil! What could I do with them?"
"Hum! what will the general say?"
"What he likes; that is nothing to me. Yes or no—do you accept the
offer I make you?"
"Oh! I accept it."
"Do you swear to be faithful to your engagements?"
"I swear."

"Now then, how long does the general reckon upon remaining in this
new encampment?"
"Ten days."
"Why, then, did you tell me that you did not know how to draw the
young girl out, having so much time before you?"
"Hum! I did not know when you would require her to be delivered
up to you?"
"That's true. Well, I give you nine days; that is to say, on the eve of
their departure the young girl must be given up to me."
"Oh! in that way——"
"Then that arrangement suits you?"
"It could not be better."
"Is it agreed?"
"Irrevocably."
"Here, then, Babbler," said the captain, giving the guide a
magnificent diamond pin which he wore in his hunting shirt, "here is
my earnest."
"Oh!" the bandit exclaimed, seizing the jewel joyfully.
"That pin," said the captain, "is a present I make you in addition to
the eight thousand dollars I will hand over to you on receiving Doña
Luz."
"You are noble and generous, captain," said the guide; "it is a
pleasure to serve you."
"Still," the captain rejoined, in a rough voice, and with a look cold as
a steel blade, "I would have you remember I am called he who kills;
and that if you deceive me, there does not exist in the prairie a place
sufficiently strong or sufficiently unknown to protect you from the
terrible effects of my vengeance.

"I know that, captain," said the half-breed, shuddering in spite of
himself; "but you may be quite satisfied I will not deceive you."
"I hope you will not! Now let us separate; your absence may be
observed. In nine days I shall be here."
"In nine days I will place the girl in your hands."
After these words the guide returned to the camp, which he entered
without being seen.
As soon as they were alone, the two men with whom the Babbler
had just made this hideous and strange bargain, retreated silently
among the underwood, through which they crawled like serpents.
They soon reached the banks of a little rivulet which ran,
unperceived and unknown, through the forest. Kennedy whistled in a
certain fashion twice.
A slight noise was heard, and a horseman, holding two horses in
hand, appeared at a few paces from the spot where they had
stopped.
"Come on, Frank," said Kennedy, "you may approach without fear."
The horsemen immediately advanced.
"What is there new?" Kennedy asked.
"Nothing very important," the horseman replied.
"I have discovered an Indian trail."
"Ah! ah!" said the captain, "numerous?"
"Rather so."
"In what direction?"
"It cuts the prairie from east to west."
"Well done, Frank, and who are these Indians?"
"As well as I can make out, they are Comanches."
The captain reflected a moment.

"Oh! it is some detachment of hunters," he said.
"Very likely," Frank replied.
The two men mounted.
"Frank and you, Kennedy," said the captain, at the expiration of a
minute, "will go to the passage of the Buffalo, and encamp in the
grotto which is there; carefully watching the movements of the
Mexicans, but in such a manner as not to be discovered."
"Be satisfied of that, captain."
"Oh; I know you are very adroit and devoted comrades, therefore I
perfectly rely upon you. Watch the Babbler, likewise; that half-breed
only inspires me with moderate confidence."
"That shall be done!"
"Farewell, then, till we meet again. You shall soon hear of me."
Notwithstanding the darkness, the three men set off at a gallop, and
were soon far in the desert, in two different directions.
CHAPTER XII.
PSYCHOLOGICAL.
The general had kept the causes which made him undertake a
journey into the prairies from the west of the United States so
profound a secret, that the persons who accompanied him had not
even a suspicion of them.
Several times already, at his command, and without any apparent
reason, the caravan had encamped in regions completely desert,
where he had passed a week, and sometimes a fortnight, without
any apparent motive for such a halt.

In these various encampments the general would set out every
morning, attended by one of the guides, and not return till evening.
What was he doing during the long hours of his absence?
For what object were these explorations made, at the end of which a
greater degree of sadness darkened his countenance?
No one knew.
During these excursions, Doña Luz led a sufficiently monotonous life,
isolated among the rude people who surrounded her. She passed
whole days seated sadly in front of her tent, or, mounted on
horseback and escorted by Captain Aguilar or the fat doctor, she
took rides near the camp, without object and without interest.
It happened this time again, exactly as it had happened at the
preceding stations of the caravan.
The young girl, abandoned by her uncle, and even by the doctor,
who was pursuing, with increasing ardour, the great research for his
imaginary plant, and set out resolutely every morning herbalizing,
was reduced to the company of Captain Aguilar.
But Captain Aguilar was, we are forced to admit, although young,
elegant and endowed with a certain relative intelligence, not a very
amusing companion for Doña Luz.
A brave soldier, with the courage of a lion, entirely devoted to the
general, to whom he owed everything, the captain entertained for
the niece of his chief great attachment and respect; he watched with
the utmost care over her safety, but he was completely
unacquainted with the means of rendering the time shorter by those
attentions and that pleasant chat which are so agreeable to girls.
This time Doña Luz did not become so ennuyée as usual. Since that
terrible night—from the time that one of those fabulous heroes
whose history and incredible feats she had so often read, Loyal
Heart, had appeared to her to save her and those who accompanied
her—a new sentiment, which she had not even thought of analyzing,

had germinated in her maiden heart, had grown by degrees, and in
a very few days had taken possession of her whole being.
The image of the hunter was incessantly present to her thoughts,
encircled with that ennobling glory which is won by the invincible
energy of the man who struggles, body to body, with some immense
danger, and forces it to acknowledge his superiority. She took delight
in recalling to her partial mind the different scenes of that tragedy of
a few hours, in which the hunter had played the principal character.
Her implacable memory, like that of all pure young girls, retraced
with incredible fidelity the smallest details of those sublime phases.
In a word, she reconstructed in her thoughts the series of events in
which the hunter had mingled, and in which he had, thanks to his
indomitable courage and his presence of mind, extricated in so
happy a fashion those he had suddenly come to succour, at the
instant when they were without hope.
The hurried manner in which the hunter had left them, disdaining
the most simple thanks, and appearing even unconcerned for those
he had saved, had chilled the girl; she was piqued more than can be
imagined by this real or affected indifference. And, consequently, she
continually revolved means to make her preserver repent that
indifference, if chance should a second time bring them together.
It is well known, although it may at the first glance appear a
paradox, that from hatred, or, at least, from curiosity to love, there is
but one step.
Doña Luz passed it at full speed, without perceiving it.
As we have said, Doña Luz had been educated in a convent, at the
gates of which the sounds of the world died away without an echo.
Her youth had passed calm and colourless, in the religious, or,
rather, superstitious practices, upon which in Mexico religion is built.
When her uncle took her from the convent to lead her with him
through the journey he meditated into the prairies, the girl was
ignorant of the most simple exigences of life, and had no more idea

of the outward world, in which she was so suddenly cast, than a
blind man has of the effulgent splendour of the sun's beams.
This ignorance, which seconded admirably the projects of the uncle,
was for the niece a stumbling block against which she twenty times
a day came into collision in spite of herself.
But, thanks to the care with which the general surrounded her, the
few weeks which passed away before their departure from Mexico
had been spent without too much pain by the young girl.
We feel called upon, however, to notice here an incident, trifling in
appearance, but which left too deep a trace in the mind of Doña Luz
not to be related.
The general was actively employed in getting together the people he
wanted for his expedition, and was therefore obliged to neglect his
niece more than he would have wished.
As he, however, feared that the young girl would be unhappy at
being left so much alone with an old duenna in the palace he
occupied, in the Calle de los Plateros, he sent her frequently to
spend her evenings at the house of a female relation who received a
select society, and with whom his niece passed her time in a
comparatively agreeable manner.
Now one evening when the assembly had been more numerous than
usual, the party did not break up till late.
At the first stroke of eleven, sounded by the ancient clock of the
convent of the Merced Doña Luz and her duenna, preceded by a
peon carrying a torch to light them, set off on their return home,
casting anxious looks, right and left, on account of the character of
the streets at that time of night. They had but a short distance to
go, when all at once, on turning the corner of the Calle San Agustin
to enter that of Plateros, four or five men of bad appearance seemed
to rise from the earth, and surrounded the two women, after having
previously, by a vigorous blow, extinguished the torch carried by the
peon.

To express the terror of the young lady at this unexpected
apparition, is impossible.
She was so frightened that, without having the strength to utter a
cry, she fell on her knees, with her hands clasped, before the
bandits.
The duenna, on the contrary, sent forth deafening screams.
The Mexican bandits, all very expeditious men, had, in the shortest
time possible, reduced the duenna to silence, by gagging her with
her own rebozo; then, with all the calmness which these worthies
bring to the exercise of their functions, assured as they are of the
impunity granted to them by that justice with which they generally
go halves, proceeded to plunder their victims.
The operation was shortened by the latter, for, so far from offering
any resistance, they tore off their jewels in the greatest haste, and
the bandits pocketed them with grins of satisfaction.
But, at the very height of this enjoyment, a sword gleamed suddenly
over their heads, and two of the bandits fell to the ground, swearing
and howling with fury.
Those who were left standing, enraged at this unaccustomed attack,
turned to avenge their companions, and rushed all together upon
the aggressor.
The latter, heedless of their numbers, made a step backwards,
placed himself on guard, and prepared to give them a welcome.
But, by chance, with the change in his position, the moonlight fell
upon his face. The bandits instantly drew back in terror, and
promptly sheathed their machetes.
"Ah, ah!" said the stranger, with a smile of contempt, as he
advanced towards them, "you recognise me, my masters, do you?
By the Virgin! I am sorry for it—I was preparing to give you a rather
sharp lesson. Is this the manner in which you execute my orders?"

The bandits remained silent, contrite and repentant, in appearance
at least.
"Come, empty your pockets, you paltry thieves, and restore to these
ladies what you have taken from them!"
Without a moment's hesitation, the thieves unbandaged the duenna,
and restored the rich booty which, an instant before, they had so
joyfully appropriated to themselves.
Doña Luz could not overcome her astonishment, she looked with the
greatest surprise at this strange man, who possessed such authority
over bandits acknowledging neither faith nor law.
"Is this really all?" he said, addressing the young lady, "are you sure
you miss nothing, señora?"
"Nothing—nothing, sir!" she replied, more dead than alive, and not
knowing at all what she said.
"Now, then, begone, you scoundrels," the stranger continued; "I will
take upon myself to be the escort of these ladies."
The bandits did not require to be twice told; they disappeared like a
flight of crows, carrying off the wounded.
As soon as he was left alone with the two women, the stranger
turned towards Doña Luz—
"Permit me, señorita," he said, with refined courtesy of manner, "to
offer you my arm as far as your palace; the fright you have just
experienced must render your steps uncertain."
Mechanically, and without reply, the young girl placed her hand
within the arm so courteously offered to her, and they moved
forward.
"When they arrived at the palace, the stranger knocked at the door,
and then taking off his hat, said,—
"Señorita, I am happy that chance has enabled me to render you a
slight service. I shall have the honour of seeing you again. I have

already, for a long time, followed your steps like your shadow. God,
who has granted me the favour of an opportunity of speaking with
you once, will, I feel assured, grant me a second, although, in a few
days, you are to set out on a long journey. Permit me then to say
not adieu, but au revoir."
After bowing humbly and gracefully to the young lady, he departed
at a rapid pace.
A fortnight after this strange adventure, of which she did not think
fit to speak to her uncle, Doña Luz quitted Mexico, without having
again seen the unknown. Only, on the eve of her departure, when
retiring to her bedchamber, she found a folded note upon her prie-
dieu. In this note were the following words, written in an elegant
hand:—
"You are going, Doña Luz! Remember that I told you I should see
you again.
"Your preserver of the Calle de los Plateros."
For a long time this strange meeting strongly occupied the mind of
the young girl; for an instant, she had even believed that Loyal Heart
and her unknown preserver were the same man; but this
supposition had soon faded away. What probability was there in it?
With that object could Loyal Heart, after having saved her, so quickly
have departed? That would have been absurd.
But, by one of those consequences (or those inconsequences,
whichever the reader pleases) of the human mind, in proportion as
the affair of Mexico was effaced from her thoughts, that of Loyal
Heart, became more prominent.
She longed to see the hunter and talk with him.
Why?
She did not herself know. To see him,—-to hear his voice,—to meet
his look, at once so soft and so proud,—nothing else; all maidens
would have done the same.

But how was she to see him again?
In reply to that question arose an impossibility, before which the
poor girl dropped her head with discouragement.
And yet something at the bottom of her heart, perhaps that voice
divine which in the reflections of love whispers to young girls, told
her that her wish would soon be accomplished.
She hoped, then?
What for?
For some unforeseen incident,—a terrible danger, perhaps,—which
might again bring them together.
True love may doubt sometimes, but it never despairs.
Four days after the establishment of the camp upon the hill, in the
evening, when retiring to her tent, Doña Luz smiled inwardly as she
looked at her uncle, who was pensively preparing to go to rest.
She had at length thought of a means of going in search of Loyal
Heart.
CHAPTER XIII.
THE BEE-HUNT.
The sun was scarcely above the horizon, when the general, whose
horse was already saddled, left the reed cabin which served him as a
sleeping apartment, and prepared to set out on his usual daily ride.
At the moment when he was putting his foot in the stirrup, a little
hand lifted the curtain of the tent, and Doña Luz appeared.
"Oh! oh! what, up already!" said the general, smiling. "So much the
better, dear child. I shall be able to have a kiss before I set out; and

that perhaps may bring me good luck," he added, stifling a sigh.
"You will not go thus, uncle," she replied, presenting her cheek,
upon which he placed a kiss.
"Why not, fair lady?" he asked gaily.
"Because I wish you to partake of something I have prepared for
you before you mount on horseback; you cannot refuse me, can
you, dear uncle?" she said, with that coaxing smile of spoilt children
which delights the hearts of old men.
"No, certainly not, dear child, upon condition that the breakfast you
offer me so gracefully be not delayed. I am rather in a hurry."
"I only ask for a few minutes," she replied, returning to the tent.
"For a few minutes be it then," said he, following her.
The young girl clapped her hands with joy.
In the twinkling of an eye, the breakfast was ready, and the general
at table with his niece. Whilst assisting her uncle, and taking great
care that he wanted for nothing, the young girl looked at him from
time to time in an embarrassed manner, and did it so evidently, that
the old soldier ended by observing it.
"It is my opinion," he said, laying down his knife and fork, and
looking at her earnestly, "that you have something to ask me, Lucita;
you know very well that I am not accustomed to refuse you
anything."
"That is true, dear uncle; but this time, I am afraid, you will be more
difficult to be prevailed upon."
"Ah! ah!" the general said, gaily; "it must be something serious,
then!"
"Quite the contrary, uncle; and yet, I confess, I am afraid you will
refuse me."
"Speak, notwithstanding, my child," said the old soldier; "speak
without fear; when you have told me what this mighty affair is, I will

soon answer you."
"Well, uncle," the girl said, blushing, but determined on her purpose,
"I am compelled to say that the residence in the camp has nothing
agreeable about it."
"I can conceive that, my child; but what do you wish me to do to
make it otherwise?"
"Everything."
"How so, dear?"
"Nay, dear, uncle, if you were always here, it would not be dull; I
should have your company."
"What you say is very amiable; but, as you know I am absent every
morning, I cannot be here, and——-
"That is exactly where the difficulty lies."
"That is true."
"But, if you were willing, it could be easily removed."
"Do you think so?"
"I am sure of it."
"Well, I don't see too clearly how, unless I remained always with
you, and that is impossible."
"Oh; there are other means that would arrange the whole affair."
"Nonsense!"
"Yes, uncle, and very simple means too."
"Well, then, darling, what are these means?"
"You will not scold me, uncle?"
"Silly child! do I ever scold you?"
"That is true! You are so kind."
"Come, then; speak out, little pet?"

Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.
More than just a book-buying platform, we strive to be a bridge
connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.
Join us on a journey of knowledge exploration, passion nurturing, and
personal growth every day!
ebookbell.com

Gitops And Kubernetes Continuous Deployment With Argo Cd Jenkins X And Flux 1st Edition Billy Yuen

More Related Content

Similar to Gitops And Kubernetes Continuous Deployment With Argo Cd Jenkins X And Flux 1st Edition Billy Yuen (20)

Recently uploaded (20)

Gitops And Kubernetes Continuous Deployment With Argo Cd Jenkins X And Flux 1st Edition Billy Yuen