A beginners guide to

• • An introduction to GitOps

• • The benefits of infrastructure automation

• • GitOps best practices

 2

Table of contents

03 Introduction 15 CI/CD pipelines with GitOps and Terraform

05 The Road to GitOps 16 The future of infrastructure automation

07 How GitOps works 17 About GitLab

09 The Benefits of GitOps

10 Common GitOps tools

11 Best practices for getting started with GitOps

Define all infrastructure as config files
Document what you can’t automate
Outline a code review and merge request process
Consider multiple environments
Make CI/CD the access point to resources
Have a repository strategy
Keep changes small

Try GitLab free for 30 days > Follow us:

 3

Introduction

As software applications become more sophisticated, This eBook will introduce the infrastructure automation
the demands on infrastructure increase. Infrastructure process of GitOps and how it offers an end-to-end
teams need to support complex deployments at solution for designing, changing, and deploying
immense scale and speed. While much of application infrastructure. In this eBook, you’ll also learn:
development has been automated, infrastructure
has remained largely a manual process requiring • How GitOps works with processes you already use
specialized teams. Instead of manual processes, is in application development
there a repeatable and reliable way to design, change, • The three components teams need to get started
and deploy software environments? Infrastructure- with GitOps
as-code (IaC) tools like Ansible and Terraform are
• GitOps best practices and workflows
a good start, but they don’t solve the entire problem.
Teams need a prescriptive workflow that puts IaC into
action automatically.

Try GitLab free for 30 days > Follow us:

 4

Organizations with a mature DevOps culture can deploy code to production the AWS portal to deploy all services manually is time-consuming, error
hundreds of times per day. While the software development lifecycle has prone, and not realistic for large organizations.
been automated, rolling out infrastructure is still largely a manual process.
IT teams struggling to keep up with more frequent deployments is not a GitOps offers a way to automate and manage infrastructure, and it does
new problem. this by using DevOps best practices that many organizations already
use, such as version control, code review, and CI/CD pipelines. Having
When physical hardware was required, infrastructure automation infrastructure described as code allows you to deploy the same service
was practically impossible. With virtualization, things got a bit easier. over and over. By using parameterization, it’s possible to deploy the
It wasn’t until the cloud went public that large infrastructures could same service, but to different environments and with different names
be completely automated with relative ease. The cloud doesn’t require and settings.
hardware and, unlike "traditional" servers and Virtual Machines (VMs),
cloud native services can be created and managed independently without
having to provision a VM or Operating System (OS).

By using scripting languages like PowerShell and Bash, IT teams are

able to deploy various services to the cloud. Automated scaling is often
included in cloud services, like serverless offerings. When scaling is
not automatic, being able to deploy another instance of your service
instantly is important.

Just because these services are available doesn’t mean teams are able to
use them effectively. AWS alone has over 200 services, and many companies
rely on dozens of them. These services often have many settings. Using

Try GitLab free for 30 days > Follow us:

 5

The road to GitOps

AWS has been publicly available since 2006, but even before that time, on-premises infrastructure
management could be a daunting task for IT teams. Various servers ran several applications and
services, and scaling up required IT to manually set up an entire server and reinstall the same
applications with the same settings. Luckily, tools were developed to make this task a little easier.

The first generation of configuration management (CM) tools, like Puppet and Chef, made it
easy to set up existing servers. IT could spin up a server or VM, install the Puppet or Chef agent,
and let the tool establish everything needed to run applications on the server. These tools ran
on on-premises servers, as well as on cloud servers.

First-generation CM tools were an efficient way to replicate all the steps to set up new production
servers. With these steps now automated, setting up new servers became a lot easier. However,
they still didn’t provision new VMs and didn’t work well with cloud native infrastructure.

Next came second-generation CM tools like Ansible and SaltStack. These tools can install software
on individual servers, just like the first​-gen CM tools, but can also provision VMs before setting them
up. For example, they can ​create​ ten EC2 instances, then install all the needed software on each of
these instances.

Try GitLab free for 30 days > Follow us:

 6

One important drawback of these CM tools is that they only provisioned and set up servers
and VMs. They don’t offer solutions for cloud native services.

Amazon CloudFormation appeared around the same as the second​-gen CM tools. It doesn’t
handle server setup, but offers the ability to use declarative code to provision an entire AWS
application architecture. There was no longer the need to click through the management console
to manually create resources. You could simply describe your infrastructure as JSON or YAML
and deploy it using the AWS Management Console, the Command-Line Interface (CLI), or the
AWS SDK. But, as an Amazon service, it only works on AWS.

Microsoft Azure offers a similar tool, the Azure Resource Manager (ARM), which allows you to
describe your infrastructure in JSON templates. But much like Amazon CloudFormation and AWS,
ARM only works with Azure services.

When private clouds and other public clouds, like Azure and Google Cloud, gained traction,
many enterprises switched to another cloud or went multicloud in order to not depend on a
single cloud platform. To address this new requirement, multicloud CM tools appeared, such
as Terraform. Simply describe your services and deploy them to multiple clouds/providers/
cloud services.

An upside to these tools is that they unlock the ability to do things like version control, code
review, and continuous integration/continuous delivery (CI/CD) on infrastructure code.

Try
Try GitLab
GitLab free
free for
for 30
30 days
days >> Follow us:
 7

How GitOps works

GitOps takes tried-and-true DevOps processes and applies them to infrastructure code.
As the name suggests, it combines Git and operations, or resource management. Git is an
open source version control system that tracks code management changes. Like DevOps,
the goal of GitOps is to use CI/CD to automatically deploy your resources by using code stored
in your Git repositories.

With GitOps, your infrastructure definition code, defined as JSON or YAML and stored in a .git
folder in a project, lives in a Git repository that serves as a single source of truth. Using Git’s
features makes it possible to see the complete change history for the organization’s infrastructure
code, and teams can roll back to an earlier version if necessary.

Git also makes it possible to do code reviews on your infrastructure. Code review is a key DevOps
practice used to ensure that bad application code doesn’t make it into production. This is just as
important for infrastructure code. Bad infrastructure code can accidentally spin up expensive cloud
infrastructure and cost the company thousands of dollars per hour. Likewise, a bad script could
take down your application, resulting in downtime for your services. Code reviews prevent these
mistakes by ensuring multiple people see every change before it’s approved.

Try
Try GitLab
GitLab free
free for
for 30
30 days
days >> Follow us:
 8

The biggest benefit to using Git and IaC is

that you can now use continuous integration
and deployment. With tools like GitLab CI/
CD, you can automatically deploy (updated)
GitOps = IaC + MRs + CI/CD
infrastructure code and automatically apply
it to your cloud environment. Resources that IaC – GitOps uses a Git repository as the single source of truth for infrastructure
have been added to the infrastructure code definitions. A Git repository is a .git folder in a project that tracks all changes
are provisioned automatically and made ready made to files in a project over time. Infrastructure as code (IaC) is the practice of
for use. Resources that were changed are keeping all infrastructure configuration stored as code. The actual desired state
updated in your cloud environment and may or may not be not stored as code (e.g., number of replicas, pods).
resources that are removed from the infra-
structure code are automatically spun down MRs – GitOps uses merge requests (MRs) as the change mechanism for all
and deleted. This allows you to write code, infrastructure updates. The MR is where teams can collaborate via reviews and
commit it to your Git repository, and take full comments and where formal approvals take place. A merge commits to your
advantage of all the benefits of the DevOps master (or trunk) branch and serves as a changelog for auditing
process, but for your infrastructure. and troubleshooting.

CI/CD – GitOps automates infrastructure updates using a Git workflow with

continuous integration and continuous delivery (CI/CD). When new code
is merged, the CI/CD pipeline enacts the change in the environment. Any
configuration drift, such as manual changes or errors, is overwritten by GitOps
automation so the environment converges on the desired state defined in Git.
GitLab uses CI/CD pipelines to manage and implement GitOps automation, but
other forms of automation such as definitions operators can be used as well.

Try GitLab free for 30 days > Follow us:

 9

The benefits of GitOps

A GitOps framework makes infrastructure automation possible, and while Less error prone. Infrastructure definition is codified and
automation has value in itself, it’s not the only advantage to GitOps. repeatable, making it less prone to human error. With code
Organizations that adopt GitOps enjoy other benefits that can make a long reviews and collaboration in merge requests, errors can
term impact. be identified and corrected before they ever make it to
production.
Collaboration on infrastructure changes. Since every
Reduced costs and downtime. Automation of infrastructure
change will go through the same change/merge request/
definition and testing eliminates manual tasks, improves
review/approval process, senior engineers can focus on
productivity, and reduces downtime due to built-in revert/
other areas beyond the critical infrastructure management.
rollback capability. Automation also allows for infrastructure
Faster time to market. Execution via code is faster than teams to better manage cloud resources, which can also
manual point and click. Test cases are automated and improve cloud costs.
repeatable, so stable environments can be delivered rapidly.
Improved access control. There's no need to give credentials
to all infrastructure components since changes are automated
Simplified auditing. When infrastructure changes are
(only CI/CD needs access).
conducted manually across a set of multiple interfaces it
can make auditing complex and time consuming. Data needs
Collaboration with compliance. In heavily regulated contexts
to be pulled from multiple places and normalized in order to
policy often dictates that the number of people who can enact
conduct the audit. Using GitOps, all changes to environments
changes to a product environment remains as small as possible.
are stored in the git log making audits simple.
With GitOps almost anyone can propose a change via merge
Less risk. All changes to infrastructure are tracked through request opening the scope of collaboration broadly while keeping
merge requests, and changes can be rolled back to a the ability to strictly limit the number of people with the ability to
previous state. merge to the production branch to maintain compliance.

Try GitLab free for 30 days > Follow us:

 10

Common GitOps tools

What makes GitOps unique is that it’s not a single product,

plugin, or platform. GitOps is a framework that helps teams
Git code repository
manage IT infrastructure through processes they already
use in application development. Popular tools are Ansible,
Terraform, and Kubernetes, but the GitOps process is Git management tool
largely technology agnostic (save for Git, of course).

GitOps is suited for a variety of scenarios. GitOps and Continuous Inegration tool
Kubernetes is a particularly good fit, for example. Kubernetes
works on all major cloud platforms and uses stateless and
Continuous Delivery tool
immutable containers. Since containerized apps running in
Kubernetes are self-contained, you don’t need to provision
and configure servers for each app. Provision Kubernetes
Container Registry
clusters and other needed infrastructure, like databases
and networking, using Terraform.
Configuration manager
Deploying stateful applications requires some additional
consideration for persisting data to external services, like
an Amazon Aurora database instance or a Redis cache. Infrastructure provisioning

While Kubernetes lends itself well to a GitOps framework,

it’s not a requirement for doing GitOps. You can use it with
Container orchestration
traditional cloud infrastructure like VMs, too. In this case,
you’d provision with Terraform and then use a CM tool like
Ansible to configure new VMs.

Try GitLab free for 30 days > Follow us:

 11

Best practices for getting started with GitOps

Define all infrastructure as config files

Firstly, make sure all the infrastructure you form of technical debt that can build over time
want to manage via GitOps is described in IaC and sabotage your GitOps strategy.
config files. Ideally, these files should be written
For teams that are used to
in declarative code. This means you describe If you’re already using IaC and you want to
making small, manual changes
the end state of what you want rather than automate it with GitOps, start by adding your
to infrastructure, adopting a
instructions of how to get there. infrastructure code to the Git repository you
process like GitOps can be a big
plan to use for GitOps.
adjustment. GitOps is a frame-
For example, use a JSON file with properties
work that requires infrastructure
describing how you want to set up your services, If you’re ​not​ using IaC, defining your existing
teams to adopt new habits, and
rather than a JavaScript file where you instruct infrastructure using config code will take
lose old habits. This can take a provider to create services for you. While somework. AWS makes it easy to create
some time and may not come many cloud provider and CM tools allow for CloudFormation config files from existing
naturally to every team. Having declarative syntax, it’s best to choose tools that resources. Terraform can import existing
best practices that are refe- are designed to be used declaratively. infrastructure from various providers, but it
renced frequently will be helpful doesn’t do 100% of the work for you.
in committing to the long term For the best results, describe all of your infra-
strategy of GitOps. structure. You may be tempted to leave out that The goal, of course, is to have all infrastructure
one service that only uses default settings and described as code, but that is a journey that
takes only a minute to set up. The fact is, that takes time. Don't get overwhelmed with trying
manual action is easy to forget when you’re to automate all your infrastructure at once.
spinning up a new environment. Others will Eat the elephant one bite at a time. If you work
likely not even know they have to deploy this iteratively, more of your infrastructure will move
one service manually. Omissions like this are a to GitOps workflows over time.

Try
Try GitLab
GitLab free
free for
for 30
30 days
days >> Follow us:
 12

Document what you can’t automate Outline a code review and merge request process
It’s not always possible to automate everything. For example, Azure has It’s important to familiarize GitOps teams with Git and code reviews.
some (usually newer) settings that are not yet added to ARM templates. Some teams already use a Git repository as a place to store config code,
A common workaround is to use PowerShell. but don’t use features like merge requests. As a starting point, take a look
at the code review guidelines for the GitLab open source project. This
Another example is when working with third-party providers. Imagine project can give you a sense for the types of information you'll want to
working with a supplier who needs to manually approve-list your IP eventually add to your code review guidelines.
addresses. An approve-list request can only be supplied by a manager.
The manual action for every new service and environment is to look up Before approving a merge request, set a minimum number of reviewers
the IP address, pass it on to your manager, and have them email it to the so all code is reviewed by at least a few members of the team.
supplier. Make sure such processes are very well-documented.
For teams new to GitOps, another option is to set up "optional reviews"
In all likelihood, you'll always have some legacy environments that need rather than set up "required blocking reviews". Since this a new process,
manual attention. Document these instances so that they’re accounted for. take some time to get used to doing code reviews and develop a good
cadence. Once teams are familiar with the toolset and practices, imple-
ment mandatory reviews to ensure code reviews happen every time.

As we've recommended elsewhere – start small and simple. If you start out
with a complex set of guidelines, no one will want to adopt your process.
Focus more on adoption than on doing what's best. Over time, iterate on
code review guidelines to make them more robust and complete.

Try GitLab free for 30 days > Follow us:

 13

Consider multiple environments Make CI/CD the access point to resources

It’s good practice to have multiple environments. One example you One practice that encourages a GitOps workflow, and reduces manual
might follow is the DTAP environments: Development, Test, Acceptance, changes to cloud infrastructure, is to make your CI/CD tooling the access
and Production. Code can be rolled out to the Development or Test point for cloud resources.
environment, after which you can test whether the services are still
available and working as expected. If they are, you can further roll out Of course, having this access during initial development can really help
your changes to the next environments. teams write their code and you may need incidental access for various
reasons. However, switching your mindset from "access unless" to
After you have rolled out your code into your environments, it’s important "access because" can help in adopting and following the GitOps process.
to keep your code in sync with your running services. Once you know
there’s a difference between your system and your configuration, you
can fix either one. A solution to this problem is to use immutable images,
such as containers, so that it’s less likely to have differences.

Tools like Chef, Puppet, and Ansible have features

like "diff alert," which notifies you when your
services differ from your configuration code. Using
tools such as Kubediff and Terradiff give you the
same features for Kubernetes and Terraform.

Try GitLab free for 30 days > Follow us:

 14

Have a repository strategy Keep changes small

Think about how you want to set up your repositories. You may want Whatever you do, always be sure to keep your commits small. This
to use a single repository for all your infrastructure. It makes sense allows for a fine-grained changelog where it’s easier to rollback separate
to use a repository for your shared resources, but keep the code for changes. At GitLab, we refer to this process as iteration, and it’s what
service-specific resources with those services. Another approach is to allows us to make changes quickly. If we take smaller steps and ship
keep resources for different projects in different repositories. This all smaller, simpler features, we get feedback sooner.
depends on the structure of your organization, your services, and your
personal preferences.

A few things to consider when deciding on a mono or multi-repo strategy:

• Do you frequently have contractors or individuals working on projects

where it may not be secure for them to have access to all code?

• Do you have multiple dependencies to consider?

• Do you want your repository to serve as a single source of truth?

Google, one of the largest tech companies in the world, uses a single
repo for all code. HashiCorp recommends that each repository containing
Terraform code be a manageable chunk of infrastructure, such as an appli-
cation, service, or specific type of infrastructure (like common networking
infrastructure). The definition of "manageable" can vary, of course.

Consider a Git branching strategy, like feature branching, so that multiple

people can work on the same repository simultaneously.

Try GitLab free for 30 days > Follow us:

 15

CI/CD pipelines with GitOps and Terraform

Set up your CI/CD to first validate infrastructure code, such as by using the Terraform validate
command or a linter for JSON files. Your infrastructure code should be handled as if it is production
code. You want your production code to be clean and consistent.

When someone commits invalid code, make sure the build or validation fails and the team is
immediately notified. This allows the team to quickly solve the issue by either applying a fix or
rolling back the commit. If you’re making small changes, that also makes it easier to find problems.

If the code is valid, the CI/CD should run any commands necessary to provision the infrastructure
defined in the config code. For example, the Terraform apply command or AWS update-stack for
CloudFormation.

To see a sample GitOps project that uses Terraform, CI/CD, and Kubernetes, you can visit our
GitOps-Demo Group. From there, we’ve provided links to Terraform security recommendations,
Terraform code to represent each configuration for three major cloud providers, and instructions
for reproducing this demo within your own group.

Go to the GitOps-Demo Group

Try GitLab free for 30 days > Follow us:

 16

The future of infrastructure automation

GitOps isn’t magic: It just takes IaC ops tools you already know and wraps them in a DevOps-style
workflow. This allows for better revision tracking, fewer costly errors, and quick, automated infra-
structure deployments that can be repeated for a multi-environment or even multicloud setup.

By adopting GitOps, organizations improve the developer experience because often-dreaded

releases become fully automated, allowing developers to focus on just their code. Teams eliminate
or minimize manual steps and make deployments repeatable and reliable.

Infrastructure maintenance often becomes a problem that takes up a lot of time. By fully automating
If you’d like to see how we
this process, infrastructure can be elastic and keep up with frequent application deployments.
can help you get started with
GitOps, sign up to try GitLab
GitOps also improves security and standardization. By practicing GitOps, developers have no need
free for 30 days.
to manually access cloud resources and additional security checks can be put in place at the code
level in CI/CD pipelines. Start your GitLab free trial

GitLab can help you get started with a GitOps workflow. From GitLab, you can manage physical,
virtual, and cloud native infrastructures (including Kubernetes and serverless technologies). GitLab
also has tight integrations with industry-leading infrastructure automation tools like Terraform,
AWS Cloud Formation, Ansible, Chef, Puppet, and others. In addition to a Git repository, GitLab
offers CI/CD, merge requests, and single sign-on simplicity so that everyone can collaborate and
deploy from one platform to any cloud provider.

Try GitLab free for 30 days > Follow us:

 17

About GitLab
GitLab is a DevOps platform built from the ground up as a single application for
all stages of the DevOps lifecycle enabling Product, Development, QA, Security,
and Operations teams to work concurrently on the same project.

GitLab provides teams a single data store, one user interface, and one permission
model across the DevOps lifecycle allowing teams to collaborate and work on a
project from a single conversation, significantly reducing cycle time and focus
exclusively on building great software quickly.

Built on Open Source, GitLab leverages the community contributions of thousands

of developers and millions of users to continuously deliver new DevOps innovations.
More than 100,000 organizations from startups to global enterprise organizations,
including Ticketmaster, Jaguar Land Rover, NASDAQ, Dish Network and Comcast
trust GitLab to deliver great software at new speeds. GitLab is the world's largest
all-remote company, with more than 1,200 team members in over 65 countries.
Try GitLab free for 30 days > Follow us: