The Docker Book
The Docker Book
James Turnbull
December 2, 2018
Page
i
Contents
List of Figures 70
List of Listings 74
Index 75
1
Chapter 1: Working with Docker images and repositories
Let’s continue our journey with Docker by learning a bit more about Docker im-
ages. A Docker image is made up of filesystems layered over each other. At the
base is a boot filesystem, bootfs, which resembles the typical Linux/Unix boot
filesystem. A Docker user will probably never interact with the boot filesystem.
Indeed, when a container has booted, it is moved into memory, and the boot
filesystem is unmounted to free up the RAM used by the initrd disk image.
So far this looks pretty much like a typical Linux virtualization stack. Indeed,
Docker next layers a root filesystem, rootfs, on top of the boot filesystem. This
rootfs can be one or more operating systems (e.g., a Debian or Ubuntu filesys-
tem).
In a more traditional Linux boot, the root filesystem is mounted read-only and
then switched to read-write after boot and an integrity check is conducted. In the
Docker world, however, the root filesystem stays in read-only mode, and Docker
takes advantage of a union mount to add more read-only filesystems onto the
root filesystem. A union mount is a mount that allows several filesystems to be
mounted at one time but appear to be one filesystem. The union mount overlays
the filesystems on top of one another so that the resulting filesystem may contain
files and subdirectories from any or all of the underlying filesystems.
Docker calls each of these filesystems images. Images can be layered on top of
one another. The image below is called the parent image and you can traverse
each layer until you reach the bottom of the image stack where the final image
is called the base image. Finally, when a container is launched from an image,
Docker mounts a read-write filesystem on top of any layers below. This is where
whatever processes we want our Docker container to run will execute.
This sounds confusing, so perhaps it is best represented by a diagram.
When Docker first starts a container, the initial read-write layer is empty. As
changes occur, they are applied to this layer; for example, if you want to change
a file, then that file will be copied from the read-only layer below into the read-
write layer. The read-only version of the file will still exist but is now hidden
underneath the copy.
This pattern is traditionally called “copy on write” and is one of the features that
makes Docker so powerful. Each read-only image layer is read-only; this image
never changes. When a container is created, Docker builds from the stack of im-
ages and then adds the read-write layer on top. That layer, combined with the
knowledge of the image layers below it and some configuration data, form the con-
tainer. As we discovered in the last chapter, containers can be changed, they have
state, and they can be started and stopped. This, and the image-layering frame-
work, allows us to quickly build images and run containers with our applications
and services.
Let’s get started with Docker images by looking at what images are available to
us on our Docker host. We can do this using the docker images command.
We see that we’ve got an image, from a repository called ubuntu. So where does
this image come from? Remember in Chapter 3, when we ran the docker run
command, that part of the process was downloading an image? In our case, it’s
the ubuntu image.
NOTE Local images live on our local Docker host in the /var/lib/docker
directory. Each image will be inside a directory named for your storage driver;
for example, aufs or devicemapper. You’ll also find all your containers in the
/var/lib/docker/containers directory.
That image was downloaded from a repository. Images live inside repositories,
and repositories live on registries. The default registry is the public registry man-
aged by Docker, Inc., Docker Hub.
TIP The Docker registry code is open source. You can also run your own
registry, as we’ll see later in this chapter. The Docker Hub product is also available
as a commercial ”behind the firewall” product called Docker Trusted Registry,
formerly Docker Enterprise Hub.
Inside Docker Hub (or on a Docker registry you run yourself), images are stored
in repositories. You can think of an image repository as being much like a Git
repository. It contains images, layers, and metadata about those images.
Each repository can contain multiple images (e.g., the ubuntu repository contains
images for Ubuntu 12.04, 12.10, 13.04, 13.10, 14.04, 16.04). Let’s get another
image from the ubuntu repository now.
Here we’ve used the docker pull command to pull down the Ubuntu 16.04 image
from the ubuntu repository.
Let’s see what our docker images command reveals now.
TIP Throughout the book we use the ubuntu image. This is a reasonably
heavyweight image, measuring a couple of hundred megabytes in size. If you’d
prefer something smaller the Alpine Linux image is recommended as extremely
lightweight, generally 5Mb in size for the base image. Its image name is alpine.
You can see we’ve now got the latest Ubuntu image and the 16.04 image. This
shows us that the ubuntu image is actually a series of images collected under a
single repository.
NOTE We call it the Ubuntu operating system, but really it is not the full
operating system. It’s a cut-down version with the bare runtime required to run
the distribution.
We identify each image inside that repository by what Docker calls tags. Each
image is being listed by the tags applied to it, so, for example, 12.04, 12.10,
quantal, or precise and so on. Each tag marks together a series of image layers
that represent a specific image (e.g., the 16.04 tag collects together all the layers
of the Ubuntu 16.04 image). This allows us to store more than one image inside
a repository.
We can refer to a specific image inside a repository by suffixing the repository
name with a colon and a tag name, for example:
This launches a container from the ubuntu:16.04 image, which is an Ubuntu 16.04
operating system.
It’s always a good idea to build a container from specific tags. That way we’ll know
exactly what the source of our container is. There are differences, for example,
between Ubuntu 14.04 and 16.04, so it would be useful to specifically state that
we’re using ubuntu:16.04 so we know exactly what we’re getting.
There are two types of repositories: user repositories, which contain images con-
tributed by Docker users, and top-level repositories, which are controlled by the
people behind Docker.
A user repository takes the form of a username and a repository name; for example,
jamtur01/puppet.
• Username: jamtur01
• Repository name: puppet
Alternatively, a top-level repository only has a repository name like ubuntu. The
top-level repositories are managed by Docker Inc and by selected vendors who pro-
vide curated base images that you can build upon (e.g., the Fedora team provides
a fedora image). The top-level repositories also represent a commitment from
vendors and Docker Inc that the images contained in them are well constructed,
secure, and up to date.
In Docker 1.8 support was also added for managing the content security of images,
essentially signed images. This is currently an optional feature and you can read
more about it on the Docker blog.
Pulling images
When we run a container from images with the docker run command, if the image
isn’t present locally already then Docker will download it from the Docker Hub.
By default, if you don’t specify a specific tag, Docker will download the latest
tag, for example:
Will download the ubuntu:latest image if it isn’t already present on the host.
Alternatively, we can use the docker pull command to pull images down our-
selves preemptively. Using docker pull saves us some time launching a container
from a new image. Let’s see that now by pulling down the ‘fedora:21 base image.
Let’s see this new image on our Docker host using the docker images command.
This time, however, let’s narrow our review of the images to only the fedora im-
ages. To do so, we can specify the image name after the docker images command.
We see that the fedora:21 image has been downloaded. We could also download
another tagged image using the docker pull command.
We can also search all of the publicly available images on Docker Hub using the
docker search command:
TIP You can also browse the available images online at Docker Hub.
Here, we’ve searched the Docker Hub for the term puppet. It’ll search images and
return:
• Repository names
• Image descriptions
• Stars - these measure the popularity of an image
• Official - an image managed by the upstream developer (e.g., the fedora
image managed by the Fedora team)
• Automated - an image built by the Docker Hub’s Automated Build process
NOTE We’ll see more about Automated Builds later in this chapter.
This will pull down the jamtur01/puppetmaster image (which, by the way, con-
tains a pre-installed Puppet master server).
We can then use this image to build a new container. Let’s do that now using the
docker run command again.
You can see we’ve launched a new container from our jamtur01/puppetmaster
image. We’ve launched the container interactively and told the container to
run the Bash shell. Once inside the container’s shell, we’ve run Facter (Puppet’s
inventory application), which was pre-installed on our image. From inside the
container, we’ve also run the puppet binary to confirm it is installed.
A big part of image building is sharing and distributing your images. We do this
by pushing them to the Docker Hub or your own registry. To facilitate this, let’s
start by creating an account on the Docker Hub. You can join Docker Hub here.
Create an account and verify your email address from the email you’ll receive after
signing up.
Now let’s test our new account from Docker. To sign into the Docker Hub you can
use the docker login command.
This command will log you into the Docker Hub and store your credentials for
future use. You can use the docker logout command to log out from a registry
server.
The first method of creating images uses the docker commit command. You can
think about this method as much like making a commit in a version control system.
We create a container, make changes to that container as you would change code,
and then commit those changes to a new image.
Let’s start by creating a container from the ubuntu image we’ve used in the past.
We’ve launched our container and then installed Apache within it. We’re going
to use this container as a web server, so we’ll want to save it in its current state.
That will save us from having to rebuild it with Apache every time we create a
new container. To do this we exit from the container, using the exit command,
and use the docker commit command.
You can see we’ve used the docker commit command and specified the ID of the
container we’ve just changed (to find that ID you could use the docker ps -l
-q command to return the ID of the last created container) as well as a target
repository and image name, here jamtur01/apache2. Of note is that the docker
commit command only commits the differences between the image the container
was created from and the current state of the container. This means updates are
lightweight.
Let’s look at our new image.
We can also provide some more data about our changes when committing our
image, including tags. For example:
Here, we’ve specified some more information while committing our new image.
We’ve added the -m option which allows us to provide a commit message explain-
ing our new image. We’ve also specified the -a option to list the author of the
image. We’ve then specified the ID of the container we’re committing. Finally,
we’ve specified the username and repository of the image, jamtur01/apache2, and
we’ve added a tag, webserver, to our image.
We can view this information about our image using the docker inspect com-
mand.
TIP You can find a full list of the docker commit flags here.
If we want to run a container from our new image, we can do so using the docker
run command.
You’ll note that we’ve specified our image with the full tag: jamtur01/apache2:
webserver.
Let’s now create a directory and an initial Dockerfile. We’re going to build a
Docker image that contains a simple web server.
$ mkdir static_web
$ cd static_web
$ touch Dockerfile
We’ve created a directory called static_web to hold our Dockerfile. This di-
rectory is our build environment, which is what Docker calls a context or build
context. Docker will upload the build context, as well as any files and directories
contained in it, to our Docker daemon when the build is run. This provides the
Docker daemon with direct access to any code, files or other data you might want
to include in the image.
We’ve also created an empty Dockerfile file to get started. Now let’s look at an
example of a Dockerfile to create a Docker image that will act as a Web server.
# Version: 0.0.1
FROM ubuntu:18.04
LABEL maintainer="[email protected]"
RUN apt-get update; apt-get install -y nginx
RUN echo 'Hi, I am in your container' \
>/var/www/html/index.html
EXPOSE 80
This means that if your Dockerfile stops for some reason (for example, if an
instruction fails to complete), you will be left with an image you can use. This is
highly useful for debugging: you can run a container from this image interactively
and then debug why your instruction failed using the last image created.
NOTE The Dockerfile also supports comments. Any line that starts with
a # is considered a comment. You can see an example of this in the first line of
our Dockerfile.
The first instruction in a Dockerfile must be FROM. The FROM instruction specifies
an existing image that the following instructions will operate on; this image is
called the base image.
In our sample Dockerfile we’ve specified the ubuntu:16.04 image as our base
image. This specification will build an image on top of an Ubuntu 16.04 base
operating system. As with running a container, you should always be specific
about exactly from which base image you are building.
Next, we’ve specified the LABEL instruction with a value of ‘maintainer=“[email protected]”,
which tells Docker who the author of the image is and what their email address
is. This is useful for specifying an owner and contact for an image.
We’ve followed these instructions with two RUN instructions. The RUN instruction
executes commands on the current image. The commands in our example: up-
dating the installed APT repositories and installing the nginx package and then
creating the /var/www/html/index.html file containing some example text. As
we’ve discovered, each of these instructions will create a new layer and, if suc-
cessful, will commit that layer and then execute the next instruction.
By default, the RUN instruction executes inside a shell using the command wrapper
/bin/sh -c. If you are running the instruction on a platform without a shell or
you wish to execute without a shell (for example, to avoid shell string munging),
you can specify the instruction in exec format:
NOTE Docker also uses the EXPOSE instruction to help link together con-
tainers, which we’ll see in Chapter 5. You can expose ports at run time with the
docker run command with the --expose option.
All of the instructions will be executed and committed and a new image returned
when we run the docker build command. Let’s try that now:
$ cd static_web
$ sudo docker build -t="jamtur01/static_web" .
Sending build context to Docker daemon 2.56 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:18.04
---> ba5877dc9bec
Step 1 : LABEL maintainer="[email protected]"
---> Running in b8ffa06f9274
---> 4c66c9dcee35
Removing intermediate container b8ffa06f9274
Step 2 : RUN apt-get update
---> Running in f331636c84f7
---> 9d938b9e0090
Removing intermediate container f331636c84f7
Step 3 : RUN apt-get install -y nginx
---> Running in 4b989d4730dd
---> 93fb180f3bc9
Removing intermediate container 4b989d4730dd
Step 4 : RUN echo 'Hi, I am in your container'
>/var/www/html/index.html
---> Running in b51bacc46eb9
---> b584f4ac1def
Removing intermediate container b51bacc46eb9
Step 5 : EXPOSE 80
---> Running in 7ff423bd1f4d
---> 22d47c8cb6e5
Successfully built 22d47c8cb6e5
We’ve used the docker build command to build our new image. We’ve specified
the -t option to mark our resulting image with a repository and a name, here the
jamtur01 repository and the image name static_web. I strongly recommend you
always name your images to make it easier to track and manage them.
You can also tag images during the build process by suffixing the tag after the
image name with a colon, for example:
TIP If you don’t specify any tag, Docker will automatically tag your image
as latest.
The trailing . tells Docker to look in the local directory to find the Dockerfile.
You can also specify a Git repository as a source for the Dockerfile as we see
here:
Here Docker assumes that there is a Dockerfile located in the root of the Git
repository.
TIP Since Docker 1.5.0 and later you can also specify a path to a file
to use as a build source using the -f flag. For example, docker build -t
"jamtur01/static_web" -f /path/to/file. The file specified doesn’t need to
But back to our docker build process. You can see that the build context has
been uploaded to the Docker daemon.
TIP If a file named .dockerignore exists in the root of the build context
then it is interpreted as a newline-separated list of exclusion patterns. Much like
a .gitignore file it excludes the listed files from being treated as part of the build
context, and therefore prevents them from being uploaded to the Docker daemon.
Globbing can be done using Go’s filepath.
Next, you can see that each instruction in the Dockerfile has been executed with
the image ID, 22d47c8cb6e5, being returned as the final output of the build pro-
cess. Each step and its associated instruction are run individually, and Docker has
committed the result of each operation before outputting that final image ID.
$ cd static_web
$ sudo docker build -t="jamtur01/static_web" .
Sending build context to Docker daemon 2.56 kB
Sending build context to Docker daemon
Step 1 : FROM ubuntu:18.04
---> 8dbd9e392a96
Step 2 : LABEL maintainer="[email protected]"
---> Running in d97e0c1cf6ea
---> 85130977028d
Step 3 : RUN apt-get update
---> Running in 85130977028d
---> 997485f46ec4
Step 4 : RUN apt-get install -y ngin
---> Running in ffca16d58fd8
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package ngin
2014/06/04 18:41:11 The command [/bin/sh -c apt-get install -y
ngin] returned a non-zero code: 100
Let’s say I want to debug this failure. I can use the docker run command to create
a container from the last step that succeeded in my Docker build, in this example
using the image ID of 997485f46ec4.
I can then try to run the apt-get install -y ngin step again with the right pack-
age name or conduct some other debugging to determine what went wrong. Once
I’ve identified the issue, I can exit the container, update my Dockerfile with the
right package name, and retry my build.
As a result of the build cache, you can build your Dockerfiles in the form of
simple templates (e.g., adding a package repository or updating packages near
the top of the file to ensure the cache is hit). I generally have the same template
set of instructions in the top of my Dockerfile, for example for Ubuntu:
FROM ubuntu:18.04
LABEL maintainer="[email protected]"
ENV REFRESHED_AT 2016-07-01
RUN apt-get -qq update
Let’s step through this new Dockerfile. Firstly, I’ve used the FROM instruction
to specify a base image of ubuntu:16.04. Next, I’ve added my MAINTAINER in-
struction to provide my contact details. I’ve then specified a new instruction, ENV.
The ENV instruction sets environment variables in the image. In this case, I’ve
specified the ENV instruction to set an environment variable called REFRESHED_AT,
showing when the template was last updated. Lastly, I’ve specified the apt-get
-qq update command in a RUN instruction. This refreshes the APT package cache
when it’s run, ensuring that the latest packages are available to install.
With my template, when I want to refresh the build, I change the date in my ENV
instruction. Docker then resets the cache when it hits that ENV instruction and runs
every subsequent instruction anew without relying on the cache. This means my
RUN apt-get update instruction is rerun and my package cache is refreshed with
the latest content. You can extend this template example for your target platform
or to fit a variety of needs. For example, for a fedora image we might:
FROM fedora:21
LABEL maintainer="[email protected]"
ENV REFRESHED_AT 2016-07-01
RUN yum -q makecache
Now let’s take a look at our new image. We can do this using the docker images
command.
If we want to drill down into how our image was created, we can use the docker
history command.
We see each of the image layers inside our new jamtur01/static_web image and
the Dockerfile instruction that created them.
Let’s launch a new container using our new image and see if what we’ve built has
worked.
Here I’ve launched a new container called static_web using the docker run com-
mand and the name of the image we’ve just created. We’ve specified the -d option,
which tells Docker to run detached in the background. This allows us to run long-
running processes like the Nginx daemon. We’ve also specified a command for
the container to run: nginx -g "daemon off;". This will launch Nginx in the
foreground to run our web server.
We’ve also specified a new flag, -p. The -p flag manages which network ports
Docker publishes at runtime. When you run a container, Docker has two methods
of assigning ports on the Docker host:
• Docker can randomly assign a high port from the range 32768 to 61000 on
the Docker host that maps to port 80 on the container.
• You can specify a specific port on the Docker host that maps to port 80 on
the container.
The docker run command will open a random port on the Docker host that will
connect to port 80 on the Docker container.
Let’s look at what port has been assigned using the docker ps command. The -l
flag tells Docker to show us the last container launched.
$ sudo docker ps -l
CONTAINER ID IMAGE ... PORTS
NAMES
6751b94bb5c0 jamtur01/static_web:latest ... 0.0.0.0:49154->80/
tcp static_web
We see that port 49154 is mapped to the container port of 80. We can get the
same information with the docker port command.
We’ve specified the container ID and the container port for which we’d like to see
the mapping, 80, and it has returned the mapped port, 49154.
Or we could use the container name too.
The -p option also allows us to be flexible about how a port is published to the
host. For example, we can specify that Docker bind the port to a specific port:
This will bind port 80 on the container to port 80 on the local host. It’s impor-
tant to be wary of this direct binding: if you’re running multiple containers, only
one container can bind a specific port on the local host. This can limit Docker’s
flexibility.
To avoid this, we could bind to a different port.
This would bind port 80 on the container to port 8080 on the local host.
We can also bind to a specific interface.
Here we’ve bound port 80 of the container to port 80 on the 127.0.0.1 interface
on the local host. We can also bind to a random port using the same structure.
Here we’ve removed the specific port to bind to on 127.0.0.1. We would now
use the docker inspect or docker port command to see which random port was
assigned to port 80 on the container.
TIP You can bind UDP ports by adding the suffix /udp to the port binding.
Docker also has a shortcut, -P, that allows us to publish all ports we’ve exposed
via EXPOSE instructions in our Dockerfile.
This would publish port 80 on a random port on our local host. It would also
publish any additional ports we had specified with other EXPOSE instructions in
the Dockerfile that built our image.
With this port number, we can now view the web server on the running container
using the IP address of our host or the localhost on 127.0.0.1.
NOTE You can find the IP address of your local host with the ifconfig or
ip addr command.
$ curl localhost:49154
Hi, I am in your container
Dockerfile instructions
We’ve already seen some of the available Dockerfile instructions, like RUN and
EXPOSE. But there are also a variety of other instructions we can put in our
Dockerfile. These include CMD, ENTRYPOINT, ADD, COPY, VOLUME, WORKDIR, USER,
ONBUILD, LABEL, STOPSIGNAL, ARG, SHELL, HEALTHCHECK and ENV. You can see a full
list of the available Dockerfile instructions here.
We’ll also see a lot more Dockerfiles in the next few chapters and see how to
build some cool applications into Docker containers.
CMD
The CMD instruction specifies the command to run when a container is launched. It
is similar to the RUN instruction, but rather than running the command when the
container is being built, it will specify the command to run when the container
is launched, much like specifying a command to run when launching a container
with the docker run command, for example:
CMD ["/bin/true"]
Lastly, it’s important to understand that we can override the CMD instruction using
the docker run command. If we specify a CMD in our Dockerfile and one on the
docker run command line, then the command line will override the Dockerfile’s
CMD instruction.
NOTE It’s also important to understand the interaction between the CMD
instruction and the ENTRYPOINT instruction. We’ll see some more details of this
below.
Let’s look at this process a little more closely. Let’s say our Dockerfile contains
the CMD:
CMD [ "/bin/bash" ]
We can build a new image (let’s call it jamtur01/test) using the docker build
command and then launch a new container from this image.
You can see here that we have specified the /bin/ps command to list running
processes. Instead of launching a shell, the container merely returned the list
of running processes and stopped, overriding the command specified in the CMD
instruction.
TIP You can only specify one CMD instruction in a Dockerfile. If more than
one is specified, then the last CMD instruction will be used. If you need to run
multiple processes or commands as part of starting a container you should use a
service management tool like Supervisor.
ENTRYPOINT
Closely related to the CMD instruction, and often confused with it, is the ENTRYPOINT
instruction. So what’s the difference between the two, and why are they both
needed? As we’ve just discovered, we can override the CMD instruction on the
docker run command line. Sometimes this isn’t great when we want a container
to behave in a certain way. The ENTRYPOINT instruction provides a command
that isn’t as easily overridden. Instead, any arguments we specify on the docker
run command line will be passed as arguments to the command specified in the
ENTRYPOINT. Let’s see an example of an ENTRYPOINT instruction.
ENTRYPOINT ["/usr/sbin/nginx"]
Like the CMD instruction, we also specify parameters by adding to the array. For
example:
NOTE As with the CMD instruction above, you can see that we’ve specified
the ENTRYPOINT command in an array to avoid any issues with the command being
prepended with /bin/sh -c.
We’ve rebuilt our image and then launched an interactive container. We specified
the argument -g "daemon off;". This argument will be passed to the command
specified in the ENTRYPOINT instruction, which will thus become /usr/sbin/nginx
-g "daemon off;". This command would then launch the Nginx daemon in the
foreground and leave the container running as a web server.
We can also combine ENTRYPOINT and CMD to do some neat things. For example,
we might want to specify the following in our Dockerfile.
ENTRYPOINT ["/usr/sbin/nginx"]
CMD ["-h"]
Now when we launch a container, any option we specify will be passed to the
Nginx daemon; for example, we could specify -g "daemon off"; as we did above
to run the daemon in the foreground. If we don’t specify anything to pass to the
container, then the -h is passed by the CMD instruction and returns the Nginx help
text: /usr/sbin/nginx -h.
This allows us to build in a default command to execute when our container is run
combined with overridable options and flags on the docker run command line.
TIP If required at runtime, you can override the ENTRYPOINT instruction using
the docker run command with --entrypoint flag.
WORKDIR
The WORKDIR instruction provides a way to set the working directory for the con-
tainer and the ENTRYPOINT and/or CMD to be executed when a container is launched
from the image.
We can use it to set the working directory for a series of instructions or for the
final container. For example, to set the working directory for a specific instruction
we might:
WORKDIR /opt/webapp/db
RUN bundle install
WORKDIR /opt/webapp
ENTRYPOINT [ "rackup" ]
Here we’ve changed into the /opt/webapp/db directory to run bundle install
and then changed into the /opt/webapp directory prior to specifying our
ENTRYPOINT instruction of rackup.
You can override the working directory at runtime with the -w flag, for example:
ENV
The ENV instruction is used to set environment variables during the image build
process. For example:
This new environment variable will be used for any subsequent RUN instructions,
as if we had specified an environment variable prefix to a command like so:
You can specify single environment variables in an ENV instruction or since Docker
1.4 you can specify multiple variables like so:
Here we’ve specified a new environment variable, TARGET_DIR, and then used its
value in a WORKDIR instruction. Our WORKDIR instruction would now be set to /opt
/app.
NOTE You can also escape environment variables when needed by prefix-
ing them with a backslash.
These environment variables will also be persisted into any containers created
from your image. So, if we were to run the env command in a container built
with the ENV RVM_PATH /home/rvm/ instruction we’d see:
root@bf42aadc7f09:~# env
. . .
RVM_PATH=/home/rvm/
. . .
You can also pass environment variables on the docker run command line using
the -e flag. These variables will only apply at runtime, for example:
Now our container has the WEB_PORT environment variable set to 8080.
USER
The USER instruction specifies a user that the image should be run as; for example:
USER nginx
This will cause containers created from the image to be run by the nginx user. We
can specify a username or a UID and group or GID. Or even a combination thereof,
for example:
USER user
USER user:group
USER uid
USER uid:gid
USER user:gid
USER uid:group
You can also override this at runtime by specifying the -u flag with the docker run
command.
TIP The default user if you don’t specify the USER instruction is root.
VOLUME
The VOLUME instruction adds volumes to any container created from the image.
A volume is a specially designated directory within one or more containers that
bypasses the Union File System to provide several useful features for persistent or
shared data:
This allows us to add data (like source code), a database, or other content into an
image without committing it to the image and allows us to share that data between
containers. This can be used to do testing with containers and an application’s
code, manage logs, or handle databases inside a container. We’ll see examples of
this in Chapters 5 and 6.
You can use the VOLUME instruction like so:
VOLUME ["/opt/project"]
This would attempt to create a mount point /opt/project to any container created
from the image.
TIP Also useful and related is the docker cp command. This allows you
to copy files to and from your containers. You can read about it in the Docker
command line documentation.
TIP We’ll see a lot more about volumes and how to use them in Chapters 5
and 6. If you’re curious you can read more about volumes in the Docker volumes
documentation.
ADD
The ADD instruction adds files and directories from our build environment into our
image; for example, when installing an application. The ADD instruction specifies
a source and a destination for the files, like so:
This ADD instruction will copy the file software.lic from the build directory to /
opt/application/software.lic in the image. The source of the file can be a URL,
filename, or directory as long as it is inside the build context or environment. You
cannot ADD files from outside the build directory or context.
When ADD’ing files Docker uses the ending character of the destination to deter-
mine what the source is. If the destination ends in a /, then it considers the source
a directory. If it doesn’t end in a /, it considers the source a file.
Lastly, the ADD instruction has some special magic for taking care of local tar
archives. If a tar archive (valid archive types include gzip, bzip2, xz) is specified
as the source file, then Docker will automatically unpack it for you:
This will unpack the latest.tar.gz archive into the /var/www/wordpress/ direc-
tory. The archive is unpacked with the same behavior as running tar with the
-x option: the output is the union of whatever exists in the destination plus the
contents of the archive. If a file or directory with the same name already exists in
the destination, it will not be overwritten.
WARNING Currently this will not work with a tar archive specified in a
URL. This is somewhat inconsistent behavior and may change in a future release.
Finally, if the destination doesn’t exist, Docker will create the full path for us,
including any directories. New files and directories will be created with a mode
of 0755 and a UID and GID of 0.
NOTE It’s also important to note that the build cache can be invalidated
by ADD instructions. If the files or directories added by an ADD instruction change
then this will invalidate the cache for all following instructions in the Dockerfile.
COPY
The COPY instruction is closely related to the ADD instruction. The key difference
is that the COPY instruction is purely focused on copying local files from the build
context and does not have any extraction or decompression capabilities.
This will copy files from the conf.d directory to the /etc/apache2/ directory.
The source of the files must be the path to a file or directory relative to the build
context, the local source directory in which your Dockerfile resides. You cannot
copy anything that is outside of this directory, because the build context is up-
loaded to the Docker daemon, and the copy takes place there. Anything outside
of the build context is not available. The destination should be an absolute path
inside the container.
Any files and directories created by the copy will have a UID and GID of 0.
If the source is a directory, the entire directory is copied, including filesystem
metadata; if the source is any other kind of file, it is copied individually along
with its metadata. In our example, the destination ends with a trailing slash /, so
it will be considered a directory and copied to the destination directory.
If the destination doesn’t exist, it is created along with all missing directories in
its path, much like how the mkdir -p command works.
LABEL
The LABEL instruction adds metadata to a Docker image. The metadata is in the
form of key/value pairs. Let’s see an example.
LABEL version="1.0"
LABEL location="New York" type="Data Center" role="Web Server"
The LABEL instruction is written in the form of label="value". You can specify
one item of metadata per label or multiple items separated with white space. We
recommend combining all your metadata in a single LABEL instruction to save
creating multiple layers with each piece of metadata. You can inspect the labels
on an image using the docker inspect command..
"Labels": {
"version": "1.0",
"location": "New York",
"type": "Data Center",
"role": "Web Server"
},
Here we see the metadata we just defined using the LABEL instruction.
STOPSIGNAL
The STOPSIGNAL instruction sets the system call signal that will be sent to the
container when you tell it to stop. This signal can be a valid number from the
kernel syscall table, for instance 9, or a signal name in the format SIGNAME, for
instance SIGKILL.
ARG
The ARG instruction defines variables that can be passed at build-time via the
docker build command. This is done using the --build-arg flag. You can only
specify build-time arguments that have been defined in the Dockerfile.
ARG build
ARG webapp_user=user
The second ARG instruction sets a default, if no value is specified for the argument
at build-time then the default is used. Let’s use one of these arguments in a docker
build now.
As the jamtur01/webapp image is built the build variable will be set to 1234 and
the webapp_user variable will inherit the default value of user.
Docker has a set of predefined ARG variables that you can use at build-time without
a corresponding ARG instruction in the Dockerfile.
HTTP_PROXY
http_proxy
HTTPS_PROXY
https_proxy
FTP_PROXY
ftp_proxy
NO_PROXY
no_proxy
To use these predefined variables, pass them using the --build-arg <variable
>=<value> flag to the docker build command.
NOTE The ARG instruction was introduced in Docker 1.9 and you can read
more about it in the Docker documentation.
SHELL
The SHELL instruction allows the default shell used for the shell form of commands
to be overridden. The default shell on Linux is ‘["/bin/sh", "-c"] and on Win-
dows is ["cmd", "/S", "/C"].
The SHELL instruction is useful on platforms such as Windows where there are
multiple shells, for example running commands in the cmd or powershell environ-
ments. Or when need to run a command on Linux in a specific shell, for example
Bash.
The SHELL instruction can be used multiple times. Each new SHELL instruction
overrides all previous SHELL instructions, and affects any subsequent instructions.
HEALTHCHECK
The HEALTHCHECK instruction tells Docker how to test a container to check that it
is still working correctly. This allows you to check things like a web site being
served or an API endpoint responding with the correct data, allowing you to iden-
tify issues that appear, even if an underlying process still appears to be running
normally.
When a container has a health check specified, it has a health status in addition
to its normal status. You can specify a health check like:
The HEALTHCHECK instruction contains options and then the command you wish to
run itself, separated by a CMD keyword.
We’ve first specified three default options:
The command after the CMD keyword can be either a shell command or an exec
array, for example as we’ve seen in the ENTRYPOINT instruction. The command
should exit with 0 to indicate health or 1 to indicate an unhealthy state. In our
CMD we’re executing curl on the localhost. If the command fails we’re exiting
with an exit code of 1, indicating an unhealthy state.
We can see the state of the health check using the docker inspect command.
The health check state and related data is stored in the .State.Health namespace
and includes current state as well as a history of previous checks and their output.
The output from each health check is also available via docker inspect.
Here we’re iterating through the array of .Log entries in the docker inspect out-
put.
There can only be one HEALTHCHECK instruction in a Dockerfile. If you list more
than one then only the last will take effect.
You can also disable any health checks specified in any base images you may have
inherited with the instruction:
HEALTHCHECK NONE
ONBUILD
The ONBUILD instruction adds triggers to images. A trigger is executed when the
image is used as the basis of another image (e.g., if you have an image that needs
source code added from a specific location that might not yet be available, or if
you need to execute a build script that is specific to the environment in which the
image is built).
The trigger inserts a new instruction in the build process, as if it were specified
right after the FROM instruction. The trigger can be any build instruction. For
example:
This would add an ONBUILD trigger to the image being created, which we see when
we run docker inspect on the image.
For example, we’ll build a new Dockerfile for an Apache2 image that we’ll call
jamtur01/apache2.
FROM ubuntu:18.04
LABEL maintainer="[email protected]"
RUN apt-get update; apt-get install -y apache2
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
ENV APACHE_RUN_DIR /var/run/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ONBUILD ADD . /var/www/
EXPOSE 80
ENTRYPOINT ["/usr/sbin/apachectl"]
CMD ["-D", "FOREGROUND"]
We now have an image with an ONBUILD instruction that uses the ADD instruction
to add the contents of the directory we’re building from to the /var/www/ directory
in our image. This could readily be our generic web application template from
which I build web applications.
Let’s try this now by building a new image called webapp from the following
Dockerfile:
FROM jamtur01/apache2
LABEL maintainer="[email protected]"
ENV APPLICATION_NAME webapp
ENV ENVIRONMENT development
We see that straight after the FROM instruction, Docker has inserted the ADD in-
struction, specified by the ONBUILD trigger, and then proceeded to execute the
remaining steps. This would allow me to always add the local source and, as I’ve
done here, specify some configuration or build information for each application;
hence, this becomes a useful template image.
The ONBUILD triggers are executed in the order specified in the parent image and
are only inherited once (i.e., by children and not grandchildren). If we built an-
other image from this new image, a grandchild of the jamtur01/apache2 image,
then the triggers would not be executed when that image is built.
NOTE There are several instructions you can’t ONBUILD: FROM, MAINTAINER,
and ONBUILD itself. This is done to prevent Inception-like recursion in Dockerfile
builds.
Once we’ve got an image, we can upload it to the Docker Hub. This allows us to
make it available for others to use. For example, we could share it with others in
our organization or make it publicly available.
NOTE The Docker Hub also has the option of private repositories. These
are a paid-for feature that allows you to store an image in a private repository
that is only available to you or anyone with whom you share it. This allows you
to have private images containing proprietary information or code you might not
want to share publicly.
We push images to the Docker Hub using the docker push command.
Let’s build an image without a user prefix and try and push it now.
$ cd static_web
$ sudo docker build --no-cache -t="static_web" .
. . .
Successfully built a312a2ed58c7
$ sudo docker push static_web
The push refers to a repository [docker.io/library/static_web]
c0121fc36460: Preparing
8591faa9900d: Preparing
9a39129ae0ac: Preparing
98305c1a8f5e: Preparing
0185b3091e8e: Preparing
ea9f151abb7e: Waiting
unauthorized: authentication required
What’s gone wrong here? We’ve tried to push our image to the repository
static_web, but Docker knows this is a root repository. Root repositories are
managed only by the Docker, Inc., team and will reject our attempt to write to
them as unauthorized. Let’s try again, rebuilding our image with a user prefix
and then pushing it.
This time, our push has worked, and we’ve written to a user repository, jamtur01
/static_web. We would write to your own user ID, which we created earlier, and
to an appropriately named image (e.g., youruser/yourimage).
We can now see our uploaded image on the Docker Hub.
TIP You can find documentation and more information on the features of
the Docker Hub here.
Automated Builds
In addition to being able to build and push our images from the command line,
the Docker Hub also allows us to define Automated Builds. We can do so by con-
necting a GitHub or BitBucket repository containing a Dockerfile to the Docker
Hub. When we push to this repository, an image build will be triggered and a new
image created. This was previously also known as a Trusted Build.
NOTE Automated Builds also work for private GitHub and BitBucket repos-
itories.
The first step in adding an Automated Build to the Docker Hub is to connect your
GitHub account or BitBucket to your Docker Hub account. To do this, navigate to
Docker Hub, sign in, click on your profile link, then click the Create -> Create
Automated Build button.
You will see a page that shows your options for linking to either GitHub or Bit-
Bucket. Click the Select button under the GitHub logo to initiate the account
linkage. You will be taken to GitHub and asked to authorize access for Docker
Hub.
On Github you have two options: Public and Private (recommended) and
Limited. Select Public and Private (recommended), and click Allow Access
to complete the authorization. You may be prompted to input your GitHub
password to confirm the access.
From here, you will be prompted to select the organization and repository from
which you want to construct an Automated Build.
Select the repository from which you wish to create an Automated Build and then
configure the build.
Specify the default branch you wish to use, and confirm the repository name.
Specify a tag you wish to apply to any resulting build, then specify the location of
the Dockerfile. The default is assumed to be the root of the repository, but you
can override this with any path.
Finally, click the Create button to add your Automated Build to the Docker Hub.
You will now see your Automated Build submitted. Click on the Build Details
link to see the status of the last build, including log output showing the build
process and any errors. A build status of Done indicates the Automated Build is
up to date. An Error status indicates a problem; you can click through to see the
log output.
NOTE You can’t push to an Automated Build using the docker push com-
mand. You can only update it by pushing updates to your GitHub or BitBucket
repository.
Deleting an image
We can also delete images when we don’t need them anymore. To do this, we’ll
use the docker rmi command.
Here we’ve deleted the jamtur01/static_web image. You can see Docker’s layer
filesystem at work here: each of the Deleted: lines represents an image layer
being deleted. If a running container is still using an image then you won’t be
able to delete it. You’ll need to stop all containers running that image, remove
them and then delete the image.
NOTE This only deletes the image locally. If you’ve previously pushed
that image to the Docker Hub, it’ll still exist there.
If you want to delete an image’s repository on the Docker Hub, you’ll need to sign
in and delete it there using the Settings -> Delete button.
We can also delete more than one image by specifying a list on the command line.
or, like the docker rm command cheat we saw in Chapter 3, we can do the same
with the docker rmi command:
The team at Docker, Inc., have open-sourced the code they use to run a Docker
registry, thus allowing us to build our own internal registry. The registry does not
currently have a user interface and is only made available as an API service.
TIP If you’re running Docker behind a proxy or corporate firewall you can
also use the HTTPS_PROXY, HTTP_PROXY, NO_PROXY options to control how Docker
connects.
Installing a registry from a Docker container is simple. Just run the Docker-
provided container like so:
This will launch a container running version 2.0 of the registry application and
bind port 5000 to the local host.
TIP If you’re running an older version of the Docker Registry, prior to 2.0,
you can use the Migrator tool to upgrade to a new registry.
So how can we make use of our new registry? Let’s see if we can upload one of
our existing images, the jamtur01/static_web image, to our new registry. First,
let’s identify the image’s ID using the docker images command.
Next we take our image ID, 22d47c8cb6e5, and tag it for our new registry. To
specify the new registry destination, we prefix the image name with the hostname
and port of our new registry. In our case, our new registry has a hostname of
docker.example.com.
After tagging our image, we can then push it to the new registry using the docker
push command:
The image is then posted in the local registry and available for us to build new
containers using the docker run command.
This is the simplest deployment of the Docker registry behind your firewall. It
doesn’t explain how to configure the registry or manage it. To find out details
like configuring authentication, how to manage the backend storage for your im-
ages and how to manage your registry see the full configuration and deployments
details in the Docker Registry deployment documentation.
Alternative Indexes
There are a variety of other services and companies out there starting to provide
custom Docker registry services.
Quay
The Quay service provides a private hosted registry that allows you to upload both
public and private containers. Unlimited public repositories are currently free.
Private repositories are available in a series of scaled plans. The Quay product
has recently been acquired by CoreOS and will be integrated into that product.
Summary
In this chapter, we’ve seen how to use and interact with Docker images and the
basics of modifying, updating, and uploading images to the Docker Hub. We’ve
also learned about using a Dockerfile to construct our own custom images. Fi-
nally, we’ve discovered how to run our own local Docker registry and some hosted
alternatives. This gives us the basis for starting to build services with Docker.
We’ll use this knowledge in the next chapter to see how we can integrate Docker
into a testing workflow and into a Continuous Integration lifecycle.
70
Listings
71
Listings
.dockerignore, 24 RUN, 20
/var/lib/docker, 4 STOPSIGNAL, 49
USER, 43
Automated Builds, 60 VOLUME, 44
Build content, 47 WORKDIR, 39
Build context, 18, 24 Running your own registry, 65
.dockerignore, 24 Security, 8
Building images, 18 setting the working directory, 39
Bypassing the Dockerfile cache, 26 specifying a Docker build source, 24
tags, 7
Context, 18 docker
build, 18, 21, 22, 49
Debugging Dockerfiles, 25
–no-cache, 26
Docker
-f, 24
Bind UDP ports, 32
context, 18
Docker Hub, 5
commit, 15
Dockerfile
history, 28
ADD, 45
images, 4, 9, 28, 66
ARG, 49
inspect, 16, 32, 48, 54
CMD, 34
login, 13
COPY, 47
logout, 14
ENTRYPOINT, 37
port, 30, 32
ENV, 40
ps, 30
EXPOSE, 21, 33
pull, 6, 9
FROM, 20
push, 57, 63, 67
LABEL, 20, 48
rm, 65
ONBUILD, 53
75
Index
exec format, 20
GitHub, 60
HTTP_PROXY, 65
HTTPS_PROXY, 65
NO_PROXY, 65
Port mapping, 21
Private repositories, 57
proxy, 65
Registry