The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove
4 likes4,620 views
Dockerfiles are great. They provide a zero-barrier-to-entry format for describing a single Docker image which is immediately clear to anyone reading them. But with that simplicity comes problems that become apparent as your adoption of Docker gathers pace. * Dockerfiles can inherit from other docker images, but images are not Dockerfiles * Dockerfile provides no built-in mechanism for creating abstractions, so as usage grows identical or similar instructions can be duplicated across many files * The Docker APi exposes a build endpoint, but the API is very course, taking Dockerfile as the transport rather than exposing the individual instructions * Dockerfiles are just that, files. So they can come from anywhere The one layer per line in a Dockerfile limitation can lead to an explosion of layers, which fail to take advantage of the promised space and performance benefits.
![A simple Dockerfile
FROM ubuntu
# Install vnc, xvfb in order to create a 'fake' display and fire
RUN apt-get update && apt-get install -y x11vnc xvfb firefox
RUN mkdir ~/.vnc
# Setup a password
RUN x11vnc -storepasswd 1234 ~/.vnc/passwd
# Autostart firefox (might not be the best way, but it does the
RUN bash -c 'echo "firefox" >> /.bashrc'
EXPOSE 5900
CMD ["x11vnc", "-forever", "-usepw", "-create"]](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-12-320.jpg)
![Commands you know
MAINTAINER <name>
RUN <command>
CMD ["executable","param1","param2"]
EXPOSE <port> [<port>...]
ADD <src>... <dest>
ENV <key> <value>
WORKDIR /path/to/workdir
USER daemon
VOLUME ["/data"]
ENTRYPOINT ["executable", "param1", “param2"]
COPY <src>... <dest>](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-14-320.jpg)
![Commands you don’t know
ONBUILD [INSTRUCTION]
STOPSIGNAL signal
ARG <name>[=<default value>]
LABEL <key>=<value> <key>=<value> <key>=<value> …
HEALTHCHECK [OPTIONS] CMD command
SHELL ["executable", "parameters"]](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-15-320.jpg)
![Simplicity
FROM scratch
COPY hello /
CMD ["/hello"]](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-21-320.jpg)
![{
"builders":[{
"type": "docker",
"image": "ubuntu",
"export_path": "image.tar"
}],
"provisioners":[
{
"type": "shell",
"inline": ["apt-get -y update; apt-get install -y puppet-co
},
{
Packer example](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-56-320.jpg)
![$ s2i create <image name> <destination directory>
$ s2i build <source location> <builder image> [<tag>] [flags]
$ s2i rebuild <image name> [<new-tag-name>]
$ s2i usage <builder image> [flags]
$ s2i build ./sinatra-app openshift/ruby-20-centos7 ruby-app
s2i example](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-58-320.jpg)
![dockerTools.buildImage {
name = "redis";
runAsRoot = ''
#!${stdenv.shell}
${dockerTools.shadowSetup}
groupadd -r redis
useradd -r -g redis -d /data -M redis
mkdir /data
chown redis:redis /data
'';
contents = [ redis ];
Nix example](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-60-320.jpg)
![rm -rf /var/lib/apt/lists/*
EXPOSE 80
CMD ["nginx"]
COPY Rockerfile /Dockerfile
TAG puppet/puppet-rocker-example
More new instructions](https://image.slidesharecdn.com/thedockerfileexplosion-160624235502/85/The-Dockerfile-Explosion-and-the-Need-for-Higher-Level-Tools-by-Gareth-Rushgrove-67-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
![Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...](https://cdn.slidesharecdn.com/ss_thumbnails/alphabet-160624232958-thumbnail.jpg?width=560&fit=bounds)
Ad
Similar to The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove (20)
Ad
More from Docker, Inc. (20)
Recently uploaded (20)
The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove
- 1. The Dockerfile explosion Gareth Rushgrove Senior Software Engineer Puppet
- 2. The Dockerfile explosion and the need for higher level tools
- 3. Introductions Who am I and what am I doing here
- 4. @garethr
- 5. (without introducing more risk) Gareth Rushgrove
- 6. Built the Puppet Docker module
- 7. Maintain the Puppet images
- 9. A brief history of Dockerfile
- 10. Docker can build images automatically by reading the instructions from a Dockerfile From the official docs at https://docs.docker.com/engine/reference/builder/
- 11. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. From the official docs at https://docs.docker.com/engine/reference/builder/
- 12. A simple Dockerfile FROM ubuntu # Install vnc, xvfb in order to create a 'fake' display and fire RUN apt-get update && apt-get install -y x11vnc xvfb firefox RUN mkdir ~/.vnc # Setup a password RUN x11vnc -storepasswd 1234 ~/.vnc/passwd # Autostart firefox (might not be the best way, but it does the RUN bash -c 'echo "firefox" >> /.bashrc' EXPOSE 5900 CMD ["x11vnc", "-forever", "-usepw", "-create"]
- 14. Commands you know MAINTAINER <name> RUN <command> CMD ["executable","param1","param2"] EXPOSE <port> [<port>...] ADD <src>... <dest> ENV <key> <value> WORKDIR /path/to/workdir USER daemon VOLUME ["/data"] ENTRYPOINT ["executable", "param1", “param2"] COPY <src>... <dest>
- 15. Commands you don’t know ONBUILD [INSTRUCTION] STOPSIGNAL signal ARG <name>[=<default value>] LABEL <key>=<value> <key>=<value> <key>=<value> … HEALTHCHECK [OPTIONS] CMD command SHELL ["executable", "parameters"]
- 16. Close ALL the issues
- 17. Although this is not a definitive move, we temporarily won't accept more patches to the Dockerfile syntax for several reasons
- 18. HEALTHCHECK coming in 1.12
- 19. SHELL coming in 1.12
- 21. Simplicity FROM scratch COPY hello / CMD ["/hello"]
- 22. Multi-platform support PS> Install-PackageProvider ContainerImage -Force PS> Install-ContainerImage -Name WindowsServerCore PS> docker images REPOSITORY TAG IMAGE ID CREA windowsservercore 10.0.14300.1000 dbfee88ee9fd 7 we
- 23. Emerging tooling
- 24. Linting
- 25. Editor support
- 26. Cross platform
- 28. Complexity RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.example.com/release-"$UBUNTU_CODENAME".deb dpkg -i release-"$UBUNTU_CODENAME".deb && rm release-"$UBUNTU_CODENAME".deb && apt-get update && apt-get install --no-install-recommends -y package=0.1.2 && apt-get clean && rm -rf /var/lib/apt/lists/*
- 31. 138,062
- 32. Only two approaches to reuse
- 35. Dockerfile is not the source of truth for your image
- 37. The Dockerfile generally works beautifully for the class of problem for which it was designed Nathan Leclair, Docker Inc
- 38. The Dockerfile is a tool for creating images, but it is not the only weapon in your arsenal Nathan Leclair, Docker Inc
- 40. If we dockerize all of our applications how many Dockerfiles is that?
- 41. If we build a complex hierarchy of Dockerfiles, how quickly can we trace/rebuild a specific image?
- 42. As best-practices develops how can we refactor our Dockefiles with confidence?
- 43. Are Dockerfiles best managed centrally or on a team-by-team basis?
- 46. Build Dockerfiles with OCAML
- 47. let base = let email = "[email protected]" in comment "Generated by OCaml Dockerfile" @@ from "ubuntu" ~tag:"trusty" @@ maintainer "Anil Madhavapeddy <%s>" email let ocaml_ubuntu_image = base @@ run "apt-get -y -qq update" @@ run "apt-get -y install ocaml ocaml-native-compilers camlp4-ext onbuild (run "apt-get -y -qq update") ;; OCAML example
- 48. With Gradle
- 49. Or Javascript
- 50. Or Scala and SBT
- 51. Or with Python
- 52. No Dockerfile to be seen
- 55. Packer
- 56. { "builders":[{ "type": "docker", "image": "ubuntu", "export_path": "image.tar" }], "provisioners":[ { "type": "shell", "inline": ["apt-get -y update; apt-get install -y puppet-co }, { Packer example
- 57. Source-to-Image
- 58. $ s2i create <image name> <destination directory> $ s2i build <source location> <builder image> [<tag>] [flags] $ s2i rebuild <image name> [<new-tag-name>] $ s2i usage <builder image> [flags] $ s2i build ./sinatra-app openshift/ruby-20-centos7 ruby-app s2i example
- 59. Nix
- 60. dockerTools.buildImage { name = "redis"; runAsRoot = '' #!${stdenv.shell} ${dockerTools.shadowSetup} groupadd -r redis useradd -r -g redis -d /data -M redis mkdir /data chown redis:redis /data ''; contents = [ redis ]; Nix example
- 61. Habitat
- 63. Rocker
- 64. Rocker adds some crucial features that are missing from Dockerfile while keeping Docker’s original design
- 65. FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "[email protected]" ENV PUPPET_AGENT_VERSION="1.5.0" UBUNTU_CODENAME="xenial" PATH=/ LABEL com.puppet.version="0.1.0" com.puppet.dockerfile="/Dockerf MOUNT /opt/puppetlabs /etc/puppetlabs /root/.gem RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.puppetlabs.com/puppetlabs-release-pc1-"$UBU Rockerfile example
- 66. FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "[email protected]" ENV PUPPET_AGENT_VERSION="1.5.0" UBUNTU_CODENAME="xenial" PATH=/ LABEL com.puppet.version="0.1.0" com.puppet.dockerfile="/Dockerf MOUNT /opt/puppetlabs /etc/puppetlabs /root/.gem RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.puppetlabs.com/puppetlabs-release-pc1-"$UBU Includes new instructions
- 67. rm -rf /var/lib/apt/lists/* EXPOSE 80 CMD ["nginx"] COPY Rockerfile /Dockerfile TAG puppet/puppet-rocker-example More new instructions
- 68. Dockramp
- 70. FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "[email protected]" ENV PUPPET_AGENT_VERSION="1.5.0" R10K_VERSION="2.2.2" UBUNTU_C PUPPET_INSTALL PUPPET_COPY_PUPPETFILE PUPPET_COPY_MANIFESTS PUPPET_RUN EXPOSE 80 Domain-specific extensions
- 71. $ cat Dockerfile | dockerfilepp FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "[email protected]" ENV PUPPET_AGENT_VERSION="1.5.0" R10K_VERSION="2.2.2" UBUNTU_COD RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.puppetlabs.com/puppetlabs-release-pc1-"$UBU dpkg -i puppetlabs-release-pc1-"$UBUNTU_CODENAME".deb && rm puppetlabs-release-pc1-"$UBUNTU_CODENAME".deb && apt-get update && Simple expansion
- 72. The future Speculation and things I’d like to see
- 74. RUN, FROM, COPY, etc. as first class API primitives
- 75. Opinionated workflow tooling around image build
- 76. Shared libraries and support for pre-processors
- 77. Complementary tools that take an organizational view of image building
- 78. Conclusions If all you take away is…
- 79. Dockerfile is a great starting point for many use cases
- 80. But we will need better tools for managing many Dockerfiles
- 81. And Dockerfile is just one interface to building images
- 82. We’ll need different types of tools for different use cases
- 83. Questions? And thanks for listening
- 84. Thank you!