Docker perl build
2 likes3,302 views
The document discusses building a lightweight Docker container for Perl by starting with a minimal base image like BusyBox, copying just the Perl installation and necessary shared libraries into the container, and setting Perl as the default command to avoid including unnecessary dependencies and tools from a full Linux distribution. It provides examples of Dockerfiles to build optimized Perl containers from Gentoo and by directly importing a tarball for minimal size and easy distribution.
![A: There is more than one way to do it.
Build perl on a light[er] weight O/S.
Single package for perl + shared libs.
Shell tools available for qx{...}.
Still pretty heavy-weight.](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-15-320.jpg)
![FROM buildpack-deps # parent container
RUN apt-get update && apt-get install -y curl procps # commands to pull perl
RUN mkdir /usr/src/perl
WORKDIR /usr/src/perl # build dir within the container
RUN curl -SL http://www.cpan.org/src/5.0/perl-5.20.0.tar.gz | tar -xz --strip-components=1
RUN ./Configure -Duse64bitall -des
&& make -j$(nproc) && TEST_JOBS=$(nproc) make test_harness
&& make install && make veryclean
WORKDIR /root
CMD ["perl5.20.0","-de0"] # /bin/sh perl5.20.0 -de0
Distro's Dockerfile](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-27-320.jpg)
![New Dockerfile
FROM jgkim/gentoo-stage3
MAINTAINER Steven Lembark <lembark@wrkhors.com>
WORKDIR /var/tmp/
RUN wget -O – http://www.cpan.org/src/5.0/perl-5.20.2.tar.gz
| gzip -dc tar | tar xf -;
RUN cd perl-5.20.2 && Configure -de -Dprefix=/opt/perl
-Dman1dir=none -Dman3dir=none
-Doptimize='-O3 -march=native -pipe' ;
RUN make -C perl-5.20.2 all test install distclean;
RUN /opt/perl/bin/h2ph -r -a -l;
CMD [ "/opt/perl/bin/perl", "-d", "-E", "42" ]](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-29-320.jpg)
![$ docker build --rm=false –tag=lembark/gentoo-perl .;
Step 0 : FROM rndevfx/gentoo-stage3-amd64-nomultilib
---> e9d0ce66148c
Step 1 : MAINTAINER Steven Lembark
---> Using cache <-- recycled image
---> 41d5480e49e7
Step 2 : WORKDIR /var/tmp
---> Using cache
---> 010d3d70ced1
...
Step 7 : RUN make all test install; <-- first execution is here.
---> Running in 949c9ddfc2ef
Step 8 : RUN /opt/perl/bin/h2ph -r -a -l;
---> Running in b084569e8fc3
-r and -a options are mutually exclusive
INFO[1342] The command [/bin/sh -c /opt/perl/bin/h2ph -r -a -l;] returned a non-zero code
oops...](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-32-320.jpg)
![And impatience
$ time docker build --rm=false --tag=lembark/perl-gentoo .
Sending build context to Docker daemon 4.096 kB
Sending build context to Docker daemon
Step 0 : FROM rndevfx/gentoo-stage3-amd64-nomultilib
...
Step 10 : CMD [ “/opt/perl/bin/perl” ]
---> Using cache
---> f7b83ecbe276
Successfully built f7b83ecbe276
real 0m0.241s
user 0m0.019s
sys 0m0.012](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-35-320.jpg)
![Choose your toppings
Default for gentoo run /bin/bash <your command>.
Save typing with:
ENTRYPOINT [ “/opt/perl/bin/perl” ]
CMD [ "-d", "-E", "42" ]
Use –entrypoint='/bin/bash' if perl fails.](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-38-320.jpg)
![The next course
Stacked images inherit the ENTRYPOINT:
FROM lembark/gentoo-perl
CMD [ "/path/to/your/program" ]
runs
/opt/perl/bin/perl /path/to/your/program;](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-39-320.jpg)
![Test containers stack
Derive tests from the package.
Add ./t to another image.
WORKDIR [ “/path/to/your/code” ]
CMD [ "/opt/perl/bin/prove" ]
Result: no tests in product image:
docker run foo/bar; run application.
docker run foo/bar-test; run base tests.](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-40-320.jpg)
![Minimal Virtual Size: single "RUN" command
FROM jgkim/gentoo-stage3
MAINTAINER Steven Lembark <lembark@wrkhors.com>
WORKDIR /var/tmp/
# better yet, put this in a shell script and RUN ./build-perl!
RUN wget -O – http://www.cpan.org/src/5.0/perl-5.20.2.tar.gz
| gzip -dc tar | tar xf -
&& cd perl-5.20.2
&& Configure -de -Dprefix=/opt/perl
-Dman1dir=none -Dman3dir=none
-Doptimize='-O3 -march=native -pipe'
&& make all test install distclean
&& cd .. && rm -rf 5.20.2 ;
RUN /opt/perl/bin/h2ph -r -a -l;
ENTRYPOINT [ "/opt/perl/bin/perl" ]
CMD [ "-d", "-E", "42" ]](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-42-320.jpg)
![Copy perl on top of busybox
Build & install into /opt/perl.
/opt/perl/Dockerfile
FROM lembark/busybox_x86
COPY [ “.”, “/opt/perl” ]
ENTRYPOINT [ “/opt/perl/bin/perl” ]
CMD [ , “-d”, “-E”, “0” ]
Check build: docker run –entrypoint='/bin/sh'](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-44-320.jpg)
![Local dir with modules
/opt/perl/5.20/Dockerfile for bare copy of perl:
$ docker build –tag='lembark/perl-5.20' /opt/perl/5.20;
FROM /lembark/busybox
MAINTAINER Steven Lembark <lembark@wrkhors.com>
COPY [ ".", "/opt/perl/5.20" ]
WORKDIR /var/tmp
ENTRYPOINT [ "/opt/perl/5.20/bin/perl" ]
CMD [ "-d", "-E", "0" ]](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-46-320.jpg)
![What did all of this get us?
Mainly an optimized, current perl.
With all [and only] the modules you need.
You also save the overhead of shipping an O/S image.
Faster startup.
Easier updates.
Simpler deployment.](https://image.slidesharecdn.com/docker-perl-build-150610172557-lva1-app6891/85/Docker-perl-build-55-320.jpg)
Docker perl build
- 1. Thinking inside a box: Dockerizing Perl Steven Lembark Workhorse Computing [email protected]
- 2. What's on your box? Pre-compiled perl distros are terrible. Bulky, slow, with annoying dependencies. Not what you need...
- 3. Example: Red Hat Enterprize 7.0 5.16 == “end of life” Doptimize=O2 g pipe fstack protectorstrong mtune=generic Dversion=5.16.3 Dmyhostname=localhost Dperladmin=root@localhost Dusethreads Duseithreads Ubincompat5005
- 4. Example: Red Hat Enterprize 7.0 5.16 == “end of life” Not exactly built for speed. Doptimize=O2 g pipe fstack protectorstrong mtune=generic Dversion=5.16.3 Dmyhostname=localhost Dperladmin=root@localhost Dusethreads Duseithreads Ubincompat5005
- 5. Example: Red Hat Enterprize 7.0 5.16 == “end of life” Not exactly built for speed. Thread overhead, even if you don't use them. Doptimize=O2 g pipe fstack protectorstrong mtune=generic Dversion=5.16.3 Dmyhostname=localhost Dperladmin=root@localhost Dusethreads Duseithreads Ubincompat5005
- 6. Fresh, not frozen perl for your architecture. optimized. only dependencies you use (need). Q: How?
- 7. Try it, you'll like it... Bad approach: virtual machines...
- 8. Try it, you'll like it... Bad approach: virtual machines... and version-dirs... and recompiling for each distro, architecture... and symlink hell(2)... …
- 9. Try it, you'll like it... False lazyness!
- 10. Tasty alternative: lxc Essentially an LPAR – we've come full circle. Use a process to isolate & localize code. Share the kernel. Light weight, fast startup, easy to ship.
- 11. Fly in the soup Ever try to use lxc? Let alone finish the manpages? Kills my appetite. RTFM lxc-attach.1 lxc-autostart.1 lxc-cgroup.1 lxc-checkconfig.1 lxc-checkpoint.1 lxc-clone.1 lxc-config.1 lxc-console.1 lxc-create.1 lxc-destroy.1 lxc-device.1 lxc-execute.1 lxc-freeze.1 lxc-info.1 lxc-ls.1 lxc-monitor.1 lxc-snapshot.1 lxc-start-ephemeral.1 lxc-start.1 lxc-stop.1 lxc-top.1 lxc-unfreeze.1 lxc-unshare.1 lxc-user-nic.1 lxc-usernet.5 lxc-usernsexec.1 lxc-wait.1 lxc.7 lxc.conf.5 lxc.container.conf.5 lxc.system.conf.5
- 12. Docker: MRE for linux 80/20 of lxc: Layered filesystem + Repository + Command line. More nutritious than exciting. Still evolving.
- 13. Catch: Docker's doc is fattening! Start with full Ubuntu image. 1.6GiB – Heart attack on a plate! Includes X11 libs, lvm, mdadm, grub, parted... perl.
- 14. Q: How can we put docker on a healthy diet? Start with something lighter-weight underneath? Keep less of it in the container? Well... since this is about perl:
- 15. A: There is more than one way to do it. Build perl on a light[er] weight O/S. Single package for perl + shared libs. Shell tools available for qx{...}. Still pretty heavy-weight.
- 16. A: There is more than one way to do it. Copy perl on top of busybox. Much leaner cuisine. Decent collection of shell tools. Shared lib's as layer or via -v.
- 17. A: There is more than one way to do it. Just perl Minimal for distribution of local perl. Lacks tools to inspect the build.
- 18. A: There is more than one way to do it. Just the application. No empty calories. Requires local perl for distributed use.
- 19. Start by getting docker. They provide shell code for the basics: $curl -sL https://get.docker.io/ | sh; $wget -qO- https://get.docker.io/ | sh; will do the deed on Fedora, Ubuntu/Debian, or Gentoo. Avoids issues with apt/yum, not much value with emerge. Need to validate kernel configs.
- 20. Be yourself Don't run as su! Add your users to “docker” in /etc/group. After that check that docker is running: $ docker ps; Get access to the repository: $ docker login;
- 21. Just a taste... Minimal base container: busybox. Good for experimenting: $ docker pull busybox Pulling repository busybox fd5373b3d938: Download complete ... f06b02872d52: Download complete
- 22. Getting inside the box /bin/sh is default entrypoint: $ docker run -t -i busybox; # <-- su in box, login id out. # ping 8.8.8.8; <-- network available. ... # exit; <-- exit docker process $ <-- original EUID
- 23. Gentoo is easy to dockerize Common solution is a “stage-3” system. Shell + libs + build tools. Not much else. About half the size of Ubuntu.
- 24. Finding a distribution Start at the docker registry https://registry.hub.docker.com/ Looking for stage-3 builds: https://registry.hub.docker.com/search?q=gentoo+stage3 I find: jgkim/gentoo-stage3 741.2 MB Reasonable start.
- 25. Grabbing an O/S Get the image: $ docker pull jgkim/gentoo-stage3; Run a container: $ docker run –rm -i -t jgkim/gentoo-stage3; # gcc --version; gcc 4.8.4 good supports “--arch=native”
- 26. Building a perl container Github has templates: http://github.com/Perl/docker-perl Dockerfiles like “5.020.000-64bit/Dockerfile”. git acquires “5.020.0-64bit-optimized” directory.
- 27. FROM buildpack-deps # parent container RUN apt-get update && apt-get install -y curl procps # commands to pull perl RUN mkdir /usr/src/perl WORKDIR /usr/src/perl # build dir within the container RUN curl -SL http://www.cpan.org/src/5.0/perl-5.20.0.tar.gz | tar -xz --strip-components=1 RUN ./Configure -Duse64bitall -des && make -j$(nproc) && TEST_JOBS=$(nproc) make test_harness && make install && make veryclean WORKDIR /root CMD ["perl5.20.0","-de0"] # /bin/sh perl5.20.0 -de0 Distro's Dockerfile
- 28. Check the local arguments $ perl -V; ... config_args='-de -Dprefix=/opt/perl/5.20 -Doptimize=-O3 -march=native -pipe' perl -MConfig -E 'say $Config{ config_args }'
- 29. New Dockerfile FROM jgkim/gentoo-stage3 MAINTAINER Steven Lembark <[email protected]> WORKDIR /var/tmp/ RUN wget -O – http://www.cpan.org/src/5.0/perl-5.20.2.tar.gz | gzip -dc tar | tar xf -; RUN cd perl-5.20.2 && Configure -de -Dprefix=/opt/perl -Dman1dir=none -Dman3dir=none -Doptimize='-O3 -march=native -pipe' ; RUN make -C perl-5.20.2 all test install distclean; RUN /opt/perl/bin/h2ph -r -a -l; CMD [ "/opt/perl/bin/perl", "-d", "-E", "42" ]
- 30. Building Perl The build takes input and optional repository tag. Input is a directory not “Dockerfile” $ cd /scratch/docker/gentoo+perl; $ docker build –tag='lembark/perl-gentoo' . ; Each step in the Dockerfile is an intermediate image.
- 31. Checking local containers Intermediate images labeled with “<none>”. These were from prior tests. This one didn't finish: $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE <none> <none> 5906d8edbb59 10 minutes ago 726.1 MB <none> <none> 10f5517fcada 23 minutes ago 726.1 MB <none> <none> 7d328e761704 26 minutes ago 725.5 MB <none> <none> 88f1e41aaed5 27 minutes ago 725.5 MB <none> <none> 28052ace7e04 28 minutes ago 725.5 MB <none> <none> 6f46836220d9 31 minutes ago 725.5 MB
- 32. $ docker build --rm=false –tag=lembark/gentoo-perl .; Step 0 : FROM rndevfx/gentoo-stage3-amd64-nomultilib ---> e9d0ce66148c Step 1 : MAINTAINER Steven Lembark ---> Using cache <-- recycled image ---> 41d5480e49e7 Step 2 : WORKDIR /var/tmp ---> Using cache ---> 010d3d70ced1 ... Step 7 : RUN make all test install; <-- first execution is here. ---> Running in 949c9ddfc2ef Step 8 : RUN /opt/perl/bin/h2ph -r -a -l; ---> Running in b084569e8fc3 -r and -a options are mutually exclusive INFO[1342] The command [/bin/sh -c /opt/perl/bin/h2ph -r -a -l;] returned a non-zero code oops...
- 33. Easy fix RUN /opt/perl/bin/h2ph -r -l /usr/inlcude;
- 34. True Lazyness More re-cycled images: ---> Using cache ---> cb4c5cd0607e Step 8 : RUN /opt/perl/bin/h2ph -r -l /usr/include; ---> Running in 92efee8c31e2 require '_h2ph_pre.ph'; ...
- 35. And impatience $ time docker build --rm=false --tag=lembark/perl-gentoo . Sending build context to Docker daemon 4.096 kB Sending build context to Docker daemon Step 0 : FROM rndevfx/gentoo-stage3-amd64-nomultilib ... Step 10 : CMD [ “/opt/perl/bin/perl” ] ---> Using cache ---> f7b83ecbe276 Successfully built f7b83ecbe276 real 0m0.241s user 0m0.019s sys 0m0.012
- 36. Docker Images: REPOSITORY VIRTUAL SIZE lemark/gentoo-perl 884.4 MB jgkim/gentoo-stage3 741.2 MB localhost:5000/lembark/busybox 1.9 MB Successful build: tagged image
- 37. Welcome to perl “CMD” gets run with an interactive container: $ docker run –rm -i -t lembark/gentoo-perl; Loading DB routines from perl5db.pl version 1.44 Editor support available. ... main::(-e:1): 0 DB<1> x $^V v5.20.2
- 38. Choose your toppings Default for gentoo run /bin/bash <your command>. Save typing with: ENTRYPOINT [ “/opt/perl/bin/perl” ] CMD [ "-d", "-E", "42" ] Use –entrypoint='/bin/bash' if perl fails.
- 39. The next course Stacked images inherit the ENTRYPOINT: FROM lembark/gentoo-perl CMD [ "/path/to/your/program" ] runs /opt/perl/bin/perl /path/to/your/program;
- 40. Test containers stack Derive tests from the package. Add ./t to another image. WORKDIR [ “/path/to/your/code” ] CMD [ "/opt/perl/bin/prove" ] Result: no tests in product image: docker run foo/bar; run application. docker run foo/bar-test; run base tests.
- 41. Minimizing virtual size Cannot remove inter-RUN space. 884.4 MB includes 100MB of /var/tmp/perl-5.20.2. Avoiding it requires a single RUN. No caching of intermediate steps. Final size 787.8 MB. Best for final construction.
- 42. Minimal Virtual Size: single "RUN" command FROM jgkim/gentoo-stage3 MAINTAINER Steven Lembark <[email protected]> WORKDIR /var/tmp/ # better yet, put this in a shell script and RUN ./build-perl! RUN wget -O – http://www.cpan.org/src/5.0/perl-5.20.2.tar.gz | gzip -dc tar | tar xf - && cd perl-5.20.2 && Configure -de -Dprefix=/opt/perl -Dman1dir=none -Dman3dir=none -Doptimize='-O3 -march=native -pipe' && make all test install distclean && cd .. && rm -rf 5.20.2 ; RUN /opt/perl/bin/h2ph -r -a -l; ENTRYPOINT [ "/opt/perl/bin/perl" ] CMD [ "-d", "-E", "42" ]
- 43. In most cases there is still a better way. Note: You already have a local O/S. Q: Why add another one to the container? A: Because we are all used to virtual machines. Time to reduce the calories...
- 44. Copy perl on top of busybox Build & install into /opt/perl. /opt/perl/Dockerfile FROM lembark/busybox_x86 COPY [ “.”, “/opt/perl” ] ENTRYPOINT [ “/opt/perl/bin/perl” ] CMD [ , “-d”, “-E”, “0” ] Check build: docker run –entrypoint='/bin/sh'
- 45. Nice combination Useful shell tools for "qx". /bin/sh in case of broken build. Smaller package: REPOSITORY VIRTUAL SIZE lembark/busybox-perl 67.5 MB lembark/gentoo-perl 787.8 MB jgkim/gentoo-stage3 741.2 MB localhost:5000/lembark/busybox 1.9 MB
- 46. Local dir with modules /opt/perl/5.20/Dockerfile for bare copy of perl: $ docker build –tag='lembark/perl-5.20' /opt/perl/5.20; FROM /lembark/busybox MAINTAINER Steven Lembark <[email protected]> COPY [ ".", "/opt/perl/5.20" ] WORKDIR /var/tmp ENTRYPOINT [ "/opt/perl/5.20/bin/perl" ] CMD [ "-d", "-E", "0" ]
- 47. Portion control for perl What if you don't want CPAN::Reporter in Docker? A: Keep a “docker-perl” install. Extract stage3 into /scratch/docker-build. Build perl “chroot /scratch/docker-build”. Single /opt/perl with vetted modules. Works for just about anything.
- 48. Nothing artificial, nothing added Skip the Dockerfile: import a tarball. "—change" inserts Dockerfile syntax into image. cd /opt/perl; find . | cpio -ov -Htar | docker import –change=”VOLUME /lib64” --tag=”lembark/perl-5.20.2”; Minimal base for Plack server.
- 49. Essential ingredients Catch, this won't run as-is: since perl needs shared libs. In my case, from /lib64. Q: Where to get them without O/S image?
- 50. One way: Share libs Run containers with read-only mount of /lib64: docker run -v /lib64:/lib64:r … Light-weight. Fast: No images to ship. Requires homogeneous lib's for distributed use. Or "-v /var/tmp/$$:/var/tmp"
- 51. One way: bundle libs “ldd” lists shared libs: COPY them into the image with perl. Build lib64 image from perl – order doesn't matter. linux-vdso.so.1 (0x00007ffdfbbcb000) libperl.so => /opt/perl/5.20/lib/5.20.2/x86_64-linux/CORE/libperl.so (0x00007f40f8868000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f40f8650000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f40f844c000) libm.so.6 => /lib64/libm.so.6 (0x00007f40f8153000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f40f7f1c000) libutil.so.1 => /lib64/libutil.so.1 (0x00007f40f7d19000) libc.so.6 => /lib64/libc.so.6 (0x00007f40f7981000) /lib64/ld-linux-x86-64.so.2 (0x00007f40f8c2b000)
- 52. Zero protein pill Use "-v" to add /lib4 and /opt/perl. Image: Single #! script to start the application. Images are a few KB. Requires homogenous install of perl, application.
- 53. One way: static perl Build perl & modules “--static”. No hetergenious server issues. perl image is larger. Best for tests: no issues with underlying images.
- 54. Result: portable, optimized, minimal perl. Static perl with busybox: 68 MB. Whole lot less than Ubuntu. Not a virtual machine. Plack web server FROM this perl. Viola!, you're up.
- 55. What did all of this get us? Mainly an optimized, current perl. With all [and only] the modules you need. You also save the overhead of shipping an O/S image. Faster startup. Easier updates. Simpler deployment.
- 56. Summary Docker makes lxc approachable. You don't always need a full O/S distro in the container. perl on busybox makes a tasty, low-calorie alternative. Use “-v” to import /lib64 for a full meal. Even testing gets simpler: derive tests from package. Say goodby to brewing perl, managing multiple versions.