Continuous Packaging is also Mandatory for DevOps
3 likes1,027 views
While DevOps are comfortable with continuous integration and automatic tests, the area of continuous packaging has not been given the attention it deserves. Even with containers, delivering an application using software packages provides multiple advantages with regards to file-based installation: it allows to manage dependencies more easily, to provide metadata, checksum, and signature mechanisms, to deal with packages repositories. But doing that in a continuous packaging approach means that the generation of these packages is fully automated and part of the build process of the software. As a consequence, it eases the various steps of a solution lifecycle (controlled impact of installation/uninstallation, identical deliveries up to the customer, avoidance of code or metadata duplication) This presentation will detail the methodological approach around continuous packaging and demonstrate how this can be put in place using an Open Source tool such as project-builder.org and how this allows the MondoRescue project to deliver packages at will for lots of distribution tuples through the same number of Docker containers.
Ad
More Related Content
What's hot (20)
Viewers also liked (18)
Ad
Similar to Continuous Packaging is also Mandatory for DevOps (20)
Ad
More from Docker, Inc. (20)
Recently uploaded (20)
Continuous Packaging is also Mandatory for DevOps
- 1. Continuous Packaging is also Mandatory for DevOps Bruno Cornec HPE Open Source & Linux Strategist
- 2. Software engineering and Unices since 1988: ● Mostly Configuration Management Systems (CMS), Build systems, quality tools, on multiple commercial Unix systems ● Discovered Open Source & Linux (OSL) & made first contributions in 1993 ● Full time on OSL since 1995, first as HPE reseller then @HPE Currently: ● HPE OSL Technology Strategist, OSL Advocate and Converged Infrastructure Ambassador, WW Linux Community Lead for the HPE Open Source Profession ● POSS conference, OpenStack.fr and AFUL board member. Conferences at WW level at LinuxCon, Linux.conf.au, Fosdem, RMLL, ... ● MondoRescue, Project-Builder.org, UUWL, PUSK & python-redfish Project Lead ● LinuxCOE, mrepo, tellico, rinse, FOSSology, collectl, Ironic contributor ● FOSSBazaar/SPDX and OSL Governance enthusiast ● Mandriva, Mageia, Fedora packager And also: ● Amateur singer (Alto / Tenor), recorder player since 1976 and Choir director since 1987, CD collector (7000+), Concerts, Photography Introducing Myself
- 3. The DevOps Continuous Delivery pipeline Infrastructure as code Continuous Delivery Pipeline Version controlled Peer reviewed Autom ated tests (lots) Gates (autom ated / m anual) Autom ated deploym ent Feedback loop (m onitoring) Codifed O peration Readiness Zuul Nodepool Continuous Packaging Change in • Executable code • Confguration • Infra / environment • Data • Monitoring • … Change in • Executable code • Confguration • Infra / environment • Data • Monitoring • … Project-Builder.org
- 5. ● Startup scripts ● Specific tools ● Functional updates ● Security updates ● Community driven or Commercial (HW certification, LTS, support) Linux distribution: A project by itself ● Coherent packages set (1k-30k) taken from upstream projects ● Package Manager ● Management tools ● Installation program
- 6. RPM/deb format advantages ● Binary and source formats available w/ multiarch support ● Native support for LSB/FHS ● Provides metadata, build procedure, patches and upstream content ● Manages installation with dependencies, upgrade, removal ● Signature/Checksum support and verification ● Deployment server availability – Fully automated ● Baseline support ● RPM Package database available to query metadata ● Provides stability and coherency from the distribution Linux packages at Docker's age ? It still make sense !
- 8. “Make upstream projects life easier with regards to packaging their software” Project-Builder.org goal
- 9. “Make my life easier with regards to packaging my software” Project-Builder.org itch
- 10. ● Packaging should be a project concern as well as coding, testing, installing, .... especially for smaller projects ● Packaging as your only way of delivery (not a dream) Minimal overhead, maximum benefit: ● Consistancy and reproduceability for devs and users ● Distribution & deployment server integration, ● Consistency with distribution and avoids dependecy hell for consumers ● Packaging as a marketing activity for the upstream project. Easy way to extend your user base, and improve your community relationship and is a “competitive advantage”. ● New mantra: “Package early, package always” ● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (whatever the tool) Benefts from Continuous Packaging From Version Control System (VCS) to Packages
- 11. ● VCS agnostic: no VCS but guys it's 21st century now, SVN, CVS, Mercurial, GIT and GIT/SVN, SVK.... ● OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 150+ distro tuples made and counting – repositories for yum, urpmi, apt. Solaris pkg. ● Build environment agnostic: local, VM (QEMU, KVM...), VE (Docker, chroot, rpmbootstrap, rinse, mock, debootstrap...), RM (build farm) ● No project impact: preserves the md5sum of the delivered upstream sources. Can be completely external to the upstream project. ● Avoids duplication of code and metadata ● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (with project- builder.org !) Project-Builder.org goals Support the Continuous Packaging vision by being agnostic
- 12. Continuous Packaging Architecture Project Build + metadata VM Build Local build RM Build Farm (may host VMs) Remote build Project Repository Local Build ServerPackagers Developers VE Build
- 13. Demonstration !
- 14. ● Easy creation of VE (Docker containers from distribution images) and VM. ● Macros system with perl variables to avoid duplication of metadata ● Repository management (apt, yum, urpmi) included ● Check validity of packages (lintian, rpmlint) ● Template creation for new projects, new versions management Project-Builder.org goodies Batteries included:-)
- 15. Docker images management Minimal OS Image Image Proj1 deps Image + pb tools Image Proj2 deps Proj1 container Proj2 container Phase 0 (tag: d-v-a) - newve • Reuse Image from Docker Trusted or Local Registry Phase 1 (tag: d-v-a-pb) - (sbx2|)setupve • adds pb tools and build account Phase 2 (tag: d-v-a-pb-proj) - prepve • adds project build dependencies Phase 3 (container) - build2ve • Build project in the Docker container
- 16. Web sites: ● http://www.project-builder.org ● http://trac.project-builder.org Projects using project-builder.org: ● http://www.project-builder.org (of course :-) ● http://www.mondorescue.org ● http://www.fossology.org ● http://www.linuxcoe.org Learn with: ● Examples from http://trac.project.org/browser/projects/ ● Articles from http://brunocornec.wordpress.com/tag/proj ect-builder.org Web resources Project-builder.org Related tools Project-Builder.org is mostly suited for upstream projects wanting to package their applications Distributions provide each their build tools: • SuSE: Open Build Service (Multi distro, Web based, BaaS) • Fedora (Koji) • Mageia (Youri, mgarepo...) Other complementary tools: • Buildbot • GitLab
- 17. “Changes are never easy to make. There is comfort and safety in tradition, but change must come, no matter how painful or expensive it may be.” Bill Hewlett
- 18. Thank You !Linus Torvalds, Richard Stallman, Eric Raymond, Nat Makarevitch, René Cougnenc, Eric Dumas, Rémy Card, Bdale Garbee, Bryan Gartner, Craig Lamparter, Lee Mayes, Gallig Renaud, Andree Leidenfrost, Phil Robb, Bob Gobeille, Martin Michlmayr among others, for their work and devotion to the Open Source Software cause... and my family for their patience :-) [email protected] Open Source and Linux Technology Strategist http://downloads.linux.hpe.com