From 4 releases per year to 4 releases per day
0 likes394 views
The document discusses the transition of the European Commission's web development team to a more efficient continuous delivery model using Drupal, aiming to increase release frequency from four times a year to four times a day. It highlights challenges such as bottlenecks and outdated practices, and outlines goals for automation, modern best practices, and the adoption of tools like Git and Composer. Achievements include reduced time to market, improved software quality, and enhanced testing and review processes.
![{
"variables": {
"site_type": "",
"mysql_password": ""
},
"builders": [{
"type": "amazon-ebs",
"source_ami": "ami-********",
"ami_name": "{{user `site_type`}}-dev {{timestamp}}",
"iam_instance_profile" : "acceptance"
}],
"provisioners":[ {
"type": "salt-masterless",
"local_state_tree": "salt"
}]
}](https://image.slidesharecdn.com/from4releasesperyearto4releasesperday-160428175832/85/From-4-releases-per-year-to-4-releases-per-day-31-320.jpg)
![[aws-cf-runstack] 2016-03-17T14:38:03+00:00: AWS::CloudFormation::Stack (CREA
[aws-cf-runstack] 2016-03-17T14:38:25+00:00: AWS::EC2::SecurityGroup (CREATE_
[aws-cf-runstack] 2016-03-17T14:38:25+00:00: AWS::EC2::EIP (CREATE_COMPLETE)
[aws-cf-runstack] 2016-03-17T14:39:02+00:00: AWS::Route53::RecordSetGroup (CR
[aws-cf-runstack] 2016-03-17T14:39:20+00:00: AWS::IAM::Role (CREATE_COMPLETE)
[aws-cf-runstack] 2016-03-17T14:39:24+00:00: AWS::IAM::Policy (CREATE_COMPLET
[aws-cf-runstack] 2016-03-17T14:41:24+00:00: AWS::IAM::InstanceProfile (CREAT
[aws-cf-runstack] 2016-03-17T14:42:20+00:00: AWS::EC2::Instance (CREATE_COMPL
[aws-cf-runstack]
WebServerIP: 52.19.236.43
WebServerURL: http://demo.subsite-starterkit.ec-europa.continuous.lu/](https://image.slidesharecdn.com/from4releasesperyearto4releasesperday-160428175832/85/From-4-releases-per-year-to-4-releases-per-day-33-320.jpg)
From 4 releases per year to 4 releases per day
- 1. FROM 4 RELEASES PER YEAR TO 4 RELEASES PER DAY
- 2. FREDERIC DEWINNE CTO & Co-Founder of continuousphp© 15+ SysAdmin/PHP consultant background PHP Certified Engineer Continuous Delivery/Deployment Evangelist @fdewinne https://www.drupal.org/u/fdewinne
- 3. PIETER FRENSSEN Drupal engineer at ONE Agency Consulting for European Commission Specialized in testing and automation @pfrenssen https://www.drupal.org/u/pfrenssen
- 4. BACKGROUND European Commission Hundreds of websites, many in static HTML Millions of pages, total is unknown Myriad of different technologies Single portal: http://ec.europa.eu
- 9. TARGET AUDIENCE 500 million people 28 countries 24 languages
- 10. WHY DRUPAL? Single technology serving all needs Cost efficient Powerful content management + data modeling Proven track record Flexible Stable, secure Scalable Large community No vendor lock-in Open source
- 11. NEXTEUROPA PLATFORM Drupal 7 distribution Base framework + optional features Multisite installation Drush make Best practices Development started 2012 https://github.com/ec-europa/platform-dev
- 12. RISING CHALLENGES Anno 2014 Growing too fast Distributed teams Outdated practices Bottlenecks Regressions
- 13. GROWING TOO FAST 100 websites, 100 more planned 200+ developers 1000+ content editors 250 modules
- 14. DISTRIBUTED TEAMS Contractors in many countries Working in different ways Different operating systems Infrastructure behind DMZ
- 15. OUTDATED PRACTICES SVN Bash scripts Shared development server Quarterly release cycle
- 16. BOTTLENECKS Manual code review Manual deployment Manual updating every 3 months
- 17. REGRESSIONS High rate of code change No systematic automated testing
- 18. GOALS Reduce bottlenecks by automation Speed up the release cycle Adopt current best practices Universal build system External access Automated testing + QA
- 20. DEFINE A TOOLSET
- 22. MOVING TO GIT https://github.com/ec-europa Move to Github Git Flow Convert from SVN Clean up history Training and support
- 23. DEPENDENCY MANAGEMENT WITH COMPOSER Dependency Manager for PHP Focused on library dependencies Define PHP version and extension requirements https://getcomposer.org
- 24. { "require": { "php": ">=5.4.0", "drupal/coder": "dev-8.x-2.x", "drupal/drupal-extension": "~3.1.5", "phing/phing": "~2.10", "drupal/phing-drush-task": "1.0", "continuousphp/phing-tasks": "~0.1.3" }, "autoload": { "psr-4": { "Drupalnexteuropa": "tests/src", "NextEuropa": "src" } } }
- 25. MANAGING TASKS WITH PHING PHing Is Not GNU make PHP Project build tool based on Apache Ant written in PHP; easily extensible by PHP developers define sequences of tasks organized in targets easily drives Drush using additional tasks https://www.drupal.org/project/phingdrushtask
- 26. <project name="My subsite" default="help"> <target name="help" description="Phing target list"> <exec executable="${phing.bin}" passthru="true"> <arg value="-l"/> </exec> </target> <target name="generate-development-makefile" description="Generate the makefile for development modules." <drushmakefile makeFile="${subsite.temporary.development.make}" coreVersion="${drupal.core.version}" projects="${development.modules.download}" defaultProjectDir="${development.modules.location}" />
- 27. CHECKING CODING RULES WITH PHPCODESNIFFER Custom ruleset Configuration generated with Phing Automated check on push $ ./vendor/bin/phpcs
- 28. AUTOMATING TESTS WITH BEHAT inspired by Cucumber uses Gherkin syntax to define specs supports several web browser drivers through extensions
- 29. Feature: User authentication In order to protect the integrity of the website As a product owner I want to make sure only authenticated users can access the site administration Scenario: Anonymous user can see the user login page Given I am not logged in When I visit "user" Then I should see the text "ECAS Login" And I should see the text "Request new password" And I should see the text "Username" And I should see the text "Password" But I should not see the text "Log out" And I should not see the text "My account"
- 30. PACKING SYSTEM IMAGES WITH PACKER immutable infrastructure approach virtualization abstraction layer versioned configuration
- 31. { "variables": { "site_type": "", "mysql_password": "" }, "builders": [{ "type": "amazon-ebs", "source_ami": "ami-********", "ami_name": "{{user `site_type`}}-dev {{timestamp}}", "iam_instance_profile" : "acceptance" }], "provisioners":[ { "type": "salt-masterless", "local_state_tree": "salt" }] }
- 32. CREATING ON-DEMAND ENVIRONMENT WITH AWS coded infrastructure infrastructure as a dependency new environments deployed in a few minutes pay per use
- 33. [aws-cf-runstack] 2016-03-17T14:38:03+00:00: AWS::CloudFormation::Stack (CREA [aws-cf-runstack] 2016-03-17T14:38:25+00:00: AWS::EC2::SecurityGroup (CREATE_ [aws-cf-runstack] 2016-03-17T14:38:25+00:00: AWS::EC2::EIP (CREATE_COMPLETE) [aws-cf-runstack] 2016-03-17T14:39:02+00:00: AWS::Route53::RecordSetGroup (CR [aws-cf-runstack] 2016-03-17T14:39:20+00:00: AWS::IAM::Role (CREATE_COMPLETE) [aws-cf-runstack] 2016-03-17T14:39:24+00:00: AWS::IAM::Policy (CREATE_COMPLET [aws-cf-runstack] 2016-03-17T14:41:24+00:00: AWS::IAM::InstanceProfile (CREAT [aws-cf-runstack] 2016-03-17T14:42:20+00:00: AWS::EC2::Instance (CREATE_COMPL [aws-cf-runstack] WebServerIP: 52.19.236.43 WebServerURL: http://demo.subsite-starterkit.ec-europa.continuous.lu/
- 34. ORCHESTRATING DELIVERY PIPELINES WITH CONTINUOUSPHP no vendor lock-in not limited by worker model possibility to run tests in parallel specialized in php simplify our delivery workflow
- 36. SKELETON https://github.com/ec-europa/subsite-starterkit Common starting point for all sites Project configuration Build system: Phing Development tools: PHPCS, Composer, Behat Continuous Integration support Good documentation
- 37. ADOPT A PRACTICE
- 38. CONTINUOUS INTEGRATION continuously merging developer branches into a common one part of extreme programming practice build and test every commit to prevent integration issues deploy every build on an integration server
- 39. CONTINUOUS DELIVERY Continuous Integration is a part of CD Code is packaged by a build server every time a change is committed any code commit may be released to customers at any point implements Scrum Project Management
- 40. CONTINUOUS DEPLOYMENT Continuous Delivery is a part of Continuous Deployment every successful build is deployed to a Production Environment any completed, working feature is delivered to production as soon as possible implements Kanban Process Management
- 41. CONTINUOUS COMPARISON Feature Feature Feature Code Build Test Release Deploy Agile Development Continuous Integration Operate Continuous Delivery Continuous Deployment
- 42. BRANCHING MODEL Master Hot Fix Release Develop Feature Feature V0.1.0 V0.1.1 V0.2.0
- 43. ENVIRONMENT MANAGEMENT Master Hot Fix Release Develop Feature Feature V0.1.0 V0.1.1 V0.2.0 Demo Demo Integration Staging Demo Pre-production Production
- 44. ACHIEVEMENTS Drastically decreased the time to market Multiple releases per day Overall so ware quality improved Bottlenecks have decreased Online & inline code review Improved test coverage catches regressions Easy "one-step" building and installing of any site User acceptance testing on ephemeral environments Early detection of common problems
- 45. THANK YOU QUESTIONS? Slides: http://slideshare.net/continuousphp/from-4- releases-per-year-to-4-releases-per-day-61474837 Follow us on Twitter: https://twitter.com/fdewinne https://twitter.com/pfrenssen