JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild
2 likes•266 views
The document discusses the optimization of CI/CD processes at Ravello Systems, focusing on lessons learned from a successful Jenkins rebuild. Key improvements included utilizing Docker for better resource distribution, implementing a dynamic authentication mechanism for security, and enhancing job management through version control and parameter sharing. Overall, the presentation emphasizes the importance of addressing developer needs and fostering a culture of continuous improvement.
JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild
- 1. Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild Europe, June 2015 Jonathann Zenou @zenoujohn
- 2. Footer #jenkinsconf Greetings fellow humans • Hi, my name is Jonathann • Living in Tel Aviv, Israel • Originally from Marseille, France • Leading the DevOps effort at Ravello Systems • [email protected] 2
- 3. #jenkinsconf Footer What is Ravello Systems ?
- 4. #jenkinsconf Footer Ravello Systems • 30 developers • 4 teams • 7 major projects • Kanban like 4
- 5. #jenkinsconf Footer A part of our stack 5
- 6. #jenkinsconf Footer What are we going to talk about ? 6
- 8. #jenkinsconf Footer Jobs Leftovers • Causes full disk • Workspace Cleanup Plugin • Can pollute your maven repo or npm cache • Solution needed : a new clean environment for every job 8
- 9. Footer #jenkinsconf Better resource distribution 9 Backend Team Frontend Team
- 10. Footer #jenkinsconf Better resource distribution 10 Backend Team Frontend Team
- 11. Footer #jenkinsconf Better resource distribution 11 Backend Team Frontend Team
- 12. Footer #jenkinsconf Better resource distribution 12 Backend Team Frontend Team
- 13. Footer #jenkinsconf Better resource distribution 13 Backend Team Frontend Team “waiting for next available executor …”
- 14. Footer #jenkinsconf Better resource distribution 14 Backend Team/Frontend Team
- 15. Footer #jenkinsconf Better resource distribution 15 Backend Team/Frontend Team
- 16. Footer #jenkinsconf Better resource distribution 16 Backend Team/Frontend Team
- 17. Footer #jenkinsconf Frontend Team leader’s reaction 17
- 18. #jenkinsconf Footer Better resource distribution • Solution needed : a rapid and scalable way to spawn dynamic environments 18
- 20. #jenkinsconf Footer Using Docker in Jenkins • Using Jenkins Docker plugin • Dedicated Docker host for each team • Hosts machine capabilities are a function of team size and requirements • All images on all hosts • Problems solved : jobs leftovers and resource distribution 20
- 21. #jenkinsconf Footer Security • No active directory or LDAP • Jim uses Dave’s account • Employee leaves the company 21
- 24. #jenkinsconf Footer Security • Solution needed: third party dynamic authentication mechanism 24
- 25. #jenkinsconf Footer Security • Using Jenkins Google Login plugin • Coupled with Matrix Authorization Strategy plugin • 2-Factor authentication • Open our Jenkins to the world 25
- 26. #jenkinsconf Footer Too much code in jobs 26
- 27. #jenkinsconf Footer Too much code in jobs • Hard to maintain • Can be easily broken • “Who touched this ??” 27
- 28. #jenkinsconf Footer Too much code in jobs • Solution: • Put your scripts in revision control (Github) • Using Jenkins Multiple SCM plugin 28
- 30. #jenkinsconf Footer Jobs Versioning • Solution: • Using Jenkins JobConfigHistory plugin • Saves old configurations • Diff between job configurations • Revert job configuration • Coupled with Jenkins SCM sync configuration plugin • Saves you whole Jenkins configuration in revision control (Github) 30
- 31. #jenkinsconf Footer Jobs data sharing 31 Backend Job Frontend Job Parameters: - Backend version: 1.1 - Frontend version: stable Parameters: - Frontend version: 1.1 - Backend version: stable
- 32. #jenkinsconf Footer Jobs data sharing • Solution needed: an interface/service that will store data that will be accessible from all the jobs • Using Jenkins Global Variable String Parameter Plugin • Cannot alter them from a job • But you can using Groovy … 32
- 34. #jenkinsconf Footer Jobs data sharing • Keep a global parameter for each of product stable version • Let all the other jobs use these values • Change the global parameter value of a product stable version automatically when this product passes a green CI • Gist link: https://gist.github.com/johnyzed/2af71090419af2b20c5a 34
- 35. #jenkinsconf Footer CI at Ravello Systems 35 build + unit tests environment creation code deployment test suites stop environments Artifactory log collection
- 36. #jenkinsconf Footer CI at Ravello Systems 36 environment creation
- 38. #jenkinsconf Footer The pieces are ready • What Do Developers Want ? • Blackbox • No duplication • C.D.R.S • Single form, with a lot of triggers • Using Multijob Plugin 38
- 39. #jenkinsconf Footer Jobs sandbox • Improving a job without: • Breaking it • Stopping other people using it • Solution needed: a parallel Jenkins environment almost identical to our production Jenkins environment to test the new changes on 39
- 40. #jenkinsconf Footer Jobs sandbox • Jenkins environment blueprint on Ravello Systems • Using Ansible: • git clone • minor changes • Running regular jobs • All that orchestrated from a Jenkins job 40
- 41. #jenkinsconf Footer Jenkins sandbox flow 41 changes on Jenkins script branch create dynamic Jenkins environment run jobs stop dynamic environment
- 43. #jenkinsconf Footer Comparing Jenkins environment 43 Old Jenkins New Jenkins Number of jobs 137 56 Number of slaves 10 0-50 Build and test duration 70 43
- 45. #jenkinsconf Footer Take-aways • Ask your developers how to make their life easier • Don’t be afraid to break things • Culture • Follow @jenkins_release • Follow @devopsreactions 45
- 47. #jenkinsconf Footer One last thing • Ravello Repo • http://www.ravellosystems.com/repo/ • http://www.ravellosystems.com/repo/blueprints/60391436 47
- 49. #jenkinsconf Footer Please Share Your Feedback • Did you find this session valuable? • Please share your thoughts in the Jenkins User Conference Mobile App. • Find the session in the app and click on the feedback area. 49