Implementing Continuous Integration to Improve Software Quality
1 like227 views
The document presents a case study on implementing Continuous Integration (CI) to enhance software quality at Rocket Software, detailing the challenges faced, the business value of CI, and an overview of the CI implementation process. Key goals include automating QA tasks, synchronizing development and QA processes, and improving release predictability. Initial results indicate significant time savings in build cycles and increased efficiency in testing and automation efforts.
Implementing Continuous Integration to Improve Software Quality
- 1. 1 Case study: Implementing Continuous Integration to Improve Software Quality Ben Wedewer, Quality Assurance Manager
- 2. 2 Abstract Continuous Integration (CI) can dramatically improve the effectiveness of the software development process, which in turn can lead to improved product quality. This session provides a case study of the CI implementation for U2 databases and tools. We will cover just what is CI, examine the challenges faced, explore the business value of CI and how it can improve quality, and view a demonstration of CI projects. ©2015 Rocket Software, Inc. All Rights Reserved.
- 3. 3 Agenda Internal analysis: key challenges for QA What is Continuous Integration (CI)? What is the business value of CI? Multivalue CI implementation Demo Initial results Lessons learned © 2015 Rocket Software, Inc. All Rights Reserved.
- 4. 4 Why Are You Here? Understand the challenges faced by Rocket in delivering quality software Demonstrate the successful CI implementation at Rocket Get you thinking about how CI can help in your environment Prior CI experience? © 2015 Rocket Software, Inc. All Rights Reserved.
- 6. 6 Background – Our World Before Continuous Integration Complex source code structure QA and development cycles out of synch Manual testing – lots of it • Regression • New feature • Bug fixes Not enough machine resources © 2015 Rocket Software, Inc. All Rights Reserved.
- 7. 7 Concept Content Planning Development Phase QA Phase Release Tasks Content Freeze Code Complete QA Complete General Availability (Late) Content Plan (start) Finished Version Long release cycle Delays Without CI implementation - long release cycle with time gaps between development and QA © 2015 Rocket Software, Inc. All Rights Reserved.
- 8. 8 Key challenges for MultiValue QA Small ratio of QA to Development Many platforms, many branches = multiplied work QA time spent manually triggering builds and fixing build issues Time from Dev check-in to product build can be long • Causes complications in debugging build and product issues Automated test results interpreted manually by an expert Manual regression testing takes the majority of QA time Limited time for QA to deeply test new features © 2015 Rocket Software, Inc. All Rights Reserved.
- 9. 9 Goals • Automation of QA tasks – build, test, reporting, packaging, etc. • Dev and QA work synched more closely • More stability in the code base over time • Early QA feedback on builds before it becomes time-consuming to fix © 2015 Rocket Software, Inc. All Rights Reserved.
- 10. 10 Goals • More predictability in scheduling • Reduced risk of schedule changes • More possible release points • Make project status known to wider audience in real-time © 2015 Rocket Software, Inc. All Rights Reserved.
- 11. 11 Concept Content Plan Development Phase QA & ID Phase Release Tasks Code Freeze QA/ID Complete General Availability Content Plan (start) Finished Version Release Candidates (more cut points) Shorter release cycles With CI implementation, we can shorten the release cycle and have shorter gaps between development and QA © 2015 Rocket Software, Inc. All Rights Reserved.
- 12. 12 What is Continuous Integration (CI)?
- 13. 13 Continuous Integration in a Nutshell Merging all code changes, building, and testing automatically Small pieces of effort, applied frequently CI is NOT simply automated builds • Automated builds = WHAT to do • CI servers = WHEN to do © 2015 Rocket Software, Inc. All Rights Reserved.
- 14. 14 Continuous Integration Steps Regular and more frequent check-ins Check-ins trigger automatic builds on all branches/platforms “Smoke” test triggered after build completes Full automatic tests triggered after passed smoke test Deployment of build to other test servers © 2015 Rocket Software, Inc. All Rights Reserved.
- 15. 15 Continuous Integration Steps © 2015 Rocket Software, Inc. All Rights Reserved.
- 16. 16 CI Prerequisites A (supported) Source Code Management tool Automated build and test suites A well-defined Release Process and workflow – you’re about to accelerate it Curiosity & patience: learn about your CI controller application Choose your CI controller application based on integration CI controller licenses: may be required to use your CI system, determine your needs Lots of hardware… © 2015 Rocket Software, Inc. All Rights Reserved.
- 17. 17 Lots of Hardware Dedicated pool of machines for CI CI Environment must be isolated • All machines are set up consistently and secured away from the “general public” CI requires an investment in hardware © 2015 Rocket Software, Inc. All Rights Reserved.
- 18. 18 Sample CI Machine Plan © 2015 Rocket Software, Inc. All Rights Reserved.
- 19. 19 What’s the Business Value of CI?
- 20. 20 CI Improves Quality Through Efficiency Gains No more pile-up of check-ins • Developers get immediate feedback on their check-ins Builds across all platforms and branches are automatic Immediate notification of build and test errors means fixes can begin right away Testable product is immediately available to QA QA spends time TESTING © 2015 Rocket Software, Inc. All Rights Reserved.
- 21. 21 CI Empowers the Entire Business © 2015 Rocket Software, Inc. All Rights Reserved. Everyone has information about builds in real time Developers are instantly aware if their change broke something Ensures that check-ins across branches are built & tested Reduces stress • Fewer surprises near the end of a release cycle • Roll back easily to a successful build Release/deployment flexibility CI shines a floodlight on remaining inefficiencies
- 22. 22 CI Allows Future Improvement Allows for self-service hotfix builds for Support Allows for developers and QA to run customized test suites on demand Foundation for further test automation • Historically difficult test cases • Nightly performance testing and benchmarking (future goal) © 2015 Rocket Software, Inc. All Rights Reserved.
- 23. 23 CI Won’t Solve Everything CI is not a cure-all Competent developers and testers still required for success • CI won’t write tests for you CI does not organize your source repositories CI simply gives the team their time back © 2015 Rocket Software, Inc. All Rights Reserved.
- 25. 25 Foundation for CI © 2015 Rocket Software, Inc. All Rights Reserved. CI JIRA Hardware AgileSVN Source consolidation
- 26. 26 U2 CI Timeline (so far…) Q3 2014 • Education & Training • Project Plan • System Design: IT • Scheduling/Blackout Q4 2014 • Bamboo Licenses / Test Environment • Environment build out • Smoke Test creation Q1 2015 • QA 70% dedicated • UNIX master templates completed for UV, UD • SB/XA, WebDE, U2 Toolkit and DBTools builds 100% complete Q2 2015 • First live builds for UniVerse Unix releases • Templates cloned to all Unix platforms Q3 & Q4 2015 • Reduction in test differences • Conversion of manual tests to automated • Ongoing improvements to Bamboo infrastructure • Planning Windows implementation • D3 implementation BlackoutPlanning Setup Go-live Maintenance These statements represent Rocket Software’s current intentions. Rocket development plans are subject to change or withdrawal without further notice. Any reliance on these statements is at the relying party’s sole risk and will not create any liability or obligation for Rocket
- 27. 27 U2 CI Implementation Build product Install on build server Deploy build to central product server Run Smoke Test Run full automated test Automatically create JIRA tickets for build and test failures Report results © 2015 Rocket Software, Inc. All Rights Reserved.
- 28. 28 U2 CI Implementation – Plan Diagram © 2015 Rocket Software, Inc. All Rights Reserved.
- 29. 29 Sample CI Machine Plan © 2015 Rocket Software, Inc. All Rights Reserved.
- 30. 30 U2 CI Implementation – Machine Diagram © 2015 Rocket Software, Inc. All Rights Reserved.
- 31. 31 Demo
- 32. 32 Demo - Dashboard © 2015 Rocket Software, Inc. All Rights Reserved.
- 33. 33 Demo – Plan Overview © 2015 Rocket Software, Inc. All Rights Reserved.
- 34. 34 Demo – Plan Details (Repositories) © 2015 Rocket Software, Inc. All Rights Reserved.
- 35. 35 Demo – Plan Details (Triggers) © 2015 Rocket Software, Inc. All Rights Reserved.
- 36. 36 Demo – Plan Details (Variables) © 2015 Rocket Software, Inc. All Rights Reserved.
- 37. 37 Demo – Plan Details (Stages) © 2015 Rocket Software, Inc. All Rights Reserved.
- 38. 38 Demo – Task Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 39. 39 Demo – Plan Activity © 2015 Rocket Software, Inc. All Rights Reserved.
- 40. 40 Demo – Build Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 41. 41 Demo – Build Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 42. 42 Demo – Build Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 43. 43 Demo – Build Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 44. 44 Demo – Build Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 45. 45 Demo – Build Details © 2015 Rocket Software, Inc. All Rights Reserved.
- 46. 46 Demo - Wallboard © 2015 Rocket Software, Inc. All Rights Reserved.
- 47. 47 Demo - Tasks © 2015 Rocket Software, Inc. All Rights Reserved.
- 48. 48 Demo - ATB2 © 2015 Rocket Software, Inc. All Rights Reserved.
- 49. 49 Demo - ATB2 © 2015 Rocket Software, Inc. All Rights Reserved.
- 51. 51 Initial Results It is so easy to build products now! Nightly automatic release builds are performing exactly as they should Able to immediately start more test automation Still working on UD/UV trunk builds to recognize the full benefit © 2015 Rocket Software, Inc. All Rights Reserved.
- 52. 52 Initial Results U2 Toolkit automatic build and test saves 2+ weeks per build cycle • Entire China QA team of 3 now able to work on Python test creation and automation WebDE builds saving 1+ weeks average of manual effort per release SB/XA builds saving 3+ weeks average of manual effort per release © 2015 Rocket Software, Inc. All Rights Reserved.
- 54. 54 Lessons Learned Infrastructure planning and IT buy-in is a necessity Utopia only exists on paper Solidify a CI template for each product before cloning to new platforms or versions Eliminate as many spurious errors and warnings from build and tests as possible © 2015 Rocket Software, Inc. All Rights Reserved.
- 55. 55 Lessons Learned Bamboo is still in its infancy Explore other CI automation products before committing Document everything Double your original estimate Maintenance required after go-live – plan for easy future maintenance © 2015 Rocket Software, Inc. All Rights Reserved.
- 56. 56 Additional Resources Rocket Aldon home http://www.rocketsoftware.com/product-families/rocket-aldon Continuous Integration wiki https://en.wikipedia.org/wiki/Continuous_integration Bamboo home https://www.atlassian.com/software/bamboo Jenkins home https://jenkins-ci.org/ ©2015 Rocket Software, Inc. All Rights Reserved.
- 57. 57 Next Steps Examine your own environment • Do you have inefficiencies? Can a CI implementation improve your development and QA processes? Research the various CI tools available and begin your own implementation ©2015 Rocket Software, Inc. All Rights Reserved.
- 58. 58 Disclaimer THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON ROCKET SOFTWARE’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY ROCKET SOFTWAREWITHOUT NOTICE. ROCKET SOFTWARE SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: • CREATING ANY WARRANTY OR REPRESENTATION FROM ROCKET SOFTWARE(OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR • ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF ROCKET SOFTWARE. ©2015 Rocket Software, Inc. All Rights Reserved.
- 59. 59 Trademarks and Acknowledgements The trademarks and service marks identified in the following list are the exclusive properties of Rocket Software, Inc. and its subsidiaries (collectively, “Rocket Software”). These marks are registered with the U.S. Patent and Trademark Office, and may be registered or pending registration in other countries. Not all trademarks owned by Rocket Software are listed. The absence of a mark from this page neither constitutes a waiver of any intellectual property rights that Rocket Software has established in its marks nor means that Rocket Software is not owner of any such marks. Aldon, CorVu, Dynamic Connect, D3, FlashConnect, Pick, mvBase, MvEnterprise, NetCure, Rocket, SystemBuilder, U2, U2 Web Development Environment, UniData, UniVerse, and wIntegrate Other company, product, and service names mentioned herein may be trademarks or service marks of others. ©2015 Rocket Software, Inc. All Rights Reserved.
- 60. 60