Quality on Submit
Download as PPTX, PDF•0 likes•1,983 views
The document describes Quality on Submit (QOS), an end-to-end software development quality process that provides instant feedback through continuous integration. QOS integrates code checking, automated building, testing, static analysis, security scanning, and data collection into a single workflow. When changes are committed to the source code management system, builds are deployed and all automated tests, analyses, and scans run in parallel. Results are stored in a database and developers are notified, with optional gamification to increase quality awareness. The process aims to find issues early at low cost while increasing transparency. SAP has implemented QOS successfully in its products.
Quality on Submit
- 1. Quality on Submit Continuous integration in practice Nir Koren Development Integration Lead, SAP Labs Israel June 2012 SAP Patent pending application title "Quality on Submit Process”. Ref: 2010P00489US
- 2. Quality On Submit End to end SW development quality process that provides instant feedback and transparency to the SCRUM teams. © 2012 SAP AG. All rights reserved. 2
- 3. Motivation THE COST OF FIXING A BUG 16,000 $ 1,000 $ 25 $ Coding Unit Test QA Testing Field Test Post release Applied Software Measurement, Capers Jones 1996 © 2012 SAP AG. All rights reserved. 3
- 4. Motivation – Cont’ The nightly build was broken no systems for today Who broke the build? Who broke the runtime? How many changes do we have today? Did Jack already check-in his stuff? What is the status of our product today? Now? Why do we fix PMD issues right before the release © 2012 SAP AG. All rights reserved. 4
- 5. Code checked-out into SCM Build Test SCM Server Integrate Developer CI Server Code Change Listen Start automated chain Deploy Scan Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 5
- 6. Build Jenkins SCM Plugin (P4, GIT, etc’) Listen to the SCM and trigger the QOS process. Compilation starts (Maven / Ant / etc’) SCM Server Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 6
- 7. Deploy A dedicated reference system is deployed with the new build results. This system is always updated to the Continuously latest code state. Deployed REF System Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 7
- 8. Unit Tests All Unit tests are detached from the build process. Supports any kind of unit tests like NUnit, JUnit, etc. Reports provided via Jenkins plugins like surefire Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 8
- 9. API Tests Run on the deployed system. Supports various methods like JMeter, soapUI, etc. Reports provided via Jenkins plugins Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 9
- 10. Java Script Tests Support JSLint for JS Static Code Analysis and JsUnit tests like Jasmine. Reports provided via Jenkins plugins Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 10
- 11. Sonar Open source static code analysis platform Rich HTML dashboards Numerous plug-ins Violations, components, hotspots, time machine and a lot more… Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 11
- 12. Fortify Licensed security test tool by HP. Scan the current static code and detects security violations. Provide rich HTML dashboards Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 12
- 13. UI Automation Set of Integration tests executed Supports various technologies like HP QTP, Selenium, Sikuli and more. Reports provided via Jenkins plugins or by any external provider. Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 13
- 14. Data saved into a DB All data is saved into a DB. QOS Cycles statistics can be generated. Developers’ typical behavior can be viewed from this DB. Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 14
- 15. Email Notification Notification sent to the implementation team Transparency to the implementation team. Elaborated status and links for all phases. Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 15
- 16. Gamification Optional connection to a Gamification system which calculate phases results and provide ranks and badges. Increase the fun in development. Increase the drive for quality. Unit API JS DB Build Deploy Sonar Fortify UI Notify Gamify Test Tests Tests Save © 2012 SAP AG. All rights reserved. 16
- 17. Product Team Transparency Daily report sent to the product team Transparency © 2012 SAP AG. All rights reserved. 17
- 18. QOS End To End Quality Instant feedback in ~15 minutes. Increased team responsibility and quality awareness Rewards development for quality Makes TDD realistic and drives towards it © 2012 SAP AG. All rights reserved. 18
- 19. Prerequisites Build Test Split your unit tests from your build. Integrate Split your build and make it short. Make your tests in short cycles. Deploy Scan Make you tests reliable Avoid false alarms © 2012 SAP AG. All rights reserved. 19
- 20. Quality On Submit in SAP • SAPPatent pending application title "Quality on Submit Process”. Ref: 2010P00489US • Run in productive on-demand and on-premise products • Proved as a best method for quality in short cycles • Awarded SAP Labs Israel as one of the lab’s strengths © 2012 SAP AG. All rights reserved. 20
- 21. Thank you! Nir Koren Development Integration Lead SAP Labs Israel | [email protected] © 2012 SAP AG. All rights reserved. 21