Leveraging the GitHub Ecosystem for Python Projects: From Hello World to Docker Deployment
- 1. Leveraging GitHub Ecosystem for Python Projects: From Hello World to Docker Deployment Eoin Halpin, Tom Halpin 16/11/2024
- 2. Agenda & Presenters 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 2
- 3. Agenda Presenters Public Service Announcement DevOps and DevSecOps GitHub Support for DevSecOps Git Workflows and Actions Conclusions Q&A 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 3
- 4. Presenters Tom Halpin: DevSecOps Enablement. Help teams move to a DevOps model in support of product-aligned value streams. Facilitate adaption of the associated culture, practices, and tools in organizations. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 4 Eoin Halpin: Data Analyst - Project Management Member of Agile, customer-facing teams focused on delivering value to stakeholders. Help organizations and customers to gain valuable insights from data.
- 5. Public Service Announcement 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 5
- 6. Public Service Announcement This talk is not a Python Development talk. This talk is a Python DevSecOps talk which will cover using the GitHub eco- system to implement DevSecOps for a Python Application. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 6
- 7. DevOps 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 7
- 8. DevOps - Definition DevOps is a cultural movement that emphasizes collaboration between software development (Dev) and IT operations (Ops) teams. The goal being to • Shorten the software development lifecycle • Improve deployment frequency • Ensure high-quality software delivery through • Automation • Continuous Integration • Continuous delivery / Continuous Deployment • Rapid Feedback Loops 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 8
- 9. DevOps – The Three Ways The three ways form a framework for achieving high performance and continuous improvement within organizations adopting DevOps. • The First Way - Principles of Flow: focuses on optimizing the flow of work from development to operations. This includes continuous delivery and integration, ensuring that code changes are released quickly and reliably. The aim is to minimize work in progress and reduce bottlenecks. • The Second Way - Principles of Feedback: emphasizes the importance of feedback loops at all stages of the development process. This includes automated testing, monitoring, and alerting to gain insights into system performance and user experience. The goal is to learn from failures and successes to improve the system continuously. • The Third Way - Principles of Continual Learning and Experimentation: encourages a culture of innovation, where teams are empowered to experiment and learn from failures. This involves creating a safe environment for risk-taking, allowing for the rapid iteration of processes and practices. The focus is on fostering collaboration, knowledge sharing, and adaptation. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 9
- 10. DevOps – Infinity Loop 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 0
- 11. DevSecOps - Definition DevSecOps - Is the practice of integrating security throughout the CI/CD process. It grew out of the DevOps movement and builds on the same foundational frameworks. - Focuses on “shifting security left” into active development instead of addressing it after code has been developed. - The objective being to strengthen security and compliance by addressing security concerns as they arise during the development process. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 1
- 12. DevSecOps – Infinity Loop 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 2
- 13. GitHub – DevSecOps Enablement 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 1 3
- 14. GitHub – Workflows & Actions In the context of DevSecOps CI/CD • GitHub Workflows are automated processes that orchestrate various stages of software development, such as building, testing, and deploying applications. • These workflows utilize GitHub Actions, which are individual tasks that perform specific functions—like running tests, checking code quality, or deploying to production • Together they allow teams using GitHub to automate and streamline their continuous integration and continuous deployment pipelines, ensuring faster, more reliable and more secure software delivery. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 4
- 15. The Application 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 1 5
- 16. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 6 genai-musings - GitHub Organization
- 17. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 7 chatting-with-ChatGPT - GitHub Repository
- 18. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 8 chatting-with-ChatGPT - OpenAI Integration Hello World for the AI Age
- 19. GitHub – Workflows / DevSecOps CI/CD 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 1 9 Plan No specific workflows. Code linter - (code quality) Spellcheck- (documentation quality) md-links - (checks for broken links in documentation) Build docker_build_push Release docker_build_push Deploy docker_push_readme (updating documentation) Operate No specific workflows. Monitor No specific workflows. Test test (unit/integration tests) coverage (code coverage analysis) bandit (SAST for Python code) codeql (SAST for code analysis) safety (SAST for dependency vulnerability checks) trivy (DAST for container image security scanning)
- 20. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 0 chatting-with-ChatGPT – GitHub Workflows
- 21. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 1 chatting-with-ChatGPT - test.yaml CI Test Workflow
- 22. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 2 chatting-with-ChatGPT - safety.yaml CI SAST Workflow
- 23. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 3 chatting-with-ChatGPT - docker-build-push.yaml CD Workflow
- 24. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 4 chatting-with- ChatGPT - docker-build- push.yaml CD Workflow – DAST (Trivy)
- 25. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 5 genai-musings - Template, Repo Template • Site - https://www.cyberdynesystems.ie • API – https://www.cyberdynesystems.ie/dev/api • API Key - https://www.cyberdynesystems.ie/dev/keys • GitHub Repository - https://github.com/genai-musings/template-repo- template • Docker Image - https://hub.docker.com/r/genaimusings/template-repo- template
- 26. GitHub – DevSecOps - Workflow Outputs 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 2 6
- 27. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 7 chatting-with-ChatGPT Actions
- 28. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 8 chatting-with-ChatGPT - spellcheck.yaml CI Spell Check Workflow - Failure
- 29. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 2 9 chatting-with-ChatGPT - test.yaml CI Unit Test Workflow - Failure
- 30. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 3 0 chatting-with-ChatGPT – docker-build-push.yaml CI Workflow – GitHub Action Node Module Depreciation Failure
- 31. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 3 1 chatting-with-ChatGPT - docker-build-push.yaml CD Workflow – DAST - Failure
- 32. 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 3 2 Docker Hub - genaimusings
- 33. Conclusions 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 3 3
- 34. Conclusions 1 6 / 1 1 / 2 0 2 4 P Y C O N I R E L A N D 2 0 2 4 3 4 This session highlighted the benefits of using the GitHub ecosystem for implementing DevSecOps and CI/CD practices for Python applications. Benefits highlighted included: • Streamlined Collaboration – through pull requests and code reviews. • Integrated SAST and DAST Security Practices – via GitHub Actions and Dependabot. • Automated Testing and Deployment – implemented using GitHub Actions speeds up delivery. • Visibility and Monitoring – insights provided into commit history, release management and deployment statuses. • Scalability and Flexibility - the support for an extensive range of integrations and tools tailored for Python applications that can be incorporated in CI/CD pipelines.
- 35. Q&A & Thanks 8 / 0 5 / 2 0 X X P Y C O N I R E L A N D 2 0 2 4 3 5