What is GitOps? How GitOps works? we discuss Key Challanges.
- 2. What is GitOps? GitOps is a standardized workflow for how to deploy, configure, monitor, update and manage infrastructure-as-code. Core idea is having a Git repository that contains declarative descriptions of the infrastructure desired in the production environment and an automated process to make production environment match the described state in the repository.
- 3. How GitOps works? ➢ Two Kind of deployment strategy : Push and Pull based ➢ Push: When someone push changes to the repo, it invokes the pipeline and pushes the changes to the infrastructure ➢ Pull: We instruct the infrastructure to actively look up for the changes made to the repo, after it find out changes made to the repo it pulls the changes and apply it internally.
- 5. Advantages Disadvantages ➢ Faster Deployment - You don’t have to switch tools to deploy your application. ➢ Easily track changes in the infrastructure, due to versioning. ➢ Faster Recovery - End to end workflows handle changes in the app. ➢ A proliferation of repositories. (large number of Repo) ➢ No help for secrets management, as number of repositories grows it is hard to manage secrets.
- 6. GitOps vs DevOps: An Interesting Comparison ➢ A GitOps model is used to manage infrastructure provisioning and software deployment. ➢ Git is the main tool used in GitOps tools. ➢ All GitOps are DevOps. ➢ The focus of DevOps model is Continuous Integration and Continuous Deployment. ➢ CI/CD pipelines are the main tool used. ➢ All DevOps are not GitOps. GitOps DevOps
- 7. Why GitOps?
- 8. ➢ One of the major benefits of using GitOps is its self-healing nature. Since git is the source of truth for the whole system, at any time if someone unintentionally modified the system it will be reverted back to keep sync with git. ➢ Also, rollback is also quite easy in this scenario. In most cases, the master branch reflects the state of the system. If there is any kind of failure in the application and it needs to be rollbacked, it can be simply done by the reverting to the previous git state. ➢ It also solves the issue of audit trail and transparency because everything is done via Pull Request. If something doesn’t work you have a single place to look for i.e. commit history. Why GitOps?
- 9. ➢ Single Source of Truth:- The first and most important principle of GitOps. It states Git is always right. You can understand the whole system just by looking at Git because it has all the ingredients right there. ➢ Everything as a Code:- It states everything should be kept in the form of code be it the application or any other component which application needed. In most cases, infrastructure is also defined in the form of code. For example:- Cloud VMs, Docker containers, Kubernetes Deployments. ➢ CI/CD Automation:- This is the feature where the magic happens. Build, Test and Deploy will happen automatically relying on the truth in the repository. Infrastructure creation and application deployment will be part of this according to the infrastructure and configuration as a code. GitOps Feature set
- 10. Securing GitOps Pipelines in Jenkins When working with GitOps pipelines in Jenkins, security is crucial to protect sensitive data and ensure a secure deployment process.
- 11. Managing Credentials Securely Use Jenkins Credentials Plugin: Store credentials (e.g., SSH keys, API tokens) securely in Jenkins using the built-in Credentials Plugin. Avoid hardcoding secrets in pipeline code or repositories. Vault Integration: Integrate Jenkins with HashiCorp Vault or similar secrets management tools to dynamically fetch credentials during pipeline execution. This minimizes the risk of exposing credentials in logs or code. IAM for Git Access: For access to repositories, use AWS IAM roles or GitHub/GitLab personal access tokens (PATs) to limit access based on least privilege principles.
- 12. Securing the Git Repository Use SSH for Git Access: Instead of HTTP(S), use SSH keys for interacting with Git repositories. SSH keys provide better security compared to usernames and passwords. Use Branch Protection Rules: Set up branch protection rules in Git to ensure only approved changes (via PR reviews, tests, etc.) can be merged into production branches. Limit Repository Access: Use fine-grained access controls to restrict repository access to only those who need it. Implement multi-factor authentication (MFA) for Git repository access.
- 13. Pipeline Security Run Pipelines in Isolated Environments: Ensure Jenkins pipelines run in isolated environments (e.g., Docker containers, Kubernetes pods) to reduce the impact of security breaches. Limit Pipeline Permissions: Use Jenkins Role-Based Access Control (RBAC) to grant different levels of permissions based on user roles (e.g., Developer, Tester, Admin). Minimize permissions to only what is necessary. Use Multi-stage Pipelines: Implement multi-stage pipelines with clear boundaries, ensuring that sensitive steps (e.g., secrets injection) are isolated and cannot be accessed by unauthorized stages.
- 14. Monitoring and Auditing Enable Jenkins Audit Logs: Enable and configure audit logging in Jenkins to track access and changes to critical resources (e.g., jobs, credentials). Tools like Audit Trail Plugin can help you keep track of pipeline executions, user actions, and changes. Track Changes in Git Repositories: Use Git hooks or CI tools to enforce checks on repository changes, ensuring that malicious changes are detected early. Implement automated security scans as part of the pipeline to catch vulnerabilities before deployment.
- 15. Tools for Auditing and Compliance in GitOps Workflows
- 16. Jenkins Audit Plugin ● The Audit Trail Plugin records all actions performed in Jenkins, such as job starts, user actions, and changes to configuration, providing detailed logs for compliance and troubleshooting. SonarQube ● SonarQube can be integrated into Jenkins pipelines for continuous static code analysis. It helps to identify vulnerabilities and enforce security policies to ensure the code meets security and compliance standards.
- 17. GitHub/GitLab Security Features ● Both GitHub and GitLab offer built-in security features such as branch protection, required reviews, and security scanning tools (e.g., Dependabot for dependency vulnerability alerts). Aqua Security / Twistlock ● Tools like Aqua Security and Twistlock can be used to audit container images and Kubernetes deployments, scanning for known vulnerabilities before and after the pipeline execution.
- 18. OWASP Dependency-Check ● OWASP Dependency-Check is a software composition analysis tool that identifies known vulnerabilities in project dependencies. It can be integrated into GitOps pipelines for vulnerability scanning as part of continuous integration. Kubernetes Auditing ● If your pipeline is managing Kubernetes resources, enable Kubernetes auditing to track every interaction with the Kubernetes API. This can provide crucial information on who changed what and when, improving security visibility.
- 19. Overview of GitOps for Multi-Environment Deployments Managing Multiple Environments: ● Separate branches or directories for Dev, Staging, and Prod. ● Consistent workflows for each environment using GitOps principles.
- 20. Benefits of GitOps for Multi-Environment Management: Consistency: Unified configurations across environments. Automation: Trigger deployments automatically via Git commits. Auditability: Full history of changes through Git versioning.
- 21. GitOps Deployment Workflow 1.Workflow Steps: ● Development (Dev): ○ Developers push changes to the dev branch. ○ Jenkins pipeline triggers build, test, and deploy to the dev environment. Staging: ● After successful testing, changes are merged into the staging branch. ● Jenkins deploys to the staging environment for QA and validation. Production (Prod): ● Approved changes are merged into the prod branch. ● Jenkins pipeline deploys to the production environment.
- 22. 2.Key Automation Tools: ● Jenkins: Automates CI/CD pipelines for deployments. ● Kubernetes Controllers (e.g., ArgoCD): Reconciles desired and actual state. 3.Visual Flow: ● Git (Dev, Staging, Prod) → Jenkins Pipelines → Environments (Dev, Staging, Prod).
- 23. 2. Tag Strategies: ● Use tags to mark specific versions (e.g., v1.0.0) for deployments. ● Tags provide immutable references for consistent and repeatable deployments. 1. Branch Strategies: ● Environment-Specific Branches: ○ dev: Active development and testing. ○ staging: QA and integration testing. ○ prod: Stable, production-ready code. Branch and Tag Strategies for Deployments
- 26. 1: Separate code and configuration repositories Maintaining separate code and configuration repositories in a GitOps workflow is a best practice that optimizes the speed, flexibility, and scalability of your CI/CD processes. It promotes independent workflows for development and operations teams, enhances collaboration, reduces friction, and improves deployment speed. It also provides greater control over environments and configurations while reducing the impact of changes in one area on the other, helping streamline development cycles and better manage complex systems. Best Practices with GitOps
- 27. 2: Use directories instead of branches Simplification: With a single branch and environment-specific directories, you eliminate the need for complex merges or cherry-picking between branches. This simplifies the process of managing changes across environments. Reduced Risk of Errors: By centralizing all configurations within one branch, you minimize the chances of mistakes when applying changes to different environments. Scalability: As your project grows, adding new environments becomes easier—simply add a new directory for the new environment, rather than managing yet another branch. Consistency: Keeping all environments within one branch prevents code drift, ensuring that your application remains consistent across environments, with only the environment-specific configurations differing. using directories instead of branches helps streamline environment management, reduces the complexity of propagating changes, and mitigates the risks of manual errors and code drift, making it a more efficient and scalable approach for managing multiple environments in software development.
- 28. 3: Take a trunk-based approach to development In order to adhere to GitOps principles, the best way to apply branches is within a trunk-based development model. Instead of creating endless branches that have to be individually configured and maintained, the trunk-based model avoids unnecessary complexity by creating branches, merging them, and then deleting them. A single branch serves as the trunk in question, and development plays out on a branch from that trunk. After a pull request and all the appropriate approvals are complete, the branch is merged into the trunk and deleted completely.
- 29. 4: Build security rules and policy management protocols In a GitOps workflow, security and policy management are foundational to maintaining a secure and compliant development and deployment pipeline. Since the trunk-based development model involves all developers working on a single branch, securing that trunk becomes critical to preventing accidental or malicious changes from affecting the entire application. Implementing branch protection rules, regular code reviews, vulnerability scanning, and automated policy enforcement ensures that changes are both safe and compliant with organizational standards. By embedding security throughout the CI/CD pipeline, teams can detect and fix issues early, reducing the risk of security breaches and enabling fast, secure delivery of features.
- 30. 5: Distribute approvals across all impacted teams With a trunk-based development model, the risks of mistakes are more concentrated in that single core branch. That’s why it’s important to ensure that any team or department whose work is impacted by a change is given the chance to approve that change. Some GitOps teams intentionally assign a minimum number of approvals to every proposed merge, but in truth any and every affected team should get the chance to review requests. Then go one step further by implementing role-based access control (RBAC) and assigning specific individuals the responsibility of implementing those fully approved changes.
- 31. Any Questions?