CI / CD / CS - Continuous Security in Kubernetes
2 likes1,245 views
The document discusses continuous integration and delivery (CI/CD) with a focus on integrating security (DevSecOps) into the deployment pipeline, particularly for Kubernetes applications. It outlines critical security measures such as building secure images, conducting vulnerability scans, and implementing strong access controls, while addressing potential risks and security issues in containerized environments. Additionally, it highlights tools and processes for maintaining security within CI/CD workflows, including incident response and forensics.
![kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods"]
verbs: ["get", "watch", "list"]
# This role binding allows "jane" to read pods in the "default" namespace.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: read-pods
namespace: default
subjects:
- kind: User
name: jane
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io](https://image.slidesharecdn.com/cicdcscontinuoussecurityonkubernetes-180626211555/85/CI-CD-CS-Continuous-Security-in-Kubernetes-29-320.jpg)
CI / CD / CS - Continuous Security in Kubernetes
- 1. CI / CD / CS
- 2. About me
- 6. Tools to deploy automatically apps into Kubernetes:
- 9. - New infrastructure, new layers, new risks - But we have seen them before: - DDoS, isolation break-out, injections - Fast pipeline: skip security? - This is an opportunity - More steps: more security onion layers https://sysdig.com/blog/7-docker-security-vulnerabilities/ Are containers secure?.
- 10. How to do security?. - Establish trust boundaries (dev vs prod) - Identify, minimise, and harden attack surfaces - Reduce scope and access - Layer protections and defenses (secure and updates) - Traceability and test
- 11. Continuous Security. - DevOps: agile and faster - Security team: less incidents DevSecOps! - Modify process to bring security, agile - Security as Code - Failure: open an issue vs break the build
- 12. Security pipeline. - Build - Shipment - Run-time
- 13. Security pipeline: Build. - Code analysis - specific vulnerabilities - licensing - style - branch policies: PR, check, merge - Test Driven Security (TDS) - OWASP ZAP Scanning
- 14. Security pipeline: Build. - Container build - Trusted base image - Restrict functionality - Restrict libraries / dependencies - Multi-stage builds - Restrict privileges - root, privileged, host, mounts - Dockerfile: USER
- 15. Security pipeline: Build. - Container scan - At the CI -> registry step - Find known vulnerabilities: inventory - CoreOS Red Hat Clair - Anchore (integrated in Sysdig Secure soon) - Red Hat OpenScap - Vuls.io - Other commercial vendors
- 17. What we are looking for?. - Package lists - Software installed manually (pip, rake, …) - Static binaries - Hashes of known vulnerabilities - Lost credentials - Docker image layers
- 18. Ubuntu: 14.04 Apache: 2.2 Wordpress: 4.6 PHP: 7.0
- 19. Ubuntu: 14.04 Apache: 2.2 Wordpress: 4.6 PHP: 7.0
- 20. Ubuntu: 14.04 Apache: 2.2 Wordpress: 4.6 PHP: 7.0
- 21. How to bring this into the pipeline?. - CI, build the image - Scan: - https://github.com/optiopay/klar - https://gitlab.com/gitlab-org/clair-scanner - https://wiki.jenkins.io/display/JENKINS/Anchore+C ontainer+Image+Scanner+Plugin - https://github.com/jenkinsci/anchore-container-sc anner-plugin - Push into the registry
- 22. Container registries. - Often they already include scanning: - DockerHub - Quay.io - GCR - ECR - ACR - many!
- 23. Security pipeline. - Build - Shipment - Run-time
- 24. Security pipeline: Shipment. - Trust - enable image signing - DOCKER_CONTENT_TRUST=1 - Restrict - registry auth and CI/CD tools - Kubernetes ValidatingAdmissionWebhook - Google Grafeas
- 26. Security pipeline. - Build - Shipment - Run-time
- 27. Security pipeline: Run-time. - Infrastructure security configuration: - Host security - Docker Engine security - Kubernetes security - Docker CIS benchmark: docker-bench - Kubernetes CIS benchmark: kube-bench
- 28. Security pipeline: Kubernetes. - RBAC - namespaces - Subjects: users and serviceAccounts - resources - Role and ClusterRole - verbs: LIST, WATCH, GET, UPDATE, PATCH, DELETE - RoleBindings and ClusterRoleBindings https://sysdig.com/blog/kubernetes-security-rbac-tls/
- 29. kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"] # This role binding allows "jane" to read pods in the "default" namespace. kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io
- 30. Security pipeline: Kubernetes. - Admission controllers: PodSecurityPolicy - Privileged/hostPID/hostIPC/hostNetwork/ hostPorts - runAsUser - volumes/allowedHostPaths/ReadOnlyRootFi lesystem - Capabilities - SELinux/AppArmor/seccomp https://sysdig.com/blog/kubernetes-security-psp-network-policy/
- 31. Security pipeline: Kubernetes. - Admission controllers - DenyEscalatingExec - NodeRestriction - PodSecurityPolicy - ValidatingAdmissionWebhooks
- 32. Security pipeline: Kubernetes. - Resource management resources: requests: memory: 512Mi limits: memory: 700Mi - Network Policies: Cilium, Calico, iptables - Audit system - TLS everywhere https://sysdig.com/blog/kubernetes-security-harden-kube-system/
- 33. Security pipeline: Run-time scanning. - Threat detection - Network inspection - Privilege escalation - Post-mortem analysis and forensics - Attacks are multiple steps - Successful attack those are unknown to us - But we just need to recognize one IoC
- 34. Seccomp. - Application syscall sandboxing - Create filter (BPF) with allowed syscalls - Failures-> log message, error return, and/or kill process - Docker runs containerized process under a seccomp profile - Notable disallowed syscalls: - clone (creating new namespaces) - reboot (reboot the host) - setns (change namespaces)
- 35. MAC: SELinux / AppArmor. - Kernel-level interception/filtering - features++ && complexity++ - Higher level: - Actors (process) - Actions (read/write on files/sockets) - Targets (files, IPs, ports) https://sysdig.com/blog/selinux-seccomp-falco-technical-discussion/
- 36. - An open-source behavioral activity monitor - Detects suspicious activity defined by a set of rules - Uses Sysdig’s flexible and powerful filtering expressions - With full support for containers/orchestration sysdig.com/opensource/falco/
- 38. - macro: proc_is_new condition: proc.duration <= 5000000000 - rule: Read secret file after startup desc: > an attempt to read any secret file (e.g. files containing user/password/authentication information) Processes might read these files at startup, but not afterwards. condition: fd.name startswith /etc/secrets and open_read and not proc_is_new output: > Sensitive file opened for reading after startup (user=%user.name command=%proc.cmdline file=%fd.name) priority: WARNING Falco real rule example.
- 40. Post-mortem and forensics. What? Where? Who? Why? logs? SSH into prod? and start messing around?
- 41. How we did this in the past?.
- 43. System calls for forensics?
- 44. Sysdig Inspect https://github.com/draios/sysdig-inspect (integrated in Sysdig Secure too)
- 45. CI/CD/CS, closing the security gap - Build - Shipment - Run-time - Forensics
- 46. Thank you!