Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Download as PPTX, PDF•4 likes•71,825 views
The document provides an overview of Kubernetes RBAC and Prometheus monitoring features in Kubernetes 1.6, highlighting new alpha features, the role-based access control configuration, and specifics on how to define roles, role bindings, and clusters. It outlines Prometheus as a monitoring system and explains how to deploy it using a Kubernetes operator, including creating service monitors and managing configurations. The document emphasizes the importance of RBAC for security and authorization in Kubernetes clusters and how Prometheus helps in monitoring the performance and health of those clusters.
![8
RBAC – Role
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods"]
verbs: ["get", "watch", "list"]](https://image.slidesharecdn.com/kubernetesrbacprometheusmonitoring-170626103349-170626113534/85/Securing-Monitoring-Your-K8s-Cluster-with-RBAC-and-Prometheus-8-320.jpg)
![10
RBAC – ClusterRole
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
# "namespace" omitted since ClusterRoles are
not namespaced
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]](https://image.slidesharecdn.com/kubernetesrbacprometheusmonitoring-170626103349-170626113534/85/Securing-Monitoring-Your-K8s-Cluster-with-RBAC-and-Prometheus-10-320.jpg)
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
- 1. 1 Securing and monitoring your K8s cluster (RBAC + Prometheus)
- 2. 2 What’s new in 1.6?
- 3. 3 What’s new in 1.6? – Alpha features •Per pod eviction •Pod Injection Policy •Custom Metrics support in the Horizontal Pod Autoscaler
- 4. 4 RBAC in Kubernetes 1.6 RBAC – Role Based Access Control • Allows fine-grained authorization configuration • Configurable via API • Permissions are purely additive, there are no deny permissions • Uses concepts of Role and Role Bindings to create assign and enforce permissions • Permissions can be scoped to: • Namespaces • Nodes • Control plane • Enabled by starting the API server with the flag • --authorization-mode=RBAC
- 5. 5 RBAC - Attributes A request has the following attributes that can be considered for authorization: • user (the user-string which a user was authenticated as). • group (the list of group names the authenticated user is a member of). • “extra” (a map of arbitrary string keys to string values, provided by the authentication layer) • whether the request is for an API resource. • the request path. • allows authorizing access to miscellaneous non-resource endpoints like /api or /healthz (see kubectl).
- 6. 6 RBAC - Attributes • The request verb. • API verbs get, list, create, update, patch, watch, proxy, redirect, delete, and deletecollection are used for resource requests • HTTP verbs get, post, put, and delete are used for non- resource requests • What resource is being accessed (for resource requests only) • What subresource is being accessed (for resource requests only) • The namespace of the object being accessed (for namespaced resource requests only) • The API group being accessed (for resource requests only); an empty string designates the core API Group
- 7. 7 RBAC – API Overview Declares 4 top level types that can be interacted with via the API or kubectl • Role • Cluster Role • RoleBinding • ClusterRoleBinding
- 8. 8 RBAC – Role kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]
- 9. 9 RBAC – RoleBinding to Role # This role binding allows "jane" to read pods in the "default" namespace. kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: read-pods namespace: default subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io
- 10. 10 RBAC – ClusterRole kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: # "namespace" omitted since ClusterRoles are not namespaced name: secret-reader rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "watch", "list"]
- 11. 11 RBAC – RoleBinding to ClusterRole # This role binding allows "dave" to read secrets in the "development" namespace. kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: read-secrets namespace: development # This only grants permissions within the "development" namespace. subjects: - kind: User name: dave apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io
- 12. 12 RBAC – ClusterRoleBinding # This cluster role binding allows anyone in the "manager" group to read secrets in any namespace. kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: read-secrets-global subjects: - kind: Group name: manager apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io
- 13. 13 RBAC – Use Cases • Limit user access to specific namespaces • Limit user permissions relative to their Role • Group users based on their Roles and assign permissions to the group • Pre-assign special hardware nodes to certain user groups and workloads • Segment your continuous deployment pipeline with permissions to deploy in to only specific namespaces.
- 14. 14 Hierarchy of reliability - Monitoring
- 15. 15 Prometheus What is Prometheus? • Monitoring system and time series database • Inspired by BorgMon • Not BorgMon • Opensource – developed at SoundCloud initially • How does it work? • Collects metrics at scale via HTTP • Easy scaling • Can handle thousands of targets, millions of timeseries
- 16. 16 Prometheus - Features • Multi-dimensional data model • Flexible query language to leverage dimensionality • No reliance on distributed storage, autonomous single server nodes • Collection happens via a pull model over HTTP • Push is also supported via an intermediary gateway • Targets can be discovered statically or via service discovery • Good graphing and dashboarding support
- 18. 18 Running Prometheus in Kubernetes • What we need • Pod specification defining how to run Prometheus • Load and manage configuration • Service specification to access Prometheus on stable IP • Options • Write own pod+service+petset+... manifests • Kubernetes Helm chart in the making https://github.com/kubernetes/charts/pull/151 • CoreOS wrote an Operator managing Prometheus and its configuration https://github.com/coreos/kube-prometheus
- 19. 19 Prometheus Operator by CoreOS • What is an Operator? • Builds on basic Kubernetes resource and controller concepts • Included application domain knowledge and best practices • Frees you from deployment and lifecycle management details • Prometheus Operator by CoreOS • Single command install • Configure manage instances using a single declarative configuration • Configuration drives the creation, configuration and management of Prometheus instances
- 20. 20 Prometheus Operator - Features • Create/Destroy • Simple Configuration • Target Services via Labels
- 21. 21 Prometheus Operator – How does it work? The Operator defines 2 Third Party Resources(TPR) • Prometheus Resource • Data Retention • Persistent Volume Claims • Number of replicas • Version • Alert Managers • Service Monitor
- 22. 22 Prometheus Operator – How does it work?
- 23. 23 Prometheus Operator - Deployment Deploy the operator $ kubectl create -f https://coreos.com/operators/prometheus/latest/prometheus -operator.yaml deployment "prometheus-operator" created Create the Prometheus TPR apiVersion: monitoring.coreos.com/v1alpha1 kind: Prometheus metadata: name: prometheus-k8s labels: prometheus: k8s spec: version: v1.3.0 $ kubectl create -f https://coreos.com/operators/prometheus/latest/prometheus -k8s.yaml
- 24. 24 Prometheus Operator – Cluster Monitoring Deploy exporters providing metrics on cluster nodes and Kubernetes business logic $ kubectl create -f https://coreos.com/operators/prometheus/latest/exporters. yaml deployment "kube-state-metrics" created service "kube-state-metrics" created daemonset "node-exporter" created service "node-exporter" created Create the ConfigMap containing the Prometheus configuration $ kubectl apply -f https://coreos.com/operators/prometheus/latest/prometheus -k8s-cm.yaml configmap "prometheus-k8s" configured
- 25. 25 Prometheus Operator – Cluster Monitoring
- 26. 26 Prometheus Operator – Service Monitoring Define how a service should be monitored apiVersion: monitoring.coreos.com/v1alpha1 kind: ServiceMonitor metadata: name: frontend labels: tier: frontend spec: selector: matchLabels: tier: frontend endpoints: - port: web # works for different port numbers as long as the name matches interval: 10s # scrape the endpoint every 10 seconds
- 27. 27 Prometheus Operator – Service Monitoring Create a Prometheus instance that includes this ServiceMonitor apiVersion: monitoring.coreos.com/v1alpha1 kind: Prometheus metadata: name: prometheus-frontend labels: prometheus: frontend spec: version: v1.3.0 # Define that all ServiceMonitor TPRs with the label `tier = frontend` should be included # into the server's configuration. serviceMonitors: - selector: matchLabels: tier: frontend
- 28. 28 Create the Service monitor and Prometheus Objects $ kubectl create -f https://coreos.com/operators/prometheus/latest/servicemon itor-frontend.yaml servicemonitor "frontend" created $ kubectl create -f https://coreos.com/operators/prometheus/latest/prometheus -frontend.yaml prometheus "prometheus-frontend" created service "prometheus-frontend" created Prometheus Operator – Service Monitoring
- 29. 29 Prometheus Operator – Service Monitoring Deploy an application that matches the label tier = frontend $ kubectl create –f ./example-app.yaml
- 30. 30 Thank You