Secret Management with Hashicorp Vault and Consul on Kubernetes

1
Secret Management with
Hashicorp Vault and Consul
on Kubernetes

2
Agenda
 What’s a secret?
 Secret management
 Hashicorp Vault
 Deployment topology
 Demo
 Q&A

3
What is a secret?
 Anything is sensitive information
• Database user, password
• API keys
• SSL keys
• Encryption keys
• Cloud credentials
• Certificates
• etc..

4
Why secret management matters
 Moving fast of software development process and infrastructure
 Micro-services based architecture: service to service communication via
tokens, API keys, certificates
 DevOps processes: multiple environments for development, testing, integration,
and production
 Cloud native application and multi-cloud infrastructure: secrets for storage,
compute, analytics, logging, etc..

5
Traditional secret management (anti-patterns)
 Storing secrets:
• Hard-coding
• Clear text config files
• SCM (git, svn)
• Unencrypted file system (laptop, NFS)
 Distributed secrets:
• Email
• Slack
• Shared folder

6
Modern secret management
 Modern secret management properties:
• Secrets are encrypted at rest
• Centrally managed
• Access control to secrets
• Auditable
• Scalable
• Easy to integrate
• High automation for creating, revoking, rotating secrets
• Supported

7
Hashicorp Vault
 Modern secret management application
 Secure secret storage (in-memory, Consul, file, and more)
 Programmatic application access (automated)
 Operator access (manual)
 Leasing, renewal, revocation
 Dynamic secrets
 Rich ALCs
 Multiple client authentication methods
 Auditing

8
Vault Features
 Secure secret storage:
• Data is encrypted in transit and at rest
• TLS for clients
• No HSM required
• Support many storages: in-memory, Filesystem, Consul, DynamoDB, Etcd,
MySQL, PostgreSQL, S3, Zookeeper, etc..

9
Vault Features (cont.)
 Leasing, renewal, and revocation:
• Every secret has a lease
• Secrets are revoked at the end of the lease unless renewed
• Secret may be revoked early by operators
 Auditing:
• Pluggable audit backends
• Request and response logging
• Secrets hashed in audit log

10
 Dynamic credentials:
• On-the-fly created credentials for each instance of an application or user
• Usually short to medium TTL
• Pluggable backends
o Supported databases: MySQL, MariaDB, PostgreSQL, MSSQL, Oracle,
MongoDB, Cassandra, Influxdb, etc...
o Cloud: AWS, Google Cloud, Azure
o SSH: one time password

11
How does dynamic credential work?

12
 Support multiple auth backends
• Token
• LDAP
• Username & password
• AppRole
• AWS
• Google Cloud
• Kubernetes
• …

13
 Support multiple auth backends
• Token
• LDAP
• Username & password
• AppRole
• AWS
• Google Cloud
• Kubernetes
• …

14
Vault Enterprise Features
Feature Open source Pro Premium
Control Groups X
Multi-factor Authentication X
Read Replicas (performance standby) X
Disaster Recovery (DR replication) X X
Replication (performance replication) X
Replication Filters X
Namespace X X
Auto unseal (AWS, Google cloud, Azure) X X X

15
Vault Enterprise Features (cont.)
Performance standby node
 Handle GET requests of key/value secrets and other requests that do not
change underlying storage

16
Disaster recovery replication (DR replication)
 Replicated content, consistent across Vault
clusters
• Secret configurations
• Access control policies
• Authentication methods
• Audit configuration
• Tokens
• Leases

17
Performance replication
 Allowing Vault to scale relatively horizontally rather than vertically
 Secondaries keep track of their own tokens and leases
 If there is a request that modifiese underlying shared state, the secondary
forwards the request to the primary to be handled; this is transparent to the
client

18
Performance replication vs DR replication
Capability DR Performance
Mirrors the configuration of a primary cluster Yes Yes
Mirrors the configuration of a primary cluster’s
backends (i.e.: auth methods, secrets engines, audit
devices, etc.)
Yes Yes
Mirrors the tokens and leases for applications and
users interacting with the primary cluster
Yes No
Allows the secondary cluster to handle client requests No Yes

19

20
Understanding Seal and Unseal
 Seal state:
• When a Vault server first boots
• When a Vault server is started
• Vault servers do not know how to decrypt data
 Unsealing is a process of constructing the master key (which is constructed by
Shamir’s secret sharing algorithms) to read decryption key to decrypt data
 Unsealed state is remained until one of two thing below happens:
• It is resealed via API
• The server is restarted

21
Deploy Vault on Kubernetes
 Consul cluster is to serve as the
storage backend for Vault since it
supports HA mode
 Only active pod replies to requests.
Standby pods will redirect requests to
the active one
 Leader election is done in storage
backend
 Refer here for details

22
Demo
 Scenario 1: Legacy applications that don’t run on Kubernetes
 Scenario 2: Legacy applications that run on Kubernetes
 Scenario 3: New applications that use static credentials
 Scenario 4: New applications that use dynamic credentials

23
Scenario 1: Legacy applications that don’t run on k8s

24

25
1. Jenkins master triggers a new build
2. Jenkins slave pulls source code from SCM, does necessary steps to
build software packages and push to Artifactory if needed
3. The slave authenticates to Vault and gets required secrets that need for
deployment steps
4. Deployment server gets software packages from Jenkins master or
Artifactory and secrets. Doing necessary steps to deploy application

26
Scenario 2: Legacy applications that run on Kubernetes

27
Scenario 3: New applications that use static credentials

28
Scenario 4: New applications that use dynamic creds

Secret Management with Hashicorp Vault and Consul on Kubernetes

Recommended

More Related Content

What's hot (20)

Similar to Secret Management with Hashicorp Vault and Consul on Kubernetes (20)

More from An Nguyen (17)

Recently uploaded (20)

Secret Management with Hashicorp Vault and Consul on Kubernetes

Editor's Notes