A Survey of Container Security in 2016: A Security Update on Container Platforms
Download as PPTX, PDF2 likes506 views
This talk is an update of container security in 2016. It describes the security measures that containers provide, shows how containers provide security measures out of box that are prone to configuration errors when running applications directly on host, and finally lists the ongoing in container security in the community.
![Installing Applications in a Container vs. Directly on Host
• Host configuration and hardening is needed whether
applications run in a container on a host, or directly on host.
This is typically a one-time operation.
• Launching applications in a container on a host [is] a recurring
operation. Container makes it easy to secure host from
applications, making security usable.
HostHost
Container
• Apache webserver is among the most
popular web servers in use today.
• We think that deploying Apache web server in
a container is more secure than deploying
directly on a host.
• Let’s compare Apache installation in a
container on a host versus directly on a host
to better understand our assertion.](https://image.slidesharecdn.com/containersecurity-10252016-update-161229162351/85/A-Survey-of-Container-Security-in-2016-A-Security-Update-on-Container-Platforms-10-320.jpg)
![Running Apache web server on a container vs. host (7/10)
Denial of service Description / mitigation CIS rules
Network Configure Apache for network DoS
mitigation.
Apache CIS
1.9
Memory systemctl MemoryLimit=500M …
Resources (e.g.,
fork)
CPU systemctl CPUShares=…
Disks systemctl IO[Read|Write]BandwidthMax=…
ulimits Set ulimits
Denial of service Description / Mitigation CIS rules
Network Open issue #26767 (WIP)
Memory docker run -m
Resources (e.g.,
fork)
docker run --pids-limit
CPU docker run --cpu-set-cpus, --cpu-shares
Disks docker run --device-[read|write]-[i|b]ps
ulimits Set ulimits for open file descriptors Docker CIS 5.18
HOST
STEP 7:
• Allow resource limits with an easy to use interface
to cgroups and ulimits for ANY container.
CONTAINER (DOCKER) HOST
Agents / daemons
1. SSH
2. Docker
3. Runc
4. Audit
5. Rsyslog
6. Monitoring
STEP 7:
• Use systemd / upstart or manually configure
resource control using cgroups and ulimits using.
Cumbersome.
7. NTP
8. DNS
9. /etc/default/docker
10. Firewalls
11. Patching agent
12. Malware agent
13. AppArmor
Agents / daemons
1. SSH
2. Audit
3. Rsyslog
4. Monitoring
5. NTP
6. DNS
7. /proc/sys/net
8. Firewalls
9. Patching agent
10. Malware agent
11. AppArmor
Goal: Prevent Apache from DoSing host or other containers
Kernel: 4.5+ “cgroups v2”](https://image.slidesharecdn.com/containersecurity-10252016-update-161229162351/85/A-Survey-of-Container-Security-in-2016-A-Security-Update-on-Container-Platforms-17-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to A Survey of Container Security in 2016: A Security Update on Container Platforms (20)
Ad
Recently uploaded (20)
A Survey of Container Security in 2016: A Security Update on Container Platforms
- 1. A Survey of Container Security in 2016: A Security Update on Container Platforms Salman A. Baset, Research Staff Member, @salman_baset Phil Estes, Senior Technical Staff Member, @estesp
- 2. Recap and Background 2015 • Unraveling Docker Security: Lessons from Production Cloud (OpenStack Summit Tokyo, October 2015) 2016 • Container Security (Docker London, June 2016) • How Secure is Your Container? (ContainerCon Berlin, August 2016) Today’s talk: • Container Security: Survey and Overview, and a developer focused comparison of secure application deployment & configuration (OpenStack Summit Barcelona, October 2016)
- 3. Outline • Container Overview • Usage and Threat Models • Container Security Evolution • Developer Focus: Running Applications (Apache) Inside Containers vs Host • Why Containers are Better for Security? • What are the New Security Issues? • Docker vs. LXC vs. Rocket (NCC group report) • Ongoing work in Container Security
- 4. Raise your hand if you think that… • Running applications in hosts (VM / baremetal) vs. running them inside containers on a VM / baremetal is: • more “secure” • no difference • less “secure” • We think that running applications inside a container is more “secure” than running them directly on a VM / baremetal • This is because containers allow deep visibility, and usable and developer-focused security. But first what is a container?
- 5. What makes a container? pid mount IPC user network uts > Linux kernel namespaces provide the isolation (hence “container”) in which we place one or more processes > Linux kernel cgroups (“Control groups”) provide resource limiting and accounting (CPU, memory, I/O bandwidth, etc.)
- 6. Container usage and threats? Single tenant, trusted code Less concern regarding malicious attack vectors Multi-tenant, (organizationally) trusted code ”Noisy neighbor” a higher priority than protection from unknowns Multi-tenant, untrusted code Similar to internet-facing application, but with addition of potential malicious “neighbors” DoS Host (use up CPU, memory, disk), Forkbomb Container, even with isolation, is able to use up host OS resources Access host/private information Container is allowed to access (maybe inadvertently) host details Kernel modification/insert module Given the shared kernel underlying containers; modifying the kernel’s configuration/code/inserting code Administrative access (API socket access) Access to the container “control” socket/system, allowing full admin privileges of host system Exploits in OS libraries/code in container Latent/unpatched exploits in libraries and/or binaries in the container’s filesystem Container provenance Validating/certifying the container image I am running is what I expect THREAT DESCRIPTION
- 7. Kernel and Container Security Evolution • 2015 • Content Addressability (image spec, phase 1) – Docker 1.6 • Default ulimits for all containers – Docker 1.6 • Docker Content Trust – (notary) image provenance/signing – Docker 1.8 • 2016 • Full migration to content addressability for images/layers – Docker 1.10 • User namespaces – Docker 1.10 • Secure computing (libseccomp) – Docker 1.10 • --pids-limit (cgroups pid limitation) – Docker 1.11 (kernel 4.3 +) • cgroups “v2” – (kernel 4.5+) • --no-new-privileges (limit process escalation) – Docker 1.11 • Storage driver quotas (limited) – Docker 1.12 • Secure by default multi-node orchestration (mutual TLS) – Docker 1.12
- 8. Kernel and – save chart until we decide -2015 2016 Kernel – 3.13 Docker – 1.6 Kernel – 4.4 Docker – 1.12 cgroup pid limits
- 9. Configuring for Security: Hard & Tedious Work! • Center for Internet Security (CIS) security “benchmarks”. • Consensus driven approach for secure configuration for host and key applications. • https://www.cisecurity.org/ • Operating Systems • Red Hat 7 CIS benchmark – v2.1.0 – June 2nd, 2016 • 304 pages • Ubuntu 14.04 CIS benchmark – v2.0.0 – Sep 30, 2016 • 296 pages • Apache HTTP Server 2.4 benchmark – v1.3.0 – May 31st, 2016 • 163 pages • Docker 1.12 CIS benchmark – v1.0.0 – August 15, 2016 • 189 pages • OWASP rules for Apache • 60 mod security rules
- 10. Installing Applications in a Container vs. Directly on Host • Host configuration and hardening is needed whether applications run in a container on a host, or directly on host. This is typically a one-time operation. • Launching applications in a container on a host [is] a recurring operation. Container makes it easy to secure host from applications, making security usable. HostHost Container • Apache webserver is among the most popular web servers in use today. • We think that deploying Apache web server in a container is more secure than deploying directly on a host. • Let’s compare Apache installation in a container on a host versus directly on a host to better understand our assertion.
- 11. Running Apache web server on a container vs. host (1/10) Filesystem Size Mounted on CIS rules /dev/disk1 100G / /dev/disk1 10G /tmp Ubuntu CIS 1.1.2 /dev/disk2 100G /var/log Ubuntu CIS 1.1.10 /dev/disk3 100G /var/log/audit Ubuntu CIS 1.1.11 /dev/disk4 100G /mnt/httpd HOST STEP 1: • Configuring host file system • Configure separate partitions for Docker • Install Docker Engine Preparing host list is not an exhaustive list and is meant as an illustration. Filesystem Size Mounted on CIS rules /dev/disk1 100G / /dev/disk1 10G /tmp Ubuntu CIS 1.1.2 /dev/disk2 100G /var/log Ubuntu CIS 1.1.10 /dev/disk3 100G /var/log/audit Ubuntu CIS 1.1.11 /dev/disk4 100G /var/lib/docker Docker CIS 1.1 /dev/disk5 100G /var/dockervols CONTAINER (DOCKER) HOST Agents / daemons SSH Docker Runc Agents / daemons SSH STEP 1: • Configuring host file system • Configure separate partitions for Apache Goal: Configure separate partition
- 12. Running Apache web server on a container vs. host (2/10) Configure Config file Description CIS rules SSH /etc/ssh/sshd_config Configure SSH properly Ubuntu CIS 5.2 User accounts /etc/passwd, /etc/shadow Configure user accounts per strict policy Ubuntu CIS 5.4 Apache user Configure Apache user and run apache under it Apache CIS 1.3 Configure Config file Description CIS rules SSH /etc/ssh/sshd_config Configure SSH properly Ubuntu CIS 5.2 User accounts /etc/passwd, /etc/shadow Configure user accounts per strict policy Ubuntu CIS 5.4 User namespace user Configure Docker engine with user namespace Docker CIS 2.8 Apache Docker Image Prepare Docker image for Apache with Apache user. Apache CIS 1.3 HOST STEP 2: • Manually configure Docker engine with user namespace if non-root user not defined in Docker image. • Use of kernel namespaces is by default. CONTAINER (DOCKER) HOST Agents / daemons SSH Docker Runc Agents / daemons SSH STEP 2: • Configure non-root user for Apache. • Manually configure kernel namespaces. Goal: Run processes as non-root utilizing namespaces. Essential for deprivileging
- 13. Running Apache web server on a container vs. host (3/10) Name Configuration files Description CIS rules Audit daemon /etc/audit/auditd.co nf Record system call activity Ubuntu CIS 4.1 Rsyslog /etc/rsyslog.conf A daemon for sending logs to a central server Ubuntu CIS 4.2 Logging and log rotation Configure log rotation to avoid disk full. Ubuntu CIS 4.3 Apache CIS 1.6 Monitoring Configure monitoring Apache CIS 1.6 Name Configuration files Description CIS rules Audit daemon /etc/audit/auditd.co nf Record system call activity Ubuntu CIS 4.1 Docker CIS 1.7-1.15 Rsyslog /etc/rsyslog.conf A daemon for sending logs to a central server Ubuntu CIS 4.2 Logging and log rotation Configure log rotation to avoid disk full. Ubuntu CIS 4.3 Docker CIS 2.12 Monitoring Configure monitoring of containers Docker CIS 6.2 HOST STEP 3: • Configure audit, log collection, rotation, monitoring • Configure audit for Docker Engine • Collect Docker Engine and container logs CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 3: • Configure audit, log collection, rotation, monitoring • Collect Apache logs Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring Goal: Track system call activity, logs, and monitoring
- 14. Running Apache web server on a container vs. host (4/10) Configure Config file Description CIS rules NTP /etc/ntp.conf Ubuntu CIS 2.2.1 DNS DNS server not enabled. Ubuntu CIS 2.2.8 /proc/sys/net /etc/sysctl.conf /proc/net Network related configs, e.g., net.ipv4.ip_forward=0 Ubuntu CIS 3.1/3.2 Firewalls iptables / security groups Configure host firewalls Ubuntu CIS 3.6 Configure Config file Description CIS rules NTP /etc/ntp.conf Ubuntu CIS 2.2.1 DNS DNS server not enabled. Ubuntu CIS 2.2.8 /proc/sys/net docker /etc/sysctl.conf /proc/net /etc/docker Network related configs, e.g., net.ipv4.ip_forward=1 --sysctl net.ipv4.ip_forward=0 Ubuntu CIS 3.1/3.2 Firewalls iptables / security groups Configure host firewalls Ubuntu CIS 3.6 HOST STEP 4: • Configure network and related services • Disable IP forwarding when starting container engine. CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 4: • Configure network and related services 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls Goal: Configure network services, firewalls, IP forwarding
- 15. Running Apache web server on a container vs. host (5/10) Name Configuration files Description CIS rules System updates /etc/apt/sources.list.d/ * Configure system updates using an agent Ubuntu CIS 1.8 Antivirus / Malware detector A daemon for detecting any malware on host Name Configuration files Description CIS rules System updates /etc/apt/sources.list.d/ * Configure system updates using an agent Ubuntu CIS 1.8 Antivirus / Malware detector A daemon for detecting any malware on host HOST STEP 5: • Configure patching, and malware / antivirus agents on host CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 5: • Configure patching, and malware / antivirus agents on host 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls 11. Patching agent 12. Malware agent Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent Goal: Configure software updates and malware / antivirus
- 16. Running Apache web server on a container vs. host (6/10) Name Configuration files Description CIS rules Configure AppArmor (MAC) Configure Apache AppArmor profile /etc/apparmod.d/* Configure apparmor daemon and all profiles in enforcing Ubuntu CIS 1.6 Apache CIS 1.12 Name Configuration files Description CIS rules Configure AppArmor (MAC) Do not disable AppArmor /etc/apparmod.d/* Configure apparmor daemon and all profiles in enforcing Ubuntu CIS 1.6 Docker CIS 5.1 HOST STEP 6: • Do not disable default Docker AppArmor profile. Applied to running containers. CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 6: • Create an AppArmor profile for Apache. Complicated! 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls 11. Patching agent 12. Malware agent 13. AppArmor Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent 11. AppArmor Goal: Configure mandatory access control for defense in depth
- 17. Running Apache web server on a container vs. host (7/10) Denial of service Description / mitigation CIS rules Network Configure Apache for network DoS mitigation. Apache CIS 1.9 Memory systemctl MemoryLimit=500M … Resources (e.g., fork) CPU systemctl CPUShares=… Disks systemctl IO[Read|Write]BandwidthMax=… ulimits Set ulimits Denial of service Description / Mitigation CIS rules Network Open issue #26767 (WIP) Memory docker run -m Resources (e.g., fork) docker run --pids-limit CPU docker run --cpu-set-cpus, --cpu-shares Disks docker run --device-[read|write]-[i|b]ps ulimits Set ulimits for open file descriptors Docker CIS 5.18 HOST STEP 7: • Allow resource limits with an easy to use interface to cgroups and ulimits for ANY container. CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 7: • Use systemd / upstart or manually configure resource control using cgroups and ulimits using. Cumbersome. 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls 11. Patching agent 12. Malware agent 13. AppArmor Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent 11. AppArmor Goal: Prevent Apache from DoSing host or other containers Kernel: 4.5+ “cgroups v2”
- 18. Running Apache web server on a container vs. host (8/10) Capabilities Description / mitigation CIS rules Various Manual configuration Capabilities Description / Mitigation CIS rules Various Load a module, mount, network admin. Docker starts container with limited capabilities. Docker CIS 5.3 HOST STEP 8: • Launch containers as non-root with limited Linux capabilities. Defaults are good. CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 8: • Manual selection and setting of capabilities is cumbersome. 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls 11. Patching agent 12. Malware agent 13. AppArmor Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent 11. AppArmor Goal: Further deprivilege running process and containers through limited Linux capabilities. Although Apache is launched by a non-root user in container as well as on host, as part of defense in depth, it is good practice to further reducing capabilities available to the user.
- 19. Running Apache web server on a container vs. host (9/10) Descriptiom CIS rules Manual configuration of seccomp Preparing host list is not an exhaustive list and is meant as an illustration. Description CIS rules Do not disable default seccomp profile Docker CIS 5.12 HOST STEP 9: • Launch containers with limited set of allowed Linux system calls. Defaults are good. CONTAINER (DOCKER) HOST Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 9: • Manually configuring undesired system calls is cumbersome. 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls 11. Patching agent 12. Malware agent 13. AppArmor Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent 11. AppArmor Goal: Further deprivilege running containers by limiting Linux system calls (Seccomp). Although Apache is launched by a non-root user in container as well as on host with limited capabilties, as part of defense in depth, it is good practice to further reduce system calls that can be invoked.
- 20. Running Apache web server on a container vs. host (10/10) STEP 10: • Configure Apache in a container. • Configuring Apache firewalls is a breeze. Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring STEP 10: • Configure Apache on host. • Potential cumbersome iptables configuration. 7. NTP 8. DNS 9. /etc/default/docker 10. Firewalls 11. Patching agent 12. Malware agent 13. AppArmor 14. Apache Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent 11. AppArmor 12. Apache Goal: Configure Apache for optimum security. Apache config description CIS rules Minimize Apache modules Apache CIS 1.2 Run Apache as a non-root user even if user name space is used Apache CIS 1.3 Apache Access Control Apache CIS 1.4 Minimize features, content, and options Apache CIS 1.5 Configure TLS Apache CIS 1.7 Configure limits Apache CIS 1.8 HOSTCONTAINER (DOCKER) HOST Not an exhaustive list and is meant as an illustration. Apache config description CIS rules Minimize Apache modules Apache CIS 1.2 Run Apache as a non-root user. Apache CIS 1.3 Apache Access Control Apache CIS 1.4 Minimize features, content, and options Apache CIS 1.5 Configure TLS Apache CIS 1.7 Configure limits Apache CIS 1.8
- 21. Running Apache web server on a container vs. host • A host is isolated from a misbehaving Apache running inside a container. Various agents running on host are likely to remain functional, allowing root-cause analysis later. • Same mechanisms are used to isolate one container from another on the same host. HostHost Container Agents / daemons 1. SSH 2. Docker 3. Runc 4. Audit 5. Rsyslog 6. Monitoring 7. NTP 8. DNS 9. /etc/default/docker /proc is mounted read only for unprivileged containers 10. Firewalls 11. Patching agent 12. Malware agent 13. AppArmor 14. Apache Agents / daemons 1. SSH 2. Audit 3. Rsyslog 4. Monitoring 5. NTP 6. DNS 7. /proc/sys/net 8. Firewalls 9. Patching agent 10. Malware agent 11. AppArmor 12. Apache
- 22. Running Application in a Container vs. Host – Defense in Depth Configurepartitions Runasnon-root, utilizinguserand kernelnamespaces Configurelog, monitoring,audit Configurenetwork, relatedservices, disableIPforwarding Configurepatching andanti-virusagents Configuremandatory accesscontrol Preventapplication fromDoSingahost Furtherdeprivilege applicationthrough subsetofLinux capabilities FurtherDeprivilege applicationbylimiting allowedsystemcalls Configureapplication security Host Container Host Manual Manual Manual per app One time (in Docker) Manual per app Manual One time Step 1 Step 2 Step 3 Manual Manual Step 4 Manual Manual Step 5 Step 6 Manual per app Out of box (in Docker) Out of box (in Docker) Step 7 Manual per app Step 8 Manual per app Out of box (in Docker) Manual per app Out of box (in Docker) Step 9 Step 10 Manual per app Manual
- 23. What are the New Security Issues When Running Applications Inside Containers? • Running containers as non-root • If a Linux user is not setup inside Dockerfile or Appc file to use the application, it is best to enable user name spaces in Docker Engine. • We recommend to always enable user namespaces in Docker Engine. • While using Docker, IP forwarding for containers should be disabled in Docker Engine. • Docker / Rocket API is powerful. Enable with caution.
- 24. Observations from Production • Application security misconfiguration is among the top causes for container compromise. • IBM Vulnerability Advisor can identify vulnerable application packages and application security misconfigurations for containers in IBM Containers platform.
- 25. Docker vs LXC vs rkt • Users/packagers won’t turn on security if it’s difficult (AppArmor profiles are hard to write; SELinux can be even harder) • Sane defaults are tricky as well - someone’s app won’t work and they will complain • Docker painstakingly tries to find a balance (e.g. DCT off by default, allowance for insecure registries) * NCC Group report “Understanding and Hardening Linux Containers”, v1.1, p. 97, section 9.13
- 26. Ongoing work in Container Security • Not yet namespaced • /dev, /proc/sys/* • However /proc is mounted as read-only • Container security work == Kernel security work • Containers share kernel and make use of kernel isolation features • Use hypervisor isolation for containers • runC is a step in this direction • Replace it with a container run-time that leverages hypervisor isolation, e.g., Intel Clear Containers • Unikernels • Fully unprivileged containers
- 27. Thank You