App container rkt
2 likes•1,468 views
This document discusses container technologies including App Container (appc) and rkt. It provides an overview of appc components like the image format, discovery, and executor. It then discusses rkt, an implementation of appc, describing its modular architecture with stages 0-2 and use of systemd and cgroups for isolation. It also touches on rkt security, networking, and integration with systemd and user namespaces.
![Example: memory isolator
“limit”:
“500M”
Application
Image Manifest
[Service]
ExecStart=
MemoryLimit=500M
systemd service file
write to
memory.limit_in_
bytes
systemd action](https://image.slidesharecdn.com/appcontainer-rkt-151124184244-lva1-app6891/85/App-container-rkt-86-320.jpg)
![Example: CPU isolator
“limit”:
“500m”
Application
Image Manifest
write to
cpu.share
systemd action
[Service]
ExecStart=
CPUShares=512
systemd service file](https://image.slidesharecdn.com/appcontainer-rkt-151124184244-lva1-app6891/85/App-container-rkt-87-320.jpg)
App container rkt
- 3. With containers what does a "Linux Distro" mean?
- 6. The Bad $ python --version Python 2.7.6 $ python app-requiring-python3.py $ python --version Python 3.4.3 $ python app-requiring-python2.py package collisions
- 7. The Bad $ cat /etc/os-release | grep ^NAME= NAME=Fedora $ rpm -i package-from-suse.rpm file /foo from install of package-from-suse.rpm conflicts with file from package-from-fedora dependency namespacing
- 8. The Good $ gpg --list-only --import /etc/apt/trusted.gpg.d/* gpg: key 2B90D010: public key "Debian Archive Automatic Signing Key (8/jessie) <[email protected]>" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) gpg: no ultimately trusted keys found users control trust
- 9. The Good $ rsync ftp.us.debian.org::debian /srv/mirrors/debian $ dpkg -i /srv/mirrors/debian/kernel-image- 3.16.0-4-amd64-di_3.16.7-ckt9-2_amd64.udeb trivial mirroring and hosting
- 10. Linux Packages 2.0 .deb and .rpm for containers
- 11. Container VS VM? ● Lightweight (100s vs 10s) ● Easy to deploy ● Less isolation?
- 12. What is container ? ● Packaging with your apps with deps ● Running in isolation (using namespace, cgroups)
- 13. Why I want to use it? ● Deploy faster ● Run faster, run everywhere ● Run in isolation
- 15. appc != rkt
- 16. Application Containers self-contained, portable (decoupled from operating system) isolated (memory, network, …)
- 17. appc principles Why are we doing this?
- 18. Open Independent GitHub organisation Contributions from Cloud Foundry, Mesosphere, Google, Red Hat (and many others!)
- 19. Simple but efficient Simple to understand and implement, but eye to optimisation (e.g. content-based caching)
- 20. Secure Cryptographic image addressing Image signing and encryption Container identity
- 21. Standards-based Well-known tools (tar, gzip, gpg, http), extensible with modern technologies (bittorrent, xz)
- 22. Composable Integrate with existing systems Non-prescriptive about build workflows OS/architecture agnostic
- 23. appc components
- 24. Image Format Application Container Image tarball of rootfs + manifest uniquely identified by ImageID (hash)
- 25. Image Discovery App name → artifact example.com/http-server coreos.com/etcd HTTPS + HTML
- 26. Executor （Runtime) grouped applications runtime environment isolators networking
- 27. Metadata Service http://$AC_METADATA_URL/acMetadata container metadata container identity (HMAC verification)
- 28. ACE validator is this executor compliant with the spec? $EXECUTOR run ace_validator.aci
- 29. appc community
- 30. github.com/ cdaylward/libappc C++ library for working with app containers
- 31. github.com/ cdaylward/nosecone C++ executor for running app containers
- 33. github.com/ 3ofcoins/jetpack FreeBSD Jails/ZFS-based executor (by @mpasternacki)
- 34. github.com/ sgotti/acido ACI toolkit (build ACIs from ACIs)
- 37. appc spec in a nutshell - Image Format (ACI) - what does an application consist of? - Image Discovery - how can an image be located? - Pods - how can applications be grouped and run? - Executor (runtime) - what does the execution environment look like?
- 38. appc status Stabilising towards first backwards compatible release
- 40. rkt an implementation of appc
- 42. rkt a modern, secure container runtime
- 44. simple CLI tool golang + Linux self-contained init system/distro agnostic
- 45. simple CLI tool no daemon no API* apps run directly under spawning process
- 49. rkt internals modular architecture execution divided into stages stage0 → stage1 → stage2
- 50. rkt (stage0) pod (stage1) bash/runit/systemd/... (invoking process) app1 (stage2) app2 (stage2)
- 51. rkt (stage0) pod (stage1) bash/runit/systemd/... (invoking process) app1 (stage2) app2 (stage2)
- 52. stage0 (rkt binary) discover, fetch, manage application images set up pod filesystems commands to manage pod lifecycle
- 53. stage0 (rkt binary) - rkt run - rkt prepare - rkt run-prepared - rkt list - rkt status - ... - rkt fetch - rkt trust - rkt image list - rkt image export - rkt image gc - ...
- 54. stage0 (rkt binary) file-based locking for concurrent operation (e.g. rkt gc, rkt list for pods) database + reference counting for images
- 55. rkt (stage0) pod (stage1) bash/runit/systemd/... (invoking process) app1 (stage2) app2 (stage2)
- 56. rkt (stage0) pod (stage1) bash/runit/systemd/... (invoking process) app1 (stage2) app2 (stage2)
- 57. stage1 execution environment for pods app process lifecycle management isolators
- 58. stage1 (swappable) binary ABI with stage0 stage0 calls an execve(stage1)
- 59. stage1 (swappable) ● default implementation ○ based on systemd-nspawn+systemd ○ Linux namespaces + cgroups for isolation ● kvm implementation ○ based on lkvm+systemd ○ hardware virtualisation for isolation ● others?
- 60. rkt (stage0) pod (stage1) bash/runit/systemd/... (invoking process) app1 (stage2) app2 (stage2)
- 61. rkt (stage0) pod (stage1) bash/runit/systemd/... (invoking process) app1 (stage2) app2 (stage2)
- 62. stage2 actual app execution independent filesystems (chroot) shared namespaces, volumes, IPC, ...
- 63. rkt + systemd The different ways rkt integrates with systemd
- 64. rkt
- 66. systemd (on host) optional "systemctl stop" just works socket activation pod-level isolators: CPUShares, MemoryLimit
- 68. systemd-nspawn default stage1, besides lkvm taking care of most of the low-level things
- 72. application app-level isolators: CPUShares, MemoryLimit chrooted
- 74. systemd-journald no changes in apps required logs in the container available from the host with journalctl -m / -M
- 76. systemd-machined register on distros using systemd machinectl {show,status,poweroff…}
- 78. cgroups
- 79. What’s a control group? (cgroup) ● group processes together ● organised in trees ● applying limits to them as a group
- 80. cgroups
- 82. List of cgroup controllers /sys/fs/cgroup/ ├─ cpu ├─ devices ├─ freezer ├─ memory ├─ ... └─ systemd
- 83. /sys/fs/cgroup/ ├─ systemd │ ├─ user.slice │ ├─ system.slice │ │ ├─ NetworkManager.service │ │ │ └─ cgroups.procs │ │ ... │ └─ machine.slice How systemd units use cgroups
- 84. │... ├─ cpu │ ├─ user.slice │ ├─ system.slice │ └─ machine.slice │ └─ machine-rkt….scope │ └─ system.slice │ └─ app.service ├─ memory │ ├─ user.slice │ ├─ system.slice │ └─ machine.slice ... /sys/fs/cgroup/ ├─ systemd │ ├─ user.slice │ ├─ system.slice │ └─ machine.slice │ └─ machine-rkt….scope │ └─ system.slice │ └─ app.service │ │ │... How systemd units use cgroups w/ containers
- 85. /sys/fs/cgroup/ ├─ systemd │ ├─ user.slice │ ├─ system.slice │ └─ machine.slice │ └─ machine-rkt….scope │ └─ system.slice │ └─ app.service │ │ │... │... ├─ cpu │ ├─ user.slice │ ├─ system.slice │ └─ machine.slice │ └─ machine-rkt….scope │ └─ system.slice │ └─ app.service ├─ memory │ ├─ user.slice │ ├─ system.slice │ └─ machine.slice ... cgroups mounted in the container RW RO
- 86. Example: memory isolator “limit”: “500M” Application Image Manifest [Service] ExecStart= MemoryLimit=500M systemd service file write to memory.limit_in_ bytes systemd action
- 87. Example: CPU isolator “limit”: “500m” Application Image Manifest write to cpu.share systemd action [Service] ExecStart= CPUShares=512 systemd service file
- 88. Unified cgroup hierarchy ● Multiple hierarchies: ○ one cgroup mount point for each controller (memory, cpu, etc.) ○ flexible but complex ○ cannot remount with a different set of controllers ○ difficult to give to containers in a safe way ● Unified hierarchy: ○ cgroup filesystem mounted only one time ○ still in development in Linux: mount with option “__DEVEL__sane_behavior” ○ initial implementation in systemd-v226 (September 2015) ○ no support in rkt yet
- 89. rkt: a few other things - rkt and security - rkt API service (new!) - rkt networking - rkt and user namespaces - rkt and production
- 90. rkt and security "secure by default"
- 91. rkt security - image signature verification - privilege separation - e.g. fetch images as non-root user - SELinux integration - kernel keyring integration (soon) - lkvm stage1 for true hardware isolation
- 92. rkt API service (new!) optional, gRPC-based API daemon exposes information on pods and images runs as unprivileged user easier integration with other projects
- 93. rkt networking plugin-based Container Networking Interface (CNI)
- 94. Container Runtime (e.g. rkt) veth macvlan ipvlan OVS Container Networking Interface (CNI)
- 95. Networking, the rkt way
- 96. Network tooling ● Linux can create pairs of virtual net interfaces ● Can be linked in a bridge container1 container2 eth0 veth1 eth0 veth2 IP masquerading via iptables eth0 bridge
- 97. rkt and user namespaces
- 98. History of Linux namespaces ✓ 1991: Linux ✓ 2002: namespaces in Linux 2.4.19 ✓ 2008: LXC ✓ 2011: systemd-nspawn ✓ 2013: user namespaces in Linux 3.8 ✓ 2013: Docker ✓ 2014: rkt … development still active
- 99. Why user namespaces? ● Better isolation ● Run applications which would need more capabilities ● Per user limits ● Future? ○ Unprivileged containers: possibility to have container without root
- 100. 0 host 65535 4,294,967,295 (32-bit range) 0 container 1 655350 container 2 User ID ranges
- 101. unmapped User ID mapping /proc/$PID/uid_map: “0 1048576 65536” host container 1048576 65536 65536 unmappedunmapped
- 102. Problems with container images Container filesystem Container filesystem Overlayfs “upper” directory Overlayfs “upper” directory Application Container Image (ACI) Application Container Image (ACI) container 1 container 2 downloading web server
- 103. Problems with container images ● Files UID / GID ● rkt currently only supports user namespaces without overlayfs ○ Performance loss: no COW from overlayfs ○ “chown -R” for every file in each container
- 104. Problems with volumes / /home/var user / /data /my-app bind mount (rw / ro) /data ● mounted in several containers ● No UID translation /data
- 105. User namespace and filesystem problem ● Possible solution: add options to mount() to apply a UID mapping ● rkt would use it when mounting: ○ the overlay rootfs ○ volumes ● Idea suggested on kernel mailing lists
- 106. rkt and production - still pre-1.0 - unstable (but stabilising) CLI and API - explicitly not recommended for production - although some early adopters
- 107. rkt v1.0.0 EOY (fingers crossed) stable API stable CLI ready to use!
- 108. Questions? github.com/coreos/rkt coreos.com/careers (soon in Berlin!) Join us!