![CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp
Running CEPH-docker: monitor
task "monitor" {
global = true
count = 2
image = "ceph/daemon"
volumes = "/var/lib/ceph:/var/lib/ceph"
args = ["mon"]
env {
MON_IP = "{{private_ipv4}}"
MON_NAME = "{{hostname}}"
KV_TYPE = "etcd"
KV_IP = "{{private_ipv4}}"
CEPH_PUBLIC_NETWORK = "$(echo {{private_ipv4}} | cut -d '.' -f 1,2,3 | awk '{print
$1 ".0/24"}')"
}
docker-args = ["--net=host"]
}](https://image.slidesharecdn.com/ceph-docker-160429080511/85/Wanting-distributed-volumes-Experiences-with-ceph-docker-8-320.jpg)
![CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp
Running CEPH-docker: ODS
task "osd" {
global = true
count = 2
image = "ceph/daemon"
volumes = [
"/var/lib/ceph/osd:/var/lib/ceph/osd",
"/var/log/ceph:/var/log/ceph"
]
args = ["osd"]
env {
HOSTNAME= "{{hostname}}"
OSD_TYPE = "directory"
OSD_JOURNAL_SIZE = "20"
KV_TYPE = "etcd"
KV_IP = "{{private_ipv4}}"
}
docker-args = ["--net=host", "--privileged=true", "--pid=host"]
}](https://image.slidesharecdn.com/ceph-docker-160429080511/85/Wanting-distributed-volumes-Experiences-with-ceph-docker-9-320.jpg)
![CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp
Running CEPH-docker: MDS
task "mds" {
count = 2
image = "ceph/daemon"
volumes = [
"/var/lib/ceph/:/var/lib/ceph/",
"/var/log/ceph:/var/log/ceph"
]
args = ["mds"]
env {
MDS_NAME= "mds-${instance}"
CEPHFS_CREATE = "1"
KV_TYPE = "etcd"
KV_IP = "{{private_ipv4}}"
}
docker-args = ["--net=host"]
}](https://image.slidesharecdn.com/ceph-docker-160429080511/85/Wanting-distributed-volumes-Experiences-with-ceph-docker-10-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Wanting distributed volumes - Experiences with ceph-docker (20)
![[EXTENDED] Ceph, Docker, Heroku Slugs, CoreOS and Deis Overview](https://cdn.slidesharecdn.com/ss_thumbnails/deis1-141107133247-conversion-gate02-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
Wanting distributed volumes - Experiences with ceph-docker
- 1. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Experiences with ceph-docker Wanting distributed volumes
- 2. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp The problem 1/2 MC MC MC Machine 1 Machine 2 Machine 3
- 3. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp The problem 2/2 HD MC HD MC HD MC Machine 1 Machine 2 Machine 3 Where to store state?
- 4. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Requirements ● No provider specifics – E.g. no AWS EBS ● Use available resources ● Be resilient against failures ● Everything in a container
- 5. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Solutions ● CEPH ● Proven in production ● Automatic balancing ● Daunting to setup but wait, ceph-docker ● GlusterFS ● Proven in production ● Manual setup
- 6. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp CEPH intro ● Distributed object storage ● Block storage ● Filesystem ● Automatic balancing using CRUSH – Object location is calculated, not stored ● Terms – Object Storage Device (OSD) – Cluster map – Monitor
- 7. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp CEPH-docker ● Source: github.com/ceph/ceph-docker ● Container: ceph/daemon – Contains monitor, osd, mds, rgw – Performs configuration setup/retrieval ● Configuration stored in KV backend – consul – etcd ●
- 8. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Running CEPH-docker: monitor task "monitor" { global = true count = 2 image = "ceph/daemon" volumes = "/var/lib/ceph:/var/lib/ceph" args = ["mon"] env { MON_IP = "{{private_ipv4}}" MON_NAME = "{{hostname}}" KV_TYPE = "etcd" KV_IP = "{{private_ipv4}}" CEPH_PUBLIC_NETWORK = "$(echo {{private_ipv4}} | cut -d '.' -f 1,2,3 | awk '{print $1 ".0/24"}')" } docker-args = ["--net=host"] }
- 9. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Running CEPH-docker: ODS task "osd" { global = true count = 2 image = "ceph/daemon" volumes = [ "/var/lib/ceph/osd:/var/lib/ceph/osd", "/var/log/ceph:/var/log/ceph" ] args = ["osd"] env { HOSTNAME= "{{hostname}}" OSD_TYPE = "directory" OSD_JOURNAL_SIZE = "20" KV_TYPE = "etcd" KV_IP = "{{private_ipv4}}" } docker-args = ["--net=host", "--privileged=true", "--pid=host"] }
- 10. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Running CEPH-docker: MDS task "mds" { count = 2 image = "ceph/daemon" volumes = [ "/var/lib/ceph/:/var/lib/ceph/", "/var/log/ceph:/var/log/ceph" ] args = ["mds"] env { MDS_NAME= "mds-${instance}" CEPHFS_CREATE = "1" KV_TYPE = "etcd" KV_IP = "{{private_ipv4}}" } docker-args = ["--net=host"] }
- 11. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Mounting volumes ● Mount CEPH FS once/host – Single point of failure – Bad for upgrades ● Mount CEPH FS / container – May use more resources – Needs recent docker ● Use docker volume plugin – Plugin process cannot be run in container
- 12. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Mount CEPH FS / container ● Uses data container pattern docker run -it –-name=mydata --net=host --privileged -v /media/ceph:/data:shared -v /usr/bin/etcdctl:/usr/bin/etcdctl pulcy/ceph-volume:latest docker run -it -–volumes-from=mydata alpine ls -al /data
- 13. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp The result ● CEPH object storage running on all machines – Using available resources ● Data still available if (some) machines goes down ● Containers get the persistent state they need – on whatever machines they're scheduled on Links ● github.com/ceph/ceph-docker ● github.com/pulcy/ceph-volume ● github.com/pulcy/j2
- 14. CEPH-docker by Ewout Prangsma github:ewoutp / twitter:ewoutp Questions ● Is it production ready? – time will tell....