![● The OpenShift Advanced Installer now supports deploying container-native storage and the registry on
container-native storage out of the box.
● All you have to do since OpenShift Container Platform 3.6 is add a few lines to your Ansible inventory file.To deploy an
OpenShift registry backed by container-native storage, first add the following variable definition in the [OSEv3:vars]
section:
openshift_hosted_registry_storage_kind=glusterfs
● And then add a new host group defining the container-native storage nodes to the inventory, for example:
[glusterfs_registry]
infra-1.lab glusterfs_devices='[ "/dev/sdd" ]'
infra-2.lab glusterfs_devices='[ "/dev/sdd" ]'
infra-3.lab glusterfs_devices='[ "/dev/sdd" ]'
Support a scaleout registry](https://image.slidesharecdn.com/jaspreet-webinar-cns-180115101044/85/Jaspreet-webinar-cns-36-320.jpg)
Jaspreet webinar-cns
- 1. Container Native Storage on Openshift Container Platform Jaspreet Kaur Senior TSE(Openshift Support)
- 2. AGENDA ● Introduction ● The Journey So Far for OpenShift Storage ● Why CNS? ● What’s shipping today ● New features in CNS 3.6 ● Ref Architectures ● Q/A
- 3. - Break the assumption that “Containers are for stateless workloads only“. Containers are not persistent by default. App data is lost when containers die. - Why is container storage a pain point? Complex. Even more complicated when container orchestration is involved and applications need to be scaled out.Provide a solution that will run GlusterFs as containers in Openshift pods - Why Red Hat Container-Native Storage (CNS)? Advanced storage capabilities, deeper integration with OpenShift, better price/performance than traditional storage CNS is open, scalable and has consistent user experience across the hybrid cloud (compliments the OpenShift value proposition). Introduction
- 4. The Journey So Far
- 5. Storage Options for OpenShift: Cloud Provider Storage AUTOMATED CONFIGURATIONNFS Amazon EBS Azure Disk GCE Disk iSCSI Ceph RBD SINGLE CONTROL PANEL GlusterFS * very new, handle with care CHOICE FOR PERSISTENT STORAGE ISCSI Amazon EBS Azure Disk GCE Disk
- 6. Storage Options for OpenShift: NFS WELL UNDERSTOOD, EASY TO DEMO PROVIDES RWX STORAGE NO HA, NOT AVAILABLE IN THE CLOUD NFS REQUIRES PRE-PROVISIONING
- 7. Storage Options for OpenShift: iSCSI WELL UNDERSTOOD IN THE DC BLOCK STORAGE, NO RWX SUPPORT NOT AVAILABLE IN THE CLOUD NFS REQUIRES PRE-PROVISIONING iSCSI
- 8. Storage Options for OpenShift: Cloud Provider Storage AVAILABLE IN THE CLOUD NOT AVAILABLE ACROSS AZs PERFORMANCE DEPENDS ON SIZE NFS Azure Disk GCE Disk BLOCK STORAGE, NO RWX SUPPORT iSCSI Amazon EBS
- 9. Storage Options for OpenShift: Cloud Provider Storage SCALABLE, HA CINDER PREFERRED FOR OCP on OSP NFS Amazon EBS Azure Disk GCE Disk iSCSI Ceph RBD BLOCK STORAGE, NO RWX SUPPORT
- 10. Storage Options for OpenShift: Cloud Provider Storage RUNS ON TOP OF OPENSHIFT GEO-REPLICATION, SNAPSHOTS* NFS Amazon EBS Azure Disk GCE Disk iSCSI Ceph RBD SCALABLE FILE STORAGE, RWX, RWO* SUPPORT GlusterFS * very new, handle with care AVAILABLE EVERYWHERE S3 OBJECT STORAGE FOR APPS ISCSI Amazon EBS Azure Disk GCE Disk
- 11. ● Storage provided by a dedicated Red Hat Gluster Storage Cluster over the network outside OpenShift ● Red Hat Gluster Storage can run bare metal or VMs ● Storage and compute can scale independently Deployment Option-1: Container Ready Storage (CRS)
- 12. Paradigm Shift Traditional Storage Management Converged Storage Management
- 13. ● Fully featured scale-out storage platform (RHGS) runs inside OpenShift in containers (kubernetes pods) ● Application and storage containers can be co-located ● Storage services are managed, scaled and upgraded like app containers Deployment Option-2: Container-Native Storage (CNS)
- 14. Why should you care ?
- 16. Storage Consolidation Container-native Storage EBS gp2 EBS gp2 EBS st1 EBS io1 vs. EBS sc1
- 17. Simplify Container Availability AVAILABILITY ZONE A AVAILABILITY ZONE B AVAILABILITY ZONE C Node Node Node Node Node Node GLOBAL STORAGE NAMESPACE
- 18. ● Storage can run as a set of micro-services inside OpenShift - no external appliance needed ● Consistent storage experience across the hybrid cloud ● Storage Consolidation ● Multi purpose (RWO, RWX, object), flexible & versatile Container Native Storage - Our Advantage
- 19. ● CNS: providing dynamic persistent storage for openshift with GlusterFS in a hyper-converged fashion ● Heketi: the high-level service interface to manage the lifecycle of volumes in multiple Gluster clusters. Openshift ⇐⇒ Heketi ⇐⇒ Gluster What is CNS … and what is Heketi?
- 20. ● openshift/kubernetes ○ dynamic glusterfs provisioner ○ glusterfs plugin ● Heketi ○ high-level service interface for gluster volume lifecycle management ● glusterfs: ○ one or more glusterfs clusters ○ running hyper-converged in openshift ● cns-deploy: ○ tool to deploy gluster and heketi into an existing openshift cluster Components of CNS
- 21. PVC(created by user) references the glusterfs provisioner. ● glusterfs provisioner extracts details from PVC ● provisioner tells heketi to create a volume if given size and type. > heketi looks for a gluster cluster that can satisfy this request > if found, heketi tells the gluster instance to create the volume. > gluster creates a volume > Heketi hands volume back to provisioner ● provisioner creates PV and puts the gluster volume details into it. ● provisioner puts glusterfs as the mount plugin into the pv ● provisioner returns PV to the caller PVC is bound to the PV and can later be in a pod by the user PV creation: glusterfs dynamic provisioner
- 22. Workflow with Dynamic Provisioning Persistent Volume Claim is submitted
- 23. Workflow with Dynamic Provisioning Openshift request volume to be created
- 24. Workflow with Dynamic Provisioning Persistent volume is created by storage system and registered with Openshift
- 25. Workflow with Dynamic Provisioning Volume can now be used by Pod to provide persistent storage
- 27. CONTAINER-NATIVE STORAGE ON OPENSHIFT INFRA
- 28. CONTAINER-NATIVE STORAGE ON OPENSHIFT POD CONTAINERIZED RED HAT GLUSTER STORAGE RHEL HOST OS RED HAT GLUSTER POD GLUSTER VOLUMES PHYSICAL DISKHOST NETWORK
- 29. CONTAINER-NATIVE STORAGE ON OPENSHIFT POD RHGS POD RHGS POD RHGS POD RHGS POD INFRA POD INFRA HEKETI RESTFUL GLUSTERFS MANAGEMENT RED HAT HEKETI POD REST API GLUSTERFS PODS
- 30. ● Container Ready Storage serving out storage to OpenShift with: ○ RHGS in stand-alone bare-metal storage clusters ○ RHGS inside VMs (VMware most popular) on bare-metal hosts ○ RHGS inside VMs fronting Enterprise Storage Arrays ■ RHGS sits between OpenShift and Storage Arrays ■ LUNs are served out as RHGS bricks ○ RHGS in AWS, Azure, Google Cloud ● Container-Native Storage ○ RHGS runs containerized inside OpenShift Container Platform (always with Heketi) ○ CNS runs anywhere OCP runs! Container Storage Deployments
- 31. Container Storage Use Cases ● Local Storage for Container Images ● OpenShift Infra (registry,logging & metrics) ● Persistent Storage for Applications
- 32. OpenShift Registry on CNS OPENSHIFT INTERNAL REGISTRY POD OPENSHIFT REGISTRY OPENSHIFT INTERNAL REGISTRY POD OPENSHIFT REGISTRY OPENSHIFT INTERNAL REGISTRY POD OPENSHIFT REGISTRY RED HAT GLUSTER CONTAINER-NATIVE STORAGE Scalable Highly-Available Automated Integrated PERSISTENT VOLUME
- 33. ● Brick multiplexing introduced in CNS 3.6 ● Optimized storage for logging and metrics. ● Support a scaleout registry. ● S3 object storage for applications. New features of CNS 3.6
- 34. Brick multiplexing in CNS 3.6 ● With brick-multiplexing, only one glusterfsd process is governing the bricks such that the amount of memory consumption of GlusterFS pods is drastically reduced and the scalability is significantly improved. ● In 3.6, we are able to support over 1,000 PersistentVolumes in a single container- native storage cluster. The amount of memory consumed increases linearly, so that 32GB of RAM are only needed at the high end of that. The rule of thumb is roughly 30-35 MB RAM per volume on each of the participating GlusterFS pods.
- 35. Optimized storage for Logging and Metrics ● Container-Native Storage version 3.6 now provides backend storage for OpenShift Logging and OpenShift Metrics with gluster- block. For the moment, the use of gluster-block in production is only supported for OpenShift Logging and Metrics services, but use of gluster- block beyond that is under qualification, and support is expected to be extended soon.
- 36. ● The OpenShift Advanced Installer now supports deploying container-native storage and the registry on container-native storage out of the box. ● All you have to do since OpenShift Container Platform 3.6 is add a few lines to your Ansible inventory file.To deploy an OpenShift registry backed by container-native storage, first add the following variable definition in the [OSEv3:vars] section: openshift_hosted_registry_storage_kind=glusterfs ● And then add a new host group defining the container-native storage nodes to the inventory, for example: [glusterfs_registry] infra-1.lab glusterfs_devices='[ "/dev/sdd" ]' infra-2.lab glusterfs_devices='[ "/dev/sdd" ]' infra-3.lab glusterfs_devices='[ "/dev/sdd" ]' Support a scaleout registry
- 37. S3 object storage for applications ● Container-Native Storage 3.6 now provides an S3 object storage interface as a TechPreview. Application developers have a ready-to-use REST API at hand to provide object storage to workloads on OpenShift, just a HTTP PUT or GET request away. ● Object storage in CNS 3.6 provides a simple yet scalable storage layer for distributed applications that were previously tied to specific cloud provider S3 object storage. These application now run with little or no modification on OpenShift.
- 38. AWS Reference Architecture Description OpenShift 3.5 with Container Native Storage on Amazon Web Services Reference Architecture: https://access.redhat.com/articles/3018151. Ansible playbooks and deplyment scripts found here: https://github.com/openshift/openshift-ansible-contrib/tree/master/reference-architecture.
- 39. VMware Reference Architecture Description OpenShift 3.6 with Container Native Storage on VMware Reference Architecture: New RA Available July 2017. Ansible playbooks and deplyment scripts found here: https://github.com/openshift/openshift-ansible-contrib/tree/master/reference-architecture.
- 40. ● https://github.com/gluster/gluster-kubernetes ● https://github.com/heketi/heketi ● https://github.com/kubernetes/kubernetes : pkg/volume/glusterfs/ ● https://github.com/gluster/glusterfs Open Source! ⇒ Upstream Bits
- 41. ● pod: group of one or more containers that form an entity ● persistent volume (PV): to be mounted by application pod ● provisioner: to provide PVs upon request ● plugin: mechanism to mount the PV, referenced in PV ● persistent volume claim (PVC): mechanism for a user to request a PV ● Access types for volumes: ○ RWO - read write once (single node) ○ RWX - read write many (multiple nodes) ○ ROX - read only many (multiple nodes) ● flavors of provisioning: dynamic and static Glossary: OpenShift/Kube Storage
- 42. THANK YOU