Red Hat Storage - Introduction to GlusterFS
21 likes15,369 views
Red Hat Storage introduces GlusterFS, an open source scale-out file system. GlusterFS provides scalable, affordable storage using commodity hardware. It allows linearly scaling performance and capacity by adding servers. GlusterFS has a global namespace and supports various protocols, enabling flexible deployment across private and public clouds. Many enterprises rely on GlusterFS for applications, virtual machines, Hadoop, and hybrid cloud solutions.
Ad
More Related Content
What's hot (20)
Similar to Red Hat Storage - Introduction to GlusterFS (20)
Ad
More from GlusterFS (6)
Ad
Recently uploaded (20)
Red Hat Storage - Introduction to GlusterFS
- 1. Red Hat Storage Introduction To GlusterFS October 2011 1
- 2. Today’s Speakers Heather Wellington Tom Trainer Program Marketing Manager Storage Product Marketing Storage Initiative Manager 2
- 3. Red Hat Acquires Gluster – What it Means for You ● Proven GlusterFS architecture ● Stability and long term viability ● Integration ● New features and functions ● Global reach ● Scalable, affordable, and flexible storage 3
- 4. What is the Gluster File System? ● Scale-out file storage software for ● Network Attached Storage (NAS) ● Object ● Big Data ● GlusterFS provides • Scalability to Petabytes & beyond • Affordability • Use of commodity hardware • Flexibility • Deploy in ANY environment • Linearly scalable performance • High availability • Unified files and objects • File system for Apache Hadoop • Superior storage economics 4
- 5. File System in User Space (FUSE) User Space • Not tied to kernel GlusterFS • No reassemblies Server (CPU/Mem) Kernel • Independence 1 TB 1 TB 1 TB 1 TB • Flexibility 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 5
- 6. Many Enterprises Rely on GlusterFS 6
- 7. GlusterFS Architecture Design Goals ● Innovation ● Eliminate metadata ● Dramatically improve performance ● Unify files and objects ● Elasticity ● Flexibility adapt to growth/reduction ● Add, delete volumes & users ● Without disruption ● Scale linearly ● Multiple dimensions ● Performance ● Capacity Performance ● Aggregated resources ● Simplicity ● Ease of management Capacity ● No complex Kernel patches ● Run in user space 7
- 8. Key Differentiators Performance ● Scalable ● No metadata server – faster file system Capacity ● Enables linear scaling of performance via elastic hashing ● Affordable ● Deploy on commodity hardware ● Flexible ● Software only ● Deploy on the infrastructure of choice ● Simultaneous files and objects ● Apache Hadoop Distributed File System (HDFS) alternative ● Modular, stackable storage OS architecture ● Data stored in native file formats 8
- 9. What is GlusterFS Elastic Hashing? ● No metadata server ● An algorithmic approach ● Unique hash tag for each file stored ● Tags stored within the file system ● Rapid file read – low latency Innovative Elastic Approach 9
- 10. Software Only – Future Proofing Storage ● Superior storage economics & flexibility ● Data center / private cloud use commodity hardware ● Public cloud – i.e. AWS, RightSacle, GoGrid, Nimbula – pay for only what you need ● No hardware lock-in ● You choose hardware vendors - at purchase time or in the future ● Any Cloud – Public, private, and hybrid ● Performance, capacity, or availability levels ● GlusterFS – not proprietary, files are stored in native formats (i.e. EXT4) 10
- 11. A Strong Open Source Foundation Global Adoption ● 200,000+ downloads ● ~16,000 /month ● 550+ registered deployments ● 45 countries ● 2,500+ registered users ● Mailing lists, Forums, etc. ● Active community ● Diverse testing environments ● Bugs identification and fixes ● Code contributions ● Member of broader ecosystem ● OpenStack ● Linux Foundation ● Open Virtualization Alliance 11
- 12. Anatomy of a GlusterFS Deployment Clients/Apps Clients/Apps Clients/Apps ● Standard clients running standard apps IP Network ● Over any standard IP network ● Access to data, as files and folders and or objects, Gluster Global Namespace (HTTP, NFS, CIFS, Gluster Native) in global namespace, Application Data Objects VMs ApacheTM HadoopTM using a variety of standard protocols VMDK VMDK ● Stored in a commoditized, virtualized, scale-out, centrally managed pool of virtual storage pool DAS, NAS 12
- 13. Unifying Private and Public Clouds Client/Apps Client/Apps Client/Apps Client/Apps Client/Apps Client/Apps Client/Apps Client/Apps Client/Apps IP Network GlusterFS Global Namespace Private Cloud Public Cloud Scale Out Performance, Capacity, and Availability Red Hat Storage Red Hat Storage Red Hat Storage Software Appliance Software Appliance Software Appliance RHEL RHEL RHEL Server Server Server (CPU/Mem) (CPU/Mem) (CPU/Mem) 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB Scale Up Capacity 1 TB 1 TB 1 TB 1 TB Replication 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 1 TB 13
- 14. CIC Electronic Signature Solutions Hybrid Cloud: Electronic Signature Solutions ● Problem ● Must leverage economics of the cloud ● Storage performance in the cloud too slow ● Need to meet demanding client SLA‟s ● Solution ● Red Hat Storage Software Appliance ● Amazon EC2 and Elastic Block Storage (EBS) • Reduced time-to- ● RightScale Cloud Management market for new ● Benefits products ● Enabled faster development and delivery of new • Meeting all client SLAs products to clients ● All SLA‟s are met with headroom to spare • Accelerating move to ● Accelerating move to the cloud the cloud ● Scale-out architecture allows for constantly changing resources to be added and accessed ● Data is highly available – allowing 24/7 client access to data 14
- 15. Common Solutions Built on GlusterFS ● Media serving (CDN) ● Large scale file storage ● Tier 2 & 3 archive ● File sharing ● High Performance Computing (HPC) FS storage ● IaaS storage layer ● Disaster recovery ● Backup & restore ● Private cloud 15
- 16. Pandora Internet Radio Private Cloud: Media Serving • Problem • Explosive user & title growth • As many as 12 file formats for each song • „Hot‟ content and long tail • Solution • Three data centers, each with a six-node • 1.2 PB of audio served GlusterFS cluster per week • Replication for high availability • 250+ TB total capacity • 13 million files • Over 50 GB/sec peak • Benefits traffic • Easily scale capacity • Centralized management; one administrator to manage day-to-day operations • No changes to application • Higher reliability 16
- 17. Brightcove Private Cloud: Media Serving • Problem • Cloud-based online video platform • Explosive customer & title growth • Massive video in multiple locations • Costs rising, esp. with HD formats • Solution • Complete scale-out based on commodity DAS/JBOD and GlusterFS • Over 1 PB currently in • Replication for high availability Gluster • 1PB total capacity • Separate 4 PB project in the works • Benefits • Easily scale capacity • Centralized management; one administrator to manage day-to-day operations • Higher reliability • Path to multi-site 17
- 18. Partners Healthcare Private Cloud: Centralized Storage as a Service • Problem • Capacity growth from 144TB to 1+PB • Multiple distributed users/departments • Multi OS access - Windows, Linux and Unix • Solution • GlusterFS Cluster • Over 500 TB • Red Hat Enterprise Linux (RHEL) • 9 Sun “Thumper” • Native CIFS/ NFS access systems in cluster • Benefits • Capacity on demand / pay as you grow • Centralized management • Higher reliability • OPEX decreased by 10X 18
- 19. Simultaneous File and Object Storage (SFO) ● SFO Defined ● As part of GlusterFS, it is the first file system that enables you to store and access data as an object and as a file ● Flexible and powerful ● Simplifies access and management of data. • Eases migration of legacy, file-based applications to object storage for use in the cloud ● Public beta available since 2011 ● Broad community testing and participation ● Selected enterprise customer engagements A breakthrough from the traditional hardware approach… 19
- 20. Traditional Hardware Approach Files Objects DB’s “VNX reminds me of my old VHS, DVD and cable box…. ….one thing fails and I’m blown out of the water.” NAS Object Block Beta Customer , 2011 Carved Up ● Proprietary Storage Pool ● Bolt-together disparate technology ● Combined hardware raises costs Traditional Monolithic Hardware ● Higher TCO Bolt-on Approach ● Paying for what you many not need (i.e. EMC VNX) ● Increased risk ● Common hardware elements can fail ● Power supplies ● Fans ● Cabling…lots of cabling 20
- 21. Software Approach to File and Object Storage Store discrete video files and move numbers of them as objects and vice versa… • Window Access • Improves Windows performance • Uses HTTP, not slower CIFS • We will still support SAMBA • Object Storage • API • Internet Protocol (IP) • ResTFul • Get/Put • Buckets • Objects seen as files • Standards based • Amazon Web Services S3 ReSTFul interface compatible • Network Attached Storage (NAS) • Access data as objects and a NAS interface to access files (NFS, CIFS, • NFS / CIFS / GlusterFS GlusterFS) • POSIX compliant • Backup to AWS • Access files within objects • Public and private clouds GlusterFS simplifies file and object storage 21
- 22. Introducing GlusterFS Compatibility for Apache Hadoop Metadata Server NameNode MapReduce MapReduce MapReduce MapReduce MapReduce GlusterFS GlusterFS GlusterFS GlusterFS GlusterFS ● Coexist, or alternative to HDFS ● NameNode metadata server eliminated ● Faster access times – faster file system ● All the features and benefits of GlusterFS 22
- 23. Why It’s Different ● No metadata server ● No performance bottleneck on data lookups - fast file access ● Reduces requirement for replicated files from 3 to 2 ● 33% capacity savings ● Built in replication ● Synchronous for inter-node replication ● Asynchronous for geo-replication ● No single point of failure ● No block size restrictions ● Ideal for small and large files ● POSIX compliant file system ● Out of the box NFS, CIFS and Gluster native access ● Expanded data access options ● File and object access to data ● Access files from your object interface and access data within objects as files ● File based applications can access data without modification 23
- 24. Major Retailer – Analytics Group Leverages Hadoop and GlusterFS ● Problem ● Performance trails off as file quantity soars NameNode server errors degrade availability Big Data ● ● Must scale beyond current performance limitations Analytics ● Solution ● Red Hat Storage ● GlusterFS alternative to HDFS • Higher performance ● Elimination of NameNode server • Greater availability ● Benefits • Lower overall costs ● Accelerating overall performance ● Scale-out architecture allows for constantly changing resources to be added and accessed ● Files are highly available – allowing 24/7access to data ● NFS and Object access to files ● Higher overall capacity utilization 24 ● Reduced storage spend
- 25. The Gluster Connector for OpenStack – July 2011 • Enables GlusterFS to be the underlying file system • Connects GlusterFS to Xen and KVM hypervisor • Unified File and Object storage OpenStack with Gluster • Highly-available, scale-out NAS • Alternative to SWIFT Mobile Apps. Web Clients. Enterprise Software Ecosystem API Layer OpenStack Prior to Gluster … Compute Simultaneous File & Object Storage SWIFT OpenStack Imaging Services 25
- 26. The Gluster Connector for OpenStack - July 2011 • Connector enables GlusterFS to be chosen as the file system • Provides: • Unified File and Object storage • Highly scalable NAS • High Availability – synchronous and asynchronous replication • Preferred, scalable alternative to SWIFT • Virtual motion of virtual machines (a.k.a. vmotion) VM VM VM VM VM VM VM VM Hypervisor Hypervisor GlusterFS GlusterFS Server Server (CPU/Mem) (CPU/Mem) Virtual storage pool 26
- 27. Red Hat Storage Deployment Options On-premise/datacenter • Red Hat Storage Software Appliance • Deploy on bare metal • Any hardware on Red Hat Hardware HCL Public cloud • Amazon Web Services (AWS) • Runs within Amazon Machine Image (AMI) • GoGrid Cloud • Gluster Server Image (GSI) for scale-out NAS on GoGrid cloud 27
- 28. Summary • GlusterFS is scale-out storage • NAS • Object • Big Data • Scalable, affordable, and flexible • Open Source • Innovative architecture provides a better way to do storage 28
- 29. Questions & Answers Your turn - ask our experts • Register to try GlusterFS here: http://www.gluster.com/trybuy/ • Follow us on twitter: @RedHatStorage • Additional resources here: http://www.gluster.com/products/resources/ • Join the community: http://www.gluster.org/ • Read our blog: http://blog.gluster.com/ Contact us at: [email protected] or 1-800-805-5215 29