Petascale Cloud Storage with GlusterFS

Aug 11, 20113 likes1,543 views

This document summarizes GlusterFS, an open source software-only solution for petabyte scale cloud storage. Key points include: - GlusterFS uses a fully distributed architecture with no single point of failure and allows scaling storage capacity linearly by adding more servers. It can support hundreds of petabytes. - Features include high availability through replication, global namespaces, and support for hundreds of petabytes of storage using Gigabit or 10 Gigabit Ethernet networks. - GlusterFS integrates with OpenStack for providing scalable cloud storage for applications, virtual machine images, and other use cases. It allows better management of VM images than object storage.

Petascale Cloud Storage
with GlusterFS

AB Periasamy | CTO Gluster, Inc.
Mon, August 1, 2011

Story of Gluster

1st meeting room
1st Office US

1st Office Bengaluru 1st Office Bengaluru
Petascale Cloud Filesystem 2

1000s of Community Deployments

Petascale Cloud Filesystem 3

Fast Growing Commercial Deployments

Petascale Cloud Filesystem 4

GlusterFS Simple Commands

# gluster peer probe HOSTNAME
# gluster volume info
# gluster volume create VOLNAME [stripe COUNT] [replica
COUNT] [transport tcp | rdma] BRICK …
# gluster volume delete VOLNAME
# gluster volume add-brick VOLNAME NEW-BRICK ...
# gluster volume rebalance VOLNAME start

Petascale Cloud Filesystem 5

Gluster Architecture Advantages
Software only
No metadata server
‘Google Storage’ for • Fully distributed architecture, no bottleneck
Everyone • Gluster Elastic Hash
• Intelligence in the SW High performance global namespace
• Leverage commodity HW • Scale out with linear performance
• Hundreds of petabytes
• Scale-out elastically • 1 GbE, 10GbE
• Replication for reliability High availability
• Software enables • Replication to survive hardware failure
virtualization • Self-healing
• Data stored in NFS-like native format
Stackable userspace design
• No kernel dependencies, simple install
• Match specific workload profiles
• Early maturity and rich functionality

Petascale Cloud Filesystem 7

Gluster GeoReplication for Hybrid Cloud

Continuously replicate data between the private and public
clouds.
Migrate VMs and applications across across geographies.

Petascale Cloud Filesystem 8

Gluster Unified File and Object Storage

Enable seamless transition of enterprise applications to the cloud

Petascale Cloud Filesystem 9

Gluster + OpenStack Integration

Provide scalable cloud storage for applications and VM images

Application Data
Gluster NAS Applications

OpenStack
Swift / REST VM Images
OpenStack
Nova
OpenStack
Glance

Petascale Cloud Filesystem 10

GlusterFS as Virtual Machine File System

Bring VMware's vMotion like capabilities to OpenStack.
Instantly boot VMs using mountable filesystem interface. Object
storage requires entire VM image to be fetched before booting.
Easily manage VM images just like any other files.
Scale OpenStack beyond VMware. No 64 TB volume limit or
2TB VM limit. GlusterFS scales to multiple Petabytes.

Petascale Cloud Filesystem 11

This document discusses storage as a service and how OpenStack provides storage options. It describes Storage as a Service and the types of storage available in OpenStack, including ephemeral, object, block, and file storage. It provides overviews of the OpenStack Object Storage (Swift) and Block Storage (Cinder) projects and how they work. The document also discusses filesystem storage in OpenStack using protocols like NFS and CIFS.

OpenStack Storage - an OverviewNilesh Bhosale

This document provides an overview of OpenStack storage components, including Cinder (block storage) and Manila (shared file storage). It describes OpenStack's mission to produce an open source cloud computing platform that is simple to implement and massively scalable. Key OpenStack components are outlined, including Horizon, Nova, Cinder, Swift, Networking, Glance, Keystone, and Oslo. Cinder provides block storage functionality like creating, deleting, and attaching volumes. Manila provides a vendor-neutral API for provisioning and attaching various file-based storage systems and allows sharing of file systems across OpenStack compute instances.

Big data analytics and docker the thrilla in manilaDean Hildebrand

This document discusses using Big Data analytics with Docker containers in a university cloud environment. It describes using OpenStack Manila to provide shared file systems across containers via NFS. IBM Spectrum Scale is used as the back-end storage for its high performance, scale-out capabilities. OpenStack Heat orchestrates the deployment of Docker instances, subnets, and data folders upon user requests. Manila shares are mounted within containers to enable big data analytics access to shared data. Challenges in integrating storage with Docker and ensuring proper resource cleanup are also outlined.

OpenNebula TechDay Boston 2015 - Future of Information Storage with ISS Super...OpenNebula Project

The document introduces ISS SuperCore and Ceph as solutions for future data storage challenges. Ceph was created as a PhD thesis in 2007 and uses pseudo-random data distribution and CRUSH algorithm to store data without relying on parity calculations or centralized controllers. It provides a reliable, autonomic, distributed object store without limitations on size or scalability. SuperCore delivers Ceph through an active storage cluster that provides resilience, replication optimization, and economy compared to traditional RAID storage systems. It allows limitless expansion, instant rebuilds, self-healing from faults, flexible data placement rules, and on-demand provisioning to economize storage use up to 40%.

Meshing OpenStack and Bare Metal Networks with EVPN - David Iles, Mellanox Te...OpenStack

Audience Level Intermediate Synopsis The latest SDN revolution is centered on creating efficient virtualized data center networks using VXLAN & EVPN. We will talk about the scale, performance, and cost advantages of using a modern controller-free virtualized network solution built on 100 Gigabit Ethernet switches with hardware based VXLAN Routing. We will explore the ease of automating such a network in an OpenStack environment and take you through a real world use case of using OpenStack Network Node bridging between a bare metal cloud (EVPN) and a fully virtualized cloud environments (orchestrated by Neutron). Speaker Bio: David has held leadership roles at 3COM, Cisco Systems, Nortel Networks, and IBM where he promoted advanced network technologies including High Speed Ethernet, Layer 4-7 switching, Virtual Machine-aware networking, and Software Defined Networking. David’s current focus is on the evolving landscape of data center networking, scale out storage, Open Networking, and cloud computing.

State of the Container EcosystemVinay Rao

This document discusses containerization and the Docker ecosystem. It begins by describing the challenges of managing different software stacks across multiple environments. It then introduces Docker as a solution that packages applications into standardized units called containers that are portable and can run anywhere. The rest of the document covers key aspects of the Docker ecosystem like orchestration tools like Kubernetes and Docker Swarm, networking solutions like Flannel and Weave, storage solutions, and security considerations. It aims to provide an overview of the container landscape and components.

OpenNebula TechDay Boston 2015 - Bringing Private Cloud Computing to HPC and ...OpenNebula Project

This document discusses bringing private cloud computing to high-performance computing (HPC) and science. It outlines the challenges of using cloud infrastructure for HPC workloads, including performance penalties from virtualization and input/output overhead. It then describes OpenNebula, an open-source tool for managing private clouds that addresses these challenges. Finally, it presents several case studies of research institutions that have implemented private HPC clouds using OpenNebula to gain efficiencies while supporting a variety of applications and user groups.

[OpenStack Days Korea 2016] How open HW and SW drives telco infrastucture inn...OpenStack Korea Community

Kang-Won Lee is a Senior Vice President at SKT's Corporate R&D Center who discussed SKT's work on 5G networks and open source/hardware projects. Some key points include: - SKT is working to make 5G networks virtualized, programmable, and flexible through projects like Mobile CORD. - They are developing disaggregated and virtualized RAN and EPC architectures using open source software and open hardware to improve scalability, reduce costs and promote flexibility. - SKT is involved in several open source and open hardware projects including All-Flash Ceph storage, T-CAP servers, and the Open Compute Project to help build software-defined data centers.

[OpenStack Day in Korea 2015] Track 2-6 - Apache Tajo on SwiftOpenStack Korea Community

Jihoon Son presents Apache Tajo, an open source data warehouse system that supports SQL queries. Tajo can run on OpenStack Swift object storage without any code modifications, addressing the need for SQL queries on Swift data. Son demonstrates how Tajo integrates with Swift, discusses configurations, and addresses the data locality problem. Advanced integration techniques like a location-aware computing model are presented to improve performance when Tajo and Swift clusters are co-located. The roadmap includes specialized storage layers for Swift and support for additional storage like Cinder and Ceph block storage.

Building a GPU-enabled OpenStack Cloud for HPC - Blair Bethwaite, Monash Univ...OpenStack

Audience Level Intermediate Synopsis M3 is the latest generation system of the MASSIVE project, an HPC facility specializing in characterization science (imaging and visualization). Using OpenStack as the compute provisioning layer, M3 is a hybrid HPC/cloud system, custom-integrated by Monash’s R@CMon Research Cloud team. Built to support Monash University’s next-gen high-throughput instrument processing requirements, M3 is half-half GPU-accelerated and CPU-only. We’ll discuss the design and tech used to build this innovative platform as well as detailing approaches and challenges to building GPU-enabled and HPC clouds. We’ll also discuss some of the software and processing pipelines that this system supports and highlight the importance of tuning for these workloads. Speaker Bio Blair Bethwaite: Blair has worked in distributed computing at Monash University for 10 years, with OpenStack for half of that. Having served as team lead, architect, administrator, user, researcher, and occasional hacker, Blair’s unique perspective as a science power-user, developer, and system architect has helped guide the evolution of the research computing engine central to Monash’s 21st Century Microscope. Lance Wilson: Lance is a mechanical engineer, who has been making tools to break things for the last 20 years. His career has moved through a number of engineering subdisciplines from manufacturing to bioengineering. Now he supports the national characterisation research community in Melbourne, Australia using OpenStack to create HPC systems solving problems too large for your laptop.

Deploying a 3 tier application using dockerparth2094

The document summarizes deploying a 3-tier application using Docker on OpenStack. It describes building Docker containers for the database and web server tiers of the application to ensure independence from the underlying architecture. The application is finally deployed on an OpenStack virtual machine. Docker containers package the application and all its dependencies, guaranteeing it will run the same regardless of the environment. OpenStack is an open-source cloud computing platform that provides processing, storage and networking resources across a data center. Docker can be integrated with OpenStack Nova as a hypervisor or orchestrated with OpenStack Heat to create containers and deploy applications on top of them.

Dell/EMC Technical Validation of BlueData EPIC with IsilonGreg Kirchoff

The BlueData EPIC™ (Elastic Private Instant Clusters) software platform solves the infrastructure challenges and limitations that can slow down and stall Big Data deployments. With EPIC software, you can spin up Hadoop or Spark clusters – with the data and analytical tools that your data scientists need – in minutes rather than months. Leveraging the power of containers and the performance of bare-metal, EPIC delivers speed, agility, and cost-efficiency for Big Data infrastructure. It works with all of the major Apache Hadoop distributions as well as Apache Spark. It integrates with each of the leading analytical applications, so your data scientists can use the tools they prefer. You can run it with any shared storage environment, so you don’t have to move your EMC Isilon Scale-out Storage Solutions for Hadoop combine a powerful yet simple and highly efficient storage platform with native Hadoop integration that allows you to accelerate analytics, gain new flexibility, and avoid the costs of a separate Hadoop infrastructure. BlueData EPIC Software combined with EMC Isilon shared storage provides a comprehensive solution for compute + storage. BlueData and Isilon share several joint customers and opportunities at leading financial services, advanced research laboratories, healthcare and media/communication organizations. This paper describes the process of validating Hadoop applications running in virtual clusters on the EPIC platform with data stored on the EMC Isilon storage device using either NFS or HDFS data access protocols

BlueData EPIC 2.0 OverviewBlueData, Inc.

This presentation provides an overview of what’s new in the 2.0 release of the BlueData EPIC software platform. BlueData’s EPIC software platform solves the infrastructure challenges and limitations that can slow down and stall on-premises Big Data deployments. With BlueData, you can spin up Hadoop or Spark clusters in minutes rather than months – with the data and analytical tools that your data scientists need. The BlueData EPIC 2.0 release leverages Docker containers to simplify Big Data clusters, supports Apache Zeppelin notebooks and other new functionality for Apache Spark, and includes an enhanced App Store that provides one-click access to Big Data distributions and analytics tools. Learn more about BlueData at http://www.bluedata.com

Hybrid cloud federationopenstackindia

This presentation proposes a HybridStack architecture that allows OpenStack workloads to burst across private and public clouds. It introduces the concept of a "pseudo-child cell" to interface OpenStack with external cloud providers like Amazon Web Services. Key features include workload migration policies, scalability across deployments, and high availability. The implemented architecture demonstrates bursting to AWS using a virtual private cloud integration. Future work includes integrating newer OpenStack services and enabling bursting between OpenStack clouds.

Kubernetes Visualization-and-Monitoring-using-Weave-scopeOpenEBS

12.07.2017 Docker Meetup - POSTGRE SQL ON KUBERNETESZalando adtech lab

Trilio for Red Hat OpenStack: The Missing Link for Cloud-Native Data ProtectionDevOps.com

Red Hat OpenStack is seeing accelerated adoption globally by financial institutions, managed service providers, NFV operators, enterprises and even small-to-medium businesses. OpenStack enables organizations of all sizes to build, manage and operate an open cloud infrastructure, and to realize operational and cost benefits by serving both cloud-native stateless apps and traditional, stateful workloads on the same cloud infrastructure. However, during their digital transformation journey, many organizations realize they are missing a way to satisfy production-grade requirements for backup and recovery. In this webinar, Sean Cohen and Boaz Michaely will discuss Trilio’s role in the cloud-native backup and recovery space, and highlight the solution's unique design features and benefits, including how Trilio enables admins and tenants to: Restore entire workloads into production after a disaster. Recover files, folders and data records after accidental damage or loss. Migrate workloads across OpenStack clouds and distributions. Quickly roll back from a problematic upgrade. Easily manage copy schedule and policies.

BACD PresentationBrian Bullard

This document discusses how 5th Planet Games, a game development company, has used CloudStack and Amazon EC2 to manage their growing infrastructure needs as their employee count and game offerings expanded over time. It describes how they initially used a few EC2 servers and an RDS database, and now utilize 25 EC2 servers, 10 database servers, and CloudStack to manage resources across multiple zones. The document outlines benefits of CloudStack like automation, cost savings, and ability to tie resources across regions together. It shares how the company uses CloudStack and lessons learned in building out their private cloud.

Rails Usergroup Hamburg: HerokuRalph von der Heyden

Heroku is a cloud platform that allows companies to host their Rails applications without owning the physical infrastructure. Some key advantages are variable costs that scale with usage and high flexibility. Deployment can be challenging due to limitations on configurations and reliance on a third party. The Heroku stack provides automatic scaling through dynos and add-ons, and uses Git-based deployment where code is pushed to build application slugs. While not suitable for all large applications, Heroku is recommended for new simple projects due to its low cost and ease of use.

Introducing the Hub for Data OrchestrationAlluxio, Inc.

The Future of Cloud Software Defined Storage with Ceph: Andrew Hatfield, Red HatOpenStack

Audience: Intermediate About: Learn how cloud storage differs to traditional storage systems and how that delivers revolutionary benefits. Starting with an overview of how Ceph integrates tightly into OpenStack, you’ll see why 62% of OpenStack users choose Ceph, we’ll then take a peek into the very near future to see how rapidly Ceph is advancing and how you’ll be able to achieve all your childhood hopes and dreams in ways you never thought possible. Speaker Bio: Andrew Hatfield – Practice Lead–Cloud Storage and Big Data, Red Hat Andrew has over 20 years experience in the IT industry across APAC, specialising in Databases, Directory Systems, Groupware, Virtualisation and Storage for Enterprise and Government organisations. When not helping customers slash costs and increase agility by moving to the software-defined storage future, he’s enjoying the subtle tones of Islay Whisky and shredding pow pow on the world’s best snowboard resorts. OpenStack Australia Day - Sydney 2016 https://events.aptira.com/openstack-australia-day-sydney-2016/

StorageArchitecturesForCloudVDIVinay Rao

Traditional data center architectures connected compute farms to storage via SANs and NAS filers, but these do not scale well for VDI environments. Solutions include using SSDs for golden images and efficient snapshotting/cloning, abandoning SANs/NASs in favor of distributed file systems, and new architectures like Virsto One and Nutanix that do not require traditional storage and improve performance and scalability.

Deep Learning and Gene Computing Acceleration with Alluxio in KubernetesAlluxio, Inc.

Eric Li, Senior Architect of Alibaba Cloud, presented on using Alluxio on Kubernetes. He discussed: 1. The challenges of deploying Alluxio on Kubernetes, including how to deploy it in a Kubernetes-native way, how applications can access data without changes, and how to achieve best Alluxio performance. 2. Optimizations made to Alluxio including a Helm chart for one-click installation, optimizations to the OSS SDK for data loading speed, and using fuse and short-circuiting for performance. 3. Best practices for using Alluxio on Kubernetes for different workloads like deep learning and genomic computing.

Containerized Storage for Containers- Kubernetes LA Meetup , July 2017OpenEBS

The Containerized Storage for Containers(OpenEBS) slides were presented in the Kubernetes Meetup , LA on July 18th , 2017. OpenEBS delivers storage and storage services to containerized environments. OpenEBS allows stateful workloads to be managed more like stateless containers. OpenEBS storage services include: per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance. To know more about OpenEBS visit our website www.openebs.io

Modernizing Your Data Platform for Analytics and AI in the Hybrid Cloud EraAlluxio, Inc.

This document discusses modernizing a data platform for analytics and AI across single, hybrid, or multi-cloud environments using Alluxio. It describes Alluxio's key features like data locality, metadata locality, asynchronous data operations, and policy-driven data management that enable consistent performance, portability, and cost savings. Examples are provided of how Alluxio can be used to transition from on-premises HDFS to object storage to hybrid cloud and multi-cloud configurations.

IXcloud, Global 1st Cloud OS, Openstack CloudKINX

This document summarizes the services provided by IXcloud, an OpenStack-based cloud service provider in Korea. It offers compute, storage, networking and other cloud services. For storage, it provides both SSD-based "Supreme" volumes and SATA-based "Standard" volumes. It allows customers to create customized cloud environments meeting their needs. IXcloud also helps customers build hybrid cloud solutions combining its cloud services with on-premise private clouds and physical servers.

Cloud InfrastructureKamruddin Nur

This document is a syllabus for a course on cloud computing platforms taught by Dr. Kamruddin Nur at Stamford University Bangladesh in October 2017. It discusses several open-source platforms for building private and hybrid clouds, including Eucalyptus, Nimbus, OpenStack, and OpenNebula. For each platform, it provides an overview of its components, architecture, and installation instructions. The document also lists reference materials and concludes by thanking the audience.

Complex Analytics with NoSQL Data Store in Real TimeNati Shalom

NOSQL are often limited in the type of queries that they can support due to the distributed nature of the data. In this session we would learn patterns on how we can overcome this limitation and combine multiple query semantics with NoSQL based engines. We will demonstrate specifically a combination of key/value, SQL like, Document model and Graph based queries as well as more advanced topic such as handling partial update and query through projection. We will also demonstrate how we can create a meshaup between those API's i.e. write fast through Key/Value API and execute complex queries on that same data through SQL query. - See more at: http://nosql2014.dataversity.net/sessionPop.cfm?confid=81&proposalid=6335#sthash.PNSZi5TJ.dpuf

Petascale Storage -- Do It Yourself!Tim Lossen

Distributed Shared Memory SystemsAnkit Gupta

Distributed shared memory (DSM) systems implement a shared memory model in distributed systems without physical shared memory. DSM provides a virtual shared address space between nodes. It overcomes high communication costs by moving data to the location of access. DSM research aims to build less expensive and larger parallel machines while eliminating programming difficulties of other architectures. DSM models include object, variable, page, and hardware-based approaches. Key issues in developing DSM involve granularity, memory coherence, replacement strategies, and avoiding thrashing. Common DSM algorithms are central server, migration, read replication, and full replication models.

More Related Content

What's hot (20)

[OpenStack Day in Korea 2015] Track 2-6 - Apache Tajo on SwiftOpenStack Korea Community

Building a GPU-enabled OpenStack Cloud for HPC - Blair Bethwaite, Monash Univ...OpenStack

Deploying a 3 tier application using dockerparth2094

Dell/EMC Technical Validation of BlueData EPIC with IsilonGreg Kirchoff

BlueData EPIC 2.0 OverviewBlueData, Inc.

Hybrid cloud federationopenstackindia

Kubernetes Visualization-and-Monitoring-using-Weave-scopeOpenEBS

12.07.2017 Docker Meetup - POSTGRE SQL ON KUBERNETESZalando adtech lab

Trilio for Red Hat OpenStack: The Missing Link for Cloud-Native Data ProtectionDevOps.com

BACD PresentationBrian Bullard

Rails Usergroup Hamburg: HerokuRalph von der Heyden

Introducing the Hub for Data OrchestrationAlluxio, Inc.

The Future of Cloud Software Defined Storage with Ceph: Andrew Hatfield, Red HatOpenStack

StorageArchitecturesForCloudVDIVinay Rao

Deep Learning and Gene Computing Acceleration with Alluxio in KubernetesAlluxio, Inc.

Containerized Storage for Containers- Kubernetes LA Meetup , July 2017OpenEBS

Modernizing Your Data Platform for Analytics and AI in the Hybrid Cloud EraAlluxio, Inc.

IXcloud, Global 1st Cloud OS, Openstack CloudKINX

Cloud InfrastructureKamruddin Nur

Complex Analytics with NoSQL Data Store in Real TimeNati Shalom

[OpenStack Day in Korea 2015] Track 2-6 - Apache Tajo on SwiftOpenStack Korea Community

Building a GPU-enabled OpenStack Cloud for HPC - Blair Bethwaite, Monash Univ...OpenStack

Deploying a 3 tier application using dockerparth2094

Dell/EMC Technical Validation of BlueData EPIC with IsilonGreg Kirchoff

BlueData EPIC 2.0 OverviewBlueData, Inc.

Hybrid cloud federationopenstackindia

Kubernetes Visualization-and-Monitoring-using-Weave-scopeOpenEBS

12.07.2017 Docker Meetup - POSTGRE SQL ON KUBERNETESZalando adtech lab

Trilio for Red Hat OpenStack: The Missing Link for Cloud-Native Data ProtectionDevOps.com

BACD PresentationBrian Bullard

Rails Usergroup Hamburg: HerokuRalph von der Heyden

Introducing the Hub for Data OrchestrationAlluxio, Inc.

The Future of Cloud Software Defined Storage with Ceph: Andrew Hatfield, Red HatOpenStack

StorageArchitecturesForCloudVDIVinay Rao

Deep Learning and Gene Computing Acceleration with Alluxio in KubernetesAlluxio, Inc.

Containerized Storage for Containers- Kubernetes LA Meetup , July 2017OpenEBS

Modernizing Your Data Platform for Analytics and AI in the Hybrid Cloud EraAlluxio, Inc.

IXcloud, Global 1st Cloud OS, Openstack CloudKINX

Cloud InfrastructureKamruddin Nur

Complex Analytics with NoSQL Data Store in Real TimeNati Shalom

Viewers also liked (20)

Petascale Storage -- Do It Yourself!Tim Lossen

Distributed Shared Memory SystemsAnkit Gupta

vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28CloudStack - Open Source Cloud Computing Project

GlusterFS is an open source scale-out NAS solution. The software is a powerful and flexible solution that simplifies the task of managing unstructured file data whether you have a few terabytes of storage or multiple petabytes. It’s no secret that unstructured data is growing like crazy, Gluster provides a solutions that scales capacity and performance as you need it and is an ideal fit for an IT environment that is increasingly virtualized and moving to the cloud. There are two key ways that GlusterFS is beneficial for cloud builders: 1. Storage layer for VMs. If you're deploying Xen or KVM VMs on a private cloud, storing them on GlusterFS gives you the ability to migrate to different hypervisors, suspend and resume quickly - even on another hypervisor, scale out far beyond what other filesystems will allow, and utilize N-way replication for DR and HA 2. Unified storage layer for applications. With GlusterFS 3.3, you will be able to access your application data stores from an object (S3, Swift-style) interface, as well as a traditional POSIX-compatible NAS interface. This unified approach gives developers and admins the ability to access the same data store using a variety of different methods. In this session, attendees will learn steps for deployment and some common use cases. Speaker Bio John Mark is an experienced veteran of all things open source and a self-described agitprop, agitator and advocate for those who volunteer countless, unpaid hours for a particular project or community. He first fell down the slippery slope of open source as a web developer at VA Linux Systems and eventually switched to the community team, beginning a career that has now lasted over ten years. Along the way, John Mark made stops at young, up-and-coming startups, such as Groundwork, Hyperic and then Gluster (later acquired by Red Hat). In between, there was a brief interlude at IDG World Expo, where he was the conference director for LinuxWorld, GridWorld and OSBC. His advice for companies who want to "do community" is to trust your community and give them the space to "just try s***." John Mark loves to perform community karaoke, and is available for weddings, funerals and Bar/Bat Mitzvahs

Software Defined storageKirillos Akram

GlusterFS is an open-source distributed file system that aggregates various storage servers over a network into one large parallel file system. It does not require a metadata server, ensuring better performance, linear scalability, and reliability compared to traditional distributed file systems. Red Hat Gluster Storage provides a scalable, reliable data management platform using GlusterFS to streamline file access across physical, virtual, and cloud environments. It supports various volume types including distributed replicated volumes which replicate data across multiple servers for high availability.

Scale out backups-with_bareos_and_glusterGluster.org

This document discusses integrating Bareos backups with the Gluster distributed file system for scalable backups. It begins with an agenda that covers the Gluster integration in Bareos, an introduction to GlusterFS, a quick start guide, an example configuration and demo, and future plans. It then provides more details on GlusterFS architecture including concepts like bricks, volumes, peers and site replication. The remainder of the document outlines quick start instructions for setting up Gluster and configuring Bareos to use the Gluster backend for scalable backups across multiple servers.

Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Gluster.org

This document discusses integrating the open source GlusterFS distributed file system with QEMU virtualization and the oVirt virtualization management platform. It provides an overview of GlusterFS and how it can be used as a virtual machine image store. It describes integrating GlusterFS with QEMU using different methods and shows performance results. It then outlines the architecture and features for managing GlusterFS volumes from within oVirt, including a REST API and upcoming integration plans. The goal is to provide an open alternative for large scale centralized management of server and desktop virtualization.

Gluster Storage Platform Installation GuideGlusterFS

Award winning scale-up and scale-out storage for XenGlusterFS

This webinar discusses the Gluster Virtual Storage Appliance for Xen which packages GlusterFS in a virtual machine container optimized for ease of use with little to no configuration required. The Virtual Appliance seamlessly integrates with existing virtualization environments such as Citrix Xen, allowing you to deploy virtual storage the same way you deploy virtual machines. Deploy on premise to create a private cloud using any certified Xen server hardware platforms and certified storage: JBOD, DAS, or SAN.

How to Install Gluster Storage PlatformGlusterFS

Introduction to GlusterFS Webinar - September 2011GlusterFS

SERC, IISc CRAY PetaFLOPS SystemAdarsh Patil

The Supercomputer Education and Research Center (SERC) at the Indian Institute of Science, Bangalore (IISc) procured the fastest supercomuter in India and 50-60th fastest in the world. These are some images in setting up the machine and its supporting datacenter requirement More details here: http://adarshpatil.in/timewarp/blog/iisc-serc-cray-supercomputer-sahasrat.html

CephFS update February 2016John Spray

The document provides an overview and update on CephFS, the distributed file system implemented in Ceph. Key points include new capabilities in the Jewel release like improved scrub/repair functionality to handle metadata damage, fine-grained authorization using new MDS auth caps, and experimental support for multiple CephFS filesystems backed by a single RADOS storage cluster. It also discusses ongoing work integrating CephFS with OpenStack Manila for shared file system provisioning and new tools like CephFSVolumeClient.

Glusterfs and openstackopenstackindia

GlusterFS is an open-source networked file system that provides scalable, distributed storage for OpenStack. It aggregates disk storage from multiple servers into a single global namespace that is accessible from anywhere. GlusterFS uses a distributed hashing technique to spread files across servers and can be configured for replication or erasure coding for redundancy. It integrates with OpenStack components like Swift, Cinder, and Glance to provide block, object, and file-level storage services for virtual machines. Future integration plans include a Cinder-like file service (FaaS) and support for Windows and NFS shares.

Future of cloud storageGlusterFS

Smb gluster devmar2013Gluster.org

The document discusses integrating Samba with Gluster and FUSE. It notes that Samba expects POSIX behaviors that Gluster can provide, like cache coherency and POSIX locking support. FUSE presents a problem as it locks away Gluster internals. The document proposes that the Samba VFS layer could allow Gluster features to be selectively enabled or disabled per access method. Building a Samba VFS module for Gluster was discussed but example code was omitted.

Lisa 2015-gluster fs-introductionGluster.org

This document summarizes GlusterFS, an open-source scale-out network filesystem. It discusses GlusterFS concepts like servers, trusted storage pools, bricks and volumes. It describes the distributed, replicated and dispersed volume types. Additional features like geo-replication, snapshots, quotas and data tiering are covered. The document provides an overview of GlusterFS architecture, components like translators and processes. It also discusses performance considerations and accessing volumes via FUSE, NFS and SMB protocols.

Scale out backups-with_bareos_and_glusterGluster.org

Red Hat Storage - Introduction to GlusterFSGlusterFS

Red Hat Storage introduces GlusterFS, an open source scale-out file system. GlusterFS provides scalable, affordable storage using commodity hardware. It allows linearly scaling performance and capacity by adding servers. GlusterFS has a global namespace and supports various protocols, enabling flexible deployment across private and public clouds. Many enterprises rely on GlusterFS for applications, virtual machines, Hadoop, and hybrid cloud solutions.

Gluster technical overviewGluster.org

Gluster is a software defined distributed storage system. It uses translators to implement storage semantics like distribution and replication across server bricks. Clients can mount volumes using FUSE or access them via NFS or REST APIs. Translators are implemented as shared libraries and define fop methods and callbacks to handle storage operations in a distributed manner across the cluster. Errors are handled by unwinding the stack and returning failure codes to clients.

Dustin Black - Red Hat Storage Server Administration Deep DiveGluster.org

Petascale Storage -- Do It Yourself!Tim Lossen

Distributed Shared Memory SystemsAnkit Gupta

vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28CloudStack - Open Source Cloud Computing Project

Software Defined storageKirillos Akram

Scale out backups-with_bareos_and_glusterGluster.org

Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Gluster.org

Gluster Storage Platform Installation GuideGlusterFS

Award winning scale-up and scale-out storage for XenGlusterFS

How to Install Gluster Storage PlatformGlusterFS

Introduction to GlusterFS Webinar - September 2011GlusterFS

SERC, IISc CRAY PetaFLOPS SystemAdarsh Patil

CephFS update February 2016John Spray

Glusterfs and openstackopenstackindia

Future of cloud storageGlusterFS

Smb gluster devmar2013Gluster.org

Lisa 2015-gluster fs-introductionGluster.org

Scale out backups-with_bareos_and_glusterGluster.org

Red Hat Storage - Introduction to GlusterFSGlusterFS

Gluster technical overviewGluster.org

Dustin Black - Red Hat Storage Server Administration Deep DiveGluster.org

Similar to Petascale Cloud Storage with GlusterFS (20)

Gluster open stack dev summit 042011Open Stack

This document describes Petascale Cloud Filesystem, a distributed file system designed by Gluster for large-scale cloud storage. It discusses Gluster's architecture advantages like being software-only, fully distributed with no single point of failure, and able to elastically scale out storage. The document also provides examples of Gluster deployments at organizations like Partners Healthcare, Pandora, and Cincinnati Bell Technology Solutions to provide centralized storage services and support private and public cloud environments.

Webinar Sept 22: Gluster Partners with Redapt to Deliver Scale-Out NAS StorageGlusterFS

Gluster has partnered with Redapt, Inc., an innovative data center architecture and infrastructure solutions provider, to integrate GlusterFS with hardware providing customers with highly-scalable NAS storage technology for on-premise, virtual and cloud environments. Gluster's storage technology enables Redapt to offer a comprehensive, cost-effective storage solution delivering the scalability, performance and reliability that companies need to effectively run their data centers. This webinar will provide an overview of the partnership, benefits of the joint solution, and include use cases of how customers today are deploying the joint solution. .

Cloud Strategies for a modern hybrid datacenter - Dec 2015Miguel Pérez Colino

This document discusses strategies for building a modern hybrid data center using Red Hat technologies. It describes Red Hat Satellite for systems management, Red Hat Enterprise Virtualization for virtualization, Red Hat CloudForms for cloud management, OpenStack for private IaaS clouds, OpenShift for containers and PaaS, and Red Hat Cloud Suite for hybrid cloud solutions. Key capabilities and features of these technologies are summarized. The document promotes using these open source solutions to improve IT efficiency, business agility and developer productivity in hybrid data center environments.

Gluster Webinar May 25: Whats New in GlusterFS 3.2GlusterFS

GlusterFS Architecture - June 30, 2011 MeetupGlusterFS

The document discusses GlusterFS, an open source distributed file system. It provides details about GlusterFS architecture, which uses a userspace filesystem design running on top of FUSE. It also summarizes GlusterFS capabilities like elastic scaling, high availability through replication, and support for various volume types including distribute, replicate, and stripe. Benchmark results show GlusterFS achieving high performance with 64 servers and 220 clients connected over InfiniBand.

V fabricoverveiw telkomAbdul Zaelani

This document summarizes VMware's Cloud Application Platform and its components. It discusses how VMware focuses on re-thinking end-user computing, modernizing application development, and evolving core infrastructure. It also outlines how vFabric helps build, run, and scale applications in the cloud through frameworks, services, and infrastructure components. Finally, it introduces Cloud Foundry as a platform as a service for deploying and scaling applications in the cloud era.

OpenNebula Conf 2014 | Cloud Automation for OpenNebula by Kishorekumar Neelam...NETWAYS

Kishore works with the engineering team in building the open source product with a future focussed cloud technical strategy for “Megam – Cloud Automation Platform “http://gomegam.com”. In his prior incarnation Kishore has worked as an Architect in complex system integration projects for Airport systems with high availability. Kishore has avid experience in architecting large scale build and packaging tools for mainframe platform integrated via thin clients and eclipse IDE.

OpenNebulaConf 2014 - Cloud Automation for OpenNebula - Kishorekumar NeelamegamOpenNebula Project

This document discusses using Megam and Opennebula to deploy applications to cloud environments in a flexible and portable way. Megam allows deploying applications to any public or private cloud, provides automated scaling, and avoids vendor lock-in. The document outlines Megam's features like deployment recipes, monitoring, and integration with development tools. It also discusses Megam's support for Docker containers, including a visual designer and "Cloud in a Box" for deploying private clouds.

Gestione gerarchica dei dati con SUSE Enterprise Storage e HPE DMFSUSE Italy

In-Ceph-tion: Deploying a Ceph cluster on DreamComputePatrick McGarry

This document discusses deploying a Ceph cluster on DreamCompute, an OpenStack-powered cloud computing service from DreamHost. It begins with an overview of Ceph's scalability and uses for object, block, and file storage. The document then discusses DreamCompute's open source infrastructure and deploying Ceph using tools like Juju. It provides details on configuring the Ceph cluster by deploying MONs, OSDs, the RGW gateway, and MDS. It concludes by discussing next steps like geo-replication and erasure coding, and opportunities to get involved with the Ceph community.

Presentatie Cisco NetApp Proact over FlexPodProact Netherlands B.V.

The document discusses the FlexPod solution from Cisco, NetApp, and Proact. FlexPod provides a converged infrastructure that combines Cisco UCS servers and networking with NetApp storage. It offers scalability, efficiency, and unified management. Proact partners with Cisco and NetApp to provide FlexPod solutions that are validated, secure, and easily managed. FlexPods allow customers to reduce costs and complexity while improving agility over traditional siloed data center architectures.

Storage Requirements and Options for Running Spark on KubernetesDataWorks Summit

In a world of serverless computing users tend to be frugal when it comes to expenditure on compute, storage and other resources. Paying for the same when they aren’t in use becomes a significant factor. Offering Spark as service on cloud presents very unique challenges. Running Spark on Kubernetes presents a lot of challenges especially around storage and persistence. Spark workloads have very unique requirements of Storage for intermediate data, long time persistence, Share file system and requirements become very tight when it same need to be offered as a service for enterprise to mange GDPR and other compliance like ISO 27001 and HIPAA certifications. This talk covers challenges involved in providing Serverless Spark Clusters share the specific issues one can encounter when running large Kubernetes clusters in production especially covering the scenarios related to persistence. This talk will help people using Kubernetes or docker runtime in production and help them understand various storage options available and which is more suitable for running Spark workloads on Kubernetes and what more can be done

High Performance Cloud ComputingNephoScale

NephoScale offers a cloud platform called NephOS that aims to provide high performance infrastructure and tools to enable collaboration between development and operations teams based on DevOps principles. NephOS provides fast virtual machines, dedicated servers, object storage, and tools like CloudScript for template-based deployment and elastic networking. NephoScale argues this solution can help customers develop, deploy, and improve applications faster with greater predictability and efficiency than other cloud providers.

Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds CapacityAvere Systems

Slides from live webinar hosted on February 16, 2017. Deploying applications locally and bursting them to the cloud for compute may seem difficult, especially when working with high-performance, critical information. However, using cloudbursts to offset peaks in demand can bring big benefits and kudos from organizational leaders always looking to do more with less. After this short webinar, you’ll be ready to: - Explain what cloud bursting is and what workloads it is best for - Identify efficiencies in applying cloud bursting to high-performance applications - Understand how cloud computing services access your data and consume it during burst cycles - Share three real-world use cases of companies leveraging cloud bursting for measurable efficiencies - Have seen a demonstration of how it works Presenters will build an actionable framework in just thirty minutes and then take questions.

Spark volume requirements 2018Rachit Arora

This document discusses storage requirements for running Spark workloads on Kubernetes. It recommends using a distributed file system like HDFS or DBFS for distributed storage and emptyDir or NFS for local temp scratch space. Logs can be stored in emptyDir or pushed to object storage. Features that would improve Spark on Kubernetes include image volumes, flexible PV to PVC mappings, encrypted volumes, and clean deletion for compliance. The document provides an overview of Spark, Kubernetes benefits, and typical Spark deployments.

OpenStack Grizzly ReleaseOpenStack

Slides: Accelerating Queries on Cloud Data LakesDATAVERSITY

Using “zero-copy” hybrid bursting on remote data to solve data lake analytics capacity and performance problems. Data scientists want answers on demand. But in today’s enterprise architectures, the reality is that most data remains on-prem, despite the promise of cloud-based analytics. Moving all that data to the cloud has typically not been possible for many reasons including cost, latency, and technical difficulty. So, what if there was a technology that would connect these on-prem environments to any major cloud platform, enabling high-powered computing without the need to move massive amounts of data? Join us for this webinar where Alex Ma of Alluxio, an open-source data orchestration platform, will discuss how a data orchestration approach offers a solution for connecting traditional on-prem data centers and cloud data lakes with other clouds and data centers. With Alluxio’s “zero-copy” burst solution, companies can bridge remote data centers and data lakes with computing frameworks in other locations, enabling them to offload, compute, and leverage the flexibility, scalability, and power of the cloud for their remote data.

Gluster Webinar: Introduction to GlusterFS v3.3GlusterFS

Looking for a high performance, scale-out NAS file system? Or are you a new user of GlusterFS and want to learn more? This webinar includes an introduction and review of the GlusterFS architecture and key features. Learn how GlusterFS is deployed in the datacenter, in the cloud, or between the two. We’ll also cover a brief update on GlusterFS v3.3 which is currently in beta. On the agenda: *Brief intro to Gluster’s History *Gluster Architecture Design Goals *Key Technical Differentiators *Gluster Elastic Hashing Algorithm *Deployment scenarios *Use Cases

Am 02 osac_kt_swiftOpenCity Community

KT ucloud storage provides an OpenStack Swift-based object storage service. Some key points: - OpenStack Swift allows for redundant, scalable storage across standardized servers capable of storing petabytes of data. It ensures data replication and integrity. - KT customized Swift for their use, adding middleware for SSL performance, integrating their single sign-on system, and enabling real-time usage reporting. - To commercialize Swift, KT deployed it based on commodity servers and disks across 5 zones with 3 replications for redundancy, using automated deployment with Chef. - While Swift is stable, deploying it on hardware can introduce problems. KT also found it difficult to increase revenue as object storage is a new concept for

KT ucloud storage, by Jaesuk AhnHui Cheng

KT ucloud storage provides an OpenStack Swift-based object storage service. Some key points: - OpenStack Swift allows for redundant, scalable storage across standardized servers and can store petabytes of data. It provides reliability through data replication and a distributed architecture with no single point of failure. - KT customized Swift for their commercial deployment, adding features like SSL performance tuning, APIs for management, and real-time usage reporting. They deployed it based on a reference architecture with multiple zones and replications. - While OpenStack Swift is stable, deploying it on hardware can introduce problems. KT had to address issues in gathering customer usage data and analyzing patterns to focus improvements. They also found it challenging to increase revenue as

Gluster open stack dev summit 042011Open Stack

Webinar Sept 22: Gluster Partners with Redapt to Deliver Scale-Out NAS StorageGlusterFS

Cloud Strategies for a modern hybrid datacenter - Dec 2015Miguel Pérez Colino

Gluster Webinar May 25: Whats New in GlusterFS 3.2GlusterFS

GlusterFS Architecture - June 30, 2011 MeetupGlusterFS

V fabricoverveiw telkomAbdul Zaelani

OpenNebula Conf 2014 | Cloud Automation for OpenNebula by Kishorekumar Neelam...NETWAYS

OpenNebulaConf 2014 - Cloud Automation for OpenNebula - Kishorekumar NeelamegamOpenNebula Project

Gestione gerarchica dei dati con SUSE Enterprise Storage e HPE DMFSUSE Italy

In-Ceph-tion: Deploying a Ceph cluster on DreamComputePatrick McGarry

Presentatie Cisco NetApp Proact over FlexPodProact Netherlands B.V.

Storage Requirements and Options for Running Spark on KubernetesDataWorks Summit

High Performance Cloud ComputingNephoScale

Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds CapacityAvere Systems

Spark volume requirements 2018Rachit Arora

OpenStack Grizzly ReleaseOpenStack

Slides: Accelerating Queries on Cloud Data LakesDATAVERSITY

Gluster Webinar: Introduction to GlusterFS v3.3GlusterFS

Am 02 osac_kt_swiftOpenCity Community

KT ucloud storage, by Jaesuk AhnHui Cheng

More from The Linux Foundation (20)

ELC2019: Static Partitioning Made SimpleThe Linux Foundation

Static partitioning is used to split an embedded system into multiple domains, each of them having access only to a portion of the hardware on the SoC. It is key to enable mixed-criticality scenarios, where a critical application, often based on a small RTOS, runs alongside a larger non-critical app, typically based on Linux. The two domains cannot interfere with each other. This talk will explain how to use Xen for static partitioning. It will introduce dom0-less, a new Xen feature written for the purpose. Dom0-less allows multiple VMs to start at boot time directly from the Xen hypervisor, decreasing boot times drastically. It makes it very easy to partition the system without virtualization overhead. Dom0 becomes unnecessary. This presentation will go into details on how to setup a Xen dom0-less system. It will show configuration examples and explain device assignment. The talk will discuss its implications for latency-sensitive and safety-critical environments.

XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...The Linux Foundation

Daniel Smith discusses TrenchBoot, a project aiming to establish a unified approach to harnessing boot integrity technologies across open source platforms. TrenchBoot will enable establishing hardware-rooted integrity during platform boot (first launch inspection), runtime (runtime inspection), and other states. For runtime inspection, TrenchBoot will develop a way to securely re-establish the integrity of Xen at any time without rebooting by dynamically launching an integrity kernel to inspect and verify Xen. The talk outlines the initial and future work of TrenchBoot to integrate these capabilities with Linux, Xen and other open source projects.

XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...The Linux Foundation

Artem will briefly cover what has been done since the first talk on Xen in Automotive domain back in 2013, what is going on now and what is still missing for broad adaptation of Xen in vehicles. The following topics will be covered: Embedded/automotive features of Xen Collaboration with AGL and GENIVI organizations for standardization Efforts on Functional Safety compliance Artem will also go over typical automotive use scenarios for Xen which may not be the same as generic computing use of hypervisor.

XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...The Linux Foundation

XPDDS19 Keynote: Unikraft Weather ReportThe Linux Foundation

In recent years unikernels have shown immense performance potential (e.g., boot times of only a few ms, image sizes of only hundreds of KBs).The fundamental drawback of unikernels is that they require that applications be manually ported to the underlying minimalistic OS, needing both expert work and often considerable amount of time. The Unikraft project provides a unikernel code base and build system that significantly simplifies the building of unikernels. In addition to support for a number CPU architectures, languages and frameworks, Unikraft provides debugging and tracing features that are generally sorely missing from unikernel projects. In this talk we will talk about these features, show a set of preliminary performance numbers, and provide a roadmap for the project's future.

XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...The Linux Foundation

XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, XilinxThe Linux Foundation

This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional. Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings. The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.

XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...The Linux Foundation

As the number of contributions grow, reviewer bandwidth becomes a bottleneck; and maintainers are always asking for more help. However, ultimately maintainers must at least Ack every patch that goes in; so if you're not a maintainer, how can you contribute? Why should anyone care about your opinion? This talk will try to lay out some advice and guidelines for non-maintainers, for how they can do code review in a way which will effectively reduce the load on maintainers when they do come to review a patch.

XPDDS19: Memories of a VM Funk - Mihai Donțu, BitdefenderThe Linux Foundation

This talk is a follow-up to our Summit 2017 presentation in which we covered our plans for Intel VMFUNC and #VE, as well as related use-cases. This year, we will provide a report on what we have accomplished in Xen 4.12, and what remains to be addressed. We will also give a brief status update of VMI on AMD hardware. The session will end with some real-world numbers of the Hypervisor Introspection solution running on Citrix Hypervisor 8.0 with #VE enabled.

OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...The Linux Foundation

Safety certification is one of the essential requirements for software to be used in highly regulated industries. Besides technical and compliance issues (such as ISO 26262 vs IEC 611508) transitioning an existing project to become more easily safety certifiable requires significant changes to development practices within an open source project. In this session, we will lay out some challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the Xen Project has followed thus far and highlight lessons learned along the way. The talk will primarily focus on necessary process, tooling changes and community challenges that can prevent progress. We will be offering an in-depth review of how Xen Project is approaching this challenging goal and try to derive lessons for other projects and contributors.

OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...The Linux Foundation

This document summarizes a discussion around enabling functional safety certification for the Xen open source hypervisor project. Key points discussed include: - Establishing a split development model with open and closed parts to balance community needs and safety requirements. - Developing reference implementations and stacks supported by multiple vendors to demonstrate safety certification feasibility. - Creating plans and processes around requirements, documentation, verification testing, and tooling integration to begin filling gaps for certification. - Addressing challenges around funding, resources, expertise, and maintaining contributions to ensure any initial work is sustainable long-term. - Taking an iterative, agile approach to make early progress while further securing necessary funding and support from interested parties.

XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, CitrixThe Linux Foundation

2018 saw fundamental shifts in security boundaries which were previously taken for granted. A lot of work has been done in the past 2 years, and largely in secret under embargo, but there is plenty more work to be done to strengthen the existing mitigations and to try to recover some performance without reopening security holes. This talk will look at speculative execution sidechannels, the work which has already been done to mitigate the security holes, and future work which hopes to bring some improvements.

XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltdThe Linux Foundation

The Arm architecture provides a set of guidelines that any software should abide by when accessing the memory with MMU off and update page-tables. Failing to do so may result in getting TLB conflicts or breaking coherency. In a previous talk ("Keeping coherency on Arm"), we focused on updating safely the stage-2 (aka P2M) page-tables. This talk will focus on the boot code and Xen memory management. During this session, we will introduce some of the guidelines and when they should be used. We will also discuss how Xen boot sequence needs to be reworked to avoid breaking the guidelines.

XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...The Linux Foundation

For many years the QEMU codebase has contained PV backends for Xen guests, giving them paravirtual access to storage, network, keyboard, mouse, etc. however these backends have not been configurable as QEMU devices as their implementation did not fully adhere to the QEMU Object Model (QOM). Particularly the PV storage backend not using proper QOM devices, or qdevs, meant that the QEMU block layer needed to maintain legacy code that was cluttering up the source. This was causing push-back from the maintainers who did not want to accept any patches relating to that Xen backend until it was 'qdevified'. In this talk, I'll explain the modifications I made to QEMU to achieve 'qdevification' of the PV storage backend, how compatibility with the libxl toolstack was maintained, and what the next steps in both QEMU and libxl development should be.

XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&DThe Linux Foundation

PCI is a local computer bus for attaching hardware devices in a computer, and is the main peripheral bus on modern x86 systems. As such, having a proper way to emulate it is crucial for Xen to be able to expose both fully emulated devices or passthrough devices to guests. This talk will focus on the current status of PCI emulation in Xen, how and where it is used, what are its main limitations and future plans to improve it in order to be more robust and modular.

XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM SystemsThe Linux Foundation

XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...The Linux Foundation

Xen is a very powerful hypervisor with a talented and diverse developers community. Despite the fact it's almost everywhere (from the Cloud to the embedded world), it can be difficult to set up and manage as a system administrator. General purpose distros have Xen packages, but that's just a start in your Xen journey: you need some tooling and knowledge to have a working and scalable platform. XCP-ng was built to overcome those issues: by bringing Xen to the masses with a fully turnkey distro with Xen as its core. It's the logical sequel to the XCP project, with a community focus from the start. We'll see how it happened, what we did, and what's next. Finally, we'll see the impact of XCP-ng on the Xen Project.

XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...The Linux Foundation

XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...The Linux Foundation

High level toolstacks for server and cloud virtualization are very mature with large communities using and supporting them. Client virtualization is a much more niche community with unique requirements when compared to those found in the server space. In this talk, we’ll introduce a client virtualization toolstack for Xen (redctl) that we are using in Redfield, a new open-source client virtualization distribution that builds upon the work done by the greater virtualization and Linux communities. We will present a case for maturing libxl’s Go bindings and discuss what advantages Go has to offer for high level toolstacks, including in the server space.

XPDDS19: Core Scheduling in Xen - Jürgen Groß, SUSEThe Linux Foundation

Today Xen is scheduling guest virtual cpus on all available physical cpus independently from each other. Recent security issues on modern processors (e.g. L1TF) require to turn off hyperthreading for best security in order to avoid leaking information from one hyperthread to the other. One way to avoid having to turn off hyperthreading is to only ever schedule virtual cpus of the same guest on one physical core at the same time. This is called core scheduling. This presentation shows results from the effort to implement core scheduling in the Xen hypervisor. The basic modifications in Xen are presented and performance numbers with core scheduling active are shown.