SlideShare a Scribd company logo
keith @ nttpc.co.jp
GlusterFS’s Three Es

!  Elastic
!  Endurable
!  Easy-to-build/use/customize
Three types of storage basis
GlusterFS as File based storage

!  NFS version 3
! Gluster Native (FUSE)
!  CIFS (with Samba)
GlusterFS as Block based storage

! iSCSI (with iSCSI Enterprise Target)
  !  … unofficial way to use.
GlusterFS as Object storage

! RESTful interface
  !  c.f. NTTPC WebAPI “T4” since 2008
Why Object Storage?

          <- Petabytes of
          Unstructured data

          Massive archives
          <- that are active
Why Object Storage?
Installation
Installation on CentOS 5
#  tar xzf gluster-glusterfs-v3.3.0qa14-0-
   g4235f7a.tar.gz
#  mv gluster-glusterfs-14eb7f1 glusterfs-3git
#  cd glusterfs-3git
#  ./configure
#  cd ..
#  tar czf glusterfs-3git.tar.gz glusterfs-3git
#  rpmbuild -ta glusterfs-3git.tar.gz
Installation on CentOS 5
#  rpm -Uvh /usr/src/redhat/RPMS/x86_64/
   glusterfs-*
#  modprobe fuse
#  tar xzf gluster-object-storage-3.3beta2.tar.gz
#  cd UFO
#  ./centos_install.sh
Installation on CentOS 5
#    Enter external IP for Storage-Server: 10.1.0.0
#    Enter Super Admin key:********
#    Enter FileSystem (Press Enter for default Glusterfs):
#    Enter Object server port (Press Enter for default 6010):
#    Enter Container server port (Press Enter for default 6011):
#    Enter Account server port (Press Enter for default 6012):
#    Enter '1' for https '2' for 'http' (Recommended is https):1
#    Do you want to generate certificate/key for https(y/n): y
Preparation of GlusterFS
#  for i in `seq 1 4`; do gluster peer probe svr$i; done
#  gluster volume create auth svr1:/tmp/auth …
#  gluster volume start auth
#  gluster volume create data svr1:/tmp/data …
#  gluster volume start data
#  gluster-object-prep -K super_admin_key -A https://10.1.0.0:443/
   auth/
#  gluster-object-add-user -a -K super_admin_key -A https://
   10.1.0.0:443/auth/ data datauser datauser_password
#  gluster-object-list -K super_admin_key -A https://10.1.0.0:443/auth/
The finished construction
              http client


   GlusterFS services         Swift
account        proxy
container      object       memcached


               GlusterFS
What you can do
What you can do

!  Authentication
!  Account level operations
!  Container level operations
!  Object level operations
Authentication
!   Authentication and Returning
    !   Auth-token
    !   Storage-Url
Account level operations
!   Displaying Container Information
!   Displaying Account Metadata
Container level operations
!   Creating Container
!   Displaying Objects of a Container
!   Displaying Container Metadata Information
!   Deleting Container
!   Updating Container Metadata
!   Setting ACLs on Container
Object level operations
!   Creating or Updating Object
!   Copying Object
!   Displaying Object Information
!   Displaying Object Metadata
!   Updating Object Metadata
!   Deleting Object
DEMO
Thanks!

More Related Content

What's hot (20)

PDF
Nick Fisk - low latency Ceph
ShapeBlue
 
PDF
Ceph - A distributed storage system
Italo Santos
 
PPTX
APACHE KAFKA / Kafka Connect / Kafka Streams
Ketan Gote
 
PDF
SeaweedFS introduction
chrislusf
 
PDF
Deep Dive into the OPC UA / DDS Gateway Specification
Gerardo Pardo-Castellote
 
PDF
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
Altinity Ltd
 
PDF
Dragon: A Distributed Object Storage at Yahoo! JAPAN (WebDB Forum 2017)
Yahoo!デベロッパーネットワーク
 
PDF
Ceph Object Storage Performance Secrets and Ceph Data Lake Solution
Karan Singh
 
PDF
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
InfluxData
 
PDF
Batch Processing at Scale with Flink & Iceberg
Flink Forward
 
PPTX
Apache Ignite vs Alluxio: Memory Speed Big Data Analytics
DataWorks Summit
 
PDF
Apache Bigtop3.2 (仮)(Open Source Conference 2022 Online/Hiroshima 発表資料)
NTT DATA Technology & Innovation
 
PPTX
Cassandra Troubleshooting 3.0
J.B. Langston
 
PDF
並行実行制御の最適化手法
Sho Nakazono
 
PDF
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
StreamNative
 
PPTX
Snowflake_Template_light-2024-redesigned.pptx
Samuel168738
 
PDF
RocksDB Performance and Reliability Practices
Yoshinori Matsunobu
 
PDF
[B11] 基礎から知るSSD(いまさら聞けないSSDの基本) by Hironobu Asano
Insight Technology, Inc.
 
PPTX
Apache BigtopによるHadoopエコシステムのパッケージング(Open Source Conference 2021 Online/Osaka...
NTT DATA Technology & Innovation
 
PDF
Ceph issue 해결 사례
Open Source Consulting
 
Nick Fisk - low latency Ceph
ShapeBlue
 
Ceph - A distributed storage system
Italo Santos
 
APACHE KAFKA / Kafka Connect / Kafka Streams
Ketan Gote
 
SeaweedFS introduction
chrislusf
 
Deep Dive into the OPC UA / DDS Gateway Specification
Gerardo Pardo-Castellote
 
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
Altinity Ltd
 
Dragon: A Distributed Object Storage at Yahoo! JAPAN (WebDB Forum 2017)
Yahoo!デベロッパーネットワーク
 
Ceph Object Storage Performance Secrets and Ceph Data Lake Solution
Karan Singh
 
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
InfluxData
 
Batch Processing at Scale with Flink & Iceberg
Flink Forward
 
Apache Ignite vs Alluxio: Memory Speed Big Data Analytics
DataWorks Summit
 
Apache Bigtop3.2 (仮)(Open Source Conference 2022 Online/Hiroshima 発表資料)
NTT DATA Technology & Innovation
 
Cassandra Troubleshooting 3.0
J.B. Langston
 
並行実行制御の最適化手法
Sho Nakazono
 
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
StreamNative
 
Snowflake_Template_light-2024-redesigned.pptx
Samuel168738
 
RocksDB Performance and Reliability Practices
Yoshinori Matsunobu
 
[B11] 基礎から知るSSD(いまさら聞けないSSDの基本) by Hironobu Asano
Insight Technology, Inc.
 
Apache BigtopによるHadoopエコシステムのパッケージング(Open Source Conference 2021 Online/Osaka...
NTT DATA Technology & Innovation
 
Ceph issue 해결 사례
Open Source Consulting
 

Similar to GlusterFS As an Object Storage (20)

PDF
Swift Install Workshop - OpenStack Conference Spring 2012
Joe Arnold
 
PDF
Null Bachaav - May 07 Attack Monitoring workshop.
Prajal Kulkarni
 
PPTX
Make stateful apps in Kubernetes a no brainer with Pure Storage and GitOps
Weaveworks
 
PDF
Kafka and kafka connect
Touraj Ebrahimi
 
PDF
GlusterFS Update and OpenStack Integration
Etsuji Nakai
 
ODP
Accessing gluster ufo_-_eco_willson
Gluster.org
 
PDF
Embulk, an open-source plugin-based parallel bulk data loader
Sadayuki Furuhashi
 
ODP
GlusterFS and Openstack Storage
Deepak Shetty
 
PPTX
Installation Openstack Swift
ymtech
 
PPTX
Baylisa - Dive Into OpenStack
Jesse Andrews
 
PDF
ITB2019 NGINX Overview and Technical Aspects - Kevin Jones
Ortus Solutions, Corp
 
PDF
Build Your Own CaaS (Container as a Service)
HungWei Chiu
 
ODP
LinuxKit Swarm Nodes
Moby Project
 
PDF
Metal-k8s presentation by Julien Girardin @ Paris Kubernetes Meetup
Laure Vergeron
 
PDF
Open Source Backup Conference 2014: Workshop bareos introduction, by Philipp ...
NETWAYS
 
PPTX
Container & kubernetes
Ted Jung
 
PDF
Aeon mike guide transparent ssl filtering
Conrad Cruz
 
PDF
Aeon mike guide transparent ssl filtering (1)
Conrad Cruz
 
PDF
Jfrog artifactory as private docker registry
Vipin Mandale
 
PDF
One-Man Ops
Jos Boumans
 
Swift Install Workshop - OpenStack Conference Spring 2012
Joe Arnold
 
Null Bachaav - May 07 Attack Monitoring workshop.
Prajal Kulkarni
 
Make stateful apps in Kubernetes a no brainer with Pure Storage and GitOps
Weaveworks
 
Kafka and kafka connect
Touraj Ebrahimi
 
GlusterFS Update and OpenStack Integration
Etsuji Nakai
 
Accessing gluster ufo_-_eco_willson
Gluster.org
 
Embulk, an open-source plugin-based parallel bulk data loader
Sadayuki Furuhashi
 
GlusterFS and Openstack Storage
Deepak Shetty
 
Installation Openstack Swift
ymtech
 
Baylisa - Dive Into OpenStack
Jesse Andrews
 
ITB2019 NGINX Overview and Technical Aspects - Kevin Jones
Ortus Solutions, Corp
 
Build Your Own CaaS (Container as a Service)
HungWei Chiu
 
LinuxKit Swarm Nodes
Moby Project
 
Metal-k8s presentation by Julien Girardin @ Paris Kubernetes Meetup
Laure Vergeron
 
Open Source Backup Conference 2014: Workshop bareos introduction, by Philipp ...
NETWAYS
 
Container & kubernetes
Ted Jung
 
Aeon mike guide transparent ssl filtering
Conrad Cruz
 
Aeon mike guide transparent ssl filtering (1)
Conrad Cruz
 
Jfrog artifactory as private docker registry
Vipin Mandale
 
One-Man Ops
Jos Boumans
 
Ad

More from Keisuke Takahashi (15)

PDF
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Keisuke Takahashi
 
PDF
パーフェクト"Elixir情報収集"
Keisuke Takahashi
 
PDF
GlusterFS Masakari Talks
Keisuke Takahashi
 
PDF
GlusterFS Updates (and more) in 第六回クラウドストレージ研究会
Keisuke Takahashi
 
PDF
Trying and evaluating the new features of GlusterFS 3.5
Keisuke Takahashi
 
PDF
Gluster Cloud Night in Tokyo 2013 -- Tips for getting started
Keisuke Takahashi
 
PDF
Big Data入門に見せかけたFluentd入門
Keisuke Takahashi
 
PDF
Gluster in Japan 2012-2013
Keisuke Takahashi
 
PDF
Creating a shared storage service with GlusterFS
Keisuke Takahashi
 
PDF
GlusterFSとInfiniBandの小話
Keisuke Takahashi
 
PDF
GlusterFS 技術と動向 2of2
Keisuke Takahashi
 
PDF
GlusterFS 技術と動向 1of2
Keisuke Takahashi
 
PDF
最新技術動向 GlusterFS (仮想化DAY, Internet Week 2011)
Keisuke Takahashi
 
PDF
GlusterFS モジュール超概論
Keisuke Takahashi
 
PDF
GlusterFS座談会テクニカルセッション
Keisuke Takahashi
 
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Keisuke Takahashi
 
パーフェクト"Elixir情報収集"
Keisuke Takahashi
 
GlusterFS Masakari Talks
Keisuke Takahashi
 
GlusterFS Updates (and more) in 第六回クラウドストレージ研究会
Keisuke Takahashi
 
Trying and evaluating the new features of GlusterFS 3.5
Keisuke Takahashi
 
Gluster Cloud Night in Tokyo 2013 -- Tips for getting started
Keisuke Takahashi
 
Big Data入門に見せかけたFluentd入門
Keisuke Takahashi
 
Gluster in Japan 2012-2013
Keisuke Takahashi
 
Creating a shared storage service with GlusterFS
Keisuke Takahashi
 
GlusterFSとInfiniBandの小話
Keisuke Takahashi
 
GlusterFS 技術と動向 2of2
Keisuke Takahashi
 
GlusterFS 技術と動向 1of2
Keisuke Takahashi
 
最新技術動向 GlusterFS (仮想化DAY, Internet Week 2011)
Keisuke Takahashi
 
GlusterFS モジュール超概論
Keisuke Takahashi
 
GlusterFS座談会テクニカルセッション
Keisuke Takahashi
 
Ad

Recently uploaded (20)

PDF
Bitcoin+ Escalando sin concesiones - Parte 1
Fernando Paredes García
 
PDF
visibel.ai Company Profile – Real-Time AI Solution for CCTV
visibelaiproject
 
PDF
How a Code Plagiarism Checker Protects Originality in Programming
Code Quiry
 
PDF
Shuen Mei Parth Sharma Boost Productivity, Innovation and Efficiency wit...
AWS Chicago
 
PPTX
AVL ( audio, visuals or led ), technology.
Rajeshwri Panchal
 
PDF
OpenInfra ID 2025 - Are Containers Dying? Rethinking Isolation with MicroVMs.pdf
Muhammad Yuga Nugraha
 
PPTX
python advanced data structure dictionary with examples python advanced data ...
sprasanna11
 
PPTX
Simplifying End-to-End Apache CloudStack Deployment with a Web-Based Automati...
ShapeBlue
 
PDF
TrustArc Webinar - Navigating Data Privacy in LATAM: Laws, Trends, and Compli...
TrustArc
 
PDF
The Past, Present & Future of Kenya's Digital Transformation
Moses Kemibaro
 
PDF
Ampere Offers Energy-Efficient Future For AI And Cloud
ShapeBlue
 
PPTX
Earn Agentblazer Status with Slack Community Patna.pptx
SanjeetMishra29
 
PDF
Productivity Management Software | Workstatus
Lovely Baghel
 
PDF
Generative AI in Healthcare: Benefits, Use Cases & Challenges
Lily Clark
 
PDF
Rethinking Security Operations - Modern SOC.pdf
Haris Chughtai
 
PDF
Integrating IIoT with SCADA in Oil & Gas A Technical Perspective.pdf
Rejig Digital
 
PPTX
Extensions Framework (XaaS) - Enabling Orchestrate Anything
ShapeBlue
 
PDF
Lecture A - AI Workflows for Banking.pdf
Dr. LAM Yat-fai (林日辉)
 
PDF
HR agent at Mediq: Lessons learned on Agent Builder & Maestro by Tacstone Tec...
UiPathCommunity
 
PDF
Trading Volume Explained by CIFDAQ- Secret Of Market Trends
CIFDAQ
 
Bitcoin+ Escalando sin concesiones - Parte 1
Fernando Paredes García
 
visibel.ai Company Profile – Real-Time AI Solution for CCTV
visibelaiproject
 
How a Code Plagiarism Checker Protects Originality in Programming
Code Quiry
 
Shuen Mei Parth Sharma Boost Productivity, Innovation and Efficiency wit...
AWS Chicago
 
AVL ( audio, visuals or led ), technology.
Rajeshwri Panchal
 
OpenInfra ID 2025 - Are Containers Dying? Rethinking Isolation with MicroVMs.pdf
Muhammad Yuga Nugraha
 
python advanced data structure dictionary with examples python advanced data ...
sprasanna11
 
Simplifying End-to-End Apache CloudStack Deployment with a Web-Based Automati...
ShapeBlue
 
TrustArc Webinar - Navigating Data Privacy in LATAM: Laws, Trends, and Compli...
TrustArc
 
The Past, Present & Future of Kenya's Digital Transformation
Moses Kemibaro
 
Ampere Offers Energy-Efficient Future For AI And Cloud
ShapeBlue
 
Earn Agentblazer Status with Slack Community Patna.pptx
SanjeetMishra29
 
Productivity Management Software | Workstatus
Lovely Baghel
 
Generative AI in Healthcare: Benefits, Use Cases & Challenges
Lily Clark
 
Rethinking Security Operations - Modern SOC.pdf
Haris Chughtai
 
Integrating IIoT with SCADA in Oil & Gas A Technical Perspective.pdf
Rejig Digital
 
Extensions Framework (XaaS) - Enabling Orchestrate Anything
ShapeBlue
 
Lecture A - AI Workflows for Banking.pdf
Dr. LAM Yat-fai (林日辉)
 
HR agent at Mediq: Lessons learned on Agent Builder & Maestro by Tacstone Tec...
UiPathCommunity
 
Trading Volume Explained by CIFDAQ- Secret Of Market Trends
CIFDAQ
 

GlusterFS As an Object Storage

  • 2. GlusterFS’s Three Es !  Elastic !  Endurable !  Easy-to-build/use/customize
  • 3. Three types of storage basis
  • 4. GlusterFS as File based storage !  NFS version 3 ! Gluster Native (FUSE) !  CIFS (with Samba)
  • 5. GlusterFS as Block based storage ! iSCSI (with iSCSI Enterprise Target) !  … unofficial way to use.
  • 6. GlusterFS as Object storage ! RESTful interface !  c.f. NTTPC WebAPI “T4” since 2008
  • 7. Why Object Storage? <- Petabytes of Unstructured data Massive archives <- that are active
  • 10. Installation on CentOS 5 #  tar xzf gluster-glusterfs-v3.3.0qa14-0- g4235f7a.tar.gz #  mv gluster-glusterfs-14eb7f1 glusterfs-3git #  cd glusterfs-3git #  ./configure #  cd .. #  tar czf glusterfs-3git.tar.gz glusterfs-3git #  rpmbuild -ta glusterfs-3git.tar.gz
  • 11. Installation on CentOS 5 #  rpm -Uvh /usr/src/redhat/RPMS/x86_64/ glusterfs-* #  modprobe fuse #  tar xzf gluster-object-storage-3.3beta2.tar.gz #  cd UFO #  ./centos_install.sh
  • 12. Installation on CentOS 5 #  Enter external IP for Storage-Server: 10.1.0.0 #  Enter Super Admin key:******** #  Enter FileSystem (Press Enter for default Glusterfs): #  Enter Object server port (Press Enter for default 6010): #  Enter Container server port (Press Enter for default 6011): #  Enter Account server port (Press Enter for default 6012): #  Enter '1' for https '2' for 'http' (Recommended is https):1 #  Do you want to generate certificate/key for https(y/n): y
  • 13. Preparation of GlusterFS #  for i in `seq 1 4`; do gluster peer probe svr$i; done #  gluster volume create auth svr1:/tmp/auth … #  gluster volume start auth #  gluster volume create data svr1:/tmp/data … #  gluster volume start data #  gluster-object-prep -K super_admin_key -A https://10.1.0.0:443/ auth/ #  gluster-object-add-user -a -K super_admin_key -A https:// 10.1.0.0:443/auth/ data datauser datauser_password #  gluster-object-list -K super_admin_key -A https://10.1.0.0:443/auth/
  • 14. The finished construction http client GlusterFS services Swift account proxy container object memcached GlusterFS
  • 16. What you can do !  Authentication !  Account level operations !  Container level operations !  Object level operations
  • 17. Authentication !   Authentication and Returning !   Auth-token !   Storage-Url
  • 18. Account level operations !   Displaying Container Information !   Displaying Account Metadata
  • 19. Container level operations !   Creating Container !   Displaying Objects of a Container !   Displaying Container Metadata Information !   Deleting Container !   Updating Container Metadata !   Setting ACLs on Container
  • 20. Object level operations !   Creating or Updating Object !   Copying Object !   Displaying Object Information !   Displaying Object Metadata !   Updating Object Metadata !   Deleting Object
  • 21. DEMO