SlideShare a Scribd company logo
The evolution of Linux file system
Gang He
ghe@suse.com
2
Agenda
• Local file system (LFS)
• Cluster file system (CFS)
• Distributed file system (DFS)
Local file system (LFS)
4
File system overview
5
File system concepts
• file descriptor (user space)
• struct file, struct dentry, struct inode, struct
address_space (kernel space)
• struct super_block,
meta data, file data,
buffer/page cache
6
Ext2 → Ext3 → Ext4
• Ext2 (1993), inspired from UFS, first popular and
stable Linux file system, but design is plain.
- File systems are getting bigger, how to look up a entry under a big
directory, how to reduce fsck time after crash ...
• Ext3 (2001), add journal, hash-tree directory indexing,
etc.
- File systems are getting bigger, have to eliminate these limitations
- Various advanced file systems impact ext3 ...
• Ext4 (2008), 48-bit addressing space, no limited
directory entries, extents, multi-block allocation,
delayed block allocation, online defragmentation, 256
bytes inode, persistent preallocation, barrier, etc.
7
Nowadays
• Ext4, will continue to be maintained due to stability and
historical reasons
• XFS, robust and scalable, good performance for large
storage, will shine in handling big file area (e.g. virtual
machine image)
• BtrFS, new design (replace ext4), inspired from ZFS,
contains many features of enterprise file system. for
examples, copy on write, own internal RAID (manage
volume ), snapshot/clone support, dynamically grow
and shrink, SSD support, etc.
Cluster file system (CFS)
9
Why CFS
• Independent storage devices (e.g. SAN).
• High availability requirement.
• How to scale out file system in
CPU,
memory,
even network bandwidth.
10
CFS common points
• File system POSIX semantics.
• Shared disk.
• Distributed lock manager (DLM).
• Cluster manager stacks.
11
CFS future
• Scale out more nodes, provide higher aggregation IO
bandwidth.
• More high availability, support online fsck, online
deframentation, online expand/shrink.
• File system level snapshot, file level clone.
• Tiered storage, SSD support.
• Deduplication.
• ...
Distributed file system (DFS)
13
Background
• Costs, storage array, fabric switches, HBA card, etc
are expensive.
• Unified storage space, linear expansion, commodity
hardware.
• Driven by Internet industry (e.g. search, picture share,
big data, etc).
• Google file system appeared (2003).
14
GFS-like DFS (HDFS, MooseFS, KFS)
15
DFS common points
• Not strictly comply with File system POSIX semantics,
most implementations are based on user-space.
• Share nothing, meta-data/file data are stored
separately, meta-data access/file data access are
separated.
• Have own local file system, a file represents a logical
data block, a data block has several copy blocks.
• Meta-data server usually load all meta data into
memory at start-up, writing logs records incremental
changes, then flushing memory to disk/merging log
and previous meta-data file gets a new checkpoint of
meta-data.
• Other algorithms: heartbeat algorithm, rack-aware,
block allocation policy, file lock management, etc.
16
Scale out
• Meta-data cluster server, e.g. GFS2, Ceph.
• Fully symmetric, no central meta-data server, e.g.
GlusterFS.
• Improved cluster management mechanism.
hearbeat/corosync → zookeeper cluster
• IO Flow Control, reduce meta-data server
dependence, costs control (ECC), etc.
VS.
17
Current trends
• Linear scale out.
• CompuStor hyper-converged systems.
• Flash technology utilization.
• High-speed network support.
• Application-Aware (e.g. VM image).
• Deduplication/Compression/Snapshot/Clone.
• Object/Block/File unified storage.
18
Ad

More Related Content

What's hot (20)

4. linux file systems
4. linux file systems4. linux file systems
4. linux file systems
Marian Marinov
 
Lesson 2 Understanding Linux File System
Lesson 2 Understanding Linux File SystemLesson 2 Understanding Linux File System
Lesson 2 Understanding Linux File System
Sadia Bashir
 
Unix & Linux File System in Operating System
Unix & Linux File System in Operating SystemUnix & Linux File System in Operating System
Unix & Linux File System in Operating System
Meghaj Mallick
 
A fast file system for unix presentation by parang saraf (cs5204 VT)
A fast file system for unix presentation by parang saraf (cs5204 VT)A fast file system for unix presentation by parang saraf (cs5204 VT)
A fast file system for unix presentation by parang saraf (cs5204 VT)
Parang Saraf
 
Linux Directory System: Introduction
Linux Directory System: IntroductionLinux Directory System: Introduction
Linux Directory System: Introduction
Amar Jukuntla
 
The linux file system structure
The linux file system structureThe linux file system structure
The linux file system structure
Teja Bheemanapally
 
Linux and windows file system
Linux and windows  file systemLinux and windows  file system
Linux and windows file system
lin yucheng
 
Linux file system nevigation
Linux file system nevigationLinux file system nevigation
Linux file system nevigation
hetaldobariya
 
Unix File System
Unix File SystemUnix File System
Unix File System
student(MCA)
 
Management file and directory in linux
Management file and directory in linuxManagement file and directory in linux
Management file and directory in linux
Zkre Saleh
 
Os
OsOs
Os
sakthibalabalamuruga
 
Mca ii os u-5 unix linux file system
Mca  ii  os u-5 unix linux file systemMca  ii  os u-5 unix linux file system
Mca ii os u-5 unix linux file system
Rai University
 
Files and directories in Linux 6
Files and directories  in Linux 6Files and directories  in Linux 6
Files and directories in Linux 6
Meenakshi Paul
 
Xfs file system for linux
Xfs file system for linuxXfs file system for linux
Xfs file system for linux
Ajay Sood
 
File system.
File system.File system.
File system.
elyza12
 
linux file system
linux file systemlinux file system
linux file system
AryaTadbir Network Designers
 
Linux course fhs file hierarchy standard
Linux   course   fhs file hierarchy standardLinux   course   fhs file hierarchy standard
Linux course fhs file hierarchy standard
Khawar Nehal [email protected]
 
File system Os
File system OsFile system Os
File system Os
Nehal Naik
 
File system
File systemFile system
File system
Harleen Johal
 
Disk and File System Management in Linux
Disk and File System Management in LinuxDisk and File System Management in Linux
Disk and File System Management in Linux
Henry Osborne
 
Lesson 2 Understanding Linux File System
Lesson 2 Understanding Linux File SystemLesson 2 Understanding Linux File System
Lesson 2 Understanding Linux File System
Sadia Bashir
 
Unix & Linux File System in Operating System
Unix & Linux File System in Operating SystemUnix & Linux File System in Operating System
Unix & Linux File System in Operating System
Meghaj Mallick
 
A fast file system for unix presentation by parang saraf (cs5204 VT)
A fast file system for unix presentation by parang saraf (cs5204 VT)A fast file system for unix presentation by parang saraf (cs5204 VT)
A fast file system for unix presentation by parang saraf (cs5204 VT)
Parang Saraf
 
Linux Directory System: Introduction
Linux Directory System: IntroductionLinux Directory System: Introduction
Linux Directory System: Introduction
Amar Jukuntla
 
The linux file system structure
The linux file system structureThe linux file system structure
The linux file system structure
Teja Bheemanapally
 
Linux and windows file system
Linux and windows  file systemLinux and windows  file system
Linux and windows file system
lin yucheng
 
Linux file system nevigation
Linux file system nevigationLinux file system nevigation
Linux file system nevigation
hetaldobariya
 
Management file and directory in linux
Management file and directory in linuxManagement file and directory in linux
Management file and directory in linux
Zkre Saleh
 
Mca ii os u-5 unix linux file system
Mca  ii  os u-5 unix linux file systemMca  ii  os u-5 unix linux file system
Mca ii os u-5 unix linux file system
Rai University
 
Files and directories in Linux 6
Files and directories  in Linux 6Files and directories  in Linux 6
Files and directories in Linux 6
Meenakshi Paul
 
Xfs file system for linux
Xfs file system for linuxXfs file system for linux
Xfs file system for linux
Ajay Sood
 
File system.
File system.File system.
File system.
elyza12
 
File system Os
File system OsFile system Os
File system Os
Nehal Naik
 
Disk and File System Management in Linux
Disk and File System Management in LinuxDisk and File System Management in Linux
Disk and File System Management in Linux
Henry Osborne
 

Similar to The evolution of linux file system (20)

Introduction to distributed file systems
Introduction to distributed file systemsIntroduction to distributed file systems
Introduction to distributed file systems
Viet-Trung TRAN
 
009709863.pdf
009709863.pdf009709863.pdf
009709863.pdf
KalsoomTahir2
 
Ch11 file system implementation
Ch11   file system implementationCh11   file system implementation
Ch11 file system implementation
Welly Dian Astika
 
file system overview in oerating system .
file system overview in oerating system .file system overview in oerating system .
file system overview in oerating system .
pavanm6360
 
Ch10 file system interface
Ch10   file system interfaceCh10   file system interface
Ch10 file system interface
Welly Dian Astika
 
operating system File - System Interface
operating system File - System Interfaceoperating system File - System Interface
operating system File - System Interface
Dr. Chandrakant Divate
 
Operating System
Operating SystemOperating System
Operating System
VijayaPriya14
 
Ankit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptx
Ankit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptxAnkit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptx
Ankit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptx
AnkitBargali
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
Ntu
 
Hadoop Distributed File System
Hadoop Distributed File SystemHadoop Distributed File System
Hadoop Distributed File System
Milad Sobhkhiz
 
Linux_File_System_Presentation________________________
Linux_File_System_Presentation________________________Linux_File_System_Presentation________________________
Linux_File_System_Presentation________________________
saurabhbquest
 
chapter10 - File structures.pdf
chapter10 - File structures.pdfchapter10 - File structures.pdf
chapter10 - File structures.pdf
satonaka3
 
Windows file system
Windows file systemWindows file system
Windows file system
sumitjain2013
 
Distributed file systems
Distributed file systemsDistributed file systems
Distributed file systems
Sri Prasanna
 
File Management & Access Control
File Management & Access Control File Management & Access Control
File Management & Access Control
YuvrajWadavale
 
Windows File Systems
Windows File SystemsWindows File Systems
Windows File Systems
primeteacher32
 
UNIT III.pptx
UNIT III.pptxUNIT III.pptx
UNIT III.pptx
YogapriyaJ1
 
Lect09
Lect09Lect09
Lect09
Vin Voro
 
Os6
Os6Os6
Os6
gopal10scs185
 
File systems for mobile phones or handheld devices
File systems for mobile phones or handheld devices File systems for mobile phones or handheld devices
File systems for mobile phones or handheld devices
Ram Kumar K R
 
Introduction to distributed file systems
Introduction to distributed file systemsIntroduction to distributed file systems
Introduction to distributed file systems
Viet-Trung TRAN
 
Ch11 file system implementation
Ch11   file system implementationCh11   file system implementation
Ch11 file system implementation
Welly Dian Astika
 
file system overview in oerating system .
file system overview in oerating system .file system overview in oerating system .
file system overview in oerating system .
pavanm6360
 
operating system File - System Interface
operating system File - System Interfaceoperating system File - System Interface
operating system File - System Interface
Dr. Chandrakant Divate
 
Ankit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptx
Ankit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptxAnkit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptx
Ankit Bargali Ouiiihhhoojpk;oihigigiS BCA-IV.pptx
AnkitBargali
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
Ntu
 
Hadoop Distributed File System
Hadoop Distributed File SystemHadoop Distributed File System
Hadoop Distributed File System
Milad Sobhkhiz
 
Linux_File_System_Presentation________________________
Linux_File_System_Presentation________________________Linux_File_System_Presentation________________________
Linux_File_System_Presentation________________________
saurabhbquest
 
chapter10 - File structures.pdf
chapter10 - File structures.pdfchapter10 - File structures.pdf
chapter10 - File structures.pdf
satonaka3
 
Distributed file systems
Distributed file systemsDistributed file systems
Distributed file systems
Sri Prasanna
 
File Management & Access Control
File Management & Access Control File Management & Access Control
File Management & Access Control
YuvrajWadavale
 
File systems for mobile phones or handheld devices
File systems for mobile phones or handheld devices File systems for mobile phones or handheld devices
File systems for mobile phones or handheld devices
Ram Kumar K R
 
Ad

Recently uploaded (20)

Revolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptxRevolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptx
nidhisingh691197
 
Adobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest VersionAdobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest Version
kashifyounis067
 
Why Orangescrum Is a Game Changer for Construction Companies in 2025
Why Orangescrum Is a Game Changer for Construction Companies in 2025Why Orangescrum Is a Game Changer for Construction Companies in 2025
Why Orangescrum Is a Game Changer for Construction Companies in 2025
Orangescrum
 
Download YouTube By Click 2025 Free Full Activated
Download YouTube By Click 2025 Free Full ActivatedDownload YouTube By Click 2025 Free Full Activated
Download YouTube By Click 2025 Free Full Activated
saniamalik72555
 
Solidworks Crack 2025 latest new + license code
Solidworks Crack 2025 latest new + license codeSolidworks Crack 2025 latest new + license code
Solidworks Crack 2025 latest new + license code
aneelaramzan63
 
Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025
kashifyounis067
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
EASEUS Partition Master Crack + License Code
EASEUS Partition Master Crack + License CodeEASEUS Partition Master Crack + License Code
EASEUS Partition Master Crack + License Code
aneelaramzan63
 
Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025
kashifyounis067
 
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Lionel Briand
 
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage DashboardsAdobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
BradBedford3
 
Top 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docxTop 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docx
Portli
 
How can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptxHow can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptx
laravinson24
 
How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...
How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...
How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...
Egor Kaleynik
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
steaveroggers
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
Expand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchangeExpand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchange
Fexle Services Pvt. Ltd.
 
Download Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With LatestDownload Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With Latest
tahirabibi60507
 
Revolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptxRevolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptx
nidhisingh691197
 
Adobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest VersionAdobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest Version
kashifyounis067
 
Why Orangescrum Is a Game Changer for Construction Companies in 2025
Why Orangescrum Is a Game Changer for Construction Companies in 2025Why Orangescrum Is a Game Changer for Construction Companies in 2025
Why Orangescrum Is a Game Changer for Construction Companies in 2025
Orangescrum
 
Download YouTube By Click 2025 Free Full Activated
Download YouTube By Click 2025 Free Full ActivatedDownload YouTube By Click 2025 Free Full Activated
Download YouTube By Click 2025 Free Full Activated
saniamalik72555
 
Solidworks Crack 2025 latest new + license code
Solidworks Crack 2025 latest new + license codeSolidworks Crack 2025 latest new + license code
Solidworks Crack 2025 latest new + license code
aneelaramzan63
 
Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025
kashifyounis067
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
EASEUS Partition Master Crack + License Code
EASEUS Partition Master Crack + License CodeEASEUS Partition Master Crack + License Code
EASEUS Partition Master Crack + License Code
aneelaramzan63
 
Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025
kashifyounis067
 
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Lionel Briand
 
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage DashboardsAdobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
BradBedford3
 
Top 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docxTop 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docx
Portli
 
How can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptxHow can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptx
laravinson24
 
How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...
How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...
How Valletta helped healthcare SaaS to transform QA and compliance to grow wi...
Egor Kaleynik
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
steaveroggers
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
Expand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchangeExpand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchange
Fexle Services Pvt. Ltd.
 
Download Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With LatestDownload Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With Latest
tahirabibi60507
 
Ad

The evolution of linux file system

  • 1. The evolution of Linux file system Gang He [email protected]
  • 2. 2 Agenda • Local file system (LFS) • Cluster file system (CFS) • Distributed file system (DFS)
  • 5. 5 File system concepts • file descriptor (user space) • struct file, struct dentry, struct inode, struct address_space (kernel space) • struct super_block, meta data, file data, buffer/page cache
  • 6. 6 Ext2 → Ext3 → Ext4 • Ext2 (1993), inspired from UFS, first popular and stable Linux file system, but design is plain. - File systems are getting bigger, how to look up a entry under a big directory, how to reduce fsck time after crash ... • Ext3 (2001), add journal, hash-tree directory indexing, etc. - File systems are getting bigger, have to eliminate these limitations - Various advanced file systems impact ext3 ... • Ext4 (2008), 48-bit addressing space, no limited directory entries, extents, multi-block allocation, delayed block allocation, online defragmentation, 256 bytes inode, persistent preallocation, barrier, etc.
  • 7. 7 Nowadays • Ext4, will continue to be maintained due to stability and historical reasons • XFS, robust and scalable, good performance for large storage, will shine in handling big file area (e.g. virtual machine image) • BtrFS, new design (replace ext4), inspired from ZFS, contains many features of enterprise file system. for examples, copy on write, own internal RAID (manage volume ), snapshot/clone support, dynamically grow and shrink, SSD support, etc.
  • 9. 9 Why CFS • Independent storage devices (e.g. SAN). • High availability requirement. • How to scale out file system in CPU, memory, even network bandwidth.
  • 10. 10 CFS common points • File system POSIX semantics. • Shared disk. • Distributed lock manager (DLM). • Cluster manager stacks.
  • 11. 11 CFS future • Scale out more nodes, provide higher aggregation IO bandwidth. • More high availability, support online fsck, online deframentation, online expand/shrink. • File system level snapshot, file level clone. • Tiered storage, SSD support. • Deduplication. • ...
  • 13. 13 Background • Costs, storage array, fabric switches, HBA card, etc are expensive. • Unified storage space, linear expansion, commodity hardware. • Driven by Internet industry (e.g. search, picture share, big data, etc). • Google file system appeared (2003).
  • 14. 14 GFS-like DFS (HDFS, MooseFS, KFS)
  • 15. 15 DFS common points • Not strictly comply with File system POSIX semantics, most implementations are based on user-space. • Share nothing, meta-data/file data are stored separately, meta-data access/file data access are separated. • Have own local file system, a file represents a logical data block, a data block has several copy blocks. • Meta-data server usually load all meta data into memory at start-up, writing logs records incremental changes, then flushing memory to disk/merging log and previous meta-data file gets a new checkpoint of meta-data. • Other algorithms: heartbeat algorithm, rack-aware, block allocation policy, file lock management, etc.
  • 16. 16 Scale out • Meta-data cluster server, e.g. GFS2, Ceph. • Fully symmetric, no central meta-data server, e.g. GlusterFS. • Improved cluster management mechanism. hearbeat/corosync → zookeeper cluster • IO Flow Control, reduce meta-data server dependence, costs control (ECC), etc. VS.
  • 17. 17 Current trends • Linear scale out. • CompuStor hyper-converged systems. • Flash technology utilization. • High-speed network support. • Application-Aware (e.g. VM image). • Deduplication/Compression/Snapshot/Clone. • Object/Block/File unified storage.
  • 18. 18