File System Implementation

Sunu Wibirama
Thursday, December 16, 2010
 Outline
 File-System Structure

 File-System Implementation

 Directory Implementation

 Allocation Methods

 Free-Space Management

 Discussion

11.

Thursday, December 16, 2010

 File System Structure
 File system is provided by OS to allow the data to be
stored, located, and retrieved easily
 Two problems on file system design:
 How the file system should look to the user
 Algorithms and data structures to map logical file
system onto the physical secondary-storage
devices

 File system organized into layers, uses features from

lower levels to create new features for use by higher
levels.

 I/O Control controls the physical device using

device driver
 Basic file system needs only to issue generic
commands to the device driver to read and write
physical block on the disk (ex. drive 1, cylinder 73,
track 2, sector 11)

11.

Thursday, December 16, 2010

 File System Implementation
 File organization module knows physical
and logical blocks, translating logical block
address to physical block address. It also
manages free-space on the disk

 Logical file system manages metadata

information (all file system structure except
the actual data or contents of the file).

 Logical file system maintains file structure

via file-control blocks (FCB)

 File control block – storage structure

consisting of information about a file

11.

Thursday, December 16, 2010

 A Typical File Control Block

11.

Thursday, December 16, 2010

 On-Disk File System Structures

 Boot control block contains info needed by system to boot

OS from that volume (UNIX: boot block, NTFS: partition boot
sector)

 Volume control block contains volume details (UNIX:

superblock, NTFS: master file table)

 Directory structure organizes the files (UNIX: inode

numbers, NTFS: master file table)

 Per-file File Control Block (FCB) contains many details

about the file

11.

Thursday, December 16, 2010

 In-Memory File System Structures
 It is used for file-system management and performance
improvement (via caching).
 The data are loaded at mount time and discarded at
dismount.
 The structures including:
 In-memory mount table: information of each mounted
volume
 In-memory directory structure
 System-wide open-file table: a copy of FCB of each open
file
 Per-process open-file table: a pointer to the appropriate
entry in the system-wide open-file table, as well as other
information based on process that uses the file.

11.

Thursday, December 16, 2010

 New File Creation Process

 Application program calls the logical file

system

 Logical file system knows the directory

structures. It allocates a new FCB.

 The system then reads the appropriate

directory into memory, updates it with the
new file name and FCB, and writes it back
to the disk.

 Now, the new created file can be used for

I/O operation, which will be explained in
the next slide

11.

Thursday, December 16, 2010

 In-Memory File System Structures

11.

Thursday, December 16, 2010

 Directory Implementation

 Directory-allocation and directory-management

algorithms significantly affects the efficiency,
performance, and reliability of the file system.

 Linear list of file names with pointer to the data

blocks.
 simple to program
 time-consuming to execute, because it requires a
linear search to create or delete file.

 Hash Table – linear list with hash data structure.

 decreases directory search time
 problem: fixed size of hash table

11.

Thursday, December 16, 2010

 Allocation Methods

 Many files are stored in the disk

 How to allocate space to these files so that disk space is utilized

effectively and files can be accessed quickly

 An allocation method refers to how disk blocks are allocated for files:

 Contiguous allocation

 Linked allocation

 Indexed allocation

11.

Thursday, December 16, 2010

 Contiguous Allocation

 Each file occupies a set of contiguous blocks on the disk

 Simple – only starting location (block #) and length (number of blocks)
are required
 Both sequential and direct access are supported
 Disadvantages:
 Wasteful of space (dynamic storage-allocation problem)
 File cannot grow
 External fragmentation: free space is broken into chunks
 One of several solutions: use a modified contiguous allocation scheme
 Extent-based file systems allocate disk blocks in extents
 An extent is a contiguous block of disks
 Extents are allocated for file allocation
 A file consists of one or more extents

11.

Thursday, December 16, 2010

 Contiguous Allocation of Disk Space

11.

Thursday, December 16, 2010

 Linked Allocation
 Linked allocation solves all problems of
contiguous allocation

 Each file is a linked list of disk blocks:

blocks may be scattered anywhere on
the disk.

 Ex: File “Jeep”

 Start at block 9
 Then: block 16, 1, 10
 Finally end at block 25

pointer (4 bytes)

1 block (512 bytes) visible part to user

508 bytes

11.

Thursday, December 16, 2010

 Linked Allocation
 Advantages:
 Free-space management system – no waste of space
 File can grow, depends on available free blocks

 Disadvantages:
 No random access (only sequential access)
 Space required for pointers (0.78 percent of the disk is being used for pointers, rather
than for information) ~> solution, uses clusters (unit of blocks)
 Reliability, pointer damage will cause unlinked blocks in a file.

 FAT (File Allocation Table): variation on linked allocation

 Located at the beginning of each volume

11.

Thursday, December 16, 2010

 File-Allocation Table

 To do random access:
1. The disk head move to the start of volume
to read the FAT
2. Find the location of the desired block
3. Move to the location of the block itself

11.

Thursday, December 16, 2010

 Indexed Allocation
 Brings all pointers together into the
index block
 Each file has its own index block, which
is an array of disk-block addresses
 Directory contains the address of index
block
 Support direct access without external
fragmentation
 Each file has its allocation for all
pointers, so that it has wasted space
greater than linked allocation (which
contains one pointer per block).
 We want the index block as small as
possible, then we have several
mechanisms:
1. Linked scheme
2. Multilevel index
3. Combined scheme

11.

Thursday, December 16, 2010

 Indexed Allocation
 Linked Scheme
 An index block is normally one disk block
 Large files -> we can link together several index blocks
 Ex.: an index block contains:
- a small header of file name
- a set of 100 disk-block addresses
- nil (for small file) or a pointer to another index block (for a large file)
 Multilevel index
 First-level index block points to second-level index blocks which in turn point to the
file blocks (see next slide)
 Combined scheme
 In unix, for example: 15 pointers in fileʼs inode
 12 first pointer: direct blocks, for small file (no more than 12 blocks). If the block
size is 4KB, then up to 48KB (12 x 4KB) can be accessed directly
 The next pointers point to indirect blocks, which implement multilevel index based
on their sequence (see next two slide)

11.

Thursday, December 16, 2010

 Multilevel Index
Back to Indexed Allocation

1st-level
index block

2nd-level
index block
file

11.

Thursday, December 16, 2010

 Combined Scheme: UNIX UFS
(4K bytes per block)
Back to Indexed Allocation

11.

Thursday, December 16, 2010

 Free-Space Management
 Free-space list concept
 Bit vector (n blocks)

0 1 2 n-1


0 ⇒ block[i] free
bit[i] =
1 ⇒ block[i] occupied

Block number calculation

{(number of bits per word) *(number of 0-value words)} +offset of first 1 bit

 Bit vector requires extra space. Example:

" " block size = 212 bytes
" " disk size = 230 bytes (1 gigabyte)
" " n (amount of bits) = 230/212 = 218 bits (or 32K bytes)

11.

Thursday, December 16, 2010

 Free-Space Management
 Linked list
 Link together all the free disk blocks
 Keeping a pointer to the first free block in a
special location on the disk and caching it in
memory.
 The first block contains a pointer to the next free
disk block...
 Must read each block to traverse list, increase I/
O operation time.

 Grouping
 Storing the address of n blocks in the first free
block.
 n-1 blocks are actually free blocks but the last
block contains the addresses of another n free
blocks.

 Counting
 Keep the address of the first free block and n of
free contiguous blocks that follow the first block.
 Each entry in free-space list consists of disk
address and a count

11.

Thursday, December 16, 2010

 ::Discussion
Thursday, December 16, 2010
 FAT 32
• FAT - 32 (File Allocation Table - 32 bits)
• Maximum size of file: 232 - 1 byte

• The last byte cannot be allocated to the

file so that no file has file size bigger than
0 x FFFFFFFF (4, 294, 967, 296)

• You can convert to NTFS, or split your

file. Each method has its advantages and
disadvantages.

Thursday, December 16, 2010

 Fragmentation

http://en.wikipedia.org/wiki/Defragmentation
Thursday, December 16, 2010
 Why Linux rarely needs
defragmentation tools?

• Does fragmentation occur in Linux?

Yes, but in very small quantity

• Block Groups: group file-data together in

‘clumps’ to manage small and large file
(remember combined scheme in indexed
allocation)

• Only write files to unused portion of the disk

that are not predictably being fragmented in
shorter time.

Thursday, December 16, 2010

 Combined Scheme

Possible to allocate bigger blocks for a file

Thursday, December 16, 2010

 Unix System
• Keep fragmentation level below 20%
• More than 20%? You certainly need to fix your hard
disk using shake-fs

• Run : e2fsck -nv /dev/sda1 as root, resulting:

Fragmented
Part

Thursday, December 16, 2010

 Implementation
(*you should have known this before...)
http://geekblog.oneandoneis2.org/index.php/2006/08/17/
why_doesn_t_linux_need_defragmenting

Empty hard disk (*simplified assumption)

Thursday, December 16, 2010
 Start with FAT file system....

Thursday, December 16, 2010

 Start with FAT file system....

I have hello.txt

Thursday, December 16, 2010

 Start with FAT file system....

I have hello.txt

OK, now add

bye.txt

Thursday, December 16, 2010

 Start with FAT file system....

I have hello.txt

OK, now add

bye.txt

I want to change
? hello.txt, dude...

Thursday, December 16, 2010

Thursday, December 16, 2010
 1st approach
Just copy, delete the original
content, and wrap it up in the
larger space.....

Thursday, December 16, 2010

 1st approach
Just copy, delete the original Or,
content, and wrap it up in the Put your extended file content to
larger space..... the next space.....

2nd approach

If the first approach requires huge read and write operation, then
the most possible approach is the second one. That’s why FAT
suffers from large fragmentation
Thursday, December 16, 2010
 What About Linux?

Initial condition

Thursday, December 16, 2010

 What About Linux?

Add bye.txt

Thursday, December 16, 2010

 What About Linux?

Change hello.txt

Thursday, December 16, 2010

 Thank You

Thursday, December 16, 2010