Operating Systems: Internals and Design Principles, 6/E William Stallings

Chapter 12 File Management

Dave Bremer Otago Polytechnic, N.Z. 2008, Prentice Hall

Roadmap
Overview
File organisation and Access File Directories File Sharing Secondary Storage Management File System Security

File Management
File management system consists of system utility programs that run as privileged applications Concerned with secondary storage

Typical Operations
File systems also provide functions which can be performed on files, typically:
Create Delete Open Close Read Write

Terms
Four terms are in common use when discussing files:
Field Record File Database

Fields and Records (SKIP)

Fields
Basic element of data Contains a single value Characterized by its length and data type

Records
Collection of related fields Treated as a unit

File and Database (SKIP)

File
Have file names Is a collection of similar records Treated as a single entity May implement access control mechanisms

Database
Collection of related data Relationships exist among elements Consists of one or more files

File Management Systems

Provides services to users and applications in the use of files
The way a user or application accesses files

Programmer does not need to develop file management software

Requirements for a General Purpose System

1. Each user should be able to create, delete, read, write and modify files 2. Each user may have controlled access to other users files 3. Each user may control what type of accesses are allowed to the users files 4. Each user should be able to restructure the users files in a form appropriate to the problem

Requirements cont.
5. Each user should be able to move data between files 6. Each user should be able to back up and recover the users files in case of damage 7. Each user should be able to access the users files by using symbolic names

Typical Software Organization

Access Method Accessing records Responsible for all file I/O initiation and termination. Exchanging blocks of data

Device Drivers (SKIP)

Lowest level Communicates directly with peripheral devices Responsible for starting I/O operations on a device Processes the completion of an I/O request

Basic File System (SKIP)

Physical I/O Primary interface with the environment outside the computer system Deals with exchanging blocks of data Concerned with the placement of blocks Concerned with buffering blocks in main memory

Basic I/O Supervisor (SKIP)

Responsible for all file I/O initiation and termination. Control structures deal with
Device I/O, Scheduling, File status.

Selects and schedules I/O with the device

Logical I/O (SKIP)

Enables users and applications to access records Provides general-purpose record I/O capability Maintains basic data about file

Access Method (SKIP)

Closest to the user Reflect different file structures Provides a standard interface between applications and the file systems and devices that hold the data Access method varies depending on the ways to access and process data for the device.

Roadmap
Overview

File organisation and Access

File Directories File Sharing Secondary Storage Management File System Security

File Organization
File Management Referring to the logical structure of records
Physical organization discussed later

Determined by the way in which files are accessed

Criteria for File Organization

Important criteria include:
Short access time Ease of update Economy of storage Simple maintenance Reliability

Priority will differ depending on the use (e.g. read-only CD vs Hard Drive)
Some may even conflict

File Organisation Types

Many exist, but usually variations of:
Pile Sequential file Indexed sequential file Indexed file Direct, or hashed, file

The Pile
Data are collected in the order they arrive
No structure

Purpose is to accumulate a mass of data and save it Records may have different fields Record access is by exhaustive search

The Sequential File

Fixed format used for records Records are the same length All fields the same (order and length) Field names and lengths are attributes of the file Key field
Uniquely identifies the record Records are stored in key sequence

Indexed Sequential File

Maintains the key characteristic of the sequential file:
records are organized in sequence based on a key field.

Two features are added:

an index to the file to support random access, and an overflow file.

Indexed File
Uses multiple indexes for different key fields
May contain an exhaustive index that contains one entry for every record in the main file May contain a partial index

When a new record is added to the main file, all of the index files must be updated.

The Direct or Hashed File

Exploits the capability found on disks to access directly any block of a known address The Direct or Hashed File
Directly access a block at a known address Key field required for each record The direct file makes use of hashing on the key value

Roadmap
Overview File organisation and Access

File Directories
File Sharing Secondary Storage Management File System Security

Contents
Contains information about files
Attributes Location Ownership

Directory itself is a file owned by the operating system Provides mapping between file names and the files themselves

Directory Elements
Basic Information
Name, Type,

Address Information
Starting address, size used, size allocated, ...

Access Control Information

Owner, access rights, ...

Usage Information
Created date, modified date, accessed date,

Directory Elements: Basic Information (SKIP)

File Name
Name as chosen by creator (user or program). Must be unique within a specific directory.

File type File Organisation

For systems that support different organizations

Directory Elements: Address Information (SKIP)

Volume
Indicates device on which file is stored

Starting Address Size Used

Current size of the file in bytes, words, or blocks

Size Allocated
The maximum size of the file

 Owner

Directory Elements: Access Control Information (SKIP)

The owner may be able to grant/deny access to other users and to change these privileges.

Access Information
May include the users name and password for each authorized user.

Permitted Actions
Controls reading, writing, executing, transmitting over a network

Directory Elements: Usage Information (SKIP)

Date Created Identity of Creator Date Last Read Access Identity of Last Reader Date Last Modified Identity of Last Modifier Date of Last Backup Current Usage
Current activity, locks, etc

Operations Performed on a Directory

A directory system should support a number of operations including:
Search Create files Deleting files Listing directory Updating directory

Two-Level Scheme for a Directory

One directory for each user and a master directory
Master directory contains entry for each user Provides address and access control information

Each user directory is a simple list of files for that user

Does not provide structure for collections of files

Hierarchical, or Tree-Structured Directory

Master directory with user directories underneath it Each user directory may have subdirectories and files as entries

Example of Tree-Structured Directory

Roadmap
Overview File organisation and Access File Directories

File Sharing
Secondary Storage Management File System Security

File Sharing
In multiuser system, allow files to be shared among users Two issues
Access rights Management of simultaneous access

Access Rights
A wide variety of access rights have been used by various systems
None Knowledge Execution Reading Appending/Updating Changing protection Deletion

User Classes
Owner
Usually the files creator, usually has full rights

Specific Users
Rights may be explicitly granted to specific users

User Groups
A set of users identified as a group

All
everyone

Simultaneous Access
User may lock entire file when it is to be updated User may lock the individual records during the update Mutual exclusion and deadlock are issues for shared access

Roadmap
Overview File organisation and Access File Directories File Sharing

Secondary Storage Management

File System Security

Secondary Storage Management

The Operating System is responsible for allocating blocks to files Two related issues
Space must be allocated to files Must keep track of the space available for allocation

File Allocation Issues

1. When a file is created is the maximum space allocated at once? 2. Space is added to a file in contiguous portions
What size should be the portion?

3. What data structure should be used to keep track of the file portions?

Preallocation vs Dynamic Allocation

Need the maximum size for the file at the time of creation Difficult to reliably estimate the maximum potential size of the file Tend to overestimated file size so as not to run out of space

Portion Size
Two extremes:
Portion large enough to hold entire file is allocated Allocate space one block at a time

Trade-off between efficiency from the point of view of a single file, or the overall system efficiency

File Allocation Method

Three methods are in common use:
Contiguous Chained Indexed

Contiguous Allocation
Single set of blocks is allocated to a file at the time of creation Only a single entry in the file allocation table
Starting block and length of the file

External fragmentation will occur

Need to perform compaction

Contiguous File Allocation

External Fragmentation

Chained Allocation
Allocation on basis of individual block Each block contains a pointer to the next block in the chain Only single entry in the file allocation table
Starting block and length of file

No external fragmentation Best for sequential files

Chained Allocation

Chained Allocation Consolidation

Indexed Allocation
File allocation table contains a separate one-level index for each file The index has one entry for each portion allocated to the file The file allocation table contains block number for the index

Indexed Allocation Method

Allocation may be either
Fixed size blocks or Variable sized blocks

Allocating by blocks eliminates external fragmentation Variable sized blocks improves locality Both cases require occasional consolidation

Indexed Allocation with Block Portions

Indexed Allocation with Variable Length Portions

Free Space Management

Just as allocated space must be managed, so must the unallocated space To perform file allocation, we need to know which blocks are available. We need a disk allocation table in addition to a file allocation table

Bit Tables
This method uses a vector containing one bit for each block on the disk. Each entry of a 0 corresponds to a free block,
and each 1 corresponds to a block in use.

Advantages:
Works well with any file allocation method Small as possible

Chained Free Portions

The free portions may be chained together by using a pointer and length value in each free portion. Negligible space overhead Suited to all file allocation methods Leads to fragmentation

Indexing
treats free space as a file and uses an index table as it would for file allocation For efficiency, the index should be on the basis of variable-size portions rather than blocks.
Thus, there is one entry in the table for every free portion on the disk.

This approach provides efficient support for all of the file allocation methods.

Free Block List

Each block is assigned a number sequentially
the list of the numbers of all free blocks is maintained in a reserved portion of the disk.

Roadmap
Overview File organisation and Access File Directories File Sharing Record Blocking Secondary Storage Management

File System Security

Access Control
By successfully logging on to a system, the user is identified The OS can then enforce rules
Granting access to files and applications (or denying)

The OS needs a rule-set to enforce

Access Matrix
One such rule set is an Access Matrix

Access Control Lists

A matrix may be decomposed by columns Giving an Access Control List (ACL) for each file.

Capability Lists
Decomposition by rows yields capability lists (or ticket)
specifies authorized objects and operations for a user.