SEG3420 File Structures and Processing

Lecture 4 – Data Transfers

Source: This notes is based on

• Chapter 3 of the textbook Panos E. Livadas, File Structure Theory
and Practice, Prentice Hall
• Lecture Notes from Prof. Wong class available at
http://www.se.cuhk.edu.hk/~seg3420/
Agenda

• Hardware
• Operating Systems
Hardware
- Overview

Primary CPU
Memory

Channel

Controller Secondary
Memory
Hardware
- Primary Storage 1/2

• Also known as the main memory

• Arranged in units called cells with typical size
of eight bits
• Each cell is assigned a unique identification
known as its address
• Stored data can be accessed randomly via
the address
• The primary storage is often known as
random access memory (RAM)
Hardware
- Primary Storage 2/2

• Programs that are being executed are stored

in the primary storage
• Example of programs
– Application programs
– Operating systems
• Primary storages are expensive and volatile
– Not suitable for massive and/or permanent
storage
Hardware
- Secondary Storage

• For massive and permanent storage

– Less volatile
– Less expensive
– More flexible
• Examples:
– Magnetic tape
– Magnetic disk
– CD
Hardware
- Central Processing Unit (CPU) 1/3

• Major functions of CPUs

– Execute machine language instructions
– Data transfer control
• The CPU contains two major units
– The control unit
– The arithmetic/logic unit
• It contains registers for temporary data storage
– General-purpose registers
– Special-purpose registers
Hardware
- Central Processing Unit (CPU) 2/3

Arithmetic/Logic Unit
Primary
Registers

Storage
Control Unit
Bus
Hardware
- Central Processing Unit (CPU) 3/3

• The CPU is connected to the primary storage

through a bus
– This allows data transfer between the CPU and
the primary memory
• Registers hold data that are immediate
needed while primary storage contains data
that are needed in the near future
Hardware
- Channel 1/7

• Purposes
– Relieve the CPU of data transfer management
• A channel
– is a special processor with its own instruction set,
instruction counter, and control logic
– is for relieving the CPU from data transfer tasks
• CPU may execute instructions directly from
cache memory for efficiency
Hardware
- Channel 2/7

Physical data transfer request

The channel’s counter with the beginning of the

Channel program is loaded to the CPU

The CPU signals the channel to

execute the channel program
Hardware
- Channel 3/7

• Example: write a physical block onto the disk

Perform a “seek”;
Find the correct block;
Write the block from address;
Hardware
- Channel 4/7

• Types of channels
– Selector
– Byte Multiplexor
– Block Multiplexor
Hardware
- Channel 5/7

• Selector Channel
– Can connected as many as 256 devices
– Only one can be selected
– The physical I/O operations are completed in
sequence
– Usually use with high-speed devices
Hardware
- Channel 6/7

• Byte Multiplexor Channel

– Can be connected to up to 256 devices
– Interleaves byte transmissions between the
various connected devices
– Does not need to wait until the completion of the
physical I/O of a device
– Uses for slow –speed devices (e.g. printers)
Hardware
- Channel 7/7

• Block Multiplexor Channel

– Can execute multiple channel programs
– Interleaves block transmissions between the
various connected devices
– Uses for high speed devices
Hardware
- Controller
Primary Storage

CPU
Interprets and executes the channel
commands to the connected secondary
Channel
storage device

Controller Controller

Secondary Secondary Secondary

Storage Storage Storage
Device Device Device
Hardware
- I/O Buffers READ request

R0 R0
WRITE request
R1 R1

R2 R2

Input Buffer Output Buffer

SD R0 R1 R2
R0
First Block
Data Area
User Area Magnetic Disk

Primary Storage
Operating Systems
- An Overview

• An operating system manages a system’s

resources
– Primary storage
– Processors
– Secondary storage
– File system
Operating Systems
- The Four Primary Components 1/6
Data Management

Device Management

Memory Management

Processor Management
Operating Systems
- The Four Primary Components 2/6

• Memory Management
– Allocate and release storage area to processes
– Storage area allocation management
Operating Systems
- The Four Primary Components 3/6

• Processor Management
– There are three components
• Job scheduler: Decides which jobs are allowed
• Processor scheduler: Decides in which order and for how
long the processor will be dedicated to jobs submitted to
the system
• Traffic controller: Keeps track of the processors and
processes status
Operating Systems
- The Four Primary Components 4/6

– Scheduling algorithms
• First Come, First Serve (FCFS) Policy: The first job gains
control until it is finished

Job Number Service Time

Assumption: The differences in arrival times are
1 100µs Negligible then

2 10µs The average waiting time = (100 + 110)/3 µs

= 70 µs
3 5µs
Operating Systems
- The Four Primary Components 5/6

• Shortest Job Next (SJN): The job that needs the least
processor time will be processed next

Job Number Service Time

Assumption: The differences in arrival times are
3 5µs Negligible then

2 10µs The average waiting time = (5 + 15)/3 µs

= 6.67 µs
3 100µs

• Great improvement
• Question: What if new jobs with less than 100 µs processing
time arrive before the third job is executed?
Operating Systems
- The Four Primary Components 6/6

• Process Sharing: The jobs share the processor for a short

period of time slice in turns controlled by the dispatcher
– Other algorithms
• Priority Scheduling
• Multilevel Queue Scheduling
• Multilevel Feedback Queue Scheduling
– Goals of scheduling algorihtms
• Fairness
• Efficiency
• Throughput
Operating Systems
- Device Management Component 1/5

• There are three components

– I/O Job Scheduler
• Decide which task the device should serve next
– I/O Device Handler
• Once the job to be serviced is chosen next, the channel
is signaled to start executing tasks assigned to the I/O
device handler
• Different devices have different device handler
– I/O Traffic Handler
• Keep track of devices, channels, and controllers statuses
Operating Systems
- Device Management Component 2/5

– Device Scheduling Algorithm

• General objectives
– Minimize the I/O request waiting time
• First-Come, First-Served (FCFS)
– Service assigned according to the order of arrival
– The list of requests is a queue
– No minimization of seek time
• Shortest Cylinder Next (SCN)
– The list of requests is ordered in an ascending order of the
cylinder number
– The one with the closest physical cylinder distance is
served next
– Implemented with a doubly-linked list approach
Operating Systems
- Device Management Component 3/5

• SCAN
– The R/W head moves in one specific direction at a
particular time
– All requests over which the R/W head passes through will
be served
 Operating Systems
- Device Management Component 4/5

SR 1 10 2 100 3 5 4 95 .

FCFS

SR . 3 5 1 10 4 95 2 100 .

SCN

SR . 3 5 1 10 4 95 2 100 .

SCAN
Operating Systems
- Device Management Component 5/5

• Question: Suppose jobs 2 through 4 arrive

before the completion of job 1, which job will
be processed next for each of the algorithms
in the previous slide?
• Which is the best policy?
Operating Systems
- Data Management Component 1/17

• Purposes
– File management
• Keep track of
– Physical file locations
– File usages
– File status
– File opening and closing
– Storage allocation
– I/O buffers
– Secondary devices and their characteristics
Operating Systems
- Data Management Component 2/17

• Access Modes and Access Techniques

– Access Mode: By which an application program
must access logical records from a physical file
– Number of access modes depends on
• Vendor of the operating system
• Secondary storage medium
• File organization
– Access modes used to access a file can be
changed
Operating Systems
- Data Management Component 3/17

– Combination of access modes allowed depends on

• The file system
• The file organization
– There are two record handling techniques that an
application program can use
• Synchronous I/O: An application is placed in the wait state
until the transfer is complete
– Access method that makes use of this is called queued access
techniques
– Access method using queued access technique automatically
blocks, deblocks, and buffers
Operating Systems
- Data Management Component 4/17

• Asynchronous I/O: An application may continue to run

depends on if other application gains control
– Access method that makes use of this is called basic access
techniques
– The application program is responsible for blocking and
deblocking
Operating Systems
- Data Management Component 5/17

• The Directory
– Each system maintain a master directory referred to
as the system directory
– Files are accessed through their pathname
– Directories are files
 Operating Systems
- Data Management Component 6/17
Root

User1 User2 User3 File0

File1 File2 Sub2

File6 File1 File5

File3 File4
Operating Systems
- Data Management Component 7/17

• A possible implementation is given in the

appendix
Operating Systems
- Data Management Component 8/17

• Device Storage Allocation

– The system must keep track of block usage
information
– One approach is to use bit map
• A sequence of bits that gives a one-to-one correspondence
between block and bit positions
– The i-th bit of the bit map corresponds to the block with
physical block number I
– If the i-th block is already occupied then the i-th bit will be set
to 1
• Amount of memory for a bit map is small and can be stored
in the primary storage area
• Does not work with blocks of variable sizes
Operating Systems
- Data Management Component 9/17

– Another approach is to use free chain

• Use the linked list approach
• Maintain a linked list of free blocks (referred to as a free
chain)
• Work with the first block for allocation and deallocation
• Work with both fixed-size and variable-size blocks
Operating Systems
- Data Management Component 10/17

• A possible implementation is given in the

appendix
Operating Systems
- Data Management Component 11/17

• File Storage Allocation

– File Allocation Table
• Use the bit map approach
• The i-th bit reflects the status of the corresponding block
• Does not work for the variable-size case
– Chain-Blocked Method
• Only need to know the address of the first block
• Can only be used efficiently for sequential access files
• Works with variable-size blocks
Operating Systems
- Data Management Component 12/17

– File Map
• A file of logical records is created for each file
– There is a one-to-one correspondence between the logical
records and the blocks in the file
– Each logical record consists of two fields about the information
of the corresponding block
» First field: Length of the block
» Second field: Address of the block
• The designer is responsible for choosing the most efficient
structure for the file map
Operating Systems
- Data Management Component 13/17

• I/O Buffers
– Two pairs of pointers for I/O buffers are maintained
by the file system
• Input buffer pointer (IBP) / Output buffer pointer (OBP)
• Current input buffer pointer (CIBP) / Current output buffer
pointer (COBP)
Operating Systems
- Data Management Component 14/17

Algorithm logical_READ;

If (IBP == CIBP)
{
Perform a physical_READ;
while (transfer_not_completed)
wait;
}
Deliver logical record pointed to by CIPB to indicate program
variable;
CIPB = CIPB + Length(delivered logical record);
If (CIPB == IPB + Size(Buffer))
CIPB = IBP;
Operating Systems
- Data Management Component 15/17
IBP
CIBP

Input Buffer

Data Area

OBP Magnetic Disk

COBP

SD R0 R1 R2

First Block
Output Buffer
Operating Systems
- Data Management Component 16/17
IBP R0

CIBP R1

R2

Input Buffer

R0

Data Area

OBP Magnetic Disk

COBP

SD R0 R1 R2

First Block
Output Buffer
Operating Systems
- Data Management Component 17/17
IBP R0

R1

CIBP R2

Input Buffer

R0

Data Area

OBP R0 Magnetic Disk

COBP
SD R0 R1 R2

First Block
Output Buffer
References

1. Panos E. Livadas, File Structure Theory

and Practice, Prentice Hall
2. Past SEG3420 Lecture Notes from Prof.
Wong’s class available at
http://www.se.cuhk.edu.hk/~seg3420/