UNIT-IV

Multiprocessor System Architectures: Introduction, Motivation for multiprocessor Systems, Basic Multiprocessor System
Architectures Multi Processor Operating Systems: Introduction, Structures of Multiprocessor Operating Systems, Operating
Design Issues, Threads, Process Synchronization, Processor Scheduling.
Distributed File Systems: Architecture, Mechanisms for Building Distributed File Systems, Design Issues

1. Multiprocessor system
Definition: A system with two or more processors (CPUs) within a single computer system is
called as multiprocessor system. The main memory (RAM) is shared by all the processors in this
system.
The main feature of this architecture is to provide high speed at low cost. These types of
systems are used when very high speed is required to process a large volume of data. These
systems are generally used in environment like satellite control, weather forecasting, etc.

Motivations for Multiprocessor Systems

The main motivations for multiprocessor system are:
 Enhanced Performance: Multiprocessor systems increase system performance in two
ways. First, concurrent execution of several tasks by different processors can increase
throughput. Second, a single task can be divided into multiple sub tasks (if parallelism
exists) and these sub tasks can be executed on parallel on different processors.
 Fault Tolerance: When one of the processor fails, the system can continue the execution
because of the availability of multiple processors.

2. Multiprocessor system Architectures

Multiprocessor systems are classified into two types based on whether a memory location can be
directly accessed by a processor or not. They are:
 Loosely coupled multiprocessor system
 Tightly coupled multiprocessor system

Loosely Coupled Multiprocessor System

In loosely-coupled multiprocessor systems, each processor has its own local memory. Memory
location of one processor cannot be directly accessed by other processor. Processors exchange data over a
high-speed communication network by sending messages via a technique known as "message passing".

1|Page Advanced Operating System Notes by Ravindar Mogili

  It uses distributed memory concept.
 It has static interconnection network.
 Data rate, throughput and security are low.
 It is less expensive.
 It operates on multiple operating systems.
 Example: Beowulf cluster
Tightly Coupled Multiprocessor System
In tightly-coupled multiprocessor system, multiple processors share a common main
memory using a shared bus system. No data distribution is required as the processors do not have
a private memory.

 It uses shared memory concept.

 It has dynamic interconnection network.
 Data rate, throughput and security are high.
 It is more expensive.
 It operates on single operating system.
 Example: Zeon processor

Based on location of main memory and its accessibility with respect to processors,
Multiprocessor systems are classified into three types. They are:
 Uniform memory access (UMA) system
 Non-Uniform memory access (NUMA) system
 No Remote Memory Access (NORMA) system

Uniform memory access (UMA) system

In UMA architecture, the main memory is located at center location such that it is equidistance
from all the processors in terms of access time. In addition to centralized shared memory, each
processor can also have private memory to cache data for higher performance.

2|Page Advanced Operating System Notes by Ravindar Mogili

 Non-Uniform memory access (NUMA) system
In NUMA architectures, main memory is physically partitioned and the partitions
are attached to the processors. All the processors share the same memory address space.
A processor can directly access the memory attached to any other processor, but the time
to access the memory attached to other processors is much higher than the time to access
its own memory partition. Examples of NUMA architectures are Cm* of CMU and
Butterfly machine of BBN Laboratories.
Memory Memory

Processor Processor

Processor Processor

Memory Memory

No Remote Memory Access (NORMA) system

In NORMA architectures, main memory is physically partitioned and the partitions are
attached to the processors. However, a processor cannot directly access memory of any other
processor. The processors must send messages over the inter-connection network to exchange
information. An example of NORMA architecture is Intel's Hypercube.

I/O device (e.g., Disk or I/O

Interface)

Memory Memory Memory

Processor Processor Processor

3. Structures of Multiprocessor Operating Systems

A system with two or more processors (CPUs) within a single computer system is called
as multiprocessor system. The operating system that coordinates these multiple processors to
execute task(s) concurrently is known as Multiprocessor Operating System.
Depending upon the control structure and its organization, there are three basic types of
multiprocessor operating system:
 Separate supervisor
 Master-slave
 Symmetric Supervision

3|Page Advanced Operating System Notes by Ravindar Mogili

Separate Supervisor Configuration
 Each processor acts as autonomous and independent system.
 Each processor has its own I/O devices and file system.
 All processors have their own kernel, supervisor and data structures.
 There are some common data structures available for communication between processors
using some synchronization mechanism like semaphores.
 As each processor acts independent, it is difficult to perform parallel execution of a single
task (that is, to breakup single task into multiple subtasks and schedule these subtasks on
multiple processors concurrently).
 This configuration is inefficient because supervisor/kernel/data structure code is
replicated for each processor.
 The failure of a processor degrade the performance because there is little coupling among
processors.

Master-Slave configuration
 In master-slave configuration, one processor behaves as a master whereas others behave
as slaves.
 The operating system is executed by only master processor and the slave processors
execute application programs.
 The master processor schedules the work and assigns to slave processors.
 Since the operating system is executed by only single processor, it is efficient and its
implementation (synchronization of shared variables, etc) is easy.
 This arrangement allows the parallel execution of a single task. The single task is divided
into subtasks and allocating these subtasks to multiple processors to execute
concurrently.
 The main limitation of this system is the failure of master processor. When master
processor fails, then slave processors cannot be utilized.
 Examples of such operating systems are Cyber-170 and DEC-10.

The Symmetric configuration

 In symmetric configuration, all processors are autonomous and treated equally.
 It follows floating master method. That is, the master “floats” from one processor to
another processor.
 There is only one copy of kernel (OS) that can be executed by all processors
concurrently. The simplest way to achieve this is to treat the entire operating system as a
critical section and allow only one processor to execute the operating system at one time.
 The execution of operating system by multiple processors can become bottleneck. This
problem can be solved by dividing OS into segments such that they have very little

4|Page Advanced Operating System Notes by Ravindar Mogili

 interaction (i.e., the sharing of variables, communication, etc.). These segments are
executed concurrently by the processors.
 This configuration allows parallel execution of single task.
 Parallel execution of different applications is achieved by maintaining a queue of ready
processors in shared memory. Each ready process is allocated to first available idle
processor until either all processors are busy or the ready queue is emptied.
 It is most difficult configuration to design and implement.
 Example of such operating system is Hydra on C.mmp.

4. Multiprocessor Operating System Design Issues

The design of a Multiprocessor operating system (OS) is complicated as compared to
uniprocessor OS because it must fulfill the following requirements.
 A multiprocessor operating system must be able to support concurrent task execution by
using multiple processors.
 It should be able to exploit the power of multiple processors.
The major design issues of multiprocessor operating systems are:
(i). Threads: Traditional processes impose too much cost for context switching to support
concurrency. But use of threads can solve this problem. A process can be divided into
multiple threads and these threads can be executed simultaneously on multiple
processors. The multiprocessor operating systems should be designed in such a way that
it should support threads concept wherever it is possible to run applications.
(ii). Process Synchronization: When multiple processes active simultaneously, then there is
a chance of two or more processes to access shared address spaces or shared I/O
resources. This may leads to deadlock. So, care must be taken to enforce mutual
exclusion to provide effective synchronization. Thus, synchronization mechanism must
be carefully designed in multiprocessor operating systems; otherwise it could result in
significant performance penalty.
(iii). Processor Scheduling: To ensure the efficient use of its hardware, a multiprocessor
operating system must be able to utilize the processors effectively while executing the
tasks. A multiprocessor operating system, in cooperation with the compiler, should detect
and exploit the parallelism in the tasks being executed. The scheduler must effectively
utilize the readily available processors to execute ready processes and threads.
(iv). Memory management: Memory management on a multiprocessor OS must deal with all
of the issues found on uniprocessor machines. In addition, the design of virtual memory
is complicated due to sharing of the main memory by many processors. The operating
system must maintain a separate map table for each processor for address translation.
When several processors share a page or segment, the operating system must enforce the
consistency of their entries in respective map tables. More care must be taken to perform
page replacement.

5|Page Advanced Operating System Notes by Ravindar Mogili

 (v). Reliability and fault tolerance: The operating system should run even any of the
processor fails. The scheduler and other portions of the operating system must recognize
the loss of a processor and restructure management tables accordingly.
(vi). Protection: As several processes are active at a time in multiprocessor operating system,
care must be taken not to influence the activity of one process by other. To provide such
protection, OS must use various mechanisms to ensure that only processes that have
gained proper authorization from the OS have to use the shared resources of the system.
(vii). Inter-process Communication: multiprocessor operating system should support variety
of models for communication among several active processes and threads.

5. Threads in Multiprocessor Operating System

A program under execution is called process. Every process maintains a data structure
known as Process Control Block (PCB). The PCB contains information such as process ID,
program counter, a stack, saved register values, code segment, data segment and open files, etc.
Maximum concurrency can be achieved by performing switch between processes (context
switching). But, it is expensive to perform switch between processes due to the volume of PCB
data. This problem can be solved by using threads.

A thread is also called a lightweight process. A process can be divided into multiple
threads. Threads represent a software approach to improving performance of operating system by
reducing the context switching overhead. Each thread belongs to exactly one process and no
thread can exist outside a process. Each thread represents a separate flow of control. The
following figure shows the working of a single-threaded and a multithreaded process.

Figure: Single process P with single thread Single process P with 3 threads

6|Page Advanced Operating System Notes by Ravindar Mogili

Threads are of two types. They are:
1. User level Thread
2. Kernel level Thread
User level threads
Threads implemented in userspace are called user threads. The kernel is unaware of them, so
they are managed and scheduled in userspace. No kernel modifications needed to support user
level threads. So, it is efficient to create/destroy them. Context switching between user threads
within the same process is extremely efficient because it does not need interaction with the
kernel. Since scheduling occurs in userspace, the scheduling policy can be more easily adjusted
to the requirements of the program's workload. User level thread provides excellent performance
as compare to kernel level thread. In addition, user-level thread has following advantages:
 No modifications in operating system are required to support user-level threads.
 They are flexible enough to customize the languages or needs of users.

There are also some disadvantages of user level threads.

 If threads in an application need more intervention of kernel, then they cannot be
implemented as user level threads.
 User threads compete with one another and do not take full advantage of multiprocessors.
 User-level thread requires that system call be non-blocking. If thread blocks because of
system call, it will prevent other runnable threads from executing.

Kernel level thread

A kernel thread is a "lightweight" unit of kernel. At least one kernel thread exists within each
process. If multiple kernel threads exist within a process, then they share the same memory and
file resources. Kernel level threads have following advantages.
 The coordination between synchronization and thread scheduling is easily managed.
 They are suitable for multi-threaded applications, such as server processes, where
interactions with kernel are frequent due to IPC, page fault, exceptions etc.
 They have less overhead.
 Kernel is sole provider of thread management operations; it has to provide any feature
needed by any application.
There are also some disadvantages of user level threads.
 Kernel level threads are too costly to create and destroy threads in Kernel.
 Context switching time of kernel level threads is too slow while user level thread is fast.
For this reason system developer prefers user level threads because of excellent
performance.

7|Page Advanced Operating System Notes by Ravindar Mogili

 First class Threads
User-level threads are flexible to implement but when more kernel support needed, their
performance is worst. Kernel level threads functionality is best, but they are not flexible
according to user needs. So, we need user-level threads with advantages of kernel-level thread.
They are known as First class thread. First class thread is a mechanism used to provide first-
class status to user level threads. This allows user-level threads to be used in any reasonable way
like traditional kernel threads. But, their implementation is done by user-level code.

To overcome the problems associated with user-level threads, three mechanisms are provided
to communicate between the kernel and the thread package. These communications occur
without any kernel traps. These mechanisms are as follows:

(i). The kernel and the thread package share important data structures.
(ii). The kernel provides the thread package with software interrupts (signals, upcalls)
whenever a scheduling decision is required.
(iii). Scheduler interfaces are provided to enable the sharing of data abstractions between
dissimilar thread packages.

Scheduler Activations
Scheduler Activation is a mechanism that enables user-level threads to have complete
control over scheduling policy. It provides kernel-level thread functionality with user-level
thread flexibility and performance.

Goals of Scheduler Activations

 No processor idles when a thread is ready.

 No higher-priority thread waits while a lower-priority thread runs.
 During any thread blocking, other threads can run.

6. Process Synchronization
Process Synchronization is a mechanism that handles concurrent access to shared
resources by processes without any inconsistencies.

The Synchronization mechanisms of a uniprocessor system do not work on

multiprocessor system. It is because if a process on a uniprocessor machine (just one CU)
requires accessing some critical kernel table, the kernel code disable interrupts before touching
the table. So, this process can do its work without any other processes touching the table before it
is finished. But, on a multiprocessor, disabling interrupts affects only the CPU doing the disable.
Other CPUs continue to run and can still touch the critical table. So, a proper new mutex
protocol must be developed for multiprocessor system.
8|Page Advanced Operating System Notes by Ravindar Mogili
i. The Test-and-Set Instruction

The test-and-set instruction atomically reads and modifies the contents of a memory location in
one memory cycle. It is defined as follows (variable m is a memory location):

function Test_and_Set(var m: boolean): boolean;

begin
test_Set:= m;
m:=true;
return(Test_Set);
end;

The Test_and_Set instruction returns the current value of variable m and sets it to true. This
instruction can be used to implement P and V operations on a binary semaphore, S, in the
following way (S is implemented as a memory location):

P(S): while( Test-and-Set(S)) do nothing; Entry section

Critical section code; Critical section

V(S): S:= false; Exit section

Initially, S is set to false. When a P(S) operation is executed for the first time, Test-and-Set(S)
returns a false value (and sets S to true) and the “while” loop of the P(S) operation terminates.
All subsequent executions of P(S) keep looping because S is true until a V(S) operation is
executed.

ii. The Swap Instruction

The swap instruction atomically exchanges the contents of two variables (e.g., memory
locations). It is defined as follows (x and y are two variables):

procedure swap(var x, y: boolean);

var temp: boolean;
begin
temp=x;
x = y;
y = temp;
end;

P and V operations can be implemented using the swap instruction in the following way (P is a
variable private to the processor and S is a memory location):

9|Page Advanced Operating System Notes by Ravindar Mogili

 P(S): p = true;
Entry section
repeat swap(S, p) until p=false;

; Critical section code; Critical section

V(S): S:= false; Exit section

The above two implementations of the P operation employ busy-waiting and therefore increase
the traffic on the interconnection network. Another problem with test-and-set and swap
instructions is that if n processors execute any of these operations on the same memory location,
the main memory will perform n such operations on the location even though only one of these
operations will succeed.

iii. The Fetch-and-Add Instruction

The fetch-and-add instruction is a multiple operation memory access instruction that atomically
adds a constant ‘c’ to a memory location ‘m’ and returns the previous contents of the memory
location. This instruction is defined as follows

Function Fetch-and-Add(m: integer; c: integer):

var temp: integer;
begin
temp = m;
m = m +c;
return (temp);
end;

An interesting property of this instruction is that it is executed by the hardware placed in

the interconnection network (not by the hardware present in the memory modules). When several
processors concurrently execute a fetch-and-add instruction on the same memory location, these
instructions are combined in the network and are executed by the network in the following way.
A single increment, which is the sum of the increments of all these instructions, is added to the
memory location. A single value is returned by the network to each of the processors, which is
an arbitrary serialization of the execution of the individual instructions. If a number of processors
simultaneously perform fetch-and-add instructions on the same memory location, the net result is
as if these instructions were executed serially in some unpredictable order.

The fetch-and-add instruction is powerful and it allows the implementation of P and V operations
on a general semaphore, S, in the following manner:

10 | P a g e Advanced Operating System Notes by Ravindar Mogili

 P(S): while (Fetch-and-Add(S, -1) < 0) do
begin Entry section
Fetch-and-Add(S, 1);
while (S = < 0) do nothing;
end;

; Critical section code; Critical section

V(S): Fetch-and-Add(S, 1); Exit section

The outer “while-do” statement ensures that only one processor succeeds in decrementing S to 0
when multiple processors try to decrement variable S. All the unsuccessful processors add 1 back
to S and again try to decrement it. The second “while-do” statement forces an unsuccessful
processor to wait (before retrying) until S is greater than 0.

iv. Implementation of Process Wait

In all the implementations of a P operation discussed thus far, several processors may
wait for the semaphore to open by executing the respective atomic machine language instructions
concurrently. This wait can be implemented in three ways:

Busy Waiting: In busy-waiting, processors continuously execute the atomic instruction to check
for the status of the shared variable. It (also called spin lock) wastes processor cycles and
degrade the system performance.

Sleep-Lock: In sleep-lock, instead of continuously spinning the lock, a process is suspended

when it fails to obtain the lock and this suspended process reactivated when the lock is freed.
This method substantially reduces network traffic due to busy-waiting.

Queueing: In queueing, a process waiting for a semaphore is placed in a global queue. A waiting
process is dequeued and activated by a V operation on the semaphore. Although queueing
eliminates network traffic and the wastage of processor cycles due to busy-waiting, it introduces
other processing overhead because the enqueue and the dequeue operations require the execution
of several instructions. Also, the queue forms a shared data structure and must be protected
against concurrent access.

v. The Compare-and-Swap Instruction

The compare-and-swap (CAS) is an atomic instruction used in multithreading to achieve
synchronization. It compares the contents of a memory location with a given value and, only if
they are the same, modifies the contents of that memory location to a new given value. This is
done as a single atomic operation. The atomicity guarantees that the new value is calculated
based on up-to-date information; if the value had been updated by another thread in the

11 | P a g e Advanced Operating System Notes by Ravindar Mogili

meantime, the write would fail. It use r1 and r2 are two registers of a processor and m is a
memory location. This instruction is defined as follows
Compare-and-Swap(var r1, r2, m: integer);
var temp: integer;
begin
temp:= m;
if temp:= rl then { m:=r2; z:= 1}
else {rl:= temp;z :=0}
end;
If the contents of r1 and m are identical, this instruction assigns the contents of r2 to m and sets z
to 1. Otherwise, it assigns the contents of m to rl and sets z to 0. Variable z is a flag that indicates
the success of the execution of the instruction.

7. Processor Scheduling in Multiprocessor OS

In a single processor system, scheduling is easy because all the processes to be executed
only on one processor. But in a multiprocessor system, scheduling is complicated because the
scheduler has to decide which process to be executed on which CPU. So, scheduling in
multiprocessor system has the following issues.
(i). Cache Corruption: When a program executing on a processor, to increase execution
speed, the cache inside the processor stores data related to executing program. But when
the executing program blocked, then other ready program may execute on this processor.
When the previous blocked program becomes ready, it may be assigned to other
processor. So, processor switching from one task to another task, makes the cache data
may not be useful. This problem is called cache corruption. It can degrade the
performance.
(ii). Context Switching Overheads: In context switching, there is a need for execution of a
large number of instructions to store and save the registers with initial values, address
space, etc. Due to this, the execution progress of applications is reduced automatically.
(In addition, context switching causes the problems of Cache Corruption.)
(iii). Preemption inside Spinlock (Controlled Critical Section): When a task is preempted
inside a critical section, other tasks spinning the lock for entering the same critical section
waste the CPU cycles. This is because they continue to be in spinning mode until the
preempted task is rescheduled and complete the execution of the critical section. This
problem degrades system performance.
To solve the above design issues in multiprocessor OS scheduling the following solutions/
strategies proposed by different researchers. They are:

(i). Co-Scheduling: In co-scheduling all runnable tasks of an application are scheduled on

the processors simultaneously. Whenever a task of an application needs to be preempted,
all the tasks of that application are preempted. In other words co-scheduling performs

12 | P a g e Advanced Operating System Notes by Ravindar Mogili

 context-switching between applications but not on tasks of several different applications.
It means a single application with multiple tasks is executed on multiple processors for a
time slice. In the next time slice, other application tasks are executed, and so on. It solves
spin-lock problem.
(ii). Smart Scheduling: The smart scheduler has two nice features. First, it avoids
preempting a task when the task is inside its critical section. Second, it avoids the
rescheduling of tasks that were busy-waiting at the time of their preemption until the task
that is executing the corresponding critical section releases it. When a task enters a
critical section, it sets a flag. The scheduler does not preempt a task if its flag is set. On
exit from a critical section, a task resets the flag.
The smart scheduler eliminates the resource waste due to a processor spinning a
lock that is held by a task preempted inside its critical section. However, it does not make
any attempt neither to reduce the overhead due to context switching nor to reduce the
performance degradation due to cache corruption.
(iii). Scheduling in the NYU Ultracomputer: Scheduling in the NYU Ultracomputer
combines the strategies of the previous two scheduling techniques. In this technique,
tasks can be formed into groups and the tasks in a group can be scheduled in any of the
following ways:
 A task can be scheduled or preempted in the normal manner.
 All the tasks in a group are scheduled or preempted simultaneously (as in co-
scheduling).
 Tasks in a group are never preempted.
In addition, a task can prevent its preemption irrespective of the scheduling policy
(one of the above three) of its group. This provision can be used to efficiently implement
a spin-lock.
This scheduling technique is flexible because it allows the selection of a variety of
scheduling policies and a different scheduling technique can be used for different task
groups. However, this scheduling technique does not reduce the overhead due to context
switching or the performance degradation due to cache corruption.
(iv). Affinity Based Scheduling: Affinity based scheduling is the first scheduling policy to
address the problem of cache corruption. In this policy, a task is scheduled on the
processor where it last executed. This policy alleviates the problem of cache corruption
because it is likely that a significant portion of the working set of that task is present in
the cache of that processor when the task is rescheduled.
Affinity based scheduling, restricts load balancing among processors because a
task cannot be scheduled on any processor. (Tasks are tied to specific processors) In this
scheduling strategy, the system suffers from load imbalance because a task may wait at a
busy processor while other processors are idle.
(v). Scheduling in the Mach Operating System: In the Mach operating system, an
application or a task consists of several threads. In the Mach operating system, all the

13 | P a g e Advanced Operating System Notes by Ravindar Mogili

 processors of a multiprocessor are grouped in disjoint sets, called processors sets. The
processors in a processor set are assigned a subset of threads for execution. These
processors use priority scheduling to execute the threads assigned to their processor set.
Threads can have priority ranging from 0 to 31, where 0 and 31 are the highest and the
lowest priorities, respectively. Each processor set has an array of 32 ready queues—one
queue to store the ready threads of each priority. When a thread with priority i becomes
ready, it is appended to the ith queue. In addition, every processor has a local ready queue
that consists of the threads that must be executed only by that processor. Clearly, it is
two-level priority scheduling: all the threads in a local queue have priority over all the
threads in the global queue and there are also priorities inside each of these two queues.
When a processor becomes idle, it selects a thread for execution in the following
manner. If the local ready queue of the processor is nonempty, it selects the highest
priority thread for execution. Otherwise, it selects the highest priority thread from the
global ready queues for execution. If both the queues (local and global) are empty, the
Processor executes a special idle thread until a thread becomes ready. When a thread runs
out of its time slice at a processor, it is preempted only if an equal or higher priority ready
thread is present. Otherwise, the thread receives another time slice at the processor. The
length of the time slice is variable and depends upon the number of ready threads. The
higher the number of ready threads, the shorter the time slice.

8. Distributed File System Architecture

Distributed file system (DFS) is a method of storing and accessing files based on a client/
server architecture. In this file system, one or more central servers store files and any number of
remote clients in the network can access these files with proper authorization rights.

Figure: Distributed File System Architecture

When the client device retrieves a file from the server, the file appears as a normal file
on the client machine, and the user is able to work with the file in the same ways as if it were
stored locally on the workstation. When the user finishes working with the file, it is returned over
the network to the server, which stores the now-altered file for retrieval at a later time.

14 | P a g e Advanced Operating System Notes by Ravindar Mogili

The two most important services present in a distributed file system are the name server and
cache manager. A name server is a process that maps names specified by clients to stored objects
such as files and directories. The mapping (also referred to as name resolution) occurs when a
process references a file or directory for the first time. A cache manager is a process that
implements file caching. In file caching, a copy of data stored at a remote file server is brought to
the client's machine when referenced by the client. Subsequent accesses to the data are
performed locally at the client, thereby reducing the access delays due to network latency. Cache
managers can be present at both clients and file servers. Cache managers at the servers cache
files in the main memory to reduce delays due to disk latency. If multiple clients are allowed to
cache a file and modify it, the copies can become inconsistent. To avoid this inconsistency
problem, cache managers at both servers and clients coordinate to perform data storage and
retrieval operations. Typically, data access in a distributed file system proceeds as shown in
below figure

A request by a process to access a data block is checked in the local cache (client cache) of the
machine (client) on which the process is running. If the block is not in the cache, then the local
disk is checked for the presence of the data block. If the block is present, then the request is
satisfied and the block is loaded into the client cache. If the block is not stored locally, then the
request is passed on to the appropriate file server (as per the name server). The server checks its
own cache for the presence of the data block before issuing a disk I/O request. The data block is
transferred to the client cache in any case and loaded to the server cache if it was missing in the
server cache.

15 | P a g e Advanced Operating System Notes by Ravindar Mogili

 9. Mechanisms for building Distributed File Systems
The basic mechanisms used for building distributed file system are:

(i). Mounting: Mount mechanisms allow the binding of different file namespaces to form a
single hierarchical namespace. In distributed system, when a remote file needs to be
accessed, a special entry known as a mount point is created at some position in the local file
namespace to bind the root of remote file name space. This mechanism is mostly used by
UNIX operating system.

(ii). Client Caching: Caching is to reduce delays in the accessing of data. In file caching, a copy
of data stored at a remote file server is brought to the client, when first time the file is
referenced by the client. Subsequent access to the same file data is performed locally at the
client, thereby reducing access delays due to network latency. Data can either be cached in
the main memory or on the local disk of the clients. Similarly, data is also cached in the
main memory (server cache) at the servers to reduce disk access latency.

(iii). Hints: Caching is an important mechanism to improve the performance of a file system;
however, the consistency of the cached items requires expensive client/server protocols.
When the cached data treated as hints, then cached data may not be completely accurate. But
when applications use the invalid cached data, hints make them to recover. For example,
after the name of a file is mapped to an address, that address is stored as a hint in the cache.
If the address later fails, it is purged from the cache. The name server is consulted to provide
the actual location of the file and the cache is updated.

(iv). Bulk Data Transfer: The data transfer in a network requires the execution of various
communication layer protocols. Data is assembled and disassembled into packets; packets
copied between the buffers of various layers, transmitted and acknowledged over the
network. For small amounts of data, the transit time is low, but, network latency is high due
to establishing peer connections, packets transmission and acknowledging each packet.
Transferring data in bulk reduces the relative cost of this overhead at the source and
destination. With this scheme, multiple consecutive data packets are transferred from

16 | P a g e Advanced Operating System Notes by Ravindar Mogili

 servers to clients (or vice versa) in a burst. At the source, multiple packets are formatted and
transmitted with one context switch from client to kernel. At the destination, a single
acknowledgement is used for the entire sequence of packets received.

(v). Encryption: A number of possible threats exist such as unauthorized access of information,
unauthorized modification of information, or unauthorized denial of resources. Encryption is
used for enforcing security in distributed systems. Encryption converts the original text into
secret code text.

10. Design issues in Distributed File Systems

(i). Naming and name resolution: A name is associated with every object such as a file or
directory. Name resolution refers to mapping a name to an object. There are three
approaches to name files in a distributed environment. They are: The first approach is to
concatenate the host name to the file name. It guarantee unique file name and does not need
any name resolution. The problem with this approach is that moving a file from one host to
another requires changes in the filename. The second approach is to mount remote
directories onto local directories. Once mounted, accessing the file becomes location
transparent and name resolution is simple. The third approach is to use single global
directory where all the files in the system belong to a single name space.

(ii). Caches on Disk or Main Memory: Caching is to reduce delays in the accessing of data. In
file caching, a copy of data stored at a remote file server is brought to the client, when first
time the file is referenced by the client. Subsequent access to the same file data is
performed locally at the client, thereby reducing access delays due to network latency. Data
can either be cached in the main memory or on the local disk of the clients. Accessing a
cache in main memory is much faster than accessing a cache on local disk.

(iii). Writing Policy: The writing policy decides when a modified cache block at a client should
be transferred to the server. The simplest policy is write-through. In write-through, all
modifications done by the applications at clients are also carried out at the servers
immediately. The main advantage of write-through is reliability. In the event of a client
crash, little information is lost. A write-through policy, however, does not take advantage
of the cache. An alternate writing policy is delayed writing policy. In delayed writing,
modifications done by applications at client are reflected at the server after some delay. The
problem with this is, in the event of a client crash, a significant amount of data can be lost.

(iv). Cache Consistency: The Cache Coherence Problem is the challenge of keeping multiple
local caches synchronized when one of the processors updates its local copy of data which
is shared among multiple caches. There are two approaches to guarantee that the cache data
is consistent.

17 | P a g e Advanced Operating System Notes by Ravindar Mogili

  In the server-initiated approach, servers inform cache managers whenever the data in
the client caches become stale. Cache managers at clients can then retrieve the new
data or invalidate the blocks containing the old data in their cache.
 In the client-initiated approach, it is the responsibility of the cache managers at the
clients to validate data with the server before returning it to the clients.
 A third approach for cache consistency is simply not to allow file caching when
concurrent-write sharing occurs. In concurrent-write sharing, a file is open at
multiple clients and at least one client has it open for writing. In this approach, the file
server has to keep track of the clients sharing a file. When concurrent-write sharing
occurs for a file, the file server informs all the clients to purge their cached data items
belonging to that file.

(v). Availability: Availability is one of the important issues in the design of distributed file
systems. The failure of servers or the communication network can severely affect the
availability of files. Replication is the primary mechanism used for enhancing the
availability of files in distributed file systems. Under replication, many copies or replicas of
files are maintained at different servers. Replication is inherently expensive because of the
extra storage space required to store the replicas and the overhead incurred in maintaining
all the replicas up to date. The most serious problems with replication are (I) how to keep
the replicas of a file consistent and (2) how to detect inconsistencies among replicas of a
file and subsequently recover from these inconsistencies.

(vi). Scalability: The issue of scalability deals with the suitability of the design of a system to
cater to the demands of a growing system. Currently, client-server organization is a
commonly used approach to structure distributed file systems. Caching, which reduces
network latency and server-load is the primary technique used in client-server organization
to improve the client response time. Caching, however, introduces the cache consistency
problem. The solution to cache consistency is discussed in the above section.

(vii). Semantics: The semantics of a file system says a read operation of a data will return the
latest write (updated) operation of that data. In distributed file systems, guaranteeing the
above semantics is difficult and expensive due to usage of cache. One solution is to allow
all the reads and writes from various clients will have to go through the server. The server
can potentially become a bottleneck and the overheads are high because of high traffic
between the server and the clients.

18 | P a g e Advanced Operating System Notes by Ravindar Mogili