Chapter 4 - Communication 

Introduction

 Interprocess communication is at the heart of all distributed systems

 Communication in distributed systems is based on message passing

as offered by the underlying network as opposed to using shared
memory

 modern distributed systems consist of thousands of processes

scattered across an unreliable network such as the Internet

 unless the primitive communication facilities of the network are

replaced by more advanced ones, development of large scale
Distributed Systems becomes extremely difficult
Here, in this chapter we will cover with

 review of how processes communicate in a network (the rules

or the protocols) and their structures

 Introduce the four widely used communication models for

distributed systems:
 Remote Procedure Call (RPC)
 Remote Method Invocation (RMI)
 Message-Oriented Middleware (MOM)

 Streams Oriented Communication

Layered Protocols
 two computers, possibly from different manufacturers, must
be able to talk to each other
 for such a communication, there has to be a standard
 The ISO OSI (Open Systems Interconnection) Reference
Model is one of such standards - 7 layers
 TCP/IP protocol suite is the other; has 4 or 5 layers
 OSI
 Open – to connect open systems or systems that are open
for communication with other open systems using standard
rules that govern the format, contents, and meaning of the
messages sent and received
 these rules are called protocols
 two types of protocols: connection-oriented and
connectionless
layers, interfaces, and protocols in the OSI model
 Physical: Physical characteristics of the media

 Data Link: Reliable data delivery across the link

 Network: Managing connections across the network
or routing
 Transport: End-to-end connection and reliability (handles
lost packets); TCP (connection-oriented),
UDP (connectionless), etc.
 Session: Managing sessions between applications
(dialog control and synchronization); rarely
supported
 Presentation: Data presentation to applications; concerned
with the syntax and semantics of the
information transmitted
 Application: Network services to applications; contains
protocols that are commonly needed by
users; FTP, HTTP, SMTP, ...
OSI Reference Model
 Application Protocols
 file transfer (FTP - File Transfer Protocol)
 HTTP - Hypertext Transfer Protocol for accessing data on the
WWW
 Middleware Protocols
 a middleware is an application that contains general-
purpose protocols to provide services
 example of middleware services
 authentication and authorization services
 distributed transactions (commit protocols; locking
mechanisms)
 middleware communication protocols (calling a
procedure or invoking an object remotely, synchronizing
streams for real-time data, multicast services)
 hence an adapted reference model for networked
communications is required
An adapted reference model for networked communication
4.1.Remote Procedure Call
 The first distributed systems were based on explicit message
exchange between processes through the use of explicit send and
receive procedures; but do not allow access transparency
 in 1984, Birrel and Nelson introduced a different way of handling
communication: RPC
 it allows a program to call a procedure located on another machine
 The protocols are independent of the underlying transport
protocols
 Use marshalling and unmarshalling to hide differences due to
hardware architectures
 When a process on machine A calls' a procedure on
machine B, the calling process on A is
suspended, and execution of the called
procedure takes place on B.
Information can be transported from the caller to the
callee in the parameters and can come back in the
procedure result.
No message passing at all is visible to the programmer.
This method is known as Remote Procedure Call, or often
just RPC.
 Remote Procedure Call (RPC) is a protocol that one
program can use to request a service from a
program located in another computer on a
network without having to understand the
network's details.
 A procedure call is also sometimes known as a
function call or a subroutine call.
 A stub in distributed computing is a piece of
code used for converting parameters passed
during a Remote Procedure Call (RPC).
 The main idea of an RPC is to allow a local
computer (client) to remotely call procedures on a
remote computer (server).
 RPC is a powerful technique for constructing distributed, client-
server based applications.
 It is based on extending the notion of conventional, or local
procedure calling, so that the called procedure need not exist in the
same address space as the calling procedure.
 The two processes may be on the same system, or they may be on
different systems with a network connecting them.
 By using RPC, programmers of distributed applications avoid the
details of the interface with the network.
 Steps of a Remote Procedure Call

1. The client procedure calls a client stub passing

parameters in the normal way.
2. The client stub marshals the parameters, builds
the message, and calls the local OS.
3. The client's OS sends the message (using the
transport layer) to the remote OS.
4. The server remote OS gives transport layer
message to a server stub.
5. The server stub demarshals the parameters and
calls the desired server routine.
6. The server routine does work and returns result
to the server stub via normal procedures.
7. The server stub marshals the return values into
the message and calls local OS.
8. The server OS (using the transport layer) sends
the message to the client's OS.
9. The client's OS gives the message to the client
stub
10. The client stub demarshals the result, and
execution returns to the client.
 Parameter Passing
 The function of the client stub is to take its
parameters, pack them into a message, and send
them to the server stub.
 Packing parameters into a message is called
parameter marshaling.
1. Passing Value Parameters
 E.g.,
consider a remote procedure add(i, j),
where i and j are integer parameters
steps involved in doing remote computation through RPC
 4.2. Remote Object (Method) Invocation (RMI)
 Resulted from object-based technology that has proven its value in
developing non-distributed applications
 it is an expansion of the RPC mechanisms
 it enhances distribution transparency as a consequence of an
object that hides its internal from the outside world by means of a
well-defined interface
 Distributed Objects
 an object encapsulates data, called the state, and the operations
on those data, called methods
 methods are made available through interfaces
 The state of an object can be manipulated only by
invoking methods.
 this allows an interface to be placed on one
machine while the object itself resides on another
machine; such an organization is referred to as a
distributed object
 the state of an object is not distributed, only the
interfaces are; such objects are also referred to as
remote objects
 the implementation of an object’s interface is
called a proxy (analogous to a client stub in RPC
systems)
 it is loaded into the client’s address space when a
client binds to a distributed object
 tasks: a proxy marshals method invocation into
messages and unmarshals reply messages to return
the result of the method invocation to the client
 a server stub, called a skeleton, unmarshals
messages and marshals replies
common organization of a remote object with client-side proxy
Binding a Client to an Object
 a process must first bind to an object before invoking its methods,
which results in a proxy being placed in the process’s address space
 binding can be implicit (directly invoke methods using only a
reference to an object) or explicit (by calling a special function)
 an object reference could contain
 network address of the machine where the object resides
 endpoint of the server
 an identification of which object
 the protocol used
 ...
 Persistence and Synchronicity in Communication
assume the communication system is organized as a computer
network shown below

general organization of a communication system in which

hosts are connected through a network
 communication can be:
 persistent or transient
 asynchronous or synchronous
 transient: a message that has been submitted for
transmission is stored by the communication system
only as long as the sending and receiving applications
are executing
 asynchronous: a sender continues immediately after it has
submitted its message for transmission
 synchronous: the sender is blocked until its message is stored in
a local buffer at the receiving host or delivered to the receiver
persistent asynchronous communication persistent synchronous communication
 the different types of communication can be
combined
 persistent asynchronous: e.g., email
 transient asynchronous: e.g., UDP, asynchronous
RPC
 in general there are six possibilities
transient asynchronous communication receipt-based transient synchronous communication

weakest form; the sender is

blocked until the message is stored
in a local buffer at the receiving
host
 response-based transient synchronous
delivery-based transient synchronous communication
communication at message delivery
 the sender is blocked until the  strongest form; the sender is
message is delivered to the blocked until it receives a reply
receiver for further processing; message from the receiver
e.g., asynchronous RPC
 4.3. Message Oriented Communication
 RPCs and RMIs are not adequate for all distributed system
applications
 the provision of access transparency may be good but they have
semantics that is not adequate for all applications
 example problems
 they assume that the receiving side is running at the time of
communication
 a client is blocked until its request has been processed
Message-Oriented Transient Communication
 many applications are built on top of the simple message-oriented
model offered by the transport layer
 standardizing the interface of the transport layer by providing a
set of primitives allows programmers to use messaging protocols
 they also allow porting applications

1. Berkley Sockets
 an example is the socket interface as used in Berkley UNIX
 a socket is a communication endpoint to which an application
can write data that are to be sent over the network, and from
which incoming data can be read.
socket primitives for TCP/IP
connection-oriented communication pattern using sockets
The Message-Passing Interface (MPI)
 sockets were designed to communicate across
networks using general-purpose protocol stacks such
as TCP/IP
 they were not designed for proprietary protocols
developed for high-speed interconnection networks;
of course portability will suffer
 MPI is designed for parallel applications and tailored
for transient communication
 MPI assumes communication takes place within a
known group of processes, where each group is
assigned an identifier (groupID)
 each process within a group is also assigned an identifier
(processID)
 a (groupID, processID) identifies the source or destination
of a message, and is used instead of a transport-level
address
 Message-Oriented Persistent Communication

 there are message-oriented middleware services,

called message-queuing systems or Message-
Oriented Middleware (MOM)
 they support persistent asynchronous
communication
 they have intermediate-term storage capacity for
messages, without requiring the sender or the
receiver to be active during message transmission
 unlike Berkley sockets and MPI, message transfer
may take minutes instead of seconds or
milliseconds
 Message-Queuing Model
 applications communicate by inserting messages in specific
queues
 it permits loosely-coupled communication
 the sender may or may not be running; similarly the receiver may
or may not be running, giving four possible combinations
Message Brokers
 how can applications understand the messages they
receive
 each receiver can not be made to understand
message formats of new applications
 hence, in a message-queuing system conversations
are handled by message brokers
 a message broker converts incoming messages to a
format that can be understood by the destination
application based on a set of rules
4.4. Stream Oriented Communication
 until now, we focused on exchanging independent and complete units
of information
 time has no effect on correctness; a system can be slow or fast
 however, there are communications where time has a critical role
 Multimedia
 media
 storage, transmission, interchange, presentation,
representation and perception of different data types:
 text, graphics, images, voice, audio, video, animation, ...
 movie: video + audio + …
 multimedia: handling of a variety of representation media
 end user pull
 information overload and starvation
 technology push
 emerging technology to integrate media
 The Challenge

 new applications
 multimedia will be pervasive in few years (as graphics)
 storage and transmission
 e.g., 2 hours uncompressed HDTV (1920×1080) movie: 1.02
TB (1920×1080x3x25x60x60x2)
 videos are extremely large, even compressed
 continuous delivery
 e.g., 30 frames/s (NTSC), 25 frames/s (PAL) for video
 guaranteed Quality of Service
 admission control
Types of Media
 two types
 discrete media: text, executable code, graphics,
images; temporal relationships between data items
are not fundamental to correctly interpret the data
 continuous media: video, audio, animation; temporal
relationships between data items are fundamental to
correctly interpret the data
 a data stream is a sequence of data units and
can be applied to discrete as well as
continuous media
 stream-oriented communication provides
facilities for the exchange of time-dependent
information (continuous media) such as
audio and video streams
 timing in transmission modes
 asynchronous transmission mode: data items are transmitted
one after the other, but no timing constraints; e.g. text transfer
 synchronous transmission mode: a maximum end-to-end delay
defined for each data unit; it is possible that data can be
transmitted faster than the maximum delay, but not slower
 isochronous transmission mode: maximum and minimum end-
to-end delay are defined; also called bounded delay jitter;
applicable for distributed multimedia systems
 a continuous data stream can be simple or complex
 simple stream: consists of a single sequence of data; e.g., mono
audio, video only
 complex stream: consists of several related simple streams that
must be synchronized; e.g., stereo audio, video consisting of
audio and video (may also contain subtitles, translation to other
languages, ...)
Quality of Service (QoS)
 QoS requirements describe what is needed from the underlying
distributed system and network to ensure acceptable delivery;
e.g. viewing experience of a user
 for continuous data, the concerns are
 timeliness: data must be delivered in time
 volume: the required throughput must be met
 reliability: a given level of loss of data must not be exceeded
 quality of perception; highly subjective
QoS Management
 can be static or dynamic
 Static QoS Management Functions
 specification
 e.g., deterministic range for timeliness, volume and
reliability categories
 negotiation
 the application may accept lower level of QoS for
lower cost
 admission control
 if this test is passed, the system has to guarantee
the promised QoS
 resource reservation
 may be necessary to provide guaranteed QoS
 Dynamic QoS Management Functions

 monitoring
 notices deviation from QoS level
 at a certain level of granularity (e.g., every 100 ms)
 policing
 detect participants not keeping themselves to the contract
 e.g., source sends faster than negotiated (e.g., 25 fps)
 maintenance
 sustaining the negotiated QoS
 e.g., the system requires more resources
 renegotiation
 client tries to adapt – may be can accept lower QoS
 MULTICAST COMMUNICATION

 multicast communication in distributed systems

is the support for sending data to multiple
receivers, also known as multicast
communication.
 An important class of communication protocols
in distributed systems is multicasting
 The basic idea is to disseminate information
from one sender to multiple receivers
End!!!