CS 3551 DISTRIBUTED

COMPUTING
Logical Time: Physical Clock Synchronization: NTP – A Framework for a System of Logical
Clocks – Scalar Time – Vector Time;
Message Ordering and Group Communication: Message Ordering Paradigms – Asynchronous
Execution with Synchronous Communication – Synchronous Program Order on Asynchronous
System – Group Communication – Causal Order – Total Order;
Global State and Snapshot Recording Algorithms: Introduction – System Model and Definitions –
Snapshot Algorithms for FIFO Channels.
How interaction takes place in distributed system? 3 types of message
passing

on receiving
on sending
when order of execution changes
the message is received wrong

both a and b executed

at the same process at
same place
interact only by the message passing between processors

tow denoteds all send and receive pairs

Message Ordering Paradigms

1. Asynchronous executions - Non - FIFO

it is not necessay to be in the

same orer as send the message

● Non-FIFO transmission : On any logical link between two nodes in the system, messages
may be delivered in any order, not necessarily first-in first-out. Such executions are also known
as non-FIFO executions.
● Although each physical link typically delivers the messages sent on it in FIFO order due to the
physical properties of the medium, a logical link may be formed as a composite of physical
links and multiple paths may exist between the two end points of the logical link.
messages can be received on any order
Message Ordering Paradigms

2. FIFO Execution
occurs at same process

casulally dependent
● On any logical link in the system, messages are necessarily delivered in the order in which
they are sent. Although the logical link is inherently non-FIFO, most network protocols
provide a connection-oriented service at the transport layer.
● Design FIFO algo over non-FIFO channel:
○ separate numbering scheme to sequence the messages on each logical channel.

○ The sender assigns and appends a sequence_num, connection_id tuple to each message.

○ The receiver uses a buffer to order the incoming messages as per the sender’s sequence numbers, and
accepts only the “next” message in sequence.
Message Ordering Paradigms

3. Causally Ordered

● If two send events s and s are related by causality ordering (not physical
time ordering), then a causally ordered execution requires that their
corresponding receive events r and r occur in the same order at all
common destinations.
3. Causally Ordered Examples
Delivery Event
● To implement CO, we distinguish between the arrival of a message and its
delivery.
● A message m that arrives in the local OS buffer at Pi may have to be delayed
until the messages that were sent to Pi causally before m was sent (the
“overtaken” messages) have arrived and are processed by the application.
● The delayed message m is then given to the application for processing.
● The event of an application processing an arrived message is referred to as a
delivery event (instead of as a receive event) for emphasis.
Other Definitions
4. Synchronous execution (SYNC)
send and receive events that occur simultaneously
Key Points
● When all the communication between pairs of processes uses synchronous
send and receive primitives, the resulting order is the synchronous order.
● As each synchronous communication involves a handshake between the
receiver and the sender, the corresponding send and receive events can be
viewed as occuring instantaneously and atomically
Synchronous program order on an asynchronous system

How non-determinism occurs in DC?

1. A receive call can receive a message from any sender who has sent a
message, if the expected sender is not specified.

2. Multiple send and receive calls which are enabled at a process can be
executed in an interchangeable order.
Rendezvous

Multiway rendezvous:- synchronous communication among an arbitrary number

of asynchronous processes.

Binary rendezvous:- synchronous communication between a pair of processes at

a time.

Support for binary rendezvous communication was first provided by programming

languages such as CSP and Ada. We consider here a subset of CSP.
Simple Command with Notation

● Each communication command may be a part of a guard Gi, and may also
appear within the statement block CLi. A guard Gi is a boolean expression.
● If a guard Gi evaluates to true then CLi is said to be enabled, otherwise CLi is
said to be disabled.
● A send command of local variable x to process Pk is denoted as “x !Pk.”
● A receive from process Pk into local variable x is denoted as “Pk ? x.”
Observations about synchronous communication under
binary rendezvous
1. For the receive command, the sender must be specified.
2. However, multiple receive commands can exist.
3. A type check on the data is implicitly performed.
4. Send and received commands may be individually disabled or enabled. A
command is disabled if it is guarded and the guard evaluates to false. The
guard would likely contain an expression on some local variables.
5. Synchronous communication is implemented by scheduling messages under
the covers using asynchronous communication.
6. Scheduling involves pairing of matching send and receive commands that are
both enabled.
7. The communication events for the control messages under the covers do not
alter the partial order of the execution.
Algorithm for binary rendezvous
Constraints:

● Schedule on-line, atomically, and in a distributed manner, i.e., the scheduling

code at any process does not know the application code of other processes.
● Schedule in a deadlock-free manner (i.e., crown-free), such that both the
sender and receiver are enabled for a message when it is scheduled.
● Schedule to satisfy the progress property (i.e., find a schedule within a
bounded number of steps) in addition to the safety (i.e., correctness) property.
Assumptions
Algorithm in Simple Steps

● To send to a lower priority process

a. The sender issues send(M)
b. blocks until ack(M) arrives
c. Thus, when sending to a lower priority process, the sender blocks waiting for the partner
process to synchronize and send an acknowledgement.
● To send to a higher priority process
a. The sender issues send(request(M)), does not block, and awaits permission.
b. When permission(M) arrives, the sender issues send(M).
Group communication
● Processes across a distributed system cooperate to solve a joint task. Often,
they need to communicate with each other as a group, and therefore there
needs to be support for group communication.
● A message broadcast is the sending of a message to all members in the
distributed system.
● If a multicast algorithm requires the sender to be a part of the destination
group, the multicast algorithm is said to be a closed group algorithm.
● If the sender of the multicast can be outside the destination group, the
multicast algorithm is said to be an open group algorithm
Causal Order
in r1,p2 process receives first before p1
Causal order has many applications such as updating replicated data, allocating
requests in a fair manner, and synchronizing multimedia streams. We explain here
the use of causal order in updating replicas of a data item in the system
Criteria must be met by a causal ordering protocol