DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

NAME OF THE SUBJECT : DISTRIBUTED SYSTEMS

Subject code : CS3551

Regulation : 2017

UNIT-II MESSAGE ORDERING & SNAPSHOTS

 UNIT II CS8603
MESSAGE ORDERING & SNAPSHOTS

MESSAGE ORDERING & SNAPSHOTS

LOGICAL TIME - Message ordering and group communication: Message ordering paradigms
–Asynchronous execution with synchronous communication –Synchronous program order on an
asynchronous system –Group communication – Causal order (CO) – Total order. Global state and
snapshot recording algorithms: Introduction –System model and definitions –Snapshot algorithms
for FIFO channels

1.2 LOGICAL TIME

Logical clocks are based on capturing chronological and causal relationships of processes and
ordering events based on these relationships.

Precise physical clocking is not possible in distributed systems. The asynchronous

distributed systems spans logical clock for coordinating the events.Three types of logical
clock are maintained in distributed systems:
 Scalar clock
 Vector clock
 Matrix clock

In a system of logical clocks, every process has a logical clock that is advanced using a set
of rules. Every event is assigned a timestamp and the causality relation between events can
be generally inferred from their timestamps.
The timestamps assigned to events obey the fundamental monotonicity property; that is, if
an event a causally affects an event b, then the timestamp of a is smaller than the timestamp
of b.

Differences between physical and logical clock

Physical Clock Logical Clock
A physical clock is a physical procedure A logical clock is a component for catching
combined with a strategy for measuring that sequential and causal connections in a dispersed
procedure to record the progression of time. framework.
The physical clocks are based on cyclic A logical clock allows global ordering on
processes such as a events from different processes.
celestial rotation.

1.15.1 A Framework for a system of logical clocks

A system of logical clocks consists of a time domain T and a logical clock C. Elements of T form
a partially ordered set over a relation <. This relation is usually called the happened before or
causal precedence.
The logical clock C is a function that maps an event e in a distributed system to an element
in the time domain T denoted as C(e).
such that
for any two events ei and ej,. eiej C(ei)< C(ej).
This monotonicity property is called the clock consistency condition.When T and C satisfy
the following condition,
Then the system of clocks is strongly consistent.

1.15.2 Implementing logical clocks

The two major issues in implanting logical clocks are:
Data structures: representation of each process
Protocols: rules for updating the data structures to ensure consistent conditions.

Data structures:
Each process pimaintains data structures with the given capabilities:
•
A local logical clock (lci), that helps process pi measure itsown progress.
•
A logical global clock (gci), that is a representation of process pi’s local view of the
logical global time. It allows this process to assignconsistent timestamps to its local events.

Protocol:
The protocol ensures that a process’s logical clock, and thus its view of theglobal time, is
managed consistently with the following rules:
Rule 1: Decides the updates of the logical clock by a process. It controls send, receive and
other operations.
Rule 2: Decides how a process updates its global logical clock to update its view of the
global time and global progress. It dictates what information about the logical time is
piggybacked in a message and how this information is used by the receiving process to
update its view of the global time.

1.3 Scalar Time

Scalar time is designed by Lamport to synchronize all the events in distributed
systems. A Lamport logical clock is an incrementing counter maintained in each process.
This logical clock has meaning only in relation to messages moving between processes.
When a process receives a message, it resynchronizes its logical clock with that sender
maintainingcausal relationship.
The Lamport’s algorithm is governed using the following rules:
 The algorithm of Lamport Timestamps can be captured in a few rules:
 All the process counters start with value 0.
 A process increments its counter for each event (internal event, message
sending, message receiving) in that process.
 When a process sends a message, it includes its (incremented) counter value with the
message.
 On receiving a message, the counter of the recipient is updated to the greater of its
current counter and the timestamp in the received message, and then incremented by
one.

 If Ci is the local clock for process Pi then,

 if a and b are two successive events in Pi, then Ci(b) = Ci(a) + d1, where d1 > 0
 if a is the sending of message m by Pi, then m is assigned timestamp tm = Ci(a)
 if b is the receipt of m by Pj, then Cj(b) = max{Cj(b), tm + d2}, where d2 > 0

Rules of Lamport’s clock

Rule 1: Ci(b) = Ci(a) + d1, where d1 > 0
Rule 2: The following actions are implemented when pi receives a message m with timestamp Cm:
a) C = max(C , C )
i i m
b)
Execute Rule 1
 c) deliver the message

Fig 1.20: Evolution of scalar time

/
Basic properties of scalar time:
1. Consistency property: Scalar clock always satisfies monotonicity. A monotonic clock
only increments its timestamp and never jump.Hence it is consistent.

2. Total Reordering:Scalar clocks order the events in distributed systems.But all the events
do not follow a common identical timestamp. Hence a tie breaking mechanism is essential to
order the events. The tie breaking is done through:
 Linearly order process identifiers.
 Process with low identifier value will be given higher priority.

The term (t, i) indicates timestamp of an event, where t is its time of occurrence and i is the
identity of the process where it occurred.
The total order relation () over two events x and y with timestamp (h, i) and (k, j) is given by:

A total order is generally used to ensure liveness properties in distributed algorithms.

3. Event Counting
If event e has a timestamp h, then h−1 represents the minimum logical duration,
counted in units of events, required before producing the event e. This is called height of the
event e. h-1 events have been produced sequentially before the event e regardless of the
processes that produced these events.

4. No strong consistency
The scalar clocks are not strongly consistent is that the logical local clock and
logical global clock of a process are squashed into one, resulting in the loss causal
dependency information among events at different processes.

1.4 Vector Time

The ordering from Lamport's clocks is not enough to guarantee that if two events
precede one another in the ordering relation they are also causally related. Vector Clocks use
a vector counter instead of an integer counter. The vector clock of a system with N processes
is a vector of N counters, one counter per process. Vector counters have to follow the
following update rules:
 Initially, all counters are zero.
 Each time a process experiences an event, it increments its own counter in the vector
by one.
  Each time a process sends a message, it includes a copy of its own (incremented)
vector in the message.
 Each time a process receives a message, it increments its own counter in the vector by
one and updates each element in its vector by taking the maximum of the value in its
own vector counter and the value in the vector in the received message.

The time domain is represented by a set of n-dimensional non-negative integer vectors in vector
time.

Rules of Vector Time

Fig 1.21: Evolution of vector scale

1.17.1 Basic properties of vector time
1. Isomorphism:
 “→” induces a partial order on the set of events that are produced by a
distributed execution.
 If events x and y are time stamped as vh and vk then,


 There is an isomorphism between the set of partially ordered events produced by a
distributed computation and their vector timestamps.
 If the process at which an event occurred is known, the test to compare
two timestamps can be simplified as:

2. Strong consistency
The system of vector clocks is strongly consistent; thus, by examining the vector timestamp
of two events, we can determine if the events are causally related.
3. Event counting
If an event e has timestamp vh, vh[j] denotes the number of events executed by process pj
that causally precede e.

Vector clock ordering relation

t[i]- timestamp of process i.

1.5 PHYSICAL CLOCK SYNCHRONIZATION: NEWTWORK TIME PROTOCOL

(NTP)
Centralized systems do not need clock synchronization, as they work under a common
clock. But the distributed systems do not follow common clock: each system functions based
on its own internal clock and its own notion of time.The time in distributed systems is
measured in the following contexts:
 The time of the day at which an event happened on a specific machine in the network.
 The time interval between two events that happened on different machines in the
network.
 The relative ordering of events that happened on different machines in the network.

Clock synchronization is the process of ensuring that physically distributed processors have a
common notion of time.

Due to different clocks rates, the clocks at various sites may diverge with time, and
periodically a clock synchrinization must be performed to correct this clock skew in
distributed systems. Clocks are synchronized to an accurate real-time standard like UTC
(Universal Coordinated Time). Clocks that must not only be synchronized with each other
but also have to adhere to physical time are termed physical clocks. This degree of
synchronization additionally enables to coordinate and schedule actions between multiple
computers connected to a common network.

1.18.1 Basic terminologies:

If Ca and Cb are two different clocks, then:

Time: The time of a clock in a machine p is given by the function Cp(t),where Cp(t)= t
for a perfect clock.

Frequency: Frequency is the rate at which a clock progresses. The frequency at time t
of clock CaisCa’(t).

Offset:Clock offset is the difference between the time reported by a clockand the real
time. The offset of the clock Ca is given by Ca(t)− t. Theoffset of clock C a relative
to Cb at time t ≥ 0 is given by Ca(t)- Cb(t)

Skew: The skew of a clock is the difference in the frequencies of the clockand the
perfect clock. The skew of a clock Ca relative to clock Cb at timet is Ca’(t)- Cb’(t).

Drift (rate): The drift of clock Ca the second derivative of the clockvalue with
respect to time. The drift is calculated as:
1.18.2 Clocking Inaccuracies
Physical clocks are synchronized to an accurate real-time standard like UTC
(Universal Coordinated Time). Due to the clock inaccuracy discussed above, a timer
(clock) is said to be working within its specification if:

- maximum skew rate.

1. Offset delay estimation

A time service for the Internet - synchronizes clients to UTC Reliability from
redundant paths, scalable, authenticates time sources Architecture. Thedesign of NTP
involves a hierarchical tree of time servers with primary serverat the root synchronizes with
the UTC. The next level contains secondaryservers, which act as a backup to the primary
server. At the lowest level isthe synchronization subnet which has the clients.

2. Clock offset and delay estimation

A source node cannot accurately estimate the local time on thetarget node due to varying
message or network delays between the nodes.This protocol employs a very common
practice of performing several trialsand chooses the trial with the minimum delay.

Fig 1.24: Behavior of clocks

Fig 1.30 a) Offset and delay estimation Fig 1.30 b) Offset and delay estimation
between processes from same server between processes from different servers

Let T1, T2, T3, T4 be the values of the four mostrecent timestamps. The clocks A and B are
stable andrunning at the same speed. Let a = T1 − T3 and b = T2 − T4. If the networkdelay
difference from A to B and from B to A, called differential delay, is
 CS8603 DS
small, the clock offset and roundtrip delay of B relative to A at time T4are
approximately given by the following:

Each NTP message includes the latest three timestamps T1, T2, andT3, while
T4 is determined upon arrival.

2.1 MESSAGE ORDERING AND GROUP COMMUNICATION

. The message ordering means the order of delivering the messages to the intended
recipients. The common message order schemes are First in First out (FIFO), non FIFO,
causal order and synchronous order. In case of group communication with multicasting, the
causal and total ordering scheme is followed The following are the notations that are widely
used in this chapter:
 Distributed systems are denoted by a graph (N, L).
 The set of events are represented by event set {E, }
 Message is denoted as mi: send and receive events as si and ri respectively.
 Send (M) and receive (M) indicates the message M send and received.
 ab denotes a and b occurs at the same process
 The send receive pairs ={(s, r) Ei x Ejcorresponds to r}

2.1.1 Message Ordering Paradigms

The message orderings are

(i) non-FIFO
(ii) FIFO
(iii) causal order
(iv) synchronous order

There is always a trade-off between concurrency and ease of use and implementation.

Asynchronous Executions
An asynchronous execution (or A-execution) is an execution (E, ≺) for which the causality relation
is a partial order.
 There cannot be any causal relationship between events in asynchronous execution.
 The messages can be delivered in any order even in non FIFO.
 Though there is a physical link that delivers the messages sent on it in FIFO order due
to the physical properties of the medium, a logical link may be formed as a composite
of physical links and multiple paths may exist between the two end points of the
logical link.
 CS8603 DS

Fig 2.1: a) non FIFO executions b) FIFO executions FIFO

executions

A FIFO execution is an A-execution in which, for all

In general on any logical link, messages are delivered in the order in which they are sent.
• To implement FIFO logical channel over a non-FIFO channel, use a separate numbering scheme to
sequence the messages.
• The sender assigns and appends a tuple to each message.
The receiver uses a buffer to order the incoming messages as per the sender’s sequence numbers, and
accepts only the “next” message in sequence

Causally Ordered (CO) executions

CO execution is an A-execution in which, for all,

 Two send events s and s’ are related by causality ordering (not physical time
ordering), then a causally ordered execution requires that their corresponding receive
events r and r’ occur in the same order at all common destinations.
 If s and s’ are not related by causality, then CO is vacuously(blankly)satisfied.
 Causal order is used in applications that update shared data, distributed shared
memory, or fair resource allocation.
 The delayed message m is then given to the application for processing. The event of
an application processing an arrived message is referred to as a delivery event.
 No message overtaken by a chain of messages between the same (sender, receiver)
pair.

If send(m1) ≺ send(m2) then for each common destination d of messages m1 and m2,
deliverd(m1) ≺deliverd(m2) must be satisfied.

Synchronous Execution

 When all the communication between pairs of processes uses synchronous send and
receives primitives, the resulting order is the synchronous order.
 The synchronous communication always involves a handshake between the receiver
and the sender, the handshake events may appear to be occurring instantaneously and
atomically.
 The instantaneous communication property of synchronous executions requires a
modified definition of the causality relation because for each (s, r) ∈ T, the send
event is not causally ordered before the receive event.
 The two events are viewed as being atomic and simultaneous, and neither event
 CS8603 DS
precedes the other.

Fig 2.2 a) Execution in an Fig 2.2 b) Equivalent synchronous

asynchronous system communication

2.2 Asynchronous execution with synchronous communication

When all the communication between pairs of processes is by using synchronous send
and receive primitives, the resulting order is synchronous order. The algorithms run on
asynchronous systems will not work in synchronous system and vice versa is also true.

Realizable Synchronous Communication (RSC)

A-execution can be realized under synchronous communication is called a realizable with
synchronous communication (RSC).

 An execution can be modeled to give a total order that extends the partial order
(E, ≺).
 In an A-execution, the messages can be made to appear instantaneous if there exist a
linear extension of the execution, such that each send event is immediately followed
by its corresponding receive event in this linear extension.

Non-separated linear extension is an extension of (E, ≺) is a linear extension of (E, ≺) such that
for each pair (s, r) ∈ T, the interval { x∈ E s ≺ x ≺ r } is empty.

A A-execution (E, ≺) is an RSC execution if and only if there exists a non-separated linear
extension of the partial order (E, ≺).
 In the non-separated linear extension, if the adjacent send event and its corresponding
receive event are viewed atomically, then that pair of events shares a common past
and a common future with each other.

Crown
Let E be an execution. A crown of size k in E is a sequence <(si, ri), i ∈{0,…, k-1}> of pairs of
corresponding send and receive events such that: s0 ≺ r1, s1 ≺ r2, sk−2 ≺ rk−1, sk−1 ≺ r0.

The crown is <(s1, r1) (s2, r2)> as we have s1 ≺ r2 and s2 ≺ r1. Cyclic dependencies
may exist in a crown. The crown criterion states that an A-computation is RSC, i.e., it can be
realized on a system with synchronous communication, if and only if it contains no crown.

Timestamp criterion for RSC execution

An execution (E, ≺) is RSC if and only if there exists a mapping from E to T (scalar
timestamps) such that
 CS8603 DS

2.2.1 Hierarchy of ordering paradigms

The orders of executions are:

 Synchronous order (SYNC)
 Causal order (CO)
 FIFO order (FIFO)
 Non FIFO order (non-FIFO)
The Execution order have the following results
 For an A-execution, A is RSC if and only if A is an S-execution.
 RSC ⊂ CO ⊂ FIFO ⊂ A
 This hierarchy is illustrated in Figure 2.3(a), and example executions of each class are
shown side-by-side in Figure 2.3(b)
 The above hierarchy implies that some executions belonging to a class X will not
belong to any of the classes included in X. The degree of concurrency is most in A
and least in SYNC.
 A program using synchronous communication is easiest to develop and verify.
 A program using non-FIFO communication, resulting in an A execution, is hardest to
design and verify.

Fig (a) Fig (b)

Fig 2.3: Hierarchy of execution classes

2.2.3 Simulations
 The events in the RSC execution are scheduled as per some non-separated linear
extension, and adjacent (s, r) events in this linear extension are executed sequentially
in the synchronous system.
 The partial order of the asynchronous execution remains unchanged.
 If an A-execution is not RSC, then there is no way to schedule the events to make
them RSC, without actually altering the partial order of the given A-execution.
 However, the following indirect strategy that does not alter the partial order can be
used.
 Each channel Ci,j is modeled by a control process Pi,j that simulates the channel buffer.
 An asynchronous communication from i to j becomes a synchronous communication
from i to Pi,j followed by a synchronous communication from Pi,j to j.
 This enables the decoupling of the sender from the receiver, a feature that is essential
 CS8603 DS
in asynchronous systems.

Fig 2.4: Modeling channels as processes to simulate an execution

using asynchronous primitives on synchronous system

Synchronous programs on asynchronous systems

 A (valid) S-execution can be trivially realized on an asynchronous system by
scheduling the messages in the order in which they appear in the S-execution.
 The partial order of the S-execution remains unchanged but the communication
occurs on an asynchronous system that uses asynchronous communication primitives.
 Once a message send event is scheduled, the middleware layer waits for
acknowledgment; after the ack is received, the synchronous send primitive completes.

2.3 SYNCHRONOUS PROGRAM ORDER ON AN ASYNCHRONOUS SYSTEM

Synchronous program order on an asynchronous system

There do not exist real systems with instantaneous communication that allows for synchronous
communication to be naturally realized. Non-determinism •
This suggests that the distributed programs are deterministic, i.e., repeated runs of the same program
will produce the same partial order.
• In many cases, programs are non-deterministic in the following senses
1. A receive call can receive a message from any sender who has sent a message, if the expected sender
is not specified.
2. Multiple send and receive calls which are enabled at a process can be executed in an interchangeable
order.
• If i sends to j, and j sends to i concurrently using blocking synchronous calls, it results in a deadlock.
• However, there is no semantic dependency between the send and immediately following receive. If
the receive call at one of the processes isscheduled before the send call, then there is no deadlock.
Rendezvous \
 Rendezvous (“meet with each other”)
 One form of group communication is called multiway rendezvous, which is a synchronous
communication among an arbitrary number of asynchronous processes.
 Rendezvous between a pair of processes at a time is called binary rendezvous as opposed to the
multiway rendezvous.
 Observations about synchronous communication under binary rendezvous: o For the receive
command, the sender must be specified eventhough the multiple recieve commands exist. o Send
and received commands may be individually disabled or enabled. o Synchronous
communication is implemented by scheduling messages using asynchronous communication.
 Scheduling involves pairing of matching send and receive commands that are both enabled.
 The communication events for the control messages do not alter the partial order of execution.

Algorithm for binary rendezvous

 Each process, has a set of tokens representing the current interactions that are enabled locally. If
 CS8603 DS
multiple interactions are enabled, a process chooses one of them and tries to “synchronize” with
the partner process.
 The scheduling messages must satisfying the following constraints: o Schedule on-line,
atomically, and in a distributed manner.
 Schedule in a deadlock-free manner (i.e., crown-free), such that both the sender and receiver are
enabled for a message when it is scheduled. \

Schedule to satisfy the progress property (i.e., find a schedule within a bounded number of steps)
in addition to the safety (i.e., correctness) property.
• Additional features of a good algorithm are: (i) symmetry or some form of fairness, i.e., not
favoring particular processes (ii) efficiency, i.e., using as few messages as possible
• A simple algorithm by Bagrodia, makes the following assumptions:
1. Receive commands are forever enabled from all processes.
2. A send command, once enabled, remains enabled until it completes.
3. To prevent deadlock, process identifiers are used to break the crowns.
4. Each process attempts to schedule only one send event at any time.
• The algorithm illustrates how crown-free message scheduling is achieved on-line.

2.4 GROUP COMMUNICATION

Group communication is done by broadcasting of messages. A message broadcast is
the sending of a message to all members in the distributed system. The communication may
be
 Multicast: A message is sent to a certain subset or a group.
 Unicasting: A point-to-point message communication.

The network layer protocol cannot provide the following functionalities:

 Application-specific ordering semantics on the order of delivery of messages.
 Adapting groups to dynamically changing membership.
 Sending multicasts to an arbitrary set of processes at each send event.
 Providing various fault-tolerance semantics.
 The multicast algorithms can be open or closed group.

Differences between closed and open group algorithms:

Closed group algorithms Open group algorithms

If sender is also one of the receiver in the If sender is not a part of the communication
multicast algorithm, then it is closed group group, then it is open group algorithm.
algorithm.
They are specific and easy to implement. They are more general, difficult to design and
expensive.
It does not support large systems where client It can support large systems.
processes have short life.

2.5 CAUSAL ORDER (CO)

In the context of group communication, there are two modes of communication:
causal order and total order. Given a system with FIFO channels, causal order needs to be
explicitly enforced by a protocol. The following two criteria must be met by a causal
ordering protocol:
 Safety: In order to prevent causal order from being violated, a message M that
arrives at a process may need to be buffered until all system wide messages sent in the
causal past of the send (M) event to that same destination have already arrived. The
arrival of a message is transparent to the application process. The delivery event
 CS8603 DS
corresponds to the receive event in the execution model.
 Liveness: A message that arrives at a process must eventually be delivered to the
process.

2.6 Causal Order (CO)

An optimal CO algorithm stores in local message logs and propagates on messages,
information of the form d is a destination of M about a messageM sent in the causal past, as
long as and only as long as:

Propagation Constraint I: it is not known that the message M is delivered to d.

Propagation Constraint II: it is not known that a message has been sent to d in the causal
future of Send(M), and hence it is not guaranteed using a reasoning based on transitivity that
the message M will be delivered to d in CO.
2.7 TOTAL ORDER

For each pair of processes Pi and Pj and for each pair of messages Mx and My that are delivered to
both the processes, Pi is delivered Mx before My if and only if Pj is delivered Mxbefore My.

Centralized Algorithm for total ordering

Each process sends the message it wants to broadcast to a centralized process, which
relays all the messages it receives to every other process over FIFO channels.

Three phase distributed algorithm

Three phases can be seen in both sender and receiver side.

Sender side

Phase 1
 In the first phase, a process multicasts the message M with a locally unique tag
and the local timestamp to the group members.

Phase 2
 The sender process awaits a reply from all the group members who respond with a
tentative proposal for a revised timestamp for that message M.
 The await call is non-blocking.

Phase 3
 The process multicasts the final timestamp to the group.

Causal Order Total Order

 CS8603 DS

csau

Fig 2.9: Sender side of three phase distributed algorithm

Receiver Side
Phase 1
 The receiver receives the message with a tentative timestamp. It updates the
variable priority that tracks the highest proposed timestamp, then revises the
proposed timestamp to the priority, and places the message with its tag and the
revised timestamp at the tail of the queue temp_Q. In the queue, the entry is marked
as undeliverable.

Phase 2
 The receiver sends the revised timestamp back to the sender. The receiver then waits
in a non-blocking manner for the final timestamp.

Phase 3
 The final timestamp is received from the multicaster. The corresponding
message entry in temp_Q is identified using the tag, and is marked as deliverable
after the revised timestamp is overwritten by the final timestamp.
 The queue is then resorted using the timestamp field of the entries as the key. As the
 CS8603 DS
queue is already sorted except for the modified entry for the message under
consideration, that message entry has to be placed in its sorted position in the
queue.
 If the message entry is at the head of the temp_Q, that entry, and all consecutive
subsequent entries that are also marked as deliverable, are dequeued from temp_Q,
and enqueued in deliver_Q.

Complexity
This algorithm uses three phases, and, to send a message to n − 1 processes, it uses 3(n – 1)
messages and incurs a delay of three message hops

2.8 GLOBAL STATE AND SNAPSHOT RECORDING ALGORITHMS

 A distributed computing system consists of processes that do not share a common
memory and communicate asynchronously with each other by message passing.
 Each component has a local state. The state of the process is the local memory and a
history of its activity.
 The state of a channel is characterized by the set of messages sent along the channel
less the messages received along the channel. The global state of a distributed system
is a collection of the local states of its components.
 If shared memory were available, an up-to-date state of the entire system would be
available to the processes sharing the memory.

2.8.1 System Model


The system consists of a collection of n processes, p1, p2,…,pn that are connected
by channels.

Let Cij denote the channel from process pi to process pj.

Processes and channels have states associated with them.

The state of a process at any time is defined by the contents of processor registers,
stacks, local memory, etc., and may be highly dependent on the local context of
the distributed application.

The state of channel Cij, denoted by SCij, is given by the set of messages in transit
in the channel.

The events that may happen are: internal event, send (send (m ij)) and receive
(rec(mij)) events.

The occurrences of events cause changes in the process state.
2.8.2 A consistent global state
The global state of a distributed system is a collection of the local states ofthe
processes and the channels. The global state is given by:

The two conditions for global state are:

Condition 1 preserves law of conservation of messages.Condition C2 states that in the

collected global state, for everyeffect, its cause must be present.

Law of conservation of messages: Every messagem ijthat is recorded as sent in the local state of a
process pi must be capturedin the state of the channel C ij or in the collected local state of the
receiver process pj.
 CS8603 DS
 In a consistent global state, every message that is recorded as received isalso recorded
as sent. Such a global state captures the notion of causalitythat a message cannot be
received if it was not sent.
 Consistent global statesare meaningful global states and inconsistent global states are
not meaningful in the sense that a distributed system can never be in an
inconsistentstate.
Interpretation in terms of cuts •
Cuts is a zig-zag line that connects a point in the space–time diagram at some arbitrary point
in the process line.
• Cut is a powerful graphical aid for representing and reasoning about the global states of a
computation.
• Left side of the cut is referred as PAST event and right is referred as FUTURE event..

 A consistent global state corresponds to a cut in which every message received in the PAST of the cut has
been sent in the PAST of that cut. Such a cut is known as a consistent cut. Example: Cut C2 in the above
figure.
 All the messages that cross the cut from the PAST to the FUTURE are captured in the corresponding
channel state.
 If the flow is from the FUTURE to the PAST is inconsistent. Example: Cut C1

2.9 SNAPSHOT ALGORITHMS FOR FIFO CHANNELS

Each distributed application has number of processes running on different physical
servers. These processes communicate with each other through messaging channels.

A snapshot captures the local states of each process along with the state of each communication channel.

Snapshots are required to:

 Checkpointing
 Collecting garbage
 Detecting deadlocks
 Debugging

2.9.1 Chandy–Lamport algorithm

 The algorithm will record a global snapshot for each process channel.
 The Chandy-Lamport algorithm uses a control message, called a marker.
 After a site has recorded its snapshot, it sends a marker along all of its outgoing
 CS8603 DS
channels before sending out any more messages.
 Since channels are FIFO, a marker separates the messages in the channel into those to
be included in the snapshot from those not to be recorded in the snapshot.
 This addresses issue I1. The role of markers in a FIFO system is to act as delimiters
for the messages in the channels so that the channel state recorded by the process
at the receiving end of the channel satisfies the condition C2.

Fig 2.10: Chandy–Lamport algorithm

Initiating a snapshot
 Process Pi initiates the snapshot
 Pi records its own state and prepares a special marker message.
 Send the marker message to all other processes.
 Start recording all incoming messages from channels Cij for j not equal to i.

Propagating a snapshot
 For all processes Pjconsider a message on channel Ckj.
 if marker message is seen for the first time:

Pjrecords own sate and marks Ckj as empty

Send the marker message to all other processes.

Record all incoming messages from channels Clj for 1 not equal to j or k.

Else add all messages from inbound channels.

Terminating a snapshot
 All processes have received a marker.
 All process have received a marker on all the N-1 incoming channels.
 A central server can gather the partial state to build a global snapshot.

Complexity
 The recording part of a single instance of the algorithm requires O(e) messages
 O(d) time - e is the number of edges in the network and d is the diameter of the network.

2.9.2 Properties of the recorded global state

 The recorded global state may not correspond to any of the global states that occurred during
the computation.
 CS8603 DS
 This happens because a process can change its state asynchronously before the markers it sent
are received by other sites and the other sites record their states.
But the system could have passed through the recorded global states in some equivalent
executions.
 The recorded global state is a valid state in an equivalent execution and if a stable property
(i.e., a property that persists) holds in the system before the snapshot algorithm begins, it
holds in the recorded global snapshot.
 Therefore, a recorded global state is useful in detecting stable properties.