Lab 2

Group Communication

Andreas Larsson

2011-01-31
 Overview
• Introduction to group communication

• Desired group communication

• Multicast communication

• Group membership service

DSII: Group Comm. 2

Coordination in distributed systems
• Coordination is needed by distributed systems
but hard to achieve:
– Events happen concurrently
– Communication links are not reliable
– Computers can crash
– New nodes can join the systems
– Asynchronous environments

 Need of an efficient way to coordinate a group

of processes

DSII: Group Comm. 3

 A Distributed System in war:
Synchronous Example
Friendly Friend
fighter

Enemy missile Friendly fighter

Radar

Missile
Ready Cancel
to Fire

Commander
Ask Prepare

DSII: Group Comm. 4

 A Distributed System in war:
Reality
Friendly Friend
fighter

Enemy missile Friendly fighter

Radar

Missile
Ready Fire
to Fire

Commander
Ask Prepare

DSII: Group Comm. 5

 Group communication
• What is a group?
– A number of processes which cooperate to
provide a service.
– An abstract identity to name a collection of
processes.

• Group Communication
– For coordination among processes of a group.

DSII: Group Comm. 6

Who Needs Group Communication?
• Highly available servers (client-server)
• Database Replication
• Multimedia Conferencing
• Online Games
• Cluster management
• …

DSII: Group Comm. 7

 Distributed Web Server

• High availability
DSII: Group Comm. 8
 Online Game

• Fault-tolerance, Order
DSII: Group Comm. 9
 Different Comm. Methods
• Unicast
– Point-to-Point Communication
– Multiple copies are sent.
• Broadcast
– One-to-All Communication
– Abuse of Network Bandwidth
• Multicast
– One-to-multiple Communication

DSII: Group Comm. 10

 Overview
• Introduction to group communication

• Desired group communication

• Multicast communication

• Group membership service

DSII: Group Comm. 11

 Desired Group Communication
• Name Abstraction

• Efficiency  Multicast

• Delivery Guarantees  Reliability, Ordering

• Dynamic Membership  Group membership service

DSII: Group Comm. 12

 Properties of Communication
• Ordering
– Total ordering, causal ordering
• Failure behavior
• Reliability
– Validity, integrity, agreement

DSII: Group Comm. 13

 Properties of Group
• Name of group
• Addresses of group members
• Dynamic group membership

• Options:
– Peer group or client-server group
– Closed or Open Group

DSII: Group Comm. 14

 Peer Group
• All the members are
equal.
• All the members send
messages to the
group.
• All the members
receive all the
messages.

DSII: Group Comm. 15

 Client-Server Group
• Replicated servers.
• Clients do not care
which server
answers.

DSII: Group Comm. 16

 Overview
• Introduction to group communication

• Desired group communication

• Multicast communication

• Group membership service

DSII: Group Comm. 17

 Multicast communication
• Use network hardware support for
broadcast or multicast when it is available.
• Send message over a distribution tree.
• Minimize the time and bandwidth
utilization

DSII: Group Comm. 18

 Reliability
Correct processes: those that never fail.

• Integrity
A correct process delivers a message at most once.
• Validity
A message from a correct process will be delivered by
the process eventually.
• Agreement
A message delivered by a correct process will be
delivered by all other correct processes in the group.

 Validity + Agreement = Liveness

DSII: Group Comm. 19

 Ordering
Assumptions: a process belongs to p1
at most one group.
p2

• FIFO p3
– if mp m’p, all correct
processes that deliver m’p will
deliver mp (that is from the p1
same sender) before m’p.
• Causal p2
– if m m’, all correct p3
processes that deliver m’ will
deliver m before m’.
• Total
p1
– if a correct process delivers m
before m’, all other correct p2
processes that deliver m’ will
deliver m before m’. p3

DSII: Group Comm. 20

 Examples
• Assumption:
– Reliable one-to-one send operation (e.g. TCP)

• Basic multicast
– Requirement:
• All correct processes will eventually deliver the message
from a correct sender.
– Implementation:
• B-multicast( g, m): p  g: send( p, m);
• On receive( m) at p: B-deliver( m) at p.
 Properties: integrity, validity.

DSII: Group Comm. 21

Basic multicast: Agreement?

1
crash 3

X
Agreement
4

DSII: Group Comm. 22

 Examples (cont.)
• Reliable multicast
– Requirements: integrity, validity, agreement
– Implementation:
• Received := {};
• R-multicast( g, m) at process p: B-multicast( g, m);
• On B-deliver( m) at process p from process q
if( m  Received)
Received := Received  {m};
if( q  p) B-multicast( g, m);
R-deliver( m);
end if

 Inefficient: each message is sent O(|g|) times to each process

DSII: Group Comm. 23

 Examples (cont.)
• FIFO-ordered multicast:
– Assumption:
• a process belongs to at most one group.
– Implementation:
• Local variables at p: Sp = 1, Rp[ |g| ]={0};
• FO-multicast( g, m) at p:
B-multicast( g, <m, Sp>);
Sp++;
• On B-deliver( <m, S>) at p from q:
if( S = Rp[q] + 1)
FO-deliver( m);
Rp[q] := S;
else if( S > Rp[q] + 1)
place <m, S> in the queue until S = Rp[q] + 1;
FO-deliver( m);
Rp[q] := S;
end if

– Your task: totally ordered multicasts.

DSII: Group Comm. 24

 Overview
• Introduction to group communication

• Desired group communication

• Multicast communication

• Group membership service

DSII: Group Comm. 25

 Group membership service
• Four tasks: Group
address
– Interface for group expansion
membership changes
– Failure detector
– Membership change Multicast
Group
membership
notification Communication management
– Group address
expansion

• Group partition:
– Primary-partition – one partition only
– Partitionable – many partiations at once

DSII: Group Comm. 26

 Group views
• Group views:
– Lists of the current ordered group members
– A new one is generated when processes join or
leave/fail.
• View delivery
– When a member is notified of a membership change
– Requirements
• Order
– if p delivers v(g) v’(g), no other process delivers v’(g) v(g).
• Integrity
– if p delivers v(g), p  v(g).
• Non-triviality
– if q joins a group and becomes indefinitely reachable from p,
eventually q is always in the view p delivers.

DSII: Group Comm. 27

 View-synchronous group comm.
• Extend the reliable multicast
semantics with group views. crash
p X
– Agreement
• Correct processes deliver the same q
set of messages in any given view
– Validity (closed group) r
• Correct processes always deliver
(p,q,r) (q,r)
the messages they send.
• p  v0(g) does not deliver m in v0(g)
 p  v1(g) for processes that
deliver m.
– Integrity

DSII: Group Comm. 28

 Examples
• Ensemble: reliable group communication
toolkit
– Previous talk

DSII: Group Comm. 29

 IP-multicast
• IP: 224.0.0.1 - 239.255.255.255
• Multicast: • Group membership
service:
– Yes: – Yes:
• efficiency • Interface for group
membership change
• Group address
expansion
– No: – No:
• Reliability • Failure detector
• Ordering • Membership change
notification

DSII: Group Comm. 30

 References
• Distributed Systems: Concepts and
Design by G. Coulouris et al., ISBN 0-201-
61918-0
– Section 4.5 Group Communication
– Section 11.4 Multicast Communication
– Section 14.2.2 Group Communication
• …

DSII: Group Comm. 31

 Lab 2: Construct a reliable and
ordered multicast
• Reliable
– Integrity, Validity, Agreement
• Ordered
– All machines should agree on the order of all
received messages.
– Total and causal order.
• No membership service
– Processes can still crash though!

DSII: Group Comm. 32

 The GUI

# of messages for stress test

Time for stress test
24-hour time format
Message display

Debug message display

DSII: Group Comm. 33

 The interface
• Provide • Use
– cast – deliver
– debug

– basicreceive – basicsend
– basicpeerdown

DSII: Group Comm. 34