Synchronization Algorithms

and Concurrent Programming

Gadi Taubenfeld
Chapter 5
Barrier Synchronization

Version: June 2014

This presentation is a Synchronization

modified version of a presentation
Algorithms thatProgramming
and Concurrent Itai Avrian and Shachar Gidron
Chapter 5 1
prepared for my Seminar in Concurrent Gadi Taubenfeld
and © 2014
Distributed Computing, 2012.
 Synchronization Algorithms
and Concurrent Programming
ISBN: 0131972596, 1st edition
AA note
note on
on the
the use
use of
of these
these ppt
ppt slides:
slides:
II am
am making
making these
these slides
slides freely
freely available
available to
to all
all (faculty,
(faculty, students,
students, readers).
readers).
They
They are
are in
in PowerPoint
PowerPoint form
form so
so you
you can
can add,
add, modify,
modify, and
and delete
delete slides
slides and
and slide
slide
content
content to
to suit
suit your
your needs.
needs. They
They obviously represent aa lot
obviously represent lot of
of work
work on
on my
my part.
part.
In
In return
return for
for use,
use, II only
only ask
ask the
the following:
following:

 That
 That you
you mention
mention their
their source,
source, after
after all,
all, II would
would like
like people
people to
to use
use mymy book!
book!
 That
 That you
you note
note that
that they
they are
are adapted
adapted from
from (or(or perhaps
perhaps identical
identical to)
to)
my
my slides,
slides, and
and note
note my
my copyright
copyright of
of this
this material.
material.

Thanks
Thanks and
and enjoy!
enjoy!
Gadi
Gadi Taubenfeld
Taubenfeld
All
All material
material copyright
copyright 2014
2014
Gadi
Gadi Taubenfeld,
Taubenfeld, All
All Rights
Rights Reserved
Reserved

:To get the most updated version of these slides go to

http://www.faculty.idc.ac.il/gadi/book.htm
Synchronization Algorithms and Concurrent Programming 2
Chapter 5
Gadi Taubenfeld © 2014
 Chapter 5
Barrier Synchronization

5.1 Barriers
5.2 Atomic Counter
5.3 Test-and-set Bits
5.4 Combining Tree Barrier*
5.5 A Tree-based Barriers
5.6 The Dissemination Barrier*
5.7 The See-Saw Barrier
5.8 Semaphores
5.9 Bibliographic Notes*
5.10 Problems*

Not covered in this presentation *

Synchronization Algorithms and Concurrent Programming 3

Chapter 5
Gadi Taubenfeld © 2014
 Definition and Motivation
Barrier Synchronization

Synchronization Algorithms and Concurrent

Chapter 5 4
Programming Gadi Taubenfeld © 2014
 What is a Barrier ?

P1
P1 P1
P1 P1
P1

P2 P2 P2

Barrier

Barrier

Barrier
P2 P2 P2

P3
P3 P3
P3 P3
P3
P4
P4 P4
P4 P4
P4

time

four processes all except Once all

approach the P4 arrive arrive, they
barrier continue

Synchronization Algorithms and Concurrent Programming 5

Chapter 5
Gadi Taubenfeld © 2014
 What is a Barrier ?

 A barrier is a coordination mechanism (an algorithm),

that forces processes which participate in a concurrent
(or distributed) algorithm to wait until each one of
them has reached a certain point in its program.

 The collection of this coordination points is called the

barrier.

 Once all the processes have reached the barrier, they

are all permitted to continue pass the barrier.

Synchronization Algorithms and Concurrent Programming 6

Chapter 5
Gadi Taubenfeld © 2014
 Example: Parallel Prefix Sum

begin a b c d e f

time

a+b+c a+b+c
end a a+b a+b+c a+b+c+d
d+e+ d+e+f+
Synchronization Algorithms and Concurrent Programming 7
Chapter 5
Gadi Taubenfeld © 2014
 Example: Parallel Prefix Sum

begin a b c d e f

a a+b c d e f

a a+b a+b+c d e f
time

a a+b a+b+c a+b+c+d e f

a a+b a+b+c a+b+c+d a+b+c+d+e f

a+b+c a+b+c
end a a+b a+b+c a+b+c+d
d+e+ d+e+f+
Synchronization Algorithms and Concurrent Programming 8
Chapter 5
Gadi Taubenfeld © 2014
 Example: Parallel Prefix Sum

begin a b c d e f

a a+b b+c c+d d+e e+f

time

a a+b a+b+c a+b+c+d b+c+d+e c+d+e+f

a+b+c a+b+c
end a a+b a+b+c a+b+c+d
d+e+ d+e+f+
Synchronization Algorithms and Concurrent Programming 9
Chapter 5
Gadi Taubenfeld © 2014
 Example: Parallel Prefix Sum

begin a b c d e f

barrier

a a+b b+c c+d d+e e+f

time
barrier

a a+b a+b+c a+b+c+d b+c+d+e c+d+e+f

a+b+c a+b+c
end a a+b a+b+c a+b+c+d
d+e+ d+e+f+
Synchronization Algorithms and Concurrent Programming 10
Chapter 5
Gadi Taubenfeld © 2014
 Example: Video
Single thread

 Assume we have a video application

 Each frame needs to be calculated,
before being displayed
 Prepare frame for display by graphics processor

while (true)
{
frame = prepare_next_frame();
frame.display();
}

Synchronization Algorithms and Concurrent Programming 11

Chapter 5
Gadi Taubenfeld © 2014
 Example: Video
Multiple threads

 Now, we have n threads running in parallel

 It makes sense to split the frame into n disjoint parts
 Each thread prepares its own parts in parallel with others
 Each thread may run on different graphical processor

Barrier globalBarrier;
i = getThreadID();
while (true)
{
frame[ i ].prepare();
globalBarrier.await();
frame[ i ].display();
}

Synchronization Algorithms and Concurrent Programming 12

Chapter 5
Gadi Taubenfeld © 2014
 Where it is needed

 Scientific & numeric computation

 Computer graphics

 Garbage collections

 Parallel computing in general

Synchronization Algorithms and Concurrent Programming 13

Chapter 5
Gadi Taubenfeld © 2014
 Various Barrier Goals

Ideally when designing barriers, we would like to have the

following properties:
 Low shared memory space complexity
 Low contention on shared objects
 Low shared memory reference per process
 No need for shared memory initialization
 Symmetric-ness (same amount of work for all processes)
 Algorithm simplicity
 Simple basic primtive
 Minimal propagation time
 Reusability of the barrier (must!)

Synchronization Algorithms and Concurrent Programming 14

Chapter 5
Gadi Taubenfeld © 2014
 Data Objects in Use
Atomic Bit
Atomic Register
Fetch-and-increment register
Test and set bits
Read-Modify-Write register
Semaphores

Synchronization Algorithms and Concurrent Programming 15

Chapter 5
Gadi Taubenfeld © 2014
 Barriers using atomic counters
Section 5.2

Atomic Bit
Atomic Register
Fetch-and-increment register / atomic counter

Chapter 5 Synchronization Algorithms and Concurrent Programming 16

Gadi Taubenfeld © 2014
 Fetch-and-increment Register
 A shared register that supports a F&I operation:
 Input: register r
 Atomic operation:
 r is incremented by 1
 the old value of r is returned

function fetch-and-increment (r : register)

orig_r := r;
r:= r + 1;
return (orig_r);
end-function

Synchronization Algorithms and Concurrent Programming 17

Chapter 5
Gadi Taubenfeld © 2014
 await macro

 For clarity, we use the await macro

 Not an operation of an object
 This is also called: “spinning”

macro await (condition : boolean condition)

repeat
cond = eval(condition);
until (cond)
end-macro

Synchronization Algorithms and Concurrent Programming 18

Chapter 5
Gadi Taubenfeld © 2014
 Simple Barrier Using an Atomic Counter
Program of a Process

shared counter: fetch and increment reg. – {0,..n}, initially = 0

go: atomic bit, initial value is immaterial
local local.go: a bit, initial value is immaterial
local.counter: register

1 local.go := go
2 local.counter := fetch-and-increment (counter)
3 if local.counter + 1 = n then
4 counter := 0
5 go := 1 - go
6 else await(local.go ≠ go) fi

Synchronization Algorithms and Concurrent Programming 19

Chapter 5
Gadi Taubenfeld © 2014
 Simple Barrier Using an Atomic Counter
Run for n=2 Processes

counter ? go ? SM

local.go ? local.go ?
P1 P2
local.counter ? local.counter ?

1 local.go := go
2 local.counter := fetch-and-increment (counter)
3 if local.counter + 1 = n then
4 counter := 0
5 go := 1 - go
6 else await(local.go ≠ go) fi

Synchronization Algorithms and Concurrent Programming 20

Chapter 5
Gadi Taubenfeld © 2014
 Simple Barrier Using an Atomic Counter
Run for n=2 Processes

counter 120 go 10 SM

local.go ?0 local.go ?0
P1 P2
local.counter ?0 local.counter ?1
P2 P1
1 local.go := go
2 local.counter := fetch-and-increment (counter)
2≠
=0+1
1+1
3 if local.counter + 1 = n then
4 counter := 0
5 go := 1 - go P1 Busy wait
6 else await(local.go ≠ go) fi

Synchronization Algorithms and Concurrent Programming 21

Chapter 5
Gadi Taubenfeld © 2014
 Simple Barrier Using an Atomic Counter
Another Run for n=2 Processes

counter 120 go 10 SM

local.go ?0 local.go ?0
P1 P2
local.counter ?0 local.counter ?1
P2 P1
1 local.go := go Counter is “fetch-
P1: 0+1≠2
2 and-increment”
local.counter := fetch-and-increment (counter)
P2: 1+1=2
3 if local.counter + 1 = n then register
4 counter := 0
5 go := 1 - go P1 Busy wait
6 else await(local.go ≠ go) fi

Synchronization Algorithms and Concurrent Programming 22

Chapter 5
Gadi Taubenfeld © 2014
 Another Algorithm Using an Atomic Counter
Program of a Process

shared counter: fetch and increment reg. – {0,..n}, initially = 0

local local.counter: register

Is this
1 local.counter := fetch-and-increment(counter)
implementation
?incorrect
2 if local.counter + 1 = n then
3 counter := 0
4 else await(counter = 0) fi

Synchronization Algorithms and Concurrent Programming 23

Chapter 5
Gadi Taubenfeld © 2014
 Simple Barrier Using an Atomic Counter

 There is high memory contention on go bit

 Reducing the contention:
 Replace the go bit with n bits: go[1],…,go[n]
 Process pi may spin only on the bit go[i]

Synchronization Algorithms and Concurrent Programming 24

Chapter 5
Gadi Taubenfeld © 2014
 A Local Spinning Counter Barrier
Program of a Process i

shared counter: fetch and increment reg. – {0,..n}, initially = 0

go[1..n]: array of atomic bits, initial values are immaterial
local local.go: a bit, initial value is immaterial
local.counter: register

1 local.go := go[i]
2 local.counter := fetch-and-increment (counter)
3 if local.counter + 1 = n then
4 counter := 0
5 for j=1 to n do go[j] := 1 – go[j] od
6 else await(local.go ≠ go[i]) fi

Synchronization Algorithms and Concurrent Programming 25

Chapter 5
Gadi Taubenfeld © 2014
 A Local Spinning Counter Barrier
Example Run for n=3 Processes

counter 0321 go ?10 ?10 ?10 SM

loc.go ?0 loc.go ?0 loc.go ?0

P1 P2 P3
loc.counter ?0 loc.counter ?1 loc.counter ?2

P3 P2 P1
1 local.go := go[i]
2 local.counter := fetch-and-increment (counter)
3=≠2+1
0+1
1+1
3 if local.counter + 1 = n then
4 counter := 0
5 for j=1 to n do go[j] := 1 – go[j] od
P1,P2
P1 Busy
Busywait
wait
6 else await(local.go ≠ go[i]) fi

Synchronization Algorithms and Concurrent Programming 26

Chapter 5
Gadi Taubenfeld © 2014
 Comparison of fetch-and-increment Barriers
Simple Barrier Simple Barrier with go array
 Pros:  Pros:
 Very Simple  Low contention on the go
 Shared memory: O(log n) array
bits  In some models:
 Takes O(1) until last  spinning is done on local
waiting p is awaken memory
 remote mem. ref.: O(1)
 Cons:
 Cons:
 Shared memory: O(n)
 High contention on the
 Still contention on the
go bit
 Contention on the counter register (*)
 Takes O(n) until last
counter register (*)
waiting p is awaken

One technique for solving this contention is the )*(

Synchronization Algorithms and Concurrent Programming
Chapter 5 Combining Tree Barriers – page 210 27
Gadi Taubenfeld © 2014
 A Barrier without Memory Initialization

 Barrier is a basic synchronization method

 To initialize shared memory, processes need to be
synchronized
 Thus, barrier may be a prerequisite for shared memory
initialization and cannot assume one
 Processes may not be implemented in the same way
 So it is desirable to reduce the dependency between them

Synchronization Algorithms and Concurrent Programming 28

Chapter 5
Gadi Taubenfeld © 2014
 A Barrier without Memory Initialization
Program of a Process

shared counter: atomic counter – {0,..n-1}, initial value is immaterial

go: atomic bit, initial value is immaterial
local local.go: a bit, initial value is immaterial
local.counter: register, initial value is immaterial

1 local.go := go // remember current value

2 local.counter := counter // remember current value
3 counter := counter +1 (mod n) // atomic increment mod n
4 repeat
5 if counter = local.counter // all processes have arrived
6 then go := 1 – local.go fi // notify all
7 until (local.go ≠ go)

Synchronization Algorithms and Concurrent Programming 29

Chapter 5
Gadi Taubenfeld © 2014
 Using Test-and-Set Bits
Section 5.3

Chapter 5 Synchronization Algorithms and Concurrent Programming 30

Gadi Taubenfeld © 2014
 Test-and-Set Bit

 Input: bit b
 Test-and-set is an atomic operation:
 b is set to 1
 the old value of b (i.e., 0 or 1) is returned

function test-and-set (b : bit)

orig_b := b;
b:= 1;
return (orig_b);
end-function

 An atomic reset operation, which sets the value to 0, is

supported
Synchronization Algorithms and Concurrent Programming 31
Chapter 5
Gadi Taubenfeld © 2014
 Test-and-Test-and-Set Bit

Operations supported:
 Test-and-set
like a test-and-set bit
 Reset

 Atomic read (test)

Synchronization Algorithms and Concurrent Programming 32

Chapter 5
Gadi Taubenfeld © 2014
 Test-and-set based Barrier

shared leader: test-and-set bit, initial value = 0

countflag: test-and-test-and-set bit, initial value = 0
go: atomic bit, initial value is immaterial
local local.go: a bit, initial value is immaterial
local.counter: register, initial value is immaterial

0 leader: test-and-set bit

0 countflag: test-and-test-set bit Local.counter: register

go: atomic register local.go: bit

Synchronization Algorithms and Concurrent Programming 33

Chapter 5
Gadi Taubenfeld © 2014
 Test-and-set based Barrier

1 local.go := go
2 if test-and-set(leader) = 0 then // the leader
3 local.counter := 0
4 repeat
5 await(countflag = 1) // a test operation
6 local.counter = local.counter + 1
7 reset(countflag)
8 until (local.counter = n - 1)
9 reset(leader)
10 go := 1 – go
11 else // the other processes
12 await(test-and-set(countflag) = 0)
13 await(local.go ≠ go)
14 fi

Synchronization Algorithms and Concurrent Programming 34

Chapter 5
Gadi Taubenfeld © 2014
 Test-and-Set Barrier leader 10 SM

P1 P2 P3 P4

leader test-and-set atomic operation

First to set the

leader bit is the
leader
Synchronization Algorithms and Concurrent Programming 35
Chapter 5
Gadi Taubenfeld © 2014
 Test-and-Set Barrier countflag 10 SM

P1 P2 P3 P4
!go

repeat

await (test-and-set atomic operation on countflag) await (countflag = 1)

until
(local.counter = n - 1)
await ( go changed ? )

All processes has P4 – the leader

arrived, change go 1230
local.counter
Synchronization
bit and exit barrier Algorithms and Concurrent Programming 36
Chapter 5
Gadi Taubenfeld © 2014
 A Barrier without Memory Initialization

Two new techniques

1. The leader count each process twice
 Needs only to count to 2n – 2
 Allows off-by-one mistakes
 Thus make memory initialization redundant

2. Asymmetric-ness
 Process has a role according to its index i
 Pros: saves bits and operations
 Cons: different processes differ in their tasks

Synchronization Algorithms and Concurrent Programming 37

Chapter 5
Gadi Taubenfeld © 2014
 Asymmetric Test-and-set based Barrier w/o M/I
program of process i
shared countflag: test-and-test-and-set bit, initial value is immaterial
go: atomic bit, initial value is immaterial
local local.go: a bit, initial value is immaterial
local.counter: atomic register, initial value is immaterial

No need for the

leader
test-and-set bit

Synchronization Algorithms and Concurrent Programming 38

Chapter 5
Gadi Taubenfeld © 2014
 Asymmetric Test-and-set based Barrier w/o M/I
program of process i
1 local.go := go
2 if i = 1 then // the leader
3 local.counter := 0
4 repeat
5 await(countflag = 1) // a test operation
6 local.counter = local.counter + 1
7 reset(countflag)
8 until (local.counter = 2n - 2)
9 go := 1 – go
10 else // the other processes
11 await(test-and-set(countflag) = 0)
12 await(test-and-set(countflag) = 0)
13 await(local.go ≠ go)
14 fi

Synchronization Algorithms and Concurrent Programming 39

Chapter 5
Gadi Taubenfeld © 2014
 Test-and-Set based Barriers
Properties
 Different object (T&S instead of F&I)

 Pros:
 Shared memory: Only bits - O(1) space
As opposed to the counter-based which requires O(log n)
 Does not require memory initialization (in the second version)

 Cons:
 Asymmetric (in the second version)
 Still high contention on countflag & go bits

Synchronization Algorithms and Concurrent Programming 40

Chapter 5
Gadi Taubenfeld © 2014
 Tree Based Barriers
Section 5.5

Chapter 5 Synchronization Algorithms and Concurrent Programming 41

Gadi Taubenfeld © 2014
 A Tree-based Barrier
 The processes are organized in a binary tree

 Each node is owned by a predetermined process

 Each process waits until its 2 children arrive, combines the

results and passes them on to its parent

 When the root learns that its 2 children have arrived, it tells

its children that they can move on

 The signal propagates down the tree until all the processes

get the message

1

2 3

4 5 6 7
Synchronization Algorithms and Concurrent Programming 42
Chapter 5
Gadi Taubenfeld © 2014
 A Tree-based Barrier
1

Assume  
𝑖 

2 3
2  𝑖  

4 5 6 7

8 9 10 11 12 13 14 15

arrive

go
2 3 4 5 6 7 8 9 10 11 12 13 14 15

Synchronization Algorithms and Concurrent Programming 43

Chapter 5
Gadi Taubenfeld © 2014
 A Tree-based Barrier
program of process i
shared arrive[2..n]: array of atomic bits, initial values = 0
go[2..n]: array of atomic bits, initial values = 0

1 if i=1 then // root

2 await(arrive[2] = 1); arrive[2] := 0
3 await(arrive[3] = 1); arrive[3] := 0
4 go[2] = 1; go[3] = 1
5 else if i ≤ (n-1)/2 then // internal node
6 await(arrive[2i] = 1); arrive[2i] := 0
7 await(arrive[2i+1] = 1); arrive[2i+1] := 0
8 arrive[i] := 1
9 await(go[i] = 1); go[i] := 0
10 go[2i] = 1; go[2i+1] := 1
11 else // leaf
12 arrive[i] := 1
13 await(go[i] = 1); go[i] := 0 fi
14 fi

Synchronization Algorithms and Concurrent Programming 44

Chapter 5
Gadi Taubenfeld © 2014
 A Tree-based Barrier
Example Run for n=7 Processes
P132 zeros
arrive[2]=1
Arrive[1]=1
!!Finished
?
arrive[6,7]
arrive[4,5]
arrive[2]
arrive[3]
Waiting for
p3 to arrive 1

Waiting for
Waiting for go[3]
p4 to arrive
go[2]

Waiting for 2 3 Waiting for

go[5]
go[4] go[7]
go[6]

4 5 6 7

arrive 01 01 01 01 01 01

go 1 1 1 1 1 1

Chapter 5
2 3 4 5 6 7 45
 A Tree-based Barrier
Pros:
Low shared memory contention
No bit is shared by more than 2 processes
Good for larger n
Fast (in comparison to local spinning)
– information from the root propagates after log(n) steps
Uses only atomic bits (no special objects)
On some models:
each process spins on a locally accessible bit
# (remote memory ref.) = O(1) per process
Cons:
Shared memory space complexity – O(n)
Asymmetric – not all the processes do the same amount of work (*)

There is a similar barrier which is symmetric, but at the cost of more )*(
. shared memory consumption -- O(nlogn) as opposed to O(n)
.See the Dissemination Barrier from Section 5.6 page 213

Synchronization Algorithms and Concurrent Programming 46

Chapter 5
Gadi Taubenfeld © 2014
 The See-Saw Barrier
Section 5.7

Chapter 5 Synchronization Algorithms and Concurrent Programming 47

Gadi Taubenfeld © 2014
 See-Saw Barrier

 Now, we’ll use a Read-Modify-Write object

 Allows to construct a symmetric barrier, that requires only
few shared bits
 This algorithm can also be used to solve the leader election
and the consensus problems

 The See-Saw barrier is based on a solution to the wake-up

problem which was proposed by M. J. Fischer, S. Moran, S.
Rudich, G. Taubenfeld (1996)

Synchronization Algorithms and Concurrent Programming 48

Chapter 5
Gadi Taubenfeld © 2014
 Read-Modify-Write Register

 Input: register r with n bits, function f(r)

 Atomic operation:
 Reads the register
 Calls function f on r, return value is written into r
 The old value of r is returned

function read-modify-write (r : register, f : function)

orig_r := r;
r := f(r);
return (orig_r);
end-function

 Usually f is custom made for the algorithm

Synchronization Algorithms and Concurrent Programming 49

Chapter 5
Gadi Taubenfeld © 2014
 Data Flow
 Tokens:
 Each process starts with 2 tokens
 Total number of tokens doesn’t change
 Each process can absorb one token or emit one token,
at a time
 See-Saw:
 One see-saw
 Can be left-up-right-down OR left-down-right-up
 Each process that enters the playground needs to get-
up on the see-saw
 Each process which is on the see-saw is either on the
left side or the right side
 Tokens are weightless

Synchronization Algorithms and Concurrent Programming 50

Chapter 5
Gadi Taubenfeld © 2014
 Data Representation
Using 2-bit read-modify-write register

Token Bit See-saw Bit

Two states: Two states:
1. no-token-present 1. left-side-down
2. token-present 2. right-side-down

P2
T: 3
2
P1
T: 1
2

Synchronization Algorithms and Concurrent Programming 51

Chapter 5
Gadi Taubenfeld © 2014
 Process State

On right
side
On left
side

Never P7
P3 Got-off
been on T:T:22
P6P5
T:T:2P2
T:22
P1
P4 T: 0
T: 2

Synchronization Algorithms and Concurrent Programming 52

Chapter 5
Gadi Taubenfeld © 2014
 Runtime Flow

  process loops until it got-off from the see-saw

Each

After it got-off, waits for the go flag

The algorithm is based on 5-rules

On each loop iteration:
According to its state, one rule is performed
Only one process at a time performs a rule
A rule is done atomically, using the RMW register
Each rule changes the tokens and/or the state of the see-saw

There can be many processes on each side (up to )

When one of the processes gets 2n tokens, it gets-off and
sets the go flag

Synchronization Algorithms and Concurrent Programming 53

Chapter 5
Gadi Taubenfeld © 2014
 Rule #1 – Start of Token-state No-token-present
RMW
Algorithm See-saw-state Right-side-down
Left-side-down

 Applicable if:
scheduled process is “never-been-on”
 Operation:
 Saves the go bit locally
 got on the up side, and swings the see-saw

P2 P1
T: 2 T: 2

Chapter 5 54
 Rule #2 – Emitter Token-state No-token-present
Token-present
RMW
See-saw-state Left-side-down

 Applicable if:
scheduled process is “down-side”, has tokens,
and token-state = no-token-present
 Operation:
 Deposit one token in the shared token-state
 If remains without tokens, got-off the see-saw, and swing
it
P1
T: 2

P2
T: 21

Chapter 5 55
 Rule #3 – Absorber Token-state No-token-present
Token-present
RMW
See-saw-state Left-side-down

 Applicable if:
scheduled process is “up-side”, and
token-state = token-present
 Operation:
 Takes the token from token-state

P1
T: 23

P2
T: 1

Chapter 5 56
 Rule #2 – Emitter Token-state No-token-present
Token-present
RMW
See-saw-state Right-side-down
Left-side-down

 Applicable if:
scheduled process is “down-side”, has tokens,
and token-state = no-token-present
 Operation:
 Deposit one token in the shared token-state
 If remains without tokens, got-off the see-saw, and swing
!
it
The process that got-off now P1
awaits the go flag T: 3

P2
T: 0
1

Chapter 5 57
 Rule #4 – Leader Token-state No-token-present
Token-present
RMW
See-saw-state Right-side-down

 Applicable if:
scheduled process is on the see-saw, and sees at
least 2n tokens
 Operation:
 Gets-off the see-saw, and flips the shared go bit

…ZZZ

P1
T: 3
P2
T: 0
Chapter 5 58
 Rule #4 – Leader Token-state No-token-present
RMW
See-saw-state Right-side-down

 Applicable if:
scheduled process is on the see-saw, and sees at
least 2n tokens
 Operation:
 Gets-off the see-saw, and flips the shared go bit

!go
…ZZZ

P1
T: 4
P2
T: 0
Chapter 5 59
 Rule #5 – End of Token-state No-token-present
the Algorithm RMW
See-saw-state Right-side-down

 Applicable if:
scheduled process notices that the go bit has been
flipped (relative to its local.go)
 Operation:
 Everybody has arrived  continue past the barrier

!go
…ZZZ

P2 P1
T: 0 T: 4
Chapter 5 60
 Important Invariants

Token Invariant
During a single episode of the see-saw barrier, the
number of tokens in the system
 is either 2n or 2n+1 (like in the test-and-set barrier)
 never changes

Balance Invariant
During a single episode of the see-saw barrier, the
number of processes on the left and on the right
side of the see-saw is
 either perfectly balanced
 or favored the down-side by 1

Synchronization Algorithms and Concurrent Programming 61

Chapter 5
Gadi Taubenfeld © 2014
 Correctness

 When all processes are on the see-saw:

 Tokens are given from the down side, until one gets-off
 By induction, at some point:
 one process will see 2n tokens
 So no deadlock.

 2n tokens can only be accumulated if all processes have

arrived, so this is a barrier.

Synchronization Algorithms and Concurrent Programming 62

Chapter 5
Gadi Taubenfeld © 2014
 Remarks

 All the logic is done inside the atomic Modify function

of the RMW register

 Needs to read and modify all the three bits atomically,

to prevent race-conditions

 Before a process applies a rule, it first checks

whether the go bit has been flipped relative to its
local.go (regardless of its current state) !!!

Synchronization Algorithms and Concurrent Programming 63

Chapter 5
Gadi Taubenfeld © 2014
 Question
 How many times does the state of the shared memory change
during one episode of the see-saw barrier?

 O(n) in the best case

 O(n2) in the worst case

Synchronization Algorithms and Concurrent Programming 64

Chapter 5
Gadi Taubenfeld © 2014
 The See-Saw Barrier

Pros:
O(1) shared memory space complexity
No need to initialize shared memory
Symmetric
Cons:
Uses custom Read-Modify-Write register
High memory contention on the RMW bits
Worst case O(n2) total shared memory
references
Complex

Synchronization Algorithms and Concurrent Programming 65

Chapter 5
Gadi Taubenfeld © 2014
The code:

Synchronization Algorithms and Concurrent Programming 66

Chapter 5
Gadi Taubenfeld © 2014
 A Barrier using Semaphores
Section 5.8

Chapter 5 Synchronization Algorithms and Concurrent Programming 67

Gadi Taubenfeld © 2014
 Semaphore
Shared object
Takes a non-negative integer value
Supports two operations:
Down
 Ifvalue > 0, the value is decremented by 1
 Otherwise, the process is blocked until the value
becomes > 0
Up – the value is incremented by 1
Incrementing, Decrementing and testing the
semaphore are executed atomically

Synchronization Algorithms and Concurrent Programming 68

Chapter 5
Gadi Taubenfeld © 2014
 Binary Semaphore
Semaphore whose value is only 0 or 1
Decrementing is identical to general
semaphore
Incrementing is equal to setting the value to 1
Initial value is assume to be 1

Can be used to implement a deadlock-free

mutual exclusion:
down(S)
critical-section
up(S)

Synchronization Algorithms and Concurrent Programming 69

Chapter 5
Gadi Taubenfeld © 2014
 Barrier using Semaphores
Algorithm for n processes
shared arrival: binary semaphore, initially 1
departure: binary semaphore, initially 0
counter: atomic register ranges over {0, …, n}, initially 0

1 down(arrival)
2 counter := counter + 1 // atomic register
3 if counter < n then up(arrival) else up(departure) fi
4 down(departure)
5 counter := counter - 1
6 if counter > 0 then up(departure) else up(arrival) fi

:Question
Would this barrier be correct if the
?shared counter won’t be an atomic register
Synchronization Algorithms and Concurrent Programming 70
Chapter 5
Gadi Taubenfeld © 2014
 Barrier using Semaphores
Properties
Pros:
Very Simple
Space complexity O(1)
Symmetric
Cons:
Required a strong object
 Requires some central manager
 High contention on the semaphores if no central manager
Propagation delay O(n)

Synchronization Algorithms and Concurrent Programming 71

Chapter 5
Gadi Taubenfeld © 2014
 Summary
Barrier Synchronization

Chapter 5 Synchronization Algorithms and Concurrent Programming 72

Gadi Taubenfeld © 2014
 Barriers we’ve seen

Simple barrier
 Based on atomic fetch-and-increment counter

Local spinning barrier

 Based on atomic fetch-and-increment counter
and go array
Test-and-Set barriers
 Based on test-and-test-and-set objects
 One version without memory initialization

Tree-based barrier
See-Saw barrier
Semaphore-based barrier

Synchronization Algorithms and Concurrent Programming 73

Chapter 5
Gadi Taubenfeld © 2014
 Conclusions

 Many possible algorithms for Barrier Synchronization

 Each has pros/cons

 Different shared objects allow various algorithms

 Choosing the correct barrier is application/platform

dependent (need to do benchmarking to know for sure).

Synchronization Algorithms and Concurrent Programming 74

Chapter 5
Gadi Taubenfeld © 2014