Search and Sort Algorithms

CME4422 – Introduction to Graph Theory

Asst. Prof. Dr. Feriştah DALKILIÇ

[email protected]
 Outline

◘ Breadth-first Search
◘ Depth-first Search
◘ Topological Sorting
 Graph-Searching Algorithms

◘ Searching a graph:
– Systematically follow the edges of a graph to visit the
vertices of the graph.
◘ Used to discover the structure of a graph.
◘ Standard graph-searching algorithms.
– Breadth-First Search (BFS).
– Depth-First Search (DFS).
– Topological Ordering (Sorting)
 Breadth-First Search

◘ Input: Graph G = (V, E), either directed or undirected, and

source vertex s  V.
◘ Output:
– d[v] = distance (smallest # of edges, or shortest path) from s to v,
for all v  V. d[v] =  if v is not reachable from s.
– [v] = u such that (u, v) is last edge on shortest path s v. u is v’s
predecessor.
– Builds breadth-first tree with root s that contains all reachable
vertices.

Definitions:
Path between vertices u and v: Sequence of vertices (v1, v2, …, vk)
such that u=v1 and v =vk, and (vi,vi+1)  E, for all 1 i  k-1.
Length of the path: Number of edges in the path.
Path is simple if no vertex is repeated.
 Breadth-first Search

◘ Expands the frontier between discovered and

undiscovered vertices uniformly across the
breadth of the frontier.

– A vertex is discovered the first time it is found during the

search.

– A vertex is finished if all vertices adjacent to it have

been discovered.
 Pseudo-Code for Breadth-First Search

 Choose a starting vertex

 Search all adjacent vertices
 Return to each adjacent vertex in turn and visit all of its
adjacent vertices

breadth-first-search
mark starting vertex as visited; put on queue
while the queue is not empty
dequeue the next node
for all unvisited vertices adjacent to this one
 mark vertex as visited
 add vertex to queue
 BFS for Shortest Paths

2 3 3
2
2 3
1 S
S 2 S
1 1
2
2 3 3

Colors the vertices to keep track of progress.

White – Undiscovered.
Gray – Discovered but not finished.
Black – Finished.
BFS(G, s) // G is the graph and s is the starting node
1 for each vertex u ∈ V[G] - {s}
2 color[u] ← WHITE // Lines 1–4 paint every vertex white,
3 d[u] ← ∞ // Set d[u] to be infinity for each vertex u,
// Set the parent of every vertex to be NIL.
4 π[u] ← NIL
5 color[s] ← GRAY // Line 5 paints the source vertex s gray
6 d[s] ← 0 // Line 6 initializes d[s] to 0,
7 π[s] ← NIL // Line 7 sets the predecessor of the source to be NIL.
8 Q ← Ø // Lines 8–9 initialize Q to the queue containing just the vertex
s.
9 ENQUEUE(Q, s)
10 while Q ≠ Ø // Lines 10-18 iterates as long as there are gray vertices.
11 u ← DEQUEUE(Q)
12 for each v ∈ Adj[u]
13 do if color[v] = WHITE // discover the undiscovered adjacent vertices
14 color[v] ← GRAY // enqueued whenever painted gray
15 d[v] ← d[u] + 1
16 π[v] ← u
17 ENQUEUE(Q, v)
18 color[u] ← BLACK // painted black whenever dequeued

Q: a queue of discovered v
white: undiscovered
color[v]: color of v
gray: discovered
d[v]: distance from s to v
black: finished
[v]: predecessor of v
Example (BFS)

r s t u
 0  

   
v w x y

Q: s
0
Example (BFS)

r s t u
1 0  

 1  
v w x y

Q: w r
1 1
Example (BFS)

r s t u
1 0 2 

 1 2 
v w x y

Q: r t x
1 2 2
Example (BFS)

r s t u
1 0 2 

2 1 2 
v w x y

Q: t x v
2 2 2
Example (BFS)

r s t u
1 0 2 3

2 1 2 
v w x y

Q: x v u
2 2 3
Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: v u y
2 3 3
Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: u y
3 3
Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: y
3
Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: 
 Example (BFS)

s
0

r w
BF Tree 1
1

v t x
2 2 2

u y
3 3
 Analysis of BFS

 Computational Complexity
 Initialization takes O(V).
 Traversal Loop
• Each vertex is enqueued and dequeued at most once, and each
operation takes O(1). So, total time for queuing is O(V).
• Adjacency list representation
– The adjacency list of each vertex is scanned at most once.
– The sum of lengths of all adjacency lists is O(E).
– Summing up over all vertices: O(V+E) linear in the size of the graph.
• Adjacency matrix representation
– Have to check all entries in matrix: O(n2)
 Space Complexity
 Queue to handle unexplored nodes
• Worst case: all nodes put on queue (if all are adjacent to first node)
• O(n)
Breadth-First Search (BFS)
 Applications of BFS

◘ Shortest paths in graphs which have equal edge weights.

◘ To compute maximum flow in Ford-Fulkerson method.
◘ Certain pattern (e.g., triangular) matching in a large graph.
◘ To test if a graph has the bipartite property or not
 Depth-first Search (DFS)

◘ Explore edges out of the most recently discovered vertex v.

◘ When all edges of v have been explored, backtrack to
explore other edges leaving the vertex from which v was
discovered (its predecessor).
◘ “Search as deep as possible first”.
◘ Continue until all vertices reachable from the original source
are discovered.
◘ If any undiscovered vertices remain, then one of them is
chosen as a new source and search is repeated from that
source.
 Depth-First Search

◘ Input: G = (V, E), directed/undirected. No source vertex is

given!
◘ Output:
– 2 timestamps on each vertex. Integers between 1 and 2|V|.

• d[v] = discovery time (v turns from white to gray)

• f [v] = finishing time (v turns from gray to black)
– [v] : predecessor of v = u, such that v was discovered
during the scan of u’s adjacency list.
◘ Uses the same coloring scheme for vertices as BFS.
 Depth-First Search

DFS follows the following rules:

1. Select an unvisited node x, visit it, and treat as the
current node;
2. Find an unvisited neighbor of the current node, visit it,
and make it the new current node;
3. If the current node has no unvisited neighbors, backtrack
to its parent, and make that parent as the new current
node;
4. Repeat steps 3 and 4 until no more nodes can be visited.
5. If there are still unvisited nodes, repeat from step 1.
 Illustration of DFS

A
A B
B
C D
I D C
E L I
J E
K H L K
F F
H J

Graph G DFS Tree

DFS(G) Lines 1–3 paint all vertices white and
1.for each vertex u  V[G] initialize their π fields to NIL.
Line 4 resets the global time counter.
2. color[u]  white Lines 5–7 check each vertex in V in turn
3. [u]  NIL and, when a white vertex is found, visit it
4.time  0 using DFS-VISIT.
5.for each vertex u  V[G] Every time DFS-VISIT(u) is called in line 7,
vertex u becomes the root of a new tree in
6. if color[u] = white the depth-first forest. When DFS returns,
7. DFS-Visit(u) every vertex u has been assigned a
discovery time d[u] and a finishing time f[u].

DFS-Visit(u) Line 1 paints u gray.

1.color[u]  GRAY Line 2 increments the global variable time,
Line 3 records the new value of time as the
2.time  time + 1 discovery time d[u].
3.d[u]  time Lines 4–7 examine each vertex v adjacent
4.for each v  Adj[u] to u and recursively visit v if it is white. As
5. if color[v] = WHITE each vertex v ∈ Adj[u] is considered in line
4, we say that edge (u, v) is explored by
6. [v]  u the depth-first search.
7. DFS-Visit(v) Finally, after every edge leaving u has been
8.color[u]  BLACK explored, lines 8–9 paint u black and record
9.f[u]  time  time + 1 the finishing time in f[u].
 Classification of Edges

◘ Tree edge: in the depth-first forest. Found by exploring

(u, v).
◘ Back edge: (u, v), where u is a descendant of v (in the
depth-first tree).
◘ Forward edge: (u, v), where v is a descendant of u, but
not a tree edge.
◘ Cross edge: any other edge. Can go between vertices
in same depth-first tree or in different depth-first trees.

Theorem:
In DFS of an undirected graph, we get only tree and back edges.
No forward or cross edges.
Example (DFS)

u v w
1/

x y z
Example (DFS)

u v w
1/ 2/

x y z
Example (DFS)

u v w
1/ 2/

3/
x y z
Example (DFS)

u v w
1/ 2/

4/ 3/
x y z
Example (DFS)

u v w
1/ 2/

4/ 3/
x y z
Example (DFS)

u v w
1/ 2/

4/5 3/
x y z
Example (DFS)

u v w
1/ 2/

4/5 3/6
x y z
Example (DFS)

u v w
1/ 2/7

4/5 3/6
x y z
Example (DFS)

u v w
1/ 2/7

F B

4/5 3/6
x y z
Example (DFS)

u v w
1/8 2/7

F B

4/5 3/6
x y z
Example (DFS)

u v w
1/8 2/7 9/

F B

4/5 3/6
x y z
Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6
x y z
Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6 10/

x y z
Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6 10/ B

x y z
Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6 10/11 B

x y z
Example (DFS)

u v w
1/8 2/7 9/12

F B C

4/5 3/6 10/11 B

x y z
 Analysis of DFS

◘ Loops on lines 1-2 and 5-7 take O(V) time,

excluding time to execute DFS-Visit.

◘ DFS-Visit is called once for each white vertex

v  V when it’s painted gray the first time.
Lines 3-6 of DFS-Visit is executed |Adj[v]|
times. The total cost of executing DFS-Visit is
v V |Adj[v]| = O(E)

◘ Total running time of DFS : O(V+E)

Depth-First Search (DFS)
 Applications of DFS

◘ Find connected components of a large graph

◘ Find bridges of a graph
◘ Solve one solution puzzles (e.g. maze)
◘ Find bi-connectivity in graphs
◘ …
 Digraphs

◘ A digraph is a graph
whose edges are all E

directed
D
– Short for “directed graph”
◘ Applications C
– one-way streets
– flights B

– task scheduling
A
 Digraph Application

◘ Scheduling: edge (a,b) means task a must be

completed before b can be started
cs21 cs22 cs46

cs51 cs53 cs52

cs161

cs131 cs141 cs121 cs171

cs151 The good life

 Topological Sorting

◘ Number vertices, so that (u,v) in E implies u < v

1
wake up
2 3
study computer sci.
eat A typical student day

4 5
nap more c.s.
7
play
8
write c.s. program 6
9 work out
bake cookies
10
sleep 11
dream about graphs
 Topological Ordering
source sink
◘ A directed acyclic graph (DAG)
is a digraph that has no directed D E
cycles.
◘ Every DAG has at least one B
source and at least one sink. DAG G
C
◘ A topological ordering of a
digraph is a numbering A
v1 , …, vn v4 v5
of the vertices such that for D E
every edge (vi , vj), we have i < j v2
B
Theorem: v3
v1 C
A digraph admits a topological Topological
ordering if and only if it is a DAG A
ordering of G
Running Time

O(n + m)
Topological Sorting Example

A
B

C
D

E F

H
G

I
 Topological Sorting Example
0
0
A
B
1

C 3

D
2
E F
2
1
H
G
B
1
A I 3

ready
indegree ordering
 Topological Sorting Example
0
1
A
B
1

C 2

D
1
E F
2
1
B H
G

1
A I 3

ready
 Topological Sorting Example
2
1
A
B
0

C 1

D
1
E F
2
1
A H
G

1
C I 3

ready
 Topological Sorting Example
2
1
A
B
3

C 0

D
1
E F
2
0
C H
G G
E
0
D I 3

ready
 Topological Sorting Example
2
1
A
B
3

C 0

D
1
E F
2
0
G H
G
E
4
D I 2

ready
 Topological Sorting Example
2
1
A
B
3

C 0

D
1
E F
1
5
E H
G

4
D I 2

ready
 Topological Sorting Example
2
1
A
B
3

C 6

D
0
E F
1
5
D H
G

4
F I 2

ready
 Topological Sorting Example
2
1
A
B
3

C 6

D
7
E F
0
5
F H
G

4
H I 1

ready
 Topological Sorting Example
2
1
A
B
3

C 6

D
7
E F
8
5
H H
G

4
I I 0

ready
 Topological Sorting Example
2
1
A
B
3

C 6

D
7
E F
8
5
I H
G

4
I 9

ready
Topological Sorting Example

2
1

3
6

5 7

8
4

9
 Applications of TS

◘ Dependency resolution
– Find compilation order of dependent classes
– Project management
– Instruction scheduling
–…
Questions?

Thank you for listening!