20CA102

Data Structures and Algorithms

UNIT III

ALGORITHMIC PROBLEM
SOLVING AND ANALYSIS
 What is an Algorithm

An algorithm is a sequence of

unambiguous instructions for
solving a problem

i.e., for obtaining a required output

for any legitimate input in a finite
amount of time
What is an Algorithm
Finding GCD of 2 Numbers
Finding GCD of 2 Numbers
Finding GCD of 2 Numbers
Finding GCD of 2 Numbers
Algorithm Design Process
 Algorithm Design Process
• Understanding the Problem
• Ascertaining the Capabilities of the
Computational Device
• Choosing between Exact and Approximate
Problem Solving
• Algorithm Design Techniques
• Designing an Algorithm and Data Structure
• Methods of Specifying and Algorithm
• Proving and Algorithm’s Correctness
• Analyzing an Algorithm
• Coding and Algorithm
 Puzzle 1
• Old World puzzle A peasant finds himself on a riverbank
with a wolf, a goat, and a head of cabbage. He needs to
transport all three to the other side of the river in his
boat. However, the boat has room for only the peasant
himself and one other item (either the wolf, the goat, or
the cabbage).

• In his absence, the wolf would eat the goat, and the goat
would eat the cabbage.

• Solve this problem for the peasant or prove it has no

solution. (Note: The peasant is a vegetarian but does not
like cabbage and hence can eat neither the goat nor the
cabbage to help him solve the problem. And it goes
without saying that the wolf is a protected species.)
 Puzzle 2
• New World puzzle There are four people who want to cross a
rickety bridge; they all begin on the same side. You have 17
minutes to get them all across to the other side. It is night, and
they have one flashlight.
• A maximum of two people can cross the bridge at one time. Any
party that crosses, either one or two people, must have the
flashlight with them.
• The flashlight must be walked back and forth; it cannot be
thrown, for example. Person 1 takes 1 minute to cross the
bridge, person 2 takes 2 minutes, person 3 takes 5 minutes, and
person 4 takes 10 minutes.
• A pair must walk together at the rate of the slower person’s
pace. (Note: According to a rumor on the Internet, interviewers
at a well-known software company located near Seattle have
given this problem to interviewees.)
 Important Problem Types
• Sorting
• Searching
• String processing
• Graph problems
• Combinatorial problems
• Geometric problems
• Numerical problems
 Sorting
• The sorting problem is to rearrange the items
of a given list in non decreasing order.
• Of course, for this problem to be meaningful,
the nature of the list items must allow such an
ordering.
• A sorting algorithm is called stable if it
preserves the relative order of any two equal
elements in its input.
• An algorithm is said to be in-place if it does
not require extra memory, except, possibly,
for a few memory units
 Searching
• The searching problem deals with finding a
given value, called a search key, in a given set
(or a multiset, which permits several elements
to have the same value).
• There are plenty of searching algorithms to
choose from.
• In applications where the underlying data
may change frequently relative to the number
of searches, searching has to be considered in
conjunction with two other operations: an
addition to and deletion from the data set of
an item.
 String Processing
• A string is a sequence of characters from an
alphabet.
• Strings of particular interest are text strings,
which comprise letters, numbers, and special
characters; bit strings, which comprise zeros
and ones; and gene sequences which can be
modeled by strings of characters from the
four-character alphabet {A, C, G, T}.
• One particular problem—that of searching for
a given word in a text— has attracted special
attention from researchers. They call it string
matching.
 Graph Problems
• One of the oldest and most interesting areas
in algorithmics is graph algorithms.
• Informally, a graph can be thought of as a
collection of points called vertices, some of
which are connected by line segments called
edges.
• Basic graph algorithms include graph-
traversal algorithms (how can one reach all
the points in a network?),
• shortest-path algorithms (what is the best
route between two cities?),
 Geometric Problems
• Geometric algorithms deal with geometric
objects such as points, lines, and polygons.
• The ancient Greeks were very much
interested in developing procedures for
solving a variety of geometric problems
• Problems of constructing simple geometric
shapes—triangles, circles, and so on.
• The closest-pair problem is self-explanatory:
given n points in the plane, find the closest
pair among them. The convex-hull problem
asks to find the smallest convex polygon that
would include all the points of a given set.
 Numerical Problems
• Numerical problems, another large special
area of applications, are problems that
involve mathematical objects of continuous
nature: solving equations and systems of
equations, computing definite integrals,
evaluating functions, and so on.
 Konigsberg bridges
• Konigsberg bridges The Konigsberg bridge
puzzle is universally accepted as the problem
that gave birth to graph theory. It was solved
by the great Swiss-born mathematician
Leonhard Euler (1707–1783).
• The problem asked whether one could, in a
single stroll, cross all seven bridges of the city
of Konigsberg exactly once and return to a
starting point.
Konigsberg bridges
Fundamental Data Structures
 Linear Data Structures
 Arrays
 Linked list – Singly Linked List and Doubly Linked
 Stack
 Queues
 Graphs
 A graph G(V,E), is defined by a pair of two sets:
 a finite nonempty set V of items called vertices
 a set E of pairs of these items called edges.
 If these pairs of vertices are unordered, i.e., a pair of
vertices (u, v) is the same as the pair (v, u), we say that the
vertices u and v are adjacent to each other and that they are
connected by the undirected edge (u, v).
 We call the vertices u and v endpoints of the edge (u, v) and
say that u and v are incident to this edge.
 We also say that the edge (u, v) is incident to its endpoints u
and v.
 A graph G is called undirected if every edge in it is undirected.
Graphs
 Graph Representations
• The adjacency matrix of a graph with n
vertices is an n × n boolean matrix with one
row and one column for each of the graph’s
vertices, in which the element in the ith row
and the jth column is equal to 1 if there is an
edge from the ith vertex to the jth vertex, and
equal to 0 if there is no such edge
Adjacency Matrix
 Graph Representations
• The adjacency lists of a graph or a digraph is a
collection of linked lists, one for each vertex,
that contain all the vertices adjacent to the
list’s vertex (i.e., all the vertices connected to
it by an edge)
List Representation
 Weighted Graphs
• A weighted graph (or weighted digraph)
is a graph (or digraph) with numbers
assigned to its edges.
• These numbers are called weights or
costs. An interest in such graphs is
motivated by numerous real-world
applications
 Paths and Cycles
• Among the many properties of graphs, two are
important for a great number of applications:
connectivity and acyclicity.
• Both are based on the notion of a path.
• A path from vertex u to vertex v of a graph G can be
defined as a sequence of adjacent (connected by an
edge) vertices that starts with u and ends with v.
• If all vertices of a path are distinct, the path is said
to be simple.
• The length of a path is the total number of vertices
in the vertex sequence defining the path minus 1,
which is the same as the number of edges in the
path
Paths and Cycles
 Algorithm Analysis Framework

• Time efficiency, also called time

complexity, indicates how fast an
algorithm in question runs.
• Space efficiency, also called space
complexity, refers to the amount of
memory units required by the algorithm
in addition to the space needed for its
input and output.
 Measuring an Input’s Size

• Almost all algorithms run longer on

larger inputs.
• There are situations, of course, where
the choice of a parameter indicating an
input size does matter.
 Units for Measuring Running Time

• The speed of a particular computer

• The quality of a program implementing the
algorithm
• The compiler used in generating the
machine code.
– Identify the most important operation of the
algorithm, called the basic operation
– Operation contributing the most to the total
running time
– compute the number of times the basic
operation is executed.
 Units for Measuring Running Time

• The speed of a particular computer

• The quality of a program implementing the
algorithm
• The compiler used in generating the
machine code.
– Identify the most important operation of the
algorithm, called the basic operation
– Operation contributing the most to the total
running time
– compute the number of times the basic
operation is executed.
 Units for Measuring Running Time

• Let cop be the execution time of an

algorithm’s basic operation on a particular
computer,
• Let C(n) be the number of times this
operation needs to be executed for this
algorithm.
• Then we can estimate the running time
T(n) of a program implementing this
algorithm on that computer by the formula
 Units for Measuring Running Time

How fast the algorithm can run, if we double the input size
Units for Measuring Running Time
Orders of Growth
 Orders of Growth

• For example, it would take about 4 . 1010

years for a computer making a trillion (1012)
operations per second to execute 2100
operations.
• Though this is incomparably faster than it
would have taken to execute 100!
operations,
• it is still longer than 4.5 billion (4.5 . 109)
years—the estimated age of the planet
Earth
Worst-Case, Best-Case, and
Average-Case Efficiencies
 Worst-Case efficiency

• The worst-case efficiency of an algorithm is its

efficiency for the worst-case input of size n,
which is an input (or inputs) of size n for which
the algorithm runs the longest among all
possible inputs of that size.
 Best-Case efficiency

• The best-case efficiency of an algorithm is its

efficiency for the best-case input of size n,
which is an input (or inputs) of size n for which
the algorithm runs the fastest among all
possible inputs of that size
 Average-case efficiency

• Neither the worst-case analysis nor its best-case

counterpart yields the necessary information about
an algorithm’s behavior on a “typical” or “random”,
is called the average-case efficiency.
• For successful search, the probability of the first
match occurring in the ith position of the list is p/n
for every i.
• In the case of an unsuccessful search, the number
of comparisons will be n with the probability of
such a search being (1− p).
 Asymptotic Notations and Basic
Efficiency Classes
• To compare and rank such orders of growth,
computer scientists use three notations: O (big
oh), (big omega), and (big theta)
 Amortized Efficiency
• Yet another type of efficiency is called
amortized efficiency.
• It applies not to a single run of an algorithm
but rather to a sequence of operations
performed on the same data structure.
• It turns out that in some situations a single
operation can be expensive, but the total
time for an entire sequence of n such
operations is always significantly better than
the worst-case efficiency of that single
operation multiplied by n.
 O-notation
• A function t (n) is said to be in O(g(n)),
denoted t (n) ∈ O(g(n)),
• if t (n) is bounded above by some
constant multiple of g(n) for all large n,
• i.e., if there exist some positive constant
c and some nonnegative integer n0
such that
t (n) ≤ cg(n) for all n ≥ n0
O-notation
 O-notation
• O(g(n)) is the set of all functions with a
lower or same order of growth as g(n)
• (to within a constant multiple, as n goes
to infinity).
• A function t (n) is said to be in (g(n)),
denoted t (n) ∈ (g(n))
• If t (n) is bounded below by some positive
constant multiple of g(n) for all large n,
• i.e., if there exist some positive constant c
and some nonnegative integer n0 such
that.
• A function t (n) is said to be in (g(n)), denoted
t (n) ∈ (g(n)),
• if t (n) is bounded both above and below by
some positive constant multiples of
• g(n) for all large n, i.e., if there exist some
positive constants c1 and c2 and some
• nonnegative integer n0 such that
Basic Efficiency Classes
Mathematical Analysis of Non recursive Algorithms

Consider the problem of finding the value of the largest

element
in a list of n numbers. For simplicity, we assume that the list is
implemented as an array. The following is pseudocode of a
standard algorithm for solving the problem.

ALGORITHM MaxElement(A[0..n − 1])

//Determines the value of the largest element in a given
array
//Input: An array A[0..n − 1] of real numbers
//Output: The value of the largest element in A
maxval ←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
return maxval
 Mathematical Analysis of Non recursive
Algorithms
• The obvious measure of an input’s size here is
the number of elements in the array, i.e., n. The
operations that are going to be executed most
often are in the algorithm’s for loop.
• There are two operations in the loop’s body:
the comparison A[i]> maxval and the
assignment maxval←A[i].
• Which of these two operations should we
consider basic?
• Since the comparison is executed on each
repetition of the loop and the assignment is
not, we should consider the comparison to be
the algorithm’s basic operation
Mathematical Analysis of Non recursive
Algorithms
General Plan for Analyzing the Time Efficiency of Non
recursive Algorithms
1. Decide on a parameter (or parameters) indicating an
input’s size.
2. Identify the algorithm’s basic operation. (As a rule, it is
located in the innermost loop.)
3. Check whether the number of times the basic operation is
executed depends only on the size of an input. If it also
depends on some additional property, the worst-case,
average-case, and, if necessary, best-case efficiencies have
to be investigated separately.
4. Set up a sum expressing the number of times the
algorithm’s basic operation is executed.
5. Using standard formulas and rules of sum manipulation,
either find a closed form formula for the count or, at the
very least, establish its order of growth
General Plan for Analyzing the Time Efficiency of Non
recursive Algorithms
 Element uniqueness problem
Check whether all the elements in a given array of n
elements are distinct. This problem can be solved by
the following straightforward algorithm.

ALGORITHM UniqueElements (A[0..n − 1])

//Determines whether all the elements in a given array are
distinct
//Input: An array A[0..n − 1]
//Output: Returns “true” if all the elements in A are distinct
// and “false” otherwise
for i ←0 to n − 2 do
for j ←i + 1 to n − 1 do
if A[i]= A[j ] return false
return true
Element uniqueness problem
 Mathematical Analysis of Recursive Algorithms

Compute the factorial function F(n) = n! for an

arbitrary nonnegative integer n

ALGORITHM F(n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0 return 1
else return F(n − 1) ∗ n
 Mathematical Analysis of Recursive Algorithms

F(n) = F(n − 1) . n, for n > 0

M(n) = M(n − 1) + 1 , for n > 0.

to compute F(n−1) to multiply F(n−1) by n

if n = 0, return 1
Mathematical Analysis of Recursive Algorithms
Mathematical Analysis of Recursive Algorithms
General Plan for Analyzing the Time Efficiency
of Recursive Algorithms
1. Decide on a parameter (or parameters)
indicating an input’s size.
2. Identify the algorithm’s basic operation
3. Check whether the number of times the basic
operation is executed can vary on different
inputs of the same size; if it can, the worst-case,
average-case, and best-case efficiencies must be
investigated separately.
4. Set up a recurrence relation, with an appropriate
initial condition, for the number of times the
basic operation is executed.
5. Solve the recurrence or, at least, ascertain the
order of growth of its solution.