DESIGN AND ANALYSIS OF

ALGORITHMS
Course Code : AR212
Credit Hours :3
Prerequisite : AR211
Instructor Information

Name :Dr. Aryaf Abdallah Aladwan

Office No. --
Tel (Ext) None
E-mail [email protected]
Office Hours Sun, Tue, Thu 10:00-11:00, 15:00-16:00
Class Times Building Day Start Time End Time Room No.
AI Sun -Tue, :0014 :0015 AI LAB
Thu
Salt Technical Sun -Tue, 12:00 :0013 8
College Thu
This course is to design efficient computer algorithms, prove their
correctness, and analyze their running times. it includes mathematical
analysis of algorithms (summations and recurrences), advanced data
structures (balanced search trees), algorithm design techniques (divide-
and-conquer, dynamic programming, and greedy algorithms), graph
algorithms (breadth-first and depth-first search, minimum spanning
trees, shortest paths).
Course Title: DESIGN AND ANALYSIS OF ALGORITHMS
Credit Hour(3)

[Pre-req. AR211
Textbook: Introduction to Algorithms, Thomas Cormen, Charls Leiserson & Ronald Rivest. 3rd Edition,
MIT Press, 2009.

Image of the textbook Cover

COURSE OBJECTIVES

The main learning objectives of the course are to:

1. Learn about the use of several design techniques and the importance of studying the complexity of
a particular algorithm
2. Computational complexity analysis of algorithms and worst-case algorithm runtime analysis using
asymptotic analysis and mathematical analysis of algorithms (summations and recursion)
3. Writing and analyzing recursive relationships for recursive algorithms
4. Describe the divide and conquer approach as a principle for algorithm design
5. Learn about searching and sorting algorithms
6. Ability to design, analyze and validate graph algorithms and shortest path algorithms
7. Ability to design and analyze algorithms that follow the approach of greedy algorithms
8. Understand dynamic programming algorithms and optimization algorithms and know when the
algorithm design situation calls for it.
9. Learn about advanced data structures
COURSE SYLLABUS

Week Course Topic

Week 1 Introduction to Algorithms, The Efficiency of Algorithms, Attributes of Algorithms. A Choice of
Algorithms. Classifications of algorithms: by implementation, by design, by field of study and by
complexity.
Week 2 Mathematical Background (summations and recurrences)
Week 3 Measuring Efficiency, Measuring the asymptotic growth of functions. The basic techniques for
manipulating bounded summations, searching techniques (linear search, binary search).
Week 4 Techniques for problem-solving: Divide and Conquer, the General Method, Recurrence Equations,
Solving Recurrence Equations (Master and iteration Methods).
Week 5 and 6 Sorting techniques based on Divide and Conquer: Merge Sort.
Quick Sort, Strassen's Matrix Multiplications.
Week 7 Other sorting techniques: Insertion sort, Selection Sort, Heap Sort
Week 8 Midterm Exam
Week 9 Graph Algorithms, Formal Definition of Graphs. Graphs Types.
Graph Terminologies, Strongly Connected Graph, Representations of Graphs.
Week 10 Graph Traversal, Breadth-First Search Algorithm, Depth-First Search Algorithm. Topological Sort
Algorithm.
Week 11 Greedy Algorithms, Minimum spanning trees
Week 12 Shortest paths algorithms
Week 13 Dynamic Programming
Week 14 Optimization Algorithms
Week 15 Advanced data structures (balanced search trees)
Week 16 Final Exam
Week 4
 Chapter 4
Recursion and Recurrence

By:
Dr. Aryaf Al-Adwan
Autonomous Systems Dept.
Design and Analysis of algorithms Course

8/28
Introduction toRecursion
❑ Recursive procedures are functions that invoke themselves
directly (call themselves from within themselves)

❑ Recursion:
. An alternative to iteration
. Recursion can be very elegant at times,
. Not inexpensive to implement...
Classic examples of recursion
. Recursive calculation of a string length, factorials, divide and
conquer, towers of Hanoi, binary searches, and more

❑ Recursive functions are used in many applied areas.

. In artificial intelligence.
. In searching data structures that are themselves "recursive" in nature, such as
trees.

9/28
Example: Add the numbers from 1 to 5

Base case
int sum (int n)
{
if (n <= 1)
return n;
else
return (n + sum(n-1)); Recursive Case
} // end sum()

Notice the recursion in the Else branch

Note the ‘return’ calls itself! (calls the method it resides in!)
Note the ‘Base Case’
Note the ‘Recursive Case’

10/2
8
int public sum (int n)
{
if (n <= 1)
return n;
else
return (n + sum(n-1));
}// end sum()

START WITH N = 5:

SUM(5): 5 + SUM(4) - A FUNCTION CALL (SUM(4)) WHOSE VALUE WE AWAIT

SUM(4): 4 +SUM(3) -A FUNCTION CALL WHOSE VALUE WE AWAIT

SUM(3): 3 + SUM(2) -A FUNCTION CALL WHOSE VALUE WE AWAIT

SUM(2): 2 + SUM(1) - A FUNCTION CALL WHOSE VALUE WEAWAIT

SUM(1): 1

11/2
8
WHEN WE GET SUM(1)=1,
THEN WE REFER TO 2 + SUM(1) equals 2 + 1 = 3. (complete)

NEXT, SUM (3) = 3+SUM(2), which is 3 + 3 OR 6; (now complete)

We now have a value for SUM(3)

NEXT, SUM (4) = 4+SUM(3), WHICH IS 4 + 6 OR 10; (now complete)

FINALLY, SUM (5) = 5+SUM(4), WHICH IS 5 + 10 OR 15 (now complete)

12/2
8
Consider: FACTORIAL FUNCTION

Where n! = n* (n-1) * (n-2) * ... 1! * 0!

Where 1! = 1 & 0! = 1 by definition.

Thus 5! = 5 * 4 * 3 * 2 * 1 = 120

13/2
8
Factorial

RECURSIVE FUNCTION IS WRITTEN AS:

int public factorial (int n)

{
if (n==1)
return (1) /* THIS IS THE BASE CASE */
else
return (n * factorial (n-1) ); /* RECURSIVE CASE */
} end factorial ()

14/2
8
 int public factorial (int n)
Factorial {
if (n==1)
return (1) /* base case */
else
main calls argument n = 7. return (n * factorial (n-1) );
factorial(7) /* recursive case
value of n at this node: returned value }// end factorial()
n=7
return (7*factorial(6)) return(7*720) = 5040 = answer!!

recursive call
n=6
return(6*factorial(5)) return(6*120) = 720

n=5
return(5*factorial(4)) return(5*24) = 120

n=4
return (4*factorial(3)) return(4*6) = 24

n=3
return (3*factorial(2)) return (3 *2) = 6

n=2
return (2*factorial(1)) return( 2 * factorial(1)) = 2 * 1 = 2 Thus factorial (2) = 2. return 2.

n=1 1 is substituted for the call

return (1) base case reached.

15/2
8
Recursive Definitions: FibonacciSeries

Definition of fibi, for integer i > 0:

• fib1 = 1

• fib2 = 1

• fibn = fibn-1 + fibn-2, for n >2

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

Chapter 7: Recursion 16
Fibonacci SeriesCode

public static int fib (int n) {

if (n <= 2)
return 1;
else
return fib(n-1) + fib(n-2);
}

This is straightforward, but an inefficient recursion ...

Chapter 7: Recursion 17
11/28
Data Structure in Recursion
❑ Stacks are Used

❑ In recursive returns we may do some / all of the following:

we pop the stack to get to:
Returned values
Returned addresses
Data areas
Branch to statement invoking function taken.

❑ Calling function regains control and continues for a particular

‘call’ or message invocation

12/28
 Efficiency ofRecursion

• Recursive method often slower than iterative; why?

• Overhead for loop repetition smaller than
• Overhead for call and return
• If easier to develop algorithm using recursion,
• Then code it as a recursive method:
• Software engineering benefit probably outweighs ...
• Reduction in efficiency
• Don’t “optimize” prematurely!

Chapter 7: Recursion 13
Recurrences

21/28
Definition

• A recurrence is an equation or inequality that describes a function in

terms of itself by using smaller inputs.

22/28
Solving arecurrence

There are 4 methods :

1) Iteration Method
2) Tree Method
3) Substitution Method
4) Master Theorem

23/28
IterationMethod
Example1
• Assume we have the following recursive function

void Test( int n)

{
if (n>0)
{
cout<<n;
Test (n-1);
}
}

24/28
FindingtheTotaltimebythisalgorithm

void Test( int n) ➔ T(n)

{
if (n>0) ➔1
{
cout<<n; ➔1
Test (n-1); ➔ T(n-1)
}
}

So, T(n) = T(n-1) +2

25/28
RecurrenceRelation

26/28
Solving the recurrence (usingiteration)

T(n) = T(n-1) + 1 ……….. Eq1

Find T(n-1) ?
T(n-1)=T(n-2)+1
So, Substitute T(n-1) in Eq1
T(n)= [ T(n-2) +1 ] +1 ➔ T(n-2) +2 ……… Eq2
Find T(n-2) ?
T(n-2)=T(n-3)+1
So, Substitute T(n-2) in Eq2
T(n)= [ T(n-3) +1 ] +2 ➔ T(n-3) +3
Continues for k times ➔ T (n-k) + k

27/28
Cont.

• T (n-k) + k
• This means that I will stay in the recursive state and stop after k steps
• So, when is the stop case?
when n=0
n-k = 0 , so n=k
• T(n)= T(n-k) + k , but n=k
• T(n)= T(n-n) +n
• T(n) = T(0) + n , but T(0)=1
• T(n)= 1+n
= n+1 ➔ O(n)

28/28
IterationMethod
Example2
• Assume we have the following recursive function

void Test( int n)

{
if (n>0)
{
for(int i=0; i<n; i++)
cout<<n;
Test (n-1);
}
}

29/28
Finding the Total time by this algorithm

void Test( int n) ➔ T(n)

{
if (n>0) ➔1
{
for(int i=0; i<n; i++) ➔ n+1
cout<<n; ➔n
Test (n-1); ➔ T(n-1)
}
}
So, T(n) = T(n-1) +2n+2
T(n) = T (n-1) +n
30/28
RecurrenceRelation

31/28
Solving the recurrence (byiteration)

T(n) = T(n-1) + n ……….. Eq1

Find T(n-1) ?
T(n-1)=T(n-2)+n-1
So, Substitute T(n-1) in Eq1
T(n)= [ T(n-2) +n-1 ] +n ➔ T(n-2) +(n-1)+n ……… Eq2
Find T(n-2) ?
T(n-2)=T(n-3)+n-2
So, Substitute T(n-2) in Eq2
T(n)= [ T(n-3) +n-2 ] +(n-1)+n ➔ T(n-3) +(n-2)+(n-1)+n
Continues for k times ➔ T (n-k) + (n-(k-1))+(n-(k-2))+ …..+(n-1)+n

32/28
Cont.

T (n-k) + (n-(k-1))+(n-(k-2))+ …..+(n-1)+n

• This means that I will stay in the recursive state and stop after k steps
• So, when is the stop case?
when n=0
n-k = 0 , so n=k
• T(n)= T (n-k) + (n-(k-1))+(n-(k-2))+ …..+(n-1)+n , but n=k
• T(n)= T(n-n) +(n-(n+1)+(n-n+2)+……+(n-1)+n
• T(n) = T(0) + 1+2+3+……+(n-1)+n , but T(0)=1
• T(n)= 1+n(n+1)/2
= n+1 ➔ O(n2)

33/28
Solving the recurrence byiteration
Example3

34/28
 Solving the recurrence byiteration
Example3
T(n) = 2T(n/2) + 4n ……….. Eq1
Find T(n/2) ?
T(n/2)=2T(n/22)+4(n/2)
So, Substitute T(n/2) in Eq1
T(n)=2 [2T(n/22)+4(n/2)] +4n ➔ T(n)=22 T(n/ 22 )+4n+ 4n ……… Eq2
Find T(n/22) ?
T(n/ 22)=2T(n/23)+4(n/ 22)
So, Substitute T(n/ 22) in Eq2
T(n)= 22 [2T(n/23)+4(n/ 22)]+4n+ 4n ➔ T(n)=23 T(n/ 23 )+4n+ 4n+4n
Continues for k times ➔ 2k T(n/ 2k )+k (4n)

35/28
Cont.

2k T(n/ 2k )+k (4n)

• This means that I will stay in the recursive state and stop after k steps
• So, when is the stop case?
when n=1
n/2k = 1 , so n=2k ➔ k=log2n
T(n)= 2k T(n/ 2k )+k (4n) , but k=log2n
• T(n)= 2log2n T(n/ 2log2n )+log2n (4n) ➔ but 2log n = nlog 2 =n
2 2
• T(n) = n T(1)+ log n (4n)
2
• T(n)= 4n+ 4n log n
2
➔ O(n log n)

36/28
Solving the recurrence using MasterTheorem

•In the analysis of algorithms, the master

theorem for divide-and-conquer recurrences
provides an asymptotic analysis (using Big O
notation) for recurrence relations of types that
occur in the analysis of many divide and
conquer algorithms.
•The name "master theorem" was popularized
by the widely used algorithms textbook
Introduction to Algorithms by Cormen,
Leiserson, Rivest, and Stein.

37/28
 GenericForm

a>=1
b>1
f(n)=O(nk log pn)
where:
• n is the size of an input problem.
• a is the number of subproblems in the recursion
• n/b is the size of each subproblem
• f (n) is the cost of the work done outside the recursive calls, which
includes the cost of dividing the problem and the cost of merging the
solutions to the subproblems.

We have to find :
1) log ba
2) k 38/28
 Cont.

• Based on these values we have 3 cases:

Case1 Case2 Case3

If (log b a) > k If (log ba) = k If (log ba) < k
Then, Then, Then,
T(n)= O( n logba ) If p>-1 ➔ T(n)= O( nk log p+1 n) If p≥0 ➔
If p=-1 ➔T(n)= O( nk loglogn) T(n)= O( nk logpn)
If p<-1 ➔T(n)= O( nk) If p<0 ➔
T(n)= O( nk)

39/28
Case 1Example

40/28
Another example onCase1

41/28
Case 2Example

42/28
Another example onCase 2

43/28
Case 3Example

44/28
 End

45/28