DESIGN AND ANALYSIS OF

ALGORITHMS
Course Code : AR212
Credit Hours :3
Prerequisite : AR211
Instructor Information

Name :Dr. Aryaf Abdallah Aladwan

Office No. --
Tel (Ext) None
E-mail [email protected]
Office Hours Sun, Tue, Thu 10:00-11:00, 15:00-16:00
Class Times Building Day Start Time End Time Room No.
AI Sun -Tue, :0014 :0015 AI LAB
Thu
Salt Technical Sun -Tue, 12:00 :0013 8
College Thu
This course is to design efficient computer algorithms, prove their
correctness, and analyze their running times. it includes mathematical
analysis of algorithms (summations and recurrences), advanced data
structures (balanced search trees), algorithm design techniques (divide-
and-conquer, dynamic programming, and greedy algorithms), graph
algorithms (breadth-first and depth-first search, minimum spanning
trees, shortest paths).
Course Title: DESIGN AND ANALYSIS OF ALGORITHMS
Credit Hour(3)

[Pre-req. AR211
Textbook: Introduction to Algorithms, Thomas Cormen, Charls Leiserson & Ronald Rivest. 3rd Edition,
MIT Press, 2009.

Image of the textbook Cover

COURSE OBJECTIVES

The main learning objectives of the course are to:

1. Learn about the use of several design techniques and the importance of studying the complexity of
a particular algorithm
2. Computational complexity analysis of algorithms and worst-case algorithm runtime analysis using
asymptotic analysis and mathematical analysis of algorithms (summations and recursion)
3. Writing and analyzing recursive relationships for recursive algorithms
4. Describe the divide and conquer approach as a principle for algorithm design
5. Learn about searching and sorting algorithms
6. Ability to design, analyze and validate graph algorithms and shortest path algorithms
7. Ability to design and analyze algorithms that follow the approach of greedy algorithms
8. Understand dynamic programming algorithms and optimization algorithms and know when the
algorithm design situation calls for it.
9. Learn about advanced data structures
COURSE SYLLABUS

Week Course Topic

Week 1 Introduction to Algorithms, The Efficiency of Algorithms, Attributes of Algorithms. A Choice of
Algorithms. Classifications of algorithms: by implementation, by design, by field of study and by
complexity.
Week 2 Mathematical Background (summations and recurrences)
Week 3 Measuring Efficiency, Measuring the asymptotic growth of functions. The basic techniques for
manipulating bounded summations, searching techniques (linear search, binary search).
Week 4 Techniques for problem-solving: Divide and Conquer, the General Method, Recurrence Equations,
Solving Recurrence Equations (Master and iteration Methods).
Week 5 and 6 Sorting techniques based on Divide and Conquer: Merge Sort.
Quick Sort, Strassen's Matrix Multiplications.
Week 7 Other sorting techniques: Insertion sort, Selection Sort, Heap Sort
Week 8 Midterm Exam
Week 9 Graph Algorithms, Formal Definition of Graphs. Graphs Types.
Graph Terminologies, Strongly Connected Graph, Representations of Graphs.
Week 10 Graph Traversal, Breadth-First Search Algorithm, Depth-First Search Algorithm. Topological Sort
Algorithm.
Week 11 Greedy Algorithms, Minimum spanning trees
Week 12 Shortest paths algorithms
Week 13 Dynamic Programming
Week 14 Optimization Algorithms
Week 15 Advanced data structures (balanced search trees)
Week 16 Final Exam
Week 3
Design and Analysis of Algorithms Course

Chapter 3
Growth of Functions
Dr. Aryaf Al-Adwan
Autonomous Systems Dept.
Design and Analysis of algorithms Course
Dr. Aryaf Al-Adwan

8
Outline

1. How fast will your program run?

2. Complexity
3. Counting operations
4. complexity analysis
5. Asymptotic Notations
6. Practical Complexities

Dr. Aryaf Al-Adwan 9

How fast will your program run?

• The running time of your program will depend upon:

1. The algorithm
2. Your implementation of the algorithm in a programming language
3. The compiler you use
4. The OS on your computer
5. Your computer hardware
6. Maybe other things: temperature outside; other programs on your computer;
• Our Motivation: analyze the running time of an algorithm as a function of only simple parameters
of the a function of the input.

Dr. Aryaf Al-Adwan 10

Complexity

• Complexity is the number of steps required to solve a problem

• The goal is to find the best algorithm to solve the problem with a less number of
steps
• Complexity of Algorithms:
1. The size of the problem is a measure of the quantity of the input data n
2. The time needed by an algorithm, expressed as a function of the size of the
problem (it solves), is called the (time) complexity of the algorithm T(n)

Dr. Aryaf Al-Adwan 11

Basic idea: counting operations

• Running Time: Number of primitive steps that are executed.

• most statements roughly require the same amount of time
1. y = m * x + b
2. c = 5 / 9 * (t - 32 )
3. z = f(x) + g(y)
• Each algorithm performs a sequence of basic operations:
1. Arithmetic: (low + high)/2
2. Comparison: if ( x > 0 ) …
3. Assignment: temp = x
4. Branching: while ( true ) { … }
Dr. Aryaf Al-Adwan 12
Example1: Sequential Search

Dr. Aryaf Al-Adwan 13

Example2

Dr. Aryaf Al-Adwan 14

Example 3

Time complexity analysis of an

algorithm is the determination of
how many times the basic operation
is done for each value of the input
size.

Dr. Aryaf Al-Adwan 15

 Growth Rates

Algorithm running times

grow at different rates

Dr. Aryaf Al-Adwan 16

Measures of Algorithm Complexity

• Let T(n) denote the number of operations required by an algorithm to solve a

given class of problems.
• Often T(n) depends on the input, in such cases one can talk about
1. Worst-case complexity,
2. Best-case complexity,
3. Average-case complexity of an algorithm
• Alternatively, one can determine bounds (upper or lower) on T(n)

Dr. Aryaf Al-Adwan 17

Cont.

• Worst-Case Running Time: the longest time for any input size of n which
provides an upper bound on running time for any input.
• Best-Case Running Time: the shortest time for any input size of n which
provides lower bound on running time for any input
• Average-Case Behavior: the expected performance averaged over all
possible inputs
1. it is generally better than worst case behavior, but sometimes it’s
roughly as bad as worst case
2. difficult to compute

Dr. Aryaf Al-Adwan 18

 Sequential Search complexity analysis

• Worst-case running time when x is not in the original array A

• in this case, while loop needs 2n + 4 comparisons + c other
operations
• So, T(n) = 2n + 4 + c ➔ Linear complexity
• Best-case running time when x is found in A[1]
• in this case, while loop needs 2 comparisons + c other
operations
• So, T(n) = 2 + c ➔ Constant complexity

Dr. Aryaf Al-Adwan 19

Cont.

• average-case running time

• assume x is equally likely to equal A[1], A[2], …, A[n]

Dr. Aryaf Al-Adwan 20

Asymptotic Notation

• Asymptotic Notation is used to describe the running time of an algorithm - how

much time an algorithm takes with a given input, n.
• There are three different notations:
• big (O)
• big Theta (Θ)
• and big Omega (Ω)

Dr. Aryaf Al-Adwan 21

Big (O) (Upper Bound)
Definition:
Ω(g(n)) = { f (n) : there exist positive constants c and n 0 such that
f (n) ≤ c*g(n) for all n ≥ n0 }

The choice of n₀ and c is not

unique. There can be many
(actually, infinitely many)
different combinations of n₀
and c that would make the
proof work.

Dr. Aryaf Al-Adwan 22

Big (O)

Dr. Aryaf Al-Adwan 23

 2n+7 < 9n for n0 >= 1
2n+7 < n2 for n0 >= 6

➔ n here means n0

1 < log n < √n < n < n log n < n2 < n3 < 2n < 3n < nn
Dr. Aryaf Al-Adwan 24
Dr. Aryaf Al-Adwan 25
Dr. Aryaf Al-Adwan 26
Big (O)

Dr. Aryaf Al-Adwan 27

Ω-notation (Omega)
(Lower Bound)

Ω-notation provides an asymptotic lower bound. For a given function g(n), we

denote by Ω(g(n)) (pronounced “big-omega of g of n” or sometimes just “omega
of g of n”).

Definition:
Ω(g(n)) = { f (n) : there exist positive constants c and n 0 such that
f (n) ≥ cg(n) for all n ≥ n0 }

Dr. Aryaf Al-Adwan 28

Ω-notation

Dr. Aryaf Al-Adwan 29

Example 3

Prove that n2+10n ≥ c1*g(n)?

Solution:
c1= 1 n0=0

n2+10n ≥ n2

n=0 0 ≥ 0
n=1 11 ≥ 1
n=2 24 ≥ 4

Dr. Aryaf Al-Adwan 30

Ω-notation

Dr. Aryaf Al-Adwan 31

Θ notation (Theta)
(Tight Bound)

Dr. Aryaf Al-Adwan 32

Θ-notation

Dr. Aryaf Al-Adwan 33

Practical Complexities

Dr. Aryaf Al-Adwan 34

Impractical Complexities

Dr. Aryaf Al-Adwan 35

Some Common Name for Complexity

Dr. Aryaf Al-Adwan 36

Exponential Functions

Dr. Aryaf Al-Adwan 37

Dr. Aryaf Al-Adwan 38