Design and Analysis of Algorithms

Course Introduction

Dr. Aimal Tariq Rextin

Department of Computing

© Algorithms
 Textbook

Introduction to Algorithms
Third Edition

Thomas H. Cormen
Charles E. Leiserson
Ronald L. Rivest
Clifford Stein

The MIT Press

© Algorithms
 Evaluation
5% Assignments
▶ 3–5 assignments
▶ grades added together, thus resulting in a weighted average
▶ some or all can be inside the classroom
▶ In order to deter copying in assignments, I sometimes take quiz from
the assignment on the day of submission.
15% Quizzes
▶ Taken to ensure students study regularly
▶ Can be announced or unannounced.

25% midterm exam

45% final exam

10% project

© Algorithms
 Academic Honesty
All students are to adhere to the highest standards of academic
honesty.
We must seriously ask ourselves why we must be ethical in our
academic conduct and should not cheat, plagiarize, or use other
unfair means.

© Algorithms
 Academic Honesty
All students are to adhere to the highest standards of academic
honesty.
We must seriously ask ourselves why we must be ethical in our
academic conduct and should not cheat, plagiarize, or use other
unfair means.

© Algorithms
 Academic Honesty
All students are to adhere to the highest standards of academic
honesty.
We must seriously ask ourselves why we must be ethical in our
academic conduct and should not cheat, plagiarize, or use other
unfair means.
Four Moral Positions( Georg Christoph Lichtenberg)
1. The Philosophical: Do good for the sake of good, out of respect for the
law.

© Algorithms
 Academic Honesty
All students are to adhere to the highest standards of academic
honesty.
We must seriously ask ourselves why we must be ethical in our
academic conduct and should not cheat, plagiarize, or use other
unfair means.
Four Moral Positions( Georg Christoph Lichtenberg)
1. The Philosophical: Do good for the sake of good, out of respect for the
law.
2. The Religious: Do good because it is the will of God, out of love of God.

© Algorithms
 Academic Honesty
All students are to adhere to the highest standards of academic
honesty.
We must seriously ask ourselves why we must be ethical in our
academic conduct and should not cheat, plagiarize, or use other
unfair means.
Four Moral Positions( Georg Christoph Lichtenberg)
1. The Philosophical: Do good for the sake of good, out of respect for the
law.
2. The Religious: Do good because it is the will of God, out of love of God.
3. The Humanistic: Do good because your well-being demands it, out of
respect for yourself.

© Algorithms
 Academic Honesty
All students are to adhere to the highest standards of academic
honesty.
We must seriously ask ourselves why we must be ethical in our
academic conduct and should not cheat, plagiarize, or use other
unfair means.
Four Moral Positions( Georg Christoph Lichtenberg)
1. The Philosophical: Do good for the sake of good, out of respect for the
law.
2. The Religious: Do good because it is the will of God, out of love of God.
3. The Humanistic: Do good because your well-being demands it, out of
respect for yourself.
4. The Political: Do good because it serves the society of which you are a
part

© Algorithms
 Why Be Honest?
Take a moment to think about this: Why would you choose to be
honest in your academic work?
Reflect on how honesty shapes your growth and integrity.

Consider making a personal promise:

I promise to myself that I will not indulge in using unfair means
in my academic work.because .............
Optional: Fill out the blank above and keep it as a promise with yourself

© Algorithms
Deadlines

© Algorithms
 Deadlines
Deadlines are firm.

© Algorithms
 Deadlines
Deadlines are firm.

Exceptions may be granted

▶ at the instructor’s discretion
▶ only for documented medical conditions or other documented
emergencies

© Algorithms
 Deadlines
Deadlines are firm.

Exceptions may be granted

▶ at the instructor’s discretion
▶ only for documented medical conditions or other documented
emergencies

Each late day will reduce the assignment’s grade by one third of the
total value of that assignment

© Algorithms
Now let’s move on to the real
interesting and fun stuff. . .

© Algorithms
 Automating Arithmetic
Human brain find many things easy that are difficult to implemented on
a computer. While tasks that are difficult for humans are easy on
computers ( arithmetic)
Humans have always been trying to make arithmetic easier and faster

© Algorithms
 Automating Arithmetic
Human brain find many things easy that are difficult to implemented on
a computer. While tasks that are difficult for humans are easy on
computers ( arithmetic)
Humans have always been trying to make arithmetic easier and faster

© Algorithms
Automating Arithmetic

© Algorithms
An Islamic Example

© Algorithms
 Babbage Difference Engine
Charles Babbage made a Mechanical Computer in 1822. It calculated
polynomial equations.

© Algorithms
 Babbage Difference Engine
Charles Babbage made a Mechanical Computer in 1822. It calculated
polynomial equations.

© Algorithms
Rigid versus Flexible Computing

© Algorithms
 Rigid versus Flexible Computing
Rigid Computing Machines are computing machines that can
perform a limited number computations

© Algorithms
 Rigid versus Flexible Computing
Rigid Computing Machines are computing machines that can
perform a limited number computations
Flexible Computing Machines are able to perform unlimited
number of computations. Today’s computer are flexible one can
load and execute various types of programs.

© Algorithms
 Rigid versus Flexible Computing
Rigid Computing Machines are computing machines that can
perform a limited number computations
Flexible Computing Machines are able to perform unlimited
number of computations. Today’s computer are flexible one can
load and execute various types of programs.

Alan Turing in early 1920s proposed first time mathematically how a

flexible computing machine will work. These are called Turing Machines
© Algorithms
 The First Computers Mid 1940s

ENIAC was used to calculate artillery projectiles and feasibility of

thermo-nuclear bombs.
It was reprogrammed by setting the various wires in a new way. This
step took a few days once the ”program” was figured on a paper.
© Algorithms
Digital Computers

© Algorithms
Modern Computers

© Algorithms
The Name and Concept of Algorithms

© Algorithms
The Name and Concept of Algorithms
The decimal numbering system (Indus
area, circa 600 AD)

© Algorithms
The Name and Concept of Algorithms
The decimal numbering system (Indus
area, circa 600 AD)

Persian mathematician Khwārizmı̄ writes

a book (Baghdad, circa 830 AD)
Book name was Aljabar Wal Muqabala:
Aljabar → Algebra

Muhammad ibn Musa

al-Khwārizmı̄

© Algorithms
The Name and Concept of Algorithms
The decimal numbering system (Indus
area, circa 600 AD)

Persian mathematician Khwārizmı̄ writes

a book (Baghdad, circa 830 AD)
Book name was Aljabar Wal Muqabala:
Aljabar → Algebra

▶ methods for adding, multiplying, and

dividing numbers (and more)

Muhammad ibn Musa

al-Khwārizmı̄

© Algorithms
The Name and Concept of Algorithms
The decimal numbering system (Indus
area, circa 600 AD)

Persian mathematician Khwārizmı̄ writes

a book (Baghdad, circa 830 AD)
Book name was Aljabar Wal Muqabala:
Aljabar → Algebra

▶ methods for adding, multiplying, and

dividing numbers (and more)

▶ these procedures were precise,

unambiguous, mechanical, efficient, and
correct
Muhammad ibn Musa
al-Khwārizmı̄

© Algorithms
The Name and Concept of Algorithms
The decimal numbering system (Indus
area, circa 600 AD)

Persian mathematician Khwārizmı̄ writes

a book (Baghdad, circa 830 AD)
Book name was Aljabar Wal Muqabala:
Aljabar → Algebra

▶ methods for adding, multiplying, and

dividing numbers (and more)

▶ these procedures were precise,

unambiguous, mechanical, efficient, and
correct
Muhammad ibn Musa
▶ they were algorithms! al-Khwārizmı̄

© Algorithms
 What Are Algorithms?
Formal Definition:
“A finite sequence of unambiguous instructions solving a well-
defined computational problem”
(Adapted from Knuth)

© Algorithms
 What Are Algorithms?
Formal Definition:
“A finite sequence of unambiguous instructions solving a well-
defined computational problem”
(Adapted from Knuth)
Core Requirements:
Input/Output: Defines computational problem
Correctness: Produces right answer for all instances
Efficiency: Time/space complexity matters

© Algorithms
 Algorithms versus Implementations
Algorithm: Mathematical entity (Pseudocode)
Program: Physical instantiation (Code + Environment)

Figure: At a kid level, this is an algorithm

© Algorithms
Why Algorithms Matter

© Algorithms
 Why Algorithms Matter
Critical Differentiators:
Exponential Gap: O(n log n) vs O(n2 ) sorts
Universality: Same algorithm works across languages
Transferability: Techniques span domains

© Algorithms
 Why Algorithms Matter
Critical Differentiators:
Exponential Gap: O(n log n) vs O(n2 ) sorts
Universality: Same algorithm works across languages
Transferability: Techniques span domains
Foundational Examples:
Binary Search (Divide-&-Conquer)
Dijkstra’s Algorithm (Greedy methods)
Dynamic Programming

This Course: How to design and analyze them

© Algorithms
 Real World Impact
PageRank (Google’s foundation)
FFT (Digital signal processing)
RSA (Modern cryptography)
Shortest path (think Google Maps)

© Algorithms
 Real World Impact
PageRank (Google’s foundation)
FFT (Digital signal processing)
RSA (Modern cryptography)
Shortest path (think Google Maps)

© Algorithms
Algorithms are the
logic
of computer programs

© Algorithms
 Classical vs AI Algorithms
Cormen-Type Algorithms AI Algorithms
Goal: Solve exact problems Goal: Learn from data patterns
Example: Sorting, shortest path Example: Image recognition
Determinism: Same input → Randomness: Often has randomness in
same result output
Correctness: Math proofs Testing: Check real-world performance
Focus: Speed/memory guaran- Focus: Works on new data, needs less
tees data

Both essential: AI uses classical algorithms as building blocks

© Algorithms
 Implementations of Algorithms
Two programs are different implementations of the same algorithms if
anyone of the following changes:

© Algorithms
 Implementations of Algorithms
Two programs are different implementations of the same algorithms if
anyone of the following changes:
The programmer
the hardware running the program
a different programming language Java versus C++

© Algorithms
 Implementations of Algorithms
Two programs are different implementations of the same algorithms if
anyone of the following changes:
The programmer
the hardware running the program
a different programming language Java versus C++
Question: In what aspect way would two implementation of the same
algorithm differ

© Algorithms
 Implementations of Algorithms
Two programs are different implementations of the same algorithms if
anyone of the following changes:
The programmer
the hardware running the program
a different programming language Java versus C++
Question: In what aspect way would two implementation of the same
algorithm differ

Time it takes to execute the program

© Algorithms
 One Problem Many Algorithms
There can be many problems for a single problem

© Algorithms
 One Problem Many Algorithms
There can be many problems for a single problem
Example
Sorting an array of numbers is one of the most common function. There
are various algorithms for it for example insertion sort is one and merge
sort is another

© Algorithms
 One Problem Many Algorithms
There can be many problems for a single problem
Example
Sorting an array of numbers is one of the most common function. There
are various algorithms for it for example insertion sort is one and merge
sort is another
Should we have any preference regarding implementing one another
algorithm ?
Yes Some are better than others

© Algorithms
Better Algorithms?

© Algorithms
 Better Algorithms?
The algorithm that finish faster are better

© Algorithms
 Better Algorithms?
The algorithm that finish faster are better
For small input (e.g. array size) it does not matter. But when the
input size is very very large size this time difference can be
significant.
▶ Example: think about searching the Call Record Database of Ufone

© Algorithms
 Time Complexity
Time an algorithm takes can be estimated by a mathematical expression
of the following form:
=c ×E
T (n)
I

© Algorithms
 Time Complexity
Time an algorithm takes can be estimated by a mathematical expression
of the following form:
=c ×E
T (n)
I
c is a constant like 2, 10, 100 etc.

© Algorithms
 Time Complexity
Time an algorithm takes can be estimated by a mathematical expression
of the following form:
=c ×E
T (n)
I
c is a constant like 2, 10, 100 etc.
it estimates the various implementation specific factors

© Algorithms
 Time Complexity
Time an algorithm takes can be estimated by a mathematical expression
of the following form:
=c ×E
T (n)
I
c is a constant like 2, 10, 100 etc.
it estimates the various implementation specific factors
E is a polynomial expression in terms of the input size, e.g.
▶ n
▶ n2
▶ nlogn
▶ etc
▶ it gives an estimate on the number instructions that need to be
executed
I is Instructions per sec

© Algorithms
 Which is important?
Question
So we now have the following factors that can influence the time taken
by some particular.
The programmer
Hardware
Programming Language
The Algorithm

© Algorithms
 Which is important?
Question
So we now have the following factors that can influence the time taken
by some particular.
The programmer
Hardware
Programming Language
The Algorithm

Question: Which of these factors is most important in performance of

the software?

© Algorithms
Algorithms are most important!

© Algorithms
Lets assume we implement two algorithms insertion sort and merge sort
We need to sort an array of 10 million numbers. i.e

n = 107

© Algorithms
Lets assume we implement two algorithms insertion sort and merge sort
We need to sort an array of 10 million numbers. i.e

n = 107

Insertion Sort Merge Sort

Num of Instructions(E) : n2 n × log n
Constant c=2 Higher with c = 50
CPU Speed(Instructions/sec) 10
10 (faster) 107

Note: Insertion sort is implemented on a computer that is 1000 times

faster than the other computer

© Algorithms
Time Taken

© Algorithms
 Time Taken
Insertion Sort
2
2 × 107
T = 10 = 2 × 104 secs = 20000secs
10 Intruc/sec

or about 5.5 hours

© Algorithms
 Time Taken
Insertion Sort
2
2 × 107
T = 10 = 2 × 104 secs = 20000secs
10 Intruc/sec

or about 5.5 hours

Merge Sort

50 × 107 log2 107

T = = 50 × 23.25 = 1163secs
107 Instruc/sec

or about 0.5 hours

© Algorithms
 Time Taken
Insertion Sort
2
2 × 107
T = 10 = 2 × 104 secs = 20000secs
10 Intruc/sec

or about 5.5 hours

Merge Sort

50 × 107 log2 107

T = = 50 × 23.25 = 1163secs
107 Instruc/sec

or about 0.5 hours

Conclusion
Algorithm is more important at large input sizes

© Algorithms
 Time Taken
Insertion Sort
2
2 × 107
T = 10 = 2 × 104 secs = 20000secs
10 Intruc/sec

or about 5.5 hours

Merge Sort

50 × 107 log2 107

T = = 50 × 23.25 = 1163secs
107 Instruc/sec

or about 0.5 hours

Conclusion
Algorithm is more important at large input sizes

This example is covered in Section 1.2 of the textbook

© Algorithms
Conclusion

© Algorithms