Algorithm Design and Analysis

Faculty of Information Technology - Computer Science Department 1

Algorithm Design and Analysis

Faculty of Information Technology - Computer Science Department 2

 Chapter 1

Introduction to Algorithm

Faculty of Information Technology - Computer Science Department 3

Objectives

• The objectives of this chapter are:

 Understand what do we mean by algorithms.
 Understand the importance of Algorithms in problem-solving process.
 Understand the correctness and efficiency of algorithms.
 Study some simple computational algorithms

Faculty of Information Technology - Computer Science Department 4

Outline

• Introduction to Data Structures and Algorithms

• Why Study Algorithms?

• The Problem-solving Process

• Applications

• Computational problems

• Writing Algorithms

• Algorithms correctness and efficiency

• Analyzing Algorithms

Faculty of Information Technology - Computer Science Department 5

 Introduction to Data Structures and Algorithms

• An algorithm is any well-defined computational procedure that takes some

value, or set of values, as input and produces some value, or set of values,
as output.
• An algorithm is a tool for solving a well-specified computational problem.

Faculty of Information Technology - Computer Science Department 6

 Why Study Algorithms?

• Necessary in any computer programming problem

 Improve algorithm efficiency: run faster, process more data, do something
that would otherwise be impossible
 Solve problems of significantly large size
 Technology only improves things by a constant factor
• Compare algorithms
• Algorithms as a field of study
 Learn about a standard set of algorithms
 Discoveries arise
 Numerous application areas
• Learn techniques of algorithm design and analysis

Faculty of Information Technology - Computer Science Department 7

 Why Study Algorithms?
Algorithm

Design Analysis
Methods/ ideas/ tricks Abstract/ mathematical computation
for developing (fast !) of algorithms
algorithms. (without actually implementing them)

• Mathematical are needed:

• To formally specify the problem
• Analysis of correctness
• Analysis of efficiency (time, memory use,…)

Faculty of Information Technology - Computer Science Department 8

The Problem-solving Process

Faculty of Information Technology - Computer Science Department 9

 Applications
• Multimedia
 CD player, DVD, MP3, JPG, DivX, HDTV
• Internet
 Packet routing, data retrieval (Google)
• Communication
 Cell-phones, e-commerce
• Computers
 Circuit layout, file systems
• Science
 Human genome
• Transportation
 Airline crew scheduling, ARAMEX and DHL deliveries

Faculty of Information Technology - Computer Science Department 10

 Computational problems

• The statement of the problem specifies in general terms the desired input/output
relationship.
• The algorithm describes a specific computational procedure for achieving that
input/output relationship.
• An instance of a problem consists of the input (satisfying whatever constraints are
imposed in the problem statement) needed to compute a solution to the problem.
• There might be different ways (algorithms) to solve a given problem!
• Usually, we use “pseudo-code” to describe algorithms
• Algorithms can be implemented in any programming language

Faculty of Information Technology - Computer Science Department 11

 Roadmap

• Different problems • Different design paradigms

• Searching – Incremental
• Sorting – Divide-and-conquer
• Graph problems – Greedy algorithms

Faculty of Information Technology - Computer Science Department 12

 Writing Algorithms

• Any algorithm should be written as:

 Name: give an appropriate name

 Input: value or set of values
 Output: value or set of values
 Algorithm:
• Step (usually steps) that can take you from the input to the output

• …

• ..

Faculty of Information Technology - Computer Science Department 13

 Algorithm Example

• Example 1: Determining if a number is prime or not.

• Name: Prime
• Input: n (an integer number)
• Output: Prime or not prime
• Algorithm:
• Think of such an algorithm here.

• 2, 5, 11, and 17 are examples of prime numbers

• 4, 9, 15, and 22 are examples of non-prime numbers
Faculty of Information Technology - Computer Science Department 14
 Algorithm Example

• Example 2:
• Name: ListSort
• Input: L (list of names of people)
• Output: L (same list sorted alphabetically). .
• Algorithm:
• Steps …

• We will see several algorithms in later chapters

Faculty of Information Technology - Computer Science Department 15

 Algorithm Example

• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
initially sum is zero
Loop from 1 to n
Add current loop counter to the sum
output sum

Faculty of Information Technology - Computer Science Department 16

 Algorithm Example

• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
sum = 0
for count =1 to n
{
sum = sum + count
}
output sum

Faculty of Information Technology - Computer Science Department 17

 Algorithm Example

• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
count = 1
sum = 0
while (count <= n)
{
sum = sum + count
count = count + 1
}
output sum
Faculty of Information Technology - Computer Science Department 18
 Algorithm Example

• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
count = 1
sum = 0
while (count <= n)
{
sum = sum + count
count = count + 1
}
output sum
Faculty of Information Technology - Computer Science Department 19
 Algorithms correctness and efficiency

• An algorithm is said to be correct if, for every input instance, it halts with the proper
output
• An incorrect algorithm might not halt at all on some input instances, or it might halt
with an answer other than the desired one.
• Algorithms must be:
• Correct: For each input produce an appropriate output
• Efficient: run as quickly as possible, and use as little memory as possible – more
about this later

Faculty of Information Technology - Computer Science Department 20

 Analyzing Algorithms
• Predict the amount of resources required:
• memory: how much space is needed?
• computational time: how fast the algorithm runs?
• FACT: running time grows with the size of the input
• Input size (number of elements in the input)
• Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary
representation of the input, vertices and edges in a graph
• Def: Running time = the number of primitive operations (steps) executed before termination
• Arithmetic operations (+, -, *), data movement, control, decision making (if, while),
comparison

Faculty of Information Technology - Computer Science Department 21

 Algorithm Example
• Example 4: We need to solve a computational problem
“Convert a weight in pounds to Kg”

• Name: ConvertPoundToKg
• Input: weight-in-pounds (weight in pound)
• Output: weight-in-Kg (weight in Kg )
• Algorithm:
 1. Read weight-in-pounds
 2. Calculate weight-in-Kg = weight-in-pounds * 0.454
 3. Print weight-in-Kg
Faculty of Information Technology - Computer Science Department 22
 Algorithm Example (cont.)
OR

• Name: ConvertPoundToKg
• Input: x (weight in pound)
• Output: y (weight in Kg)
• Algorithm:
 1. Read x
 2. Calculate y = x * 0.454
 3. Print y

Faculty of Information Technology - Computer Science Department 23

 Incorrect Algorithm example

• Name: ConvertPoundToKg
• Input: weight-in-pounds (weight in pound)
• Output: weight-in-Kg (weight in Kg )
• Algorithm:
 1. Read weight-in-pounds
 2. Calculate weight-in-Kg = weight-in-pounds / 0.454
 3. Print weight-in-Kg

Faculty of Information Technology - Computer Science Department 24

 Inefficient Algorithm example
• Name: ConvertPoundToKg
• Input: weight-in-pounds (weight in pound)
• Output: weight-in-Kg (weight in Kg )
• Algorithm:
 1. Read weight-in-pounds
 2. weight-in-Kg = 0
 3. Loop until weight-in-pounds =0
 4. if weight-in-pounds > 1
 5. weight-in-Kg = weight-in-Kg + 0.454
 6. weight-in-pounds = weight-in-pounds -1
 7. else
 8. weight-in-Kg = weight-in-Kg + weight-in-pounds *0.454
 9. weight-in-pounds =0
 10. Print weight-in-Kg

Faculty of Information Technology - Computer Science Department 25

 Another Simple Algorithm
Example 5:
• Name: FindMinInArray
• INPUT: T (a sequence of n numbers T[1], T[2], …, T[n])
• OUTPUT: min (the smallest number among them)
• Algorithm:

Faculty of Information Technology - Computer Science Department 26

 Algorithm Example

• Example 1 again
 Name: Prime
 Input: n (an integer number)
 Output: Prime or not prime
 Algorithm:

Faculty of Information Technology - Computer Science Department 27

 Greatest Common Divisor
• Definition: The GCD of two natural numbers x, and y is the largest integer j that divides both (without
remainder).
i.e. mod(x, j)=0, mod(y, j)=0, and j is the largest integer with this property.
• mod(a, b) gives the remainder of the division of a over b

• Examples:
 GCD(8,12) = 4
 GCD(12,8) = 4
 GCD(10, 25) = 5
 GCD(23, 19) = 1

Faculty of Information Technology - Computer Science Department 28

 Greatest Common Divisor
Example 6: GCD Simple algorithm:
• Name: GreatestCommonDivisor1 (or GCD1)
• Input: x, y (two natural numbers)
• Output: x (Greatest Common Divisor of x and y)
• Algorithm:

Faculty of Information Technology - Computer Science Department 29

 Euclid’s GCD Algorithm
Example 7: GCD Euclid’s algorithm
 Name: GreatestCommonDivisor2 (or GCD2)
 Input: x, y (two natural numbers)
 Output: x (Greatest Common Divisor of x and y)
 Algorithm:
while (y != 0)
{
t = mod(x, y)
x=y
y=t
}
Output x

Faculty of Information Technology - Computer Science Department 30

 Euclid’s GCD Algorithm

OR
 Name: GreatestCommonDivisor2 (or GCD2)
 Input: x, y (two natural numbers)
 Output: x (Greatest Common Divisor of x and y)
 Algorithm:

GCD(x, y)
{
while (y != 0)
{
t = mod(x, y)
x=y
y=t
}
Output x
Faculty of Information Technology - Computer Science Department 31
}
 Euclid’s GCD Algorithm

OR
 Name: GreatestCommonDivisor2 (or GCD2)
 Input: x, y (two natural numbers)
 Output: x (Greatest Common Divisor of x and y)
 Algorithm:
GCD(x, y)
{
if y = 0 return x
else return GCD( y, x mod y)
}

Faculty of Information Technology - Computer Science Department 32