CNG315 Algorithms

Introduction to Algorithms

Instructor: Dr. Şükrü Eraslan

Lecture 1 1
 Algorithms

• An algorithm is any well-defined computational

procedure that takes some value, or a set of
values, as input and produces some value, or a
set of values, as output.
• An algorithm is thus a sequence of
computational steps that transform the input
into the output.

Lecture 1 2
 Algorithm as a tool for solving a well-
specified computational problem
• Sorting Problem

32 41 59 26 43 58 5

5 26 32 41 43 58 59

Lecture 1 3
 Algorithms

Purpose of studying algorithms:

• Choose the best algorithm (or appropriate
algorithms)
• Learn techniques to develop new algorithms

Which algorithm is the best depends on:

• The size of the problem
• The type of the input – the extend to which the items
are already somewhat sorted
• The kind of storage device to be used: main memory
disks, or tapes
Lecture 1 4
 Designing algorithms

1. Put the pieces of the puzzles together

2. Choose data structures
3. Select the basic approaches to the solution of
the problem

Lecture 1 5
 Designing algorithms

• It must be correct
• It must terminate
• It must be efficient
– Design algorithms which minimise the cost!
– Implementations of an algorithm must run as fast
as possible
– How is this measured?
• Running time

Lecture 1 6
 Running time of a program

• Amount of time required for a program to

execute for a given input
• If measured experimentally,
– Dependent on hardware, operating system and
software
– Answer will be in milliseconds, seconds, minutes,
hours, …

Lecture 1 7
 Running time of an algorithm

• Answer will not be in seconds or minutes

– Seconds and minutes cannot be generalised!
• Instead, we count the number of operations
carried out
• Result will be a formula in terms of some input
size, n
• Takes into account all possible inputs
• Example statement on running time:
– Running time of algorithm A is T(n) = 2n + 3

Lecture 1 8
 Describing algorithms

• Note: First, we need to agree on how to

describe or specify an algorithm
• Algorithms are intended for humans (programs
are intended for computers)
• Descriptions should be high-level explanations
that combine natural language and familiar
programming structures: Pseudo-code

Lecture 1 9
 Pseudo-code example

Algorithm arrayMax(A,n):
Input: An array A storing n integers.
Output: The maximum element in A.
currentMax  A[0]
for i  1 to n - 1 do
if currentMax < A[i] then
currentMax  A[i]
return currentMax

Lecture 1 10
 Pseudo-code conventions

• General Algorithm Structure

• Statements
• Expressions
• Control Structures

Lecture 1 11
 Algorithm Structure

Algorithm name(param1, param2,...):

Input : input elements
Output : output elements

statements…

Lecture 1 12
 Statements

• Assignment: use  instead of =

• Method or function call:
– object.method(arguments)
– function(arguments)
• Return statement:
– return expression

Lecture 1 13
 Control Structures

• Decision structures
if ... then ... [else ...]
• While loops
while ... do …
• Repeat loops
repeat ... until ...
• For loop
for ... do …

Lecture 1 14
 Expressions

• Standard math symbols

+ - * / ( )
• Relational operators
= > < <= >= !=
• Boolean operators
and or not
• Assignment operator

• Array indexing
A[i]

Lecture 1 15
 General rules on pseudo-code

• Should communicate high-level ideas

• No implementation details
• Should be clear and informative

Lecture 1 16
 Back to running time

• Once an algorithm has been described in

pseudo-code, we can now count the primitive
operations carried out by the algorithm
• Primitive operations: assignment, calls,
arithmetic operations, comparisons, array
accesses, return statements, etc.

Lecture 1 17
 The problem of sorting

Input: sequence <a1, a2, …, an> of numbers

Output: permutation <a'1, a'2, …, a'n> such that
a'1≤ a'2 ≤…≤ a'n

• Example Case:
– Input: 8 2 4 9 3 6
– Output: 2 3 4 6 8 9

Lecture 1 18
 Insertion Sort

1 i j n
A:
key
sorted
Lecture 1 19
 Example of Insertion Sort

8 2 4 9 3 6

Lecture 1 20
 Example of Insertion Sort

8 2 4 9 3 6

Lecture 1 21
Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6

Lecture 1 22
Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6

Lecture 1 23
Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

Lecture 1 24
 Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

Lecture 1 25
Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

Lecture 1 26
Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

Lecture 1 27
Example of Insertion Sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

Lecture 1 28
Example of Insertion Sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
Lecture 1 29
 Running time of Insertion Sort

• The running time depends on the input

– An already sorted sequence is easier to sort
• Major Simplifying Convention
– Parameterize the running time by the size of the
input
– TA(n) = time of A on length n inputs

Lecture 1 30
 Kinds of running time analyses

• Best-case: (NEVER)
– Cheat with a slow algorithm that works fast on
some input.
• Average-case: (Sometimes)
– T(n) = expected time of algorithm over all inputs
of size n.
– Need assumption of statistical distribution of
inputs.
• Worst-case: (Usually)
– T(n) = maximum time of algorithm on any input of
size n.
– Generally, we seek upper bounds on the running
Lecture 1 31
time, to have a guarantee of performance.
Running time of Insertion Sort

Lecture 1 32
Additional Note: Sigma Symbol

Lecture 1 33
 Additional Note: Sigma Symbol

• Useful formulas!
𝑛

∑ 0=0
𝑖 =1

∑ 𝑐=𝑐 ∗ 𝑛
𝑖=1

𝑛
𝑛 ∗(𝑛+1)
∑ 𝑖= 2
𝑖 =1

Lecture 1 34
 Running time of Insertion Sort

Best case: The array is already sorted.

• Always find that A[i] ≤ key upon the first time
the while loop test is run (when i = j − 1)
• All tj are 1
• Running time is
T(n) = c1*n + c2*(n − 1) + c4*(n − 1) + c5*(n −
1) + c8*(n − 1)
= (c1 + c2 + c4 + c5 + c8)*n − (c2 + c4 + c5 +
c8)
• Can express T(n) as an+b for constants a and b
⇒ T(n) is a linear function of n
Lecture 1 35
 Running time of Insertion Sort

Worst case: The array is in reverse sorted order.

• Always find that A[i] > key in while loop test
• Have to compare key with all elements to the left
of the jth position ⇒ compare with j − 1 elements
• Since the while loop exits because i reaches 0,
there is one additional test after the j − 1 tests ⇒
tj = j

Lecture 1 36
Running time of Insertion Sort

Lecture 1 37
 Running time of Insertion Sort

• Running time is

• Can express T(n) as an2 + bn + c for constants

a, b, c ⇒ T(n) is a quadratic function of n.

Lecture 1 38