CSE 317: Design and Analysis of

Algorithms

Shahid Hussain
Weeks 1 & 2: Fall 2024

1
Introduction
Finding the maximum from a sequence

• Given a sequence of numbers x1 , . . . , xn , we want to find

the maximum
• Consider the following algorithm
findmax(x1 , x2 , . . . , xn )

m ← xn
k ←n−1

Yes
k=0 Output m

No

Yes
xk > m m ← xk

No

k ←k−1

2
Finding the maximum from a sequence

Algorithm: findmax
Input: A sequence of numbers x1 , . . . , xn
Output: The maximum number m in the sequence
1. m = xk
2. k =n−1
3. while k > 0
4. if xk > m
5. m = xk
6. k =k−1
7. return m
Let m = 4 and x1 = 2.3, x2 = 7.1, x3 = 4.0, and x4 = 5.9. We
can trace the algorithm (flow chart) with this data. Following is
the trace 3
Finding the maximum from a sequence

• Let m = 4 and x1 = 2.3, x2 = 7.1, x3 = 4.0, and x4 = 5.9.

We can trace the algorithm (flow chart) with this data.
Following is the trace

m ← 5.9,
 7.1
k ← 3
, 2
, 1
, 0
• Is this algorithm correct?
• We employ invariant
• For the above algorithm we can place the invariant
m = max(xk+1 , . . . , xn ) before the first decision (the if
statement) and it will always remain true

4
Time Complexity
Time Complexity

• Time in an important resource while designing algorithms

• It is meaningless to say that an Algorithm A, when
presented with input x, runs in time y seconds
• We, therefore, measure time in number of elementary
operations such as arithmetic operations (addition and
subtraction), comparisons, etc

5
Growth of Functions

30
log n
n
n log n
20 n2
n3

10

0 2 4 6 8

6
Oh and Other Notations
The Big-Omicron Notation

• Let f and g be two functions, f, g : N → N

• We say that f (n) is in big-omicron of g(n), i.e.,
f (n) = O(g(n)), if and only if, there exists constants c and
n0 such that

f (n) ≤ c · g(n), for all n ≥ n0

• Consequently, if limn→∞ f (n)/g(n) exists, then

f (n) = O(g(n))

7
The Big-Omega Notation

• Let f and g be two functions, f, g : N → N

• We say that f (n) is in big-omega of g(n), i.e.,
f (n) = Ω(g(n)), if and only if, there exists constants c and
n0 such that

f (n) ≥ c · g(n), for all n ≥ n0

• Consequently, if limn→∞ f (n)/g(n) exists, then

f (n)
lim ̸= 0 implies f (n) = Ω(g(n))
n→∞ g(n)

• Informally, f (n) = Ω(g(n)) if and only if g(n) = O(f (n))

8
The Big-Theta Notation

• Let f and g be two functions, f, g : N → N

• We say that f (n) is in big-theta of g(n), i.e.,
f (n) = Θ(g(n)), if and only if, there exists constants c1 , c2 ,
and n0 such that

c1 · g(n) ≤ f (n) ≤ c2 · g(n), for all n ≥ n0

• Consequently, if limn→∞ f (n)/g(n) exists, then

f (n)
lim = c implies f (n) = Θ(g(n))
n→∞ g(n)

• Importantly, f (n) = Θ(g(n)) if and only if f (n) = O(g(n))

and f (n) = Ω(g(n))

9
The small-omicron Notation

• Let f and g be two functions, f, g : N → N

• We say that f (n) is in small-omicron of g(n), i.e.,
f (n) = o(g(n)), if and only if, there exists a constant n0
such that for all possible values of c > 0

f (n) < c · g(n), for all n ≥ n0

• Consequently, the limn→∞ f (n)/g(n) exists and equal to

zero i.e.,

f (n)
lim = 0 implies f (n) = o(g(n)).
n→∞ g(n)

10
The small-omega Notation

• Let f and g be two functions, f, g : N → N

• We say that f (n) is in small-omega of g(n), i.e.,
f (n) = ω(g(n)), if and only if, there exists a constant n0
such that for all possible values of c > 0

f (n) > c · g(n), for all n ≥ n0

• Consequently, the limn→∞ f (n)/g(n) is infinite i.e.,

f (n)
lim = ∞ implies f (n) = ω(g(n)).
n→∞ g(n)

11
Examples

1. Let f (n) = 10n3 + 20n then f (n) = O(n3 ) also

f (n) = Ω(n3 ) therefore f (n) = Θ(n3 )
2. In general, let f (n) = ak nk + ak−1 nk−1 + · · · + a1 n + a0 ,
then f (n) = Θ(nk )
3. Let f (n) = logb n2 then f (n) = 2 logb n therefore
f (n) = Θ(log n)
4. For any fixed constant k, log nk = Θ(log n)
5. Any constant function is O(1), Ω(1), and Θ(1)
6. 2n = Θ(2n+1 ), this is an example of many functions that
satisfy f (n) = Θ(f (n + 1))

12
More Examples

Pn
1. Consider the series j=1 log j, clearly:
n
X n
X
log j ≤ log n = O(n log n)
j=1 j=1
Pn
2. We also prove that j=1 log j = Ω(n log n), therefore:
n
X
log j = Θ(n log n)
j=1

13
Problems: State True or False

1. n3 = O(n2 )
√
2. n log n = O(n n)
√
3. n2 (1 + n) = O(n2 log n)
√
4. log n + n = O(n)
√
5. n log n = O(n)
6. log n = O(1/n)
√ √
7. n + n = O( n log n)

14
Problems: Proof Questions

1. n2 − 3n − 18 = Ω(n)
2. 2n = O(n!)
3. n! = Ω(2n )
4. Does nlog n = O((log n)n )? Prove it
2
5. Prove or disprove: 2(1+O(1/n)) = 2 + O(1/n)

15
Searching
Linear Search

• Given a sequence A of n ≥ 0 pair-wise different numbers

a1 , . . . , an , and a number x
• We want to find the index i such that ai = x

Algorithm: linearsearch
Input: A sequence of numbers a1 , . . . , an and a number x
Output: The index i such that ai = x, 0 otherwise

1. for j = 1 to n
2. if aj = x return j
3. return 0

16
Linear Search

• The Algorithm linearsearch is correct, why?

• The time complexity of the algorithm is O(n), why?
• The space complexity of the algorithm is O(1), why?
• This algorithm gets its name because of its worst-case
running time, which is linear in terms of the size of its input
• This algorithm does not depend on the order of the
elements in the sequence

17
Binary Search

• Given a sequence A of n ≥ 0 pair-wise different numbers

a1 , . . . , an , such that a1 < a2 < · · · < an , and a number x
• We want to find the index i such that ai = x
• Now we can exploit the fact that the sequence is sorted

18
Binary Search

Algorithm: binarysearch
Input: A sequence of numbers a1 , . . . , an and a number x
Output: The index i such that ai = x, 0 otherwise
Iterative Version Recursive Version

1. l = 1, r = n 1. m = ⌊(l + r)/2⌋
2. while l ≤ r 2. if am = x return m
3. m = ⌊(l + r)/2⌋ 3. else if am > x return
binarysearch(a1 , . . . , am−1 , x)
4. if am = x return m
4. else return
5. if am < x l = m + 1
binarysearch(am+1 , . . . , an , x)
6. else r = m − 1
7. return 0 19
Binar Search

• The Algorithm binarysearch is correct, why?

• The time complexity of the algorithm is O(log n), why?
• The space complexity of the algorithm is O(1), why?

20
Sorting
Sorting

• Given a sequence A of n ≥ 0 numbers a1 , . . . , an

• We want to sort the sequence in non-decreasing order i.e.,
a1 ≤ a2 ≤ · · · ≤ an
• There are several ways to sort a sequence, some are more
efficient than others
• Here we look at some of the comparison based sorting
algorithms

21
Selection Sort

Algorithm: selectionsort
Input: A sequence of numbers A = ⟨a1 , . . . , an ⟩
Output: A permutation of A such that ai1 ≤ ai2 ≤ · · · ≤ ain
1. let B = ⟨⟩
2. for i = 1 to n
3. m = findmin(ai , . . . , an )
4. B = B ◦ ⟨am ⟩
5. am = ai
6. return B

• The findmin makes n − i comparisons to find the minimum

• The time complexity of selectionsort is therefore
n + (n − 1) + · · · + 1 = Θ(n2 )
22
Insertion Sort

Algorithm: insertionsort
Input: A sequence of numbers A = ⟨a1 , . . . , an ⟩
Output: A permutation of A such that ai1 ≤ ai2 ≤ · · · ≤ ain

1. for i = 2 to n
2. j=i
3. while j > 1 and aj < aj−1
4. aj , aj−1 = aj−1 , aj
5. j =j−1
6. return A

• The time complexity of insertionsort is Θ(n2 )

23
Insertion Sort with Binary Search

Algorithm: binary-insertionsort
Input: A sequence of numbers A = ⟨a1 , . . . , an ⟩
Output: A permutation of A such that ai1 ≤ ai2 ≤ · · · ≤ ain
1. for i = 2 to n
2. j = i, l = 1, r = j
3. while l ≤ r
4. m = ⌊(l + r)/2⌋
5. if aj < am then r = m − 1
6. else l = m + 1
7. aj , aj−1 = aj−1 , aj
8. return A

• The time complexity of binary-insertionsort is Θ(n2 )

(but slightly better than insertionsort) 24
Bubble Sort

Algorithm: bubblesort
Input: A sequence of numbers A = ⟨a1 , . . . , an ⟩
Output: A permutation of A such that ai1 ≤ ai2 ≤ · · · ≤ ain

1. for i = 1 to n
2. for j = 1 to n − i
3. if aj > aj+1 then aj , aj+1 = aj+1 , aj
4. return A

• The time complexity of bubblesort is Θ(n2 ) becasue

n X
n−i
X n(n − 1)
number of comparisons = 1=
2
i=1 j=1

25