Chapter 2 – Algorithms complexity

Algorithms and dynamic data structures

A. Chikhaoui 2023 - 2024

Arrays

What’s an array?
An array is a fixed-size collection of elements of the same type. The elements of the
array are stored sequentially and can be accessed using their index.

In C, we can declare an array as follows:

float arr[n]; //n is an integer indicating the length of the array.

1 17
Arrays

Types of arrays
1. Single Dimensional Arrays: Single dimensional array or 1-D array is the simplest form
of arrays. It consists of elements of similar types and these elements can be accessed
through their indices.
▶ Example: double my_array1[20];
2. Multi-dimensional Arrays: These arrays consist of elements that are array themselves.
2-D arrays are the most common type of multi-dimensional array that is used in the C
language. However, the number of dimensions can be more than 2.
▶ Example 1 (2-d array): int my_array1[5][10];
▶ Example 2 (3-d array): float my_array2[10][20][5];

2 17
Arrays

Array operations Generally, there are two classes of algorithms on arrays:

1. Algorithms for searching for a given value
▶ unsorted array: sequential search
▶ sorted array: binary search
2. Algorithms for carrying out a given treatment
▶ inserting/deleting/updating a value or a set of values
▶ sorting an array
bubble sort
selection Sort
merge sort
quick sort

3 17
Binary search

Problem: how to search a word that begins with N in a dictionary?

We could start at the beginning and keep flipping pages until we get to the Ns. That’s
we call sequential search.
However, it makes more sense for it to begin somewhere in the middle.
This is a search problem and binary search algorithm is used to solve it.
Binary search is an algorithm; its input is a sorted array. If the element we’re looking
for is in that array, binary search returns the position where it’s located. Otherwise,
binary search returns null.

4 17
Binary search

The basic steps to perform Binary Search are:

1. Read the element from the user.
2. Begin with the middle element of the whole array as a search key.
3. If the value of the search key is equal to the element then return an index of the search
key.
4. Or if the value of the search key is less than the element in the middle of the interval,
narrow the interval to the lower half.
5. Otherwise, narrow it to the upper half.
6. Repeatedly check from the second point until the value is found or the interval is empty.

5 17
Binary search VI: exercice

Suppose we have a dictionary containing 240,000 words. In the worst case, how many
steps do you think each search will take?
Suppose you have a sorted array of 128 names, and you’re searching through it using
binary search. What’s the maximum number of steps it would take?
Suppose you double the size of the array. What’s the maximum number of steps now?

6 17
Algorithmic complexity

For any given problem, there are often numerous ways to write an algorithm that solves
it. Choosing the best algorithm involves analyzing its performance.
Different algorithms have different costs in terms of:
▶ execution time: number of operations performed by the algorithm.
▶ memory size: amount of memory required for the algorithm to complete.
These two concepts are called the time and space complexity of the algorithm.
Generally, the most important critiria to evaluate an algorithm is its time complexity.

7 17
Time complexity

The execution time of a program depends on the following factors:

▶ program data,
▶ quality of the code generated by the compiler,
▶ machine (speed and type of instructions),
▶ complexity of the algorithm.
Time complexity tells us by how much the execution time grows as we increase the amount of
data.

instructions
Number of
tion
nc
y fu
xit
ple
m
Co
Input size

By analyzing the time complexity of an algorithm, we want to be able to answer questions like:
If we double the data, do we double the execution time, do we quadruple it, or something else
entirely?

8 17
Time complexity

Example: Calculate the number of instructions of the following algorithm which tries to find an
element x in table tab of n element:
for (int i = 0; i < n; i++) {
if (x == tab[i])
return true;
}
▶ number of instructions = comparison ∗(n + 1) + assignment ∗(n + 1) + comparison ∗n +
return
▶ The execution time of the different operations (comparison, assignment, etc) is assumed to
be the same. Let call this constant time c. So, the execution time T(n) of this algorithm is:
T (n) = c x (n + 1) + c x (n + 1) + c x n + c
▶ Since the duration c is not really defined, we can simply ignore the constants.
So, T (n) = 3n + 3

9 17
Types of complexities

Worst case complexity: it is the maximum number of operations taken by the

algorithm to perform on a data set of size n.
Best case complexity: it is the minimum number of operations taken by the algorithm
to perform on a data set of size n.
Average case complexity: it is the average of the complexities of the algorithm on a
dataset of size n.
When building a model for the execution time of an algorithm, we often focus on the
worst case to provide an upper bound.

10 17
Big O-notation

The Big-O notation (called also Landau notation) is a mathematical notation, used to
compare the rate of convergence of functions.
Let n → f (n) and n → g(n) be 2 functions defined over the natural numbers.
▶ We say that f = O(g) if and only if ∃n0 ∈ N+ , ∃c ∈ R∗+ , ∀n ≥ n0 , f (n) ≤ c ∗ g(n).
▶ In other words, lim f (n)/g(n) = c, which means that, f (n)/g(n) is bounded when n
n→∞
approaches infinity.
▶ g(n) is an asymptotic upper bound for f(n).

11 17
Big O-notation

Example 1:
▶ n2 + 1 is O(n2 ).
In fact, for n0 = 1 and c = 2, we have forall n ≥ n0 , n2 + 1 ≤ 2n2
▶ The definition is satisfied, so n2 + 1 = O(n2 ).
Example 2:
▶ f (n) = n4 + 100n2 + 5 = O(n4 ).
▶ for n0 = 11 and c = 2, we have forall n ≥ n0 , n4 + 100n2 + 5 ≤ 2n4
The Big-O notation makes it possible to represent the algorithmic complexity by taking
into account only the dominant term.
There is no unique set of values for n0 and c in proving the asymptotic bounds.
Example: 100n + 5 = O(n)
▶ Solution 1: n0 = 5 and c = 101.
▶ Soultion 2: n0 = 1 and c = 105

12 17
Common complexity classes

Here are some common complexity classes from the smallest to the biggest:
O(1): constant.
O(log(n)): logarithmic.
O(n): linear.
O(nlog(n)): linearithmic.
O(n2 ): quadratic.
O(n3 ): cubic.
2n : exponential.
n!: factorial.

13 17
Common complexity classes

Table bellow shows how long algorithms that use f(n) operations take to run on a
computer, where each operation costs one nanosecond (10−9 seconds).

14 17
Operations

Sum rule: O(f (n) + g(n)) = O(max(f (n), g(n)).

Product rule: O(f (n)) ∗ O(g(n)) = O(f (n) ∗ g(n)).
Transitivity rule: if f (n) = O(g(n)) and g(n) = O(h(n)), then f (n) = O(h(n)).

15 17
Complexity of iterative algorithms

Basic operations (assignment, comparison, rad, write, evaluation of a simple

expressions): O(1).
Sequence of steps: sum rule. It is therefore the time of the sequence that has the
longest execution time.
Alternative (if/else, switch): take the worst case.
Loops: Product rule. It is the product of the number of iterations of the loop by the
greatest possible time for an execution of the body.

16 17
 Exercice 1
Calculate the complexity of Selection sort algorithm:
void selection_sort(int arr[],int n)
{
int i, j;
int min;/* index of minimum */
int elt_min;
for (i =0;i<n; i++) {
min = i;
for (j = i +1;j<n; j++){
if (arr[j]< arr[min])
{
min = j;
}
}
elt_min = arr[min];
arr[min] = arr[i];
arr[i] = elt_min;
}
}
17 / 17