CS-250

Data structures & Algorithms

Complexity of Search Algorithms

Search Algorithms
Two Approaches:
Sequential Search
 Sequential Search on Unsorted Data
 Sequential Search on Sorted Data

Binary Search
Sequential Search
Every element in the array will be examined
sequentially, starting from the first element.
The process will be repeated until the last element
of the array or until the searched data is found.
Sequential Search Implementation
int SequenceSearch(int search_key,int array [ ],int array_size )
{
int index =-1; //-1 means record is not found operation 1

for (int p = 10; p < n+1

operation array_size;
times p++ )
n times
{
if ( search_key == array[p] ){ n times
index = p;//assign current array index
break; 2
operations
}//end if
} //end for
return index;
} //end function

1+1+n+1 +n+n+2
=3n+4
Sequential Search Analysis
Searching time for sequential search is O(n).
If the searched key is located at the end of the list or
the key is not found, then the loop will be repeated
based on the number of element in the list, O(n).
If the list can be found at index 0, then searching time
is, O(1).
Improvement of Basic Sequential Search
The efficiency of basic search technique can be
improved by searching on a sorted list.
For searching on ascending list, the search key will be
compared one by one until :
1.the searched key is found.
2.Or until the searched key value is smaller than the
item compared in the list.
Sequential Search on Sorted Data
Implementation
int SortedSeqSearch (int search_key,int array[ ],int array_size)
{
int index = -1; //-1 means record not found
for (int p = 0; p < array_size; p++ )
{
if (search_key < array [p])
break;
else if (search_key == array[p])
{
index = p;
break;
}
}
return index;
}
Sorted Sequential Search Analysis
Searching time for sorted sequential search is O(n)
only if the searched key is located at the end of the
list or the key is greater than (or less than for
descending arrangement) all the elements in the
list.
Binary Search
Consider a list in ascending sorted order.
Search process is started at the middle of the list,
then the algorithm determine which half the item is in
(because the list is sorted).
It divides the working range in half with a single test.
By repeating the procedure, the result is an efficient
search algorithm  O(log2 n).
Binary Search
Starts by comparing the search key with the element at the
middle
If the value matches, it will be return to the calling function
(index = MIDDLE)
If the search key < the middle element, search will be focused on
the elements from the first element to the element before the
middle element (MIDDLE -1)
If the search key > the middle element, search will only be
focused on the elements from after the middle element
(MIDDLE +1) to the last element.
Search is repeated until the searched key is found or the last
element in the subset is traversed (LEFT>RIGHT ).
Binary Search Implementation
int search(int a[], int beg, int end, int item)
{
if(beg > end)
return -1;
int mid = (beg + end)/2;
if(a[mid]==item)
return mid;
else if(item < a[mid])
search(a,beg,mid-1,item); 1 execution + T(n/2)
else
search(a , mid+1 , end , item); }

}
If array is of size 8 then we can divide it into 3 equal
halves as
23=8
If array is of size n then we can divide it into k equal
halves as
2k=n (1)
T(1)=1 means constant time (2)
only one comparison is required
T(n)=2+T(n/2)
=2+[2+T(n/4)]
=4+[T(n/4)]
=4+[2+T(n/8)]
=6+[T(n/8)]
=2*3 +[T(n/23)
=2*k +[T(n/2k) (3)
Applying log on both sides of equation (1)
log(2k)=log(n)
k log(2)=log(n)
k=log(n)/log(2) (4)
in logarithm log(n)/log(2)=log2n
Therefore (3) becomes
k= log2n (5)
Applying (1) and (5) , equation(3) becomes
2*log2n +[T(n/n)]
2*log2n +[T(1)]
2*log2n +1
Ignoring the constants the order of binary search becomes

O(log2n)
SORTING
Bubble Sort
Go through multiple passes over the array.
In every pass:
Compare adjacent elements in the list.
Exchange the elements if they are out of order.
Each pass moves the largest (or smallest) elements to
the end of the array
Repeating this process in several passes eventually
sorts the array into ascending (or descending) order.
Bubble sort complexity is O(n2) and only suitable to
sort array with small size of data.
Bubble Sort Implementation
void BubbleSort(int data[], int listSize)
{
int pass, tempValue;
for ( pass = 1;pass < listSize; pass++ )
{
for (int x = 0; x < listSize - pass; x++)
if ( data[x]>data[x+1] )
{
tempValue = data[x];
data[x] = data[x+1];
data[x+1] = tempValue;
}
}
}
Bubble Sort Analysis
The number of comparison between elements and the number of
exchange between elements determine the efficiency of Bubble Sort
algorithm.
Generally, the number of comparisons between elements in Bubble Sort
can be stated as follows:
(n-1)+(n-2)+….+2+1=n(n-1)/2  O(n2)
In any cases, (worst case, best case or average case) the number of
comparisons between elements is the same.
The difference can be seen in the number of swapping elements. Worst
case has maximum number of swapping, while best case has no
swapping since all data is already in the right position.
If the list is already sorted, the next pass shouldn't be continued and the
sorting process should stop.
Improved Bubble Sort
void BubbleSort(int data[], int listSize)
{
int pass, tempValue;
int sorted = 0;
int pass=1;
while(pass<listSize && !sorted)
{
sorted=1;
for (int x = 0; x < listSize - pass; x++)
if ( data[x]>data[x+1] )
{
tempValue = data[x];
data[x] = data[x+1];
data[x+1] = tempValue;
sorted =0;
}
}pass++;
}
Improved Bubble Sort Analysis
Complexity is measured based on time consuming
operations to compare and swap elements.
Number of comparisons
a for loop embedded inside a while loop
Worst Case (n‐1)+(n‐2)+(n‐3) …+1 , or O(n2)
Best Case – (n‐1) or O(n)
Number of Swaps
inside a conditional ‐> #swaps data dependent !!
Best Case O(n)
Worst Case (n‐1)+(n‐2)+(n‐3) …+1 , or O(n2)
SELECTION SORT
The Selectionsort Algorithm
Start by
finding the 70
70
smallest 60
60
entry.
50
50
40
40
30
30
20
20
10
10
00
[1]
[1]
[0] [2]
[2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Start by
finding the 70
70
smallest 60
60
entry.
50
50
Swap the
40
40
smallest
30
30
entry with
20
20
the first
entry. 10
10
00
[1]
[1]
[0] [2]
[2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Start by
finding the 70
smallest 60
entry.
50
Swap the
40
smallest
30
entry with
20
the first
entry. 10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
Part of the 60
array is now 50
sorted.
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
Find the 60
smallest 50
element in
40
the unsorted
30
side.
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
Find the 60
smallest 50
element in
40
the unsorted
30
side.
20
Swap with
10
the front of
the unsorted 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
side.
The Selectionsort Algorithm
Sorted side Unsorted side

70
We have 60
increased the 50
size of the
40
sorted side by
one element. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
The process 60
continues... 50
Smallest
from
40 unsorted
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side Unsorted side

70
The process 60
continues... 50 w ap
S th
wi t
40 n
fro
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
Sorted side
is bigger Sorted side Unsorted side

70
The process 60
continues... 50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The Selectionsort Algorithm
The process Sorted side Unsorted side

keeps adding one 70

more number to 60
the sorted side.
50
The sorted side
40
has the smallest
30
numbers,
20
arranged from
small to large. 10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Selectionsort Algorithm
We can stop Sorted side Unsorted side

when the 70
70
unsorted side 60
60
has just one
50
50
number, since
40
40
that number
must be the 30
30

largest number. 20
20
10
10
00
[1]
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The Selectionsort Algorithm
The array is now
sorted. 70
We repeatedly 60
selected the 50
smallest
40
element, and
30
moved this
20
element to the
front of the 10

unsorted side. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
Selection Sort
Strategy
Choose the largest/smallest item in the array and place
the item in its correct place
Choose the next largest/next smallest item in the array
and place the item in its correct place.
Repeat the process until all items are sorted.
Does not depend on the initial arrangement of the
data
Only appropriate for small n ‐ O(n2) algorithm
Selection Sort Implementation
void selectionSort(int Data[], int size)
{
for(i=0;i<size-1;i++)
{
int minIndex = i;
int minval=Data[i];
for (int p=i+1;p <size; p++)
{ if (Data[p]< Data[minIndex])
{minval = array[p];
minIndex = p;}
}
Data[minIndex]=Data[i];
Data[i]=minval;}
}
Let size of array is 5
Value of i No of No of swaps
comparisons
0 4 1
1 3 1
2 2 1
3 1 1

# of comparisons= (n-1)+(n-2)+(n-3)+….+1
=n(n-1)/2
= n2
# of swaps= n-1
Selection Sort Analysis
 For an array with size n, the external loop will iterate from 1 to n-1.
 n-1  O(n)
 For each iteration, to find the minimum number in subarray,
 Therefore the total comparison for Selection Sort for all iterations is (n‐
1) + (n‐2) + ….. 2 + 1.
 Generally, the number of comparisons between elements in Selection
Sort can be stated as:
 (n-1)+(n-2)+….+2+1=n(n-1)/2O(n2)
 In any case of Selection Sort (worst case, best case or average case) the
number of comparisons between elements is the same.
INSERTION SORT
Insertion Sort
Strategy
Take multiple passes over the array
Partition the array into two regions: sorted and unsorted
 Take each item from the unsorted region and insert it into its
correct order in the sorted region
 Find the next unsorted element and insert it in correct place,
relative to the ones already sorted.
Appropriate for small arrays
The Insertionsort Algorithm
Sorted side Unsorted side

The sorted 70

side starts 60
with just the 50
first element, 40
which is not 30
necessarily 20
the smallest
10
element.
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

The sorted 70

side grows by 60
taking the 50
front element 40
from the 30
unsorted 20
side...
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

...and 70

inserting it in 60
the place that 50
keeps the 40
sorted side 30
arranged 20
from small to
10
large.
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

In this 70

example, the 60
new element 50
goes in front 40
of the 30
element that 20
was already
10
in the sorted
0
side. [1] [2] [3] [4] [5] [6]
[0] [1] [2] [3] [4] [5]
The Insertionsort Algorithm
Sorted side Unsorted side

Sometimes 70

we are lucky 60
and the new 50
inserted item 40
doesn't need 30
to move at 20
all.
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
The Insertionsort Algorithm
Sorted side Unsorted side

Sometimes 70

we are lucky 60
twice in a 50
row. 40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How to Insert One Element
Copy the Sorted side Unsorted side
new element
70
to a separate
60
location.
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
 Shift
elements in
70
the sorted
60
side, creating
an open 50

space for the 40

new element. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
 Shift
elements in
70
the sorted
60
side, creating
an open 50

space for the 40

new element. 30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[6]
[5]
3] [4] [5] [6]
 How to Insert One Element
 Continue
shifting
70
elements...
60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[5]
[4] [6]
[6]
[5]
3] [4] [5] [6]
 How to Insert One Element
 Continue
shifting
70
elements...
60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[4]
[3] [5]
[5]
[4] [6]
[6]
[5]
3] [4] [5] [6]
 How to Insert One Element
 ...until you
reach the
70
location for
60
the new
element. 50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
Copy the Sorted side Unsorted side
new element
70
back into the
60
array, at the
correct 50

location. 40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How to Insert One Element
The last Sorted side Unsorted side

element 70
must also be 60
inserted.
50
Start by
40
copying it...
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
 How many
shifts will 70
70
occur before 60
60
we copy this
50
50
element back
40
40
into the array?
30
30
20
20
10
10
00
[1]
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
  Four items
are shifted. 70
60
50
40
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
  Four items
are shifted. 70
And then
60
the element is
50
copied back
40
into the array.
30
20
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
3] [4] [5] [6]
Insertion Sort Implementation
void insertionSort(int data[],int n)
{
int item;
int pass, insertIndex;
for(pass=1;pass<n;pass++)
{ item = data[pass];
insertIndex = pass;
while((insertIndex >0)&&(data[insertIndex-1]>item))
{
data[insertIndex]= data[insertIndex -1];
insertIndex --;
}
data[insertIndex] = item;
}
}
Value of pass No of No of Total
comparisons movements
1 1 1 2
2 2 2 4
3 3 3 6
4 4 4 8
n-1 n-1 n-1 2(n-1)

=2+4+6+8….+2(n-1)
=2(1+2+3+4+…n-1)
=2[n(n-1)/2]
=n(n-1)
=O(n2)
 Insertion Sort Analysis
For best case:
The number of comparison is n‐1 which gives linear time complexity
 O(n)
The number of swaps are zero, which gives constant complexity 
O(1)
Worst case for insertion sort is when we have totally unsorted
data. In each pass, the number of iteration for while loop is
maximum.
The number of comparisons are: (n-1)+(n-2) + …. + 2 + 1 = n(n-1)/2
 O(n2)
The number of swaps are: (n-1)+(n-2) + …. + 2 + 1 = n(n-1)/2  O(n2)