Welcome to data structures and algorithms.

We are
going to learn today some basic terminology regarding data structures and the
notations
that you would be following in the rest of this course. We will begin with some
very
simple definitions. An algorithm which is an outline of the steps that a program
has
to take or any computational procedure has to take. A program on the other
hand is an
implementation of an algorithm and it could be in any programming language.
Data structure
is the way we need to organize the data, so that it can be used effectively by the
program.
So you are all familiar with certain data structures, an array or a list for instance.
In this course you will be seeing a lot more data structures in this course and you
will see how to use them in various algorithms. We will take a particular problem,
try to
solve that problem and in the process develop data structures, the best way of
organizing
the data, associated with that problem. What is an algorithmic problem? An
algorithmic
problem is essentially, that you have a certain specifications of an input as is
given here
and you specify what the output should be like. And a specification of an input
could be, here is one specification, a sorted, non decreasing sequence of natural
numbers of
non-zero, finite length. That's an input, that's a completely specified input. I have
given 2 examples here of inputs which meet the specification and I have not
given any output specification yet here.
Now what is an instance? These are 2 instances of the input. This is the
specification for
the input and you can have any possible instance, you can take any sequence of
sorted, non-decreasing
numbers and that would form an input instance. So there are many input
instances possible
here.
An algorithm is essentially, describing the actions that one should take on the
input instance to get the output as desired, as is specified. And again there can
be infinitely
many input instances and there can be infinitely many algorithms for solving
certain problem.
Each one of you could do it in a slightly different way.
That brings the notion of good algorithm. If there are so many different
algorithms for solving a certain problem, what is a good algorithm? Good
algorithm for us is an efficient
algorithm. Anything that is efficient is good. What is efficient? Efficient is
something,
which has small running time and takes less memory. These will be the two
measures of
efficiency that we will be working with. There could also be other measures of
efficiency. But these are the only two things we would be considering in this
course. And most of
our time we would be spending with the running time really. Space, of course we
will be analyzing
the space and most of the time would be spent in worrying about the running
time of an algorithm.
And we would be interested in the efficiency of algorithms, as a function of the
input
size. So clearly you can imagine that, if I have a small input and my algorithm
running on
that input or my program running on that input will take less amount of time. If
the input becomes 10 times larger, then the time taken by the program would
also grow. It may it
becomes 10 times, may be it becomes 20 times or may be it becomes 100 times,
I do not know. It is this behavior of the increase in the running time, with the
increase in the size
of input that would also be of our interest to us. We will come to all of these in a
short
while as we go through these slides. How does one measure running time? So I
said efficiency, running time, very important.
How does one measure the running time of an algorithm? One way would be, I
have put down
here as an experimental study. You have a certain algorithm and you have to
implement that algorithm, which means you have to write a program in a certain
programming language.
You run the program with varying data sets, some smaller, some larger data sets,
some
would be of some kinds and some would be of different kinds, so varying
composition. And then you clock the time that the program takes and clock does
not mean that you should sit
down near stopwatch. Perhaps you can use the system utility like let's say
System. Current Time Millis (), to clock the time that the program takes and then
from that you try and
figure out, how good your algorithms is, so that is what one would call an
experimental
study of the algorithm. This has certain limitations. So I put them down. First you
have to implement the algorithm
to be able to determine how good your algorithm is you have to Implement it and
that is already a huge overhead, considerable amount of time has to be spent in
doing this. When your experiments
can be done only on a limited set of inputs. After all I said the number of
instances is infinitely large and you can run your experiment only on a small set
of instances and that
might not be really indicative of the time that your algorithm is taking for other
inputs, which you have not considered in your experiment.
Further if you have two algorithms and you have to decide, which one is better
you have
to use exactly the same platforms to do the comparison. Platform I mean both
the hardware
and software environment. Because as you can imagine, different machines
would make a difference,
not just different machines in fact even the users who are working on that
system at that particular point would make a difference on the running time of
an algorithm. It becomes
very messy, if you have to do it this way. What we are going to do in the part of
this course, in fact in this very first lecture
is to develop the general methodology, which will help us to analyze running
time of algorithms.
We are going to do it as follows: we are going to first develop a high level
description
of an algorithm, a way of describing an algorithm and we are going to use this
description to
figure out the running time and not to implement it to any system.
A methodology would help us take into account of all possible input instances
and it would
help us and also it will allow us to evaluate the efficiency of the algorithm in a
way that
it is independent of the environment, independent of the platform that we are
using.
I said we will give the high level description of the algorithm. This very first point
here, we will give a high level description. So, what is this? So this brings me to
what? We
will call this Pseudo-code. And this is how we are going to be specifying all our
algorithms for the purposes of this
course. Here is an example of pseudo code and you might have seen this in your
earlier courses
also. What is this algorithm doing? This algorithm takes an array A, which stores
an integer
in it and it is trying to find the maximum element in this array. What I have
written
here is not a program because I think the syntax is all wrong. But it is pseudo-
code;
it is a mixture of natural language and some high-level programming concepts.
I am going to use a for loop, do loop, I am going to use if-then-else statement
and I am going to use a while loop. But I will not bother about whether there
should be a semicolon
here or there should be colon here and I am not going to bother about those are
things are required by the compiler but for our understanding this is completely
clear what this program
is doing. So what it is doing? It is keeping track of the maximum variable in a
variable
called current max which is initialized to the first element of the array. And Then
it
is going to run through the remaining element of the array, compare them with
the current maximum element. Current Max ← A [0]. If the current maximum
element is less than the
current element, then it would update the current max. A[i] becomes the new
max and
then when the loop terminates we would just return current max. If current Max
< A[i] then current Max ← A[i] return current Max It is a very simple algorithm
but just with this pseudo-code, you are able to understand
what it is doing. This will not run on any computer since it is the pseudo-code,
but it conveys the idea or the concepts any question up to this point? That is
what I saying most
structured pseudo code, most structured than usual course but it is less for than
formal
programming. And How pseudo-code will look like? We will use standard numeric
and Boolean expressions
in it. Instead of the assignment operator which is '=' in java, I will use ← and
instead of
the equality operator, an equality relationship in java which is '= =' the same in
C, I will
just use '='. I will declare methods with the algorithmic name and the parameter
it
takes. Algorithm name (param 1, param2) I will use all kinds of programming
construct like if ...then statement, if ...then... [else]
statement, while ... do, repeat ...until, for ... do and to index array I will say A[i],
A [i, j]. It should be clear in what it is doing. I will use return when the procedure
terminates and return value will tell, what the value
returned by the particular procedure or a function. When I have to make a call to
a
method, I will specify that with the name of the method and the argument and
what is the object that is used. Any question to this point. What was the object?
This is specifies the type of the value returned by the particular method. You will
see more
of this, when we come across more pseudo-code. How do we analyze algorithms?
First we identify
what are the primitive operations in our pseudo-code. What is a primitive
operation? It is a low
level operation. Example is a data movement in ,I do an assignment from one to
another,
I do a control statement which is a branch (if... then ...else) subroutine call or
return.
I do arithmetic operations or logical operations these are all we called as a
primitive operation.
• Data movement (assign) • Control (branch, subroutine call, return) • Arithmetic
an logical operations (e.g. addition, comparison) In my pseudo code, I just
inspect the pseudo code and count the number of primitive operations
that are executed by an algorithm. Let us see an example of sorting. You all know
what
sorting is, the input is some sequence of numbers and output is a permutation of
the
sequence which is in non decreasing order. What are
the requirements for the output? It should be in non-decreasing order and it
should be
the permutation of the input. Any set of numbers which are in non-decreasing
order does not make an output. Algorithm should
sort the numbers that were given to it and not just produce the sequence of
numbers as an increasing order. Clearly the running time will depends upon,
number of elements (n)
and often it depends upon, how sorted these numbers are. If they are already in
sorted
order then the algorithm will not take a long time. It also depends upon the
particular
algorithm we use. The running time would depend upon all these things. The first
sorting technique
we use is the one that you have used very often. Let us say when you are playing
game of cards.
What is the strategy you follow, when you are picking up a set of cards that have
been
dealt out to you? You like to keep them in a sorted order in your hand. You start
with
the empty hand and you pick up the first card, then you take the next card and
insert it
at the appropriate place. Suppose if I have some five cards in your hand already,
let us say 2, 7, 9, jack and
queen. Then I getting 8, so I am going to put it between 7 and 9. That is the right
place it has to be placed in. I am inserting it at the appropriate place and that is
why
this technique is called insertion sort. I keep on doing this, till I have picked up
all the cards and inserted in the appropriate place.
So this is the pseudo-code for insertion sort. I will give an array of integers A
contain
input and output is a permutation of the original numbers, such that it is sorted.
The output
is also going to be in the same array. A [1]≤ A [2]≤ _ ≤ A[n] This is the input,
output specification. I am going to have 2 variables or indices i
and j. The array is going to be sorted from a [1] through a [j-1]. The jth Location
is
an element which I have to insert appropriately to the right place Clearly j has to
vary from
2 to n. For j ←2 to n I am going to look at jth element and I put that in key. Key
←A[j] I have to insert
A [j] or the key in to the sorted sequence which is A [1] through A [j-1]. i.e. A [1_j-
1]
I am going to use the index i to do this. What is index i going to do? Index i is
going
to run down from j-1 down to 1. We going to decrease index i, which is what we
are doing
in here in this while do loop.
It starts with the value j-1.And what I am going to do? I have to insert 7 and I am
going
to move 9 to 7th location, because 9 is more than 7. Then I compare 7 with 8 and
8 is still
greater than 7, so I will move it right. Then I compare 7 with 6. As 6 is smaller
than 7,Now
I found the right place for 7, I would put 7 here ,that exactly what is happening
here.
I run through this loop, till I find an element which is less than key. Key is the
element
which I am trying to insert. This loop will continue while the element, I am
consider is more than key and this loop will terminate, when I see an element
which is less than key
or the loop will terminate when I reach i=0. While i >0 and A[i] > key do A [i+1]
← A[i]
That means I have moved everything to the right and I should insert the element
at the very first place and what am I doing Here? I am just shifting the element
one step to
the right. Do A [i+1] ← A[i] Note that I have to insert 7 at the right place, so I
shift 9 right to 1 step. 9th location
becomes empty, then I shift 8 to 1 step, so this 8th location becomes empty and
now I
can put 7 here. i + 1 is the index, which would be the empty location eventually
and
i put the key there. A [i+1]← key All of you can implement it. May be you would
have
implemented it in a slightly different way, that would give you a different
program, but the algorithm is essentially the same. You are going to find the right
place for the
element and insert it.Now Let us analyze this algorithm.
I have put down the algorithm on the left (There is a small mistake here there
should
be a left arrow Please make a correction on that) .What we are going to do?
A [i+1] ← A key Let us count. Key ← A[j] I ← j-1 These are all my primitive
operations. Here I have to do primitive operations, why because
I am comparing i with 0 and I am comparing A[i] with key, I am also taking and,
so there
are three primitive operations. while i >0 and A[i] > key
Each of the operation takes a certain amount of time, depending upon the
computer system
you have.C1,C2,C3,C4,C5,C6 just Reflect or just represent the amount of time
taken for
these operations and they can be in any units. And here I am counting the
number of times,
each of these operations is executed is done in this entire program.
Why this operation is done n times? I start by assigning j =2 then assign 3,
4,5,6,7 and
go up to n. Then when I increment it once and check that there is one more, so I
have
counted it as n times. There might be small errors in n and n + 1, that is not very
important.
So this roughly n times we have to do this operation. How about this operation?
Key ← A[j] I am going to do exactly n-1 times once for 2,
once for3, once for 4 up to n. That is why this operation is being done up to n-1
times.
Just leave the comment statement. Again the operation will be done exactly n-1
times.
We have to look at how many times I come to this statement. While I >0 and A[i]
> key
tj -reflects the Counts the number of times I have to shift an element to the right,
when
I am inserting the jth card in to my hand. In the previous example when I am
inserting
7, I had to shift 2 elements 8 and 9. is going to count that quantity and that is
the number
of times I am going to reach A[i] part of my while loop. While I >0 and A[i] >key
I will be checking this condition for many times. For one iteration or for the jth
iteration
of this for loop, I am going to reach this condition for tj times. The total number
of
times I am saying that condition is the sum of tj as j goes from 2 to n. ∑nj=2
tj ,while
I >0 and A[i] > key, do A[i+1] ← A[i] Every time I see (A[i] >key) condition I also
come to A[i], I am going to be see this condition,
I am going to come this statement one more time then, I come here because you
know the last time i see the statement I would exit out of here. That is why this is
tj -1 where
j going from 2 to n.∑nj=2 (t j-1)
A [i+1] ← A key. This statement here is not a part of the while loop (this is an
assignment
operation please correct this). So this statement is part of the for loop is done
exactly n
minus one times as the other statement. So the total time this procedure takes if
you
knew what this constant work can be computed. You do not know what tj is. tj is
quantity
which depends upon your instance and not problem there is a difference here.
Problem is one
of sorting. The instance is a set of numbers, the sequence of numbers that have
given to
you. Thus tj depends upon the instance. Let us see the difference that tj makes.
If the input was already sorted, then tj is always
1(tj=1). I just have to compare the element with the last element and if it is
larger
than the last element, I would not have to do anything. tj is always a 1 if the
input
is already in increasing order. What happens when the input is in decreasing
order? If the input is in decreasing order,
then the number that I am trying to insert is going to be smaller than all the
numbers
that that i already have because the input is in decreasing order the number I am
trying
to insert is smaller than the numbers I have sorted in my array .What am I going
to do?
I am going to compare with the 1st element,2nd element,3rd element, 4th
element and all the way up to the element. When I am trying to insert the tj
element, I am going to end up
in comparing with all the other j elements in the array. In that case when tj is
equal
to j, note that the quantity becomes its summation of j, where j goes from 2 to n.
It is of the
kind and the running time n square of this algorithm would be some constant
time plus
some other constant times n minus some other constant.
n(C1+C2+C3+C7)+∑nj=2 t j(C4+C5+C6)-(C2+C3+C5+C6+C7) Thus the
behavior of this running time is more like n square. We come to this point
later, when we talk about asymptotic analysis but this is what I meant by f(n
square). On
the other hand in the best case when tj=1, the sum is just n or n-1 and in that
case
the total time is n times some constant plus n-1 times some constant minus
some constant
which is roughly n times some constant. So what we call linear time algorithm.
On an average what would you expect? In the best case it is something like that
you have to compare only against one element you have to compare only
against one element and in
the worst case you have to compare about j elements. In the average case would
expect that it would take compare against half of this element In an average
case if you were
to take it as you comparing again j/2 , even when the summation of j/2 where j
goes from
2 to n, what will this be? This will be roughly by (n square/4) and it behaves like n
square(
and will come to these points in a minute). This is what I mean by the best, worst
and
average case. I take the size of input, suppose if I am interested in sorting n
numbers and
I look at all possible instances of these n numbers. It may be infinitely many,
again it is not clear about how to do that. What is worst
case? The worst case is defined as the maximum possible time that your
algorithm would take for any instance of that size. So these are all the
instances are of the same size. The best case would be the smallest time that
your algorithm
takes and the average would be the average of all infinite bars. That was for the
input
for 1size of size n, that would give the values, from that we can compute worst
case, best
case and the average case. If I would consider inputs of all sizes then I can
create a plot
for each inputs size and I could figure out the worst case, best case and an
average case.
Then I would get such a monotonically increasing plots. It is clear that as the size
of the
input increases, the time taken by your algorithm will increase. It is not going to
happen that
your input size become larger and it takes lesser time.
Which of this is the easiest to work with? Worst case is the one we will use the
most.
For the purpose of this course this is the only measure we will be working with.
Why
is the worst case used often? First it provides an upper bound and it tells you
how long your
algorithm is going to take in the worst case. Many algorithms occurs fairly often.
Quite often it is the case that the worst case that
for many instances the time taken by the algorithm is close to the worst case .So
that average
case essentially becomes as bad as the worst case In fact for the previous
example that
we saw average case was like n square squared and worst case was also n
squared there were
differences in the constant but it was roughly the same.The average case might
be very difficult
quantity to compete because as you said average case if you have to compute
look at all possible instances and then take some kind of average .Or you have to
say like, when my input instance
is drawn from a certain distribution and the expected time my algorithm will take
is typically
a much harder quantity to work and to compute with.
The worst case is the measure of interest in which we will be working with.
Asymptotic
analysis is the kind of thing that we have been doing so far as n and n square
and the
goal of this is to analyze the running time while getting rid of superficial details.
We would like to say that an algorithm, which has the running time of some
constant times
squared is the same as an algorithm which has a running time of some other
constant
times ,because this constant is typically something which would be dependent
upon the
hardware that your using. 3n2 = n2 In the previous exampleC1,C2, and C3 would
depend upon the computer system, the hardware,
the compiler and many factors. We are not interested to distinguish between
such algorithms.
Both of these algorithms, one which has the running time of 3 n square and
another with running time n square have a quadratic behavior. When the input
size doubles the running time
of both of the algorithm increases four fold.
That is the thing which is of interest to us. We are interested in capturing how the
running time of algorithm increases, with the size of the input in the limit. This is
the crucial point here and that is what the symptotic analysis is all about here. In
the
limit how does the running time of this algorithm increase in input size.
That brings us to something that some of you might have seen before the "big-
oh" O-notation.
If I have functions f(n) , g (n) and n represents the input size. f (n) measures the
time taken
by that algorithm. f (n) and g (n) are non-negative functions and also non-
decreasing, because
as the input size increases, the running time taken by the algorithm would also
increase.
Both of these are non-decreasing functions of n and we say that f (n) is O (g (n)),
if
there exist constants c and , such that f (n)≤ c times of g (n)≥ n0 .
f (n) =O(g(n) f (n) c g(n) for n ≥ n0 What does it mean? I have drawn two
functions. The function in red is f (n) and g (n) is
some other function. The function in green is some constant times of g (n). As
you can
see beyond the point , c (g (n)) is always larger than that of f (n). This is the way
it continues even beyond. Then we would say that f (n) is O (g (n) or f (n) is order
(g
(n)). f (n) = O (g(n)) Few examples would clarify this and we will see those
examples. The function f (n) =2n+6
and g (n) =n. If you look at these two functions 2n+6 is always larger than
n and you might be wondering why this 2n+6 is a non-linear function. That is
because the scale here is an exponential scale. The scale increases by 2 on y-
axis and similarly
on x-axis. The red colored line is n and the blue line is 2n and the above next line
is
4n. As you can see beyond the dotted line f (n) is less than 4 times of n. Hence
the
constant c is 4 and would be this point of crossing beyond which 4n becomes
larger than
2n+6.
At what point does 4n becomes larger than 2n+6. It is three. So becomes three.
Then
we say that f (n) which is 2n+6 is O (n). 2n+6 = O (n)
Let us look at another example. The function in red is g (n) which is n and any
constant
time g (n) which is as same scale as in the previous slide. Any constant time g
(n) will be just the same straight line displaced by suitable amount. The green
line will be 4
times n and it depends upon the intercept, but you're n2 would be like the line
which
is blue in color. So there is no constant c such that n2 < c (n).
Can you find out a constant c so that n2 < c (n) for n more than . We cannot find
it.
Any constant that you choose, I can pick a larger n such that this is violated and
so
it is not the case that n2 is O (n).
How does one figure out these things? This is the very simple rule. Suppose this
is my
function 50 n log n, I just drop all constants and the lower order terms. Forget the
constant
50 and I get n log n. This function 50 n log n is O (n log n). In the function 7n-3, I
drop the constant and lower order terms, I get 7n-3 as O (n). I have some
complicated function like 8n2 log n+ 5n2 +n in which I just drop all lower
order terms. This is the fastest growing term because this has n2 as well as log n
in it.
I just drop n2 , n term and also I drop my constant and get n2 log n. This function
is
O ( log n). There is a constant c such that this quantity this large sum here is less
than c times n square log n for n larger than sum n0. In the limit this quantity
(8n2 log
n+5n2 +n) will be less than some constant times this quantity (O (n2 log n)). You
can
figure out what should be the value of c and n0 , for that to happen.
This is a common error. The function 50 n log n is also O (n5). Whether it is yes or
no. It is yes, because this quantity (50 n log n) in fact is ≤ 50 times n5 always,
for all n and that is just a constant so this is O(n5). But when we use the O-
notation we
try and provide as strong amount as possible instead of saying this statement is
true we
will rather call this as O (n log n)). We will see more of this in subsequent slides.
How are we going to use the O-notation? We are going to express the number of
primitive
operations that are executed during run of the program as a function of the input
size.
We are going to use O-notation for that. If I have an algorithm which takes the
number
of primitive operations as O (n) and some other algorithm for which the number
of primitive operations is O (n2 ). Then clearly the first algorithm is better than
the second. Why because
as the input size doubles then the running time of the algorithm is also going to
double,
while the running time of O (n2 ) algorithm will increase four fold.
Similarly our algorithm which has the running time of O (log n) is better than the
one which has running time of O (n). Thus we have a hierarchy of functions in the
order of log
n, n2,n3,n4, .
There is a word of caution here. You might have an algorithm whose running time
is 1,000,000
n, because you may be doing some other operations. I cannot see how you
would create such an
algorithm, but you might have an algorithm of this running time. 1,000,000n is O
(n),
because this is ≤ some constant time n and you might have some other
algorithm with the running time of 2n2 . Hence from what I said before, you
would say
that 1,000,000 n algorithm is better than 2n2 . The one with the linear running
time
which is O (n) running time is better than O (n2). It is true but in the limit and the
limit is achieved very late when n is really large. For small instances this 2 might
actually
take less amount of time than your 1,000,000 n. You have to be careful about the
constants
also. We will do some examples of asymptotic analysis. I have a pseudo code and
I have an array of
n numbers sitting in an array called x and I have to output an array A, in which
the
element A[i] is the average of the numbers X [0] through X[i]. One way of doing
it is,
I basically have a for loop in which I compute each element of the array A. To
compute A
[10],what should I do? I just have to sum up X [0] through X [10], which I am
doing
here. For j ← 0 to I do A ← a + X[j] A[i]← a/ (i+1)
To compute A [10], i is taking the value 10 and I am running the index j from 0-
10. I
am summing up the value of X from X [0] - X [10] in this accumulator a and then
I am eventually
dividing the value of this accumulator with 11, because it is from X [0] to X [10].
That
gives me the number I should have in A [10]. I am going to repeat this for
11,12,13,14
and for all the elements.
It is an algorithm and let us compute the running time. This is one step. It is
executed
for i number of times and initially i take a value from 0,1,2,3 and all the way up
to
n-1. This entire thing is done n times. This gives you the total running time of
roughly
. a ← a+ X[j] This one step is getting executed times and this is the dominant
thing. How many times
the steps given below are executed? A[i] ← a/ (j+1) a ← 0 These steps are
executed for n times. a ← a + X[j] But the step mentioned above
is getting executed roughly for some constant n2 times. Thus the running time of
the algorithm
is O (n2). It is a very simple problem but you can have a better solution.
What is a better solution? We will have a variable S in which we would keep
accumulating
the X[i]. Initially S=0. When I compute A[i], which I already have in S, X [0]
through X
[i-1] because they used that at the last step. That is the problem here.
a ← a +X[j] Every time we are computing X. First we are computing X [0] + X [1],
then we are computing
X [0] + X [1] +X [2] and goes on. It is a kind of repeating computations. Why
should
we do that? We will have a single variable which will keep track of the sum of the
prefixes.
S at this point (s← s+x[i]), when I am in the jth run of this loop has some of X [0]
through X [i-1] and then some X[i] in it. To compute jth element, I just need to
divide
this sum by i +1. S ←S +X[i] A[i] ← S/ (i+1) I keep this accumulator(S) around
with me.
When I finish the jth iteration of this loop, I have an S, the sum X [0] through X[i].
I
can reuse it for the next step. How much time does this take? In each run of this
loop I am just doing two primitive
operations that makes an order n times, because this loop is executed n times. I
have been
using this freely linear and quadratic, but the slide given below just tells you the
other
terms I might be using. Linear is when an algorithm has an asymptotic running
time of O (n), then we call it as
a linear algorithm. If it has asymptotic running time of n2 , we called it as a
quadratic and
logarithmic if it is log n. It is polynomial if it is nk for some constant k.
Algorithm is called exponential if it has running time of (a n), where a is some
number more than 1. Till now I have introduced only the big-oh notation, we also
have the big-omega
notation and big-theta notation. The "big-Omega" notation provides a lower
bound. The function
f (n) is omega of g (n), f (n) =Ω (g(n))
If constant time g (n) is always less than f(n), earlier that was more than f(n) but
now it is less than f(n) in the limit, beyond a certain as the picture given below
illustrates.
c g (n) f (n) for n f (n) is more than c (g(n)) beyond the point . That case we will
say that f (n) is omega of g (n).
f (n) =Ω (g (n)) In θ notation f (n) is θ (g (n), if there exist constant and such that
f (n) is sandwiched
between C1 g (n) and C2 g (n). Beyond a certain point, f (n) lies between 1
constant time
g (n) and another constant time of g (n). Then f (n) is θ (g (n)) where f (n) grows
like g (n) in the limit. Another way of thinking of it is, f (n) is θ (g (n)). If f (n) is
O (g (n)) and it also Ω (g (n). There are two more related asymptotic notations,
one
is called "Little-oh" notation and the other is called "Little-omega" notation. They
are the non-tight analogs of Big-oh and Big-omega. It is best to understand this
through the
analogy of real numbers. When I say that f (n) is O (g (n)) and iam really saying
that the function f in some
sense is less than or equal to g, that in fact what use in definition or f (n) is less
than c (g (n). The analogy with the real numbers is when the number is less than
or equal to
another number. is for and is for =. θ (g (n) is function and f=g are real numbers.
If these are real numbers, you can talk of equality but you cannot talk of equality
for
a function unless they are equal. Little-oh corresponds to strictly less than g and
Little-omega
corresponds to strictly more. We are not going to use these, infact we will use
Big-oh. You
should be very clear with that part.
The formal definition for Little-oh is that, for every constant c there should exist
some
such that f (n) is < c (g(n) for n > n0 . f (n) ≤ c (g(n)) for n ≥ n0 How it is
different
from Big- oh? In that case I said, there exist c and such that this is true. Here we
will
say for every c there should exist an n0 .
This is just one slide I had put up to show what the differences between these
functions is like. I have an algorithm whose running times are like 400n, 20n log
n, 2 ,2n2,n4
and 2n . Also I have listed out, the largest problem size that you can solve in 1
second
or 1 minute or 1 hour. The largest problem size that you can solve is roughly
2500.
Say you using some some constant, so it lets say 2500if you had this has running
time let
say your this is what the problem size would be like 4096. Why did you say that
4096 is
larger than 2500, although 20n log n is the worst running time than 400n,
because of the
constant. You can see the differences happening. If it is 2n2 then the problem
size is 707
and when it is the problem size is 19. See the behavior as the time increases. An
hour is 3600seconds and there is a huge increase
in the size of the problem you solve, if it is linear time algorithm. Still there is a
large increase, when it is n log n algorithm and not so large increase when it is
an n2 algorithm and almost no increase when it is 2n algorithm. If you have an
algorithm whose
running time is something like 2n , you cannot solve for problem of more than
size 100. It
will take millions of years to solve it. So that is the behavior that bothers us that
is the behavior we are interested in course
that is why a asymptotic analysis is what we we considering most of it. So any
questions
till this point so with that we are going to stop this lecture. Today we have looked
at asymptotic analysis and some initial notation and terminology that we be
following with this course.