Data Structure - Lecture 1 - Introduction.pdf

Course: Data Structures
Khushil Saini
Lecturer, Division of Computer Engineering
Netaji Subhas Institute of Technology, New Delhi

What this course is about ?
Data structures: Ways to organize data for
efficient storage and efficient manipulation
Employment of these data structures in the
design of efficient algorithms

Why do we need them ?
More powerful computers  more complex
applications.
Example: Index of billions of pages by Search
Engines (Google) !
Number of features in Mobile Phones increasing
day-by-day
Computers take on more and more complex
tasks  Demand more calculations
Data Structures organize data  more efficient
programs.

Organizing Data
Any organization for a collection of records
can be searched, processed in any order, or
modified.
The choice of data structure and algorithm
can make the difference between a program
running in a few seconds or many days.
Efficient data structures
Efficient algorithms

Data Structure Example
Say, we need to read some topic in a book
having 1500 pages. We looked for this topic in
the index and got the information about the
page number of its occurrence in the book.
Say, it is at page number 867. We need to go
to that page.
What to do?

Example (cont’d)
What to do?
Sol. 1 Sequential : Start from page 1 and keep on moving from
one page to next till we reach page 867. How many steps ?
or
Sol. 2 Binary Searching:
- Pages are in Sorted order.
- Look only half at a time.
Another Ex. Search 25 in : 5 8 12 14 15 18 23 25 27
25 ? 15 15 18 23 25 27
25 ? 23 23 25 27
25 ? 25
How many steps?

Some example Data Structures
List Stack Tree

What will you learn?
What are some of the common data structures
What are some ways to implement them
How to analyze their efficiency
How to use them to solve practical problems

What do you need ?
Some Programming experience with C / C++
Textbook
Data Structures Using C and C++ : Tenenbaum A. M.,
Langsam Y., Augenstein M. J. , Pearson Education
A Computer to write, compile and run your C/C++
programs
Windows/Linux Operating System
C/C++ Compiler (gcc)
Debugger (gdb)

Major Topics
• Algorithm Analysis
• ADTs
• Queues and Lists
• Stacks
• Trees
• Heaps / Priority Queues
• Binary Search Trees
• Hashing / Dictionaries
• Sorting
• Graphs and graph algorithms

Algorithm Analysis
What is Algorithm?
a clearly specified set of simple instructions to be followed to
solve a problem
• Takes a set of values, as input and
• produces a value, or set of values, as output
May be specified
• In English
• As a computer program
• As a pseudo-code
Program = algorithms + data structures

Need for Algorithm Analysis
Writing a working program is not good
enough
The program may be inefficient!
If the program is run on a large data set, then
the running time becomes an issue
Lets refer to previous example of Searching a Page
in a book

Example: Selection Problem
Given a list of N numbers, determine the kth
largest, where k  N.
Algorithm 1:
(1) Read N numbers into an array
(2) Sort the array in decreasing order by some
simple algorithm ( Bubble Sort)
(3) Return the element in position k

Example: Selection Problem…
Algorithm 2:
(1) Read the first k elements into an array and sort
them in decreasing order
(2) Each remaining element is read one by one
• If smaller than the kth element, then it is ignored
• Otherwise, it is placed in its correct spot in the array,
bumping one element out of the array.
(3) The element in the kth position is returned as
the answer.

Example: Selection Problem…
Which algorithm is better when
N =100 and k = 100?
N =100 and k = 1?
What happens when N = 1,000,000 and k =
500,000?
There exist better algorithms

Algorithm Analysis
We only analyze correct algorithms
An algorithm is correct
If, for every input instance, it halts with the correct output
Incorrect algorithms
Might not halt at all on some input instances
Might halt with other than the desired answer
Analyzing an algorithm
Predicting the resources that the algorithm requires
Resources include
• Memory
• Computational time (usually most important)
• Communication bandwidth

Algorithm Analysis…
Factors affecting the running time
computer
compiler
algorithm used
input to the algorithm
• The content of the input affects the running time
• typically, the input size (number of items in the input) is the
main consideration
– E.g. sorting problem  the number of items to be sorted
– E.g. multiply two matrices together  the total number of
elements in the two matrices
Machine model assumed
Instructions are executed one after another, with no
concurrent operations  Not parallel computers

Example


N
i
i
1
3
Calculate
Lines 1 and 4 count for one unit each
Line 3: executed N times, each time four units
Line 2: (1 for initialization, N+1 for all the tests, N for
all the increments) total 2N + 2
total cost: 6N + 4  O(N)
1
2
3
4
1
2
3
4
1
2N+2
4N
1

Worst- / average- / best-case
Worst-case running time of an algorithm
The longest running time for any input of size n
An upper bound on the running time for any input
 guarantee that the algorithm will never take longer
Example: Sort a set of numbers in increasing order; and the
data is in decreasing order
The worst case can occur fairly often
• E.g. in searching a database for a particular piece of
information
Best-case running time
sort a set of numbers in increasing order; and the data is
already in increasing order
Average-case running time
May be difficult to define what “average” means

Running-time of algorithms
Bounds are for the algorithms, rather than
programs
programs are just implementations of an
algorithm, and almost always the details of the
program do not affect the bounds
Bounds are for algorithms, rather than
problems
A problem can be solved with several algorithms,
some are more efficient than others

Asymptotic notation: Big-Oh
f(N) = O(g(N))
There are positive constants c and n0 such
that
f(N)  c g(N) when N  n0
The growth rate of f(N) is less than or equal
to the growth rate of g(N)
g(N) is an upper bound on f(N)

Big-Oh: example
Let f(N) = 2N2. Then
f(N) = O(N4)
f(N) = O(N3)
f(N) = O(N2) (best answer, asymptotically tight)
O(N2): reads “order N-squared” or “Big-Oh N-
squared”

Some rules
When considering the growth rate of a function using
Big-Oh :- Ignore the lower order terms and the
coefficients of the highest-order term
If T1(N) = O(f(N) and T2(N) = O(g(N)), then
T1(N) + T2(N) = max(O(f(N)), O(g(N))),
T1(N) * T2(N) = O(f(N) * g(N))

Big Oh: more examples
N2 / 2 – 3N = O(N2)
1 + 4N = O(N)
7N2 + 10N + 3 = O(N2) = O(N3)
log10 N = log2 N / log2 10 = O(log2 N) = O(log N)
sin N = O(1); 10 = O(1), 1010 = O(1)
log N + N = O(N)
N = O(2N), but 2N is not O(N)
210N is not O(2N)

Big-Omega
• f(N) = (g(N))
• There are positive constants c and n0 such
that
f(N)  c g(N) when N  n0
• The growth rate of f(N) is greater than or
equal to the growth rate of g(N).

Big-Omega: examples
Let f(N) = 2N2. Then
f(N) = (N)
f(N) = (N2) (best answer)

Big-Theta
f(N) = (g(N)) iff
f(N) = O(g(N)) and f(N) = (g(N))
The growth rate of f(N) equals the growth rate
of g(N)
Example: Let f(N)=N2 , g(N)=2N2
Since f(N) = O(g(N)) and f(N) = (g(N)),
thus f(N) = (g(N)).
Big-Theta means the bound is the tightest
possible.

Growth rates …
Doubling the input size
f(N) = c  f(2N) = f(N) = c
f(N) = log N  f(2N) = f(N) + log 2
f(N) = N  f(2N) = 2 f(N)
f(N) = N2  f(2N) = 4 f(N)
f(N) = N3  f(2N) = 8 f(N)
f(N) = 2N  f(2N) = f2(N)
Advantages of algorithm analysis
To eliminate bad algorithms early
pinpoints the bottlenecks, which are worth coding
carefully

General Rules
For loops
at most the running time of the statements inside
the for-loop (including tests) times the number of
iterations.
Nested for loops
the running time of the statement multiplied by the
product of the sizes of all the for-loops.
O(N2)

General rules (cont’d)
Consecutive statements
These just add
O(N) + O(N2) = O(N2)

Other Control Statements
while loop: Analyze like a for loop.
if statement: Take greater complexity of
then/else clauses.
switch statement: Take complexity of most
expensive case.
Subroutine call: Complexity of the subroutine.

Example
sum1 = 0;
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
sum1++;
O(n2)
Loop index depends on outer loop index
sum2 = 0;
for (i=1; i<=n; i++)
for (j=1; j<=i; j++)
sum2++;
N(N+1)/2 = O(n2)

Example
sum1 = 0;
for (k=1; k<=n; k*=2)
for (j=1; j<=n; j++)
sum1++;
O(nlogn)

Run-time for Recursive Algorithms
T(n) is defined recursively in terms of T(k),
k<n
The recurrence relations allow T(n) to be
“unwound” recursively into some base cases
(e.g., T(0) or T(1)).
Examples:
Factorial
Fibonacci

Example: Factorial (Recursive)
)
(
*
)
1
(
*
)
1
(
)
1
(
....
)
3
(
)
2
(
)
1
(
)
(
n
O
d
n
c
d
n
T
d
d
d
n
T
d
d
n
T
d
n
T
n
T




















int factorial (int n) {
if (n<=1) return 1;
else return n * factorial(n-1);
}

36
Example: Factorial (Iterative)
int factorial1(int n) {
if (n<=1) return 1;
else {
fact = 1;
for (k=2;k<=n;k++)
fact *= k;
return fact;
}
}
Both algorithms are O(n).
)
1
(
O
)
1
(
O
)
(n
O

Recursive v/s Iterative
Overhead required to manage the stack and
the relative slowness of function calls
Some types of problems whose solutions are
inherently recursive: Tree Traversal, Divide
and Conquer, Depth First Search etc.
Solution would be much simpler using recursion

Data Structure - Lecture 1 - Introduction.pdf

Recommended

More Related Content

Similar to Data Structure - Lecture 1 - Introduction.pdf (20)

Recently uploaded (20)

Data Structure - Lecture 1 - Introduction.pdf