DSA (1)
DSA (1)
Data
S t ru c tu r es
Lect-1-
Introduction to
Algorithms
P re s e nt ed B y : R a m a R a ni
R e se a rch S c hola r,
D e p t. of C om p ut e r S cie nce ,
P u nj a bi Un ive rs it y , P a t ia la .
Section-A
Analysis of Algorithms: In t r o d u c t io n t o A lg o r it h m s ,
th e ir d es ign an d a n aly s is , c o m p a r is o n o f a lg o r it h m s ,
co m p lex it y m ea s u r e in t er m s o f t im e a n d s p a c e.
Title:
INTRODUCTION
▶ A n a lg o r it h m is an e fficie n t m e th o d t h a t ca n b e e x p re s s e d
w it h in fin it e am o u n t o f Tim e an d s pa ce .
▶ Th e im po r ta n t as pe c t s o f alg o rit h m d e s ig n in c lu de cr e at in g
an e fficie n t alg o r it h m t o s olve a p ro b le m in an e fficie n t w ay
u s in g m in im u m t im e an d s p ace .
▶A w e ll-d e fin e d c om p u t at io n al pr oc e d u re t h at t ak e s
s om e valu e , o r s e t o f va lu e s , as an d p ro du c e s
s om e valu e , o r s e t o f va lu e s , as
▶W r it t e n in a ps e u do c od e w h ich c an b e im p le m e n t e d
in t h e la n g u ag e o f p r og r am m e r’ s ch o ice .
▶P s e u d o co de r e fe rs to an in fo rm al h ig h -le ve l de s cr ipt io n o f t h e
o pe r at in g pr in cip le o f a c om p u t e r p ro g ram o r alg o rit h m .
▶It u s e s s tr u ct u r al co n ve n t io n s o f a s t an d ar d p ro g r am m in g
lan g u a g e in t e n de d fo r h u m an r e ad in g r at h e r th a n t h e m ac h in e
re ad in g .
Advantages of Pseudocode
A lg o rit h m P r o g ram
1 .A t d e s ig n ph a s e 1 .A t Im ple m e n t a tio n ph a s e
2 .N a t u ral lan g u a g e 2 .w rit t e n in a n y
p ro g ra m m in g
lan g u a g e
3 .P e r s o n s h ou ld h ave 3 .P ro g ra m m e r
D o m ain k n ow le d g e
4 .A n alyz e 4 .Te s tin g
ALGORITHM SPECIFICATION
A lg o rit h m can b e d e s cr ib e d (R e p re s e n t) in fo u r w ay s .
1 .N at u ra l la n g u ag e lik e E n g lis h :
W h e n t h is w ay is c h oo s e s , c are s h o u ld b e t ak e n , w e
s h o u ld e n s u r e th a t e a ch & e ve r y s t a te m e n t is d e fin ite .
(n o a m b ig u it y)
2 . G ra ph ic re p re s e n ta t io n ca lle d flo w ch ar t :
T h is m e t h o d w ill w o rk w e ll w h e n th e alg o rit h m is s m all&
s im ple .
3 . P s e u d o -co d e M e th o d :
In th is m e t h o d, w e s h ou ld t yp ically de s cr ib e alg o rit h m s as
pr og r am ,w h ic h r e s e m b le s la n g u ag e lik e P as cal &
A lg o l(A lg o rit h m ic La n g u ag e ) .
4 .P ro g ram m in g L an g u ag e :
w e h ave t o u s e p ro g r am m in g la n g u ag e t o w rit e alg o rit h m s lik e
C , C + + ,JA V A e t c.
PSEUDO-CODE CONVENTIONS
1. C om m e n t s b e g in w ith // an d c o n tin u e u n t il t h e e n d o f lin e .
2. B lo ck s a re in d ic at e d w it h m at ch in g b ra ce s { a n d }.
3. A n id e n t ifie r b e g in s w ith a le t t e r. Th e d at a t yp e s o f varia b le s
ar e n ot e x p lic it ly de c lare d .
n o de = r e co rd
{
d at a t yp e 1 da t a 1 ;
d at a t y pe n da t a n ;
n od e * lin k ;
}
4 . Th e r e are t w o B oo le an va lu e s TRUE a n d FALSE.
L o g ica l O p e r at or s
AND, OR, NOT
R e lat io n al O p e rat o rs
<, <=,>,>=, =, !=
5 . A s s ig n m e n t of valu e s t o va riab le s is do n e u s in g t h e
as s ig n m e n t s t a te m e n t. <V a riab le > :=
<e x p re s s ion > ;
6 . C o m po u n d d at a t yp e s ca n b e fo r m e d w ith r e co rd s . H e re is an
e x am p le ,
N od e . R e co rd
{
d at a t yp e – 1 d at a-1 ;
.
.
.
d at a t yp e – n d at a – n ;
n o de * lin k ;
}
H e re lin k is a po in t e r t o t h e re c o rd t y pe n o de . In d iv id u al
da ta it e m s o f a re c o rd c an b e ac ce s s e d w it h a n d p e rio d.
7 . T he f ollow ing lo op ing s t a te m e nt s a re em p loy e d .
F o r, w h ile a n d re p e a t- un til W hile Loo p:
W hile < c ond it ion > d o
{
< s t at e m en t- 1 >
..
..
<s t a te m e nt -n >
}
For Loop:
F or v a ria ble : = va lu e- 1 t o v al ue -2 s t ep s t ep d o
{
< s ta t em e nt - 1 >
.
.
.
< s ta t em e nt - n>
}
repeat-until:
re pe a t
<s t at e m e n t -1 >
.
.
.
<s t at e m e n t -n >
u n t il<co n d itio n >
8 . A co n d itio n al s t a te m e n t h as th e fo llow in g fo rm s .
C as e
{
: < co n d it io n -1 > : < s t at e m e n t -1 >
.
.
.
: < co n d it io n -n > : <s t at e m e n t -n >
: e ls e : < s t at e m e n t -n + 1 >
}
9 . In p u t an d ou t p u t ar e do n e u s in g t h e in s t ru c tio n s
re ad & w r ite . N o fo rm a t is u s e d t o s pe cify t h e s iz e of
in p u t or ou t p u t q u a n tit ie s
1 0 . T h e re is o n ly on e t yp e o f p ro c e du r e : A lg or it h m , t h e
h e ad in g t a k e s t h e fo r m ,
A lg o rit h m N am e (P ara m e t e r lis t s )
co n s id e r a n e x am ple , t h e follo w in g a lg or ith m fie ld s &
re t u rn s th e m a x im u m o f n g ive n n u m b e r s :
1. alg o rit h m M ax (A ,n )
2. / / A is a n a rr ay o f s iz e n
3. {
4. R e s u lt := A [ 1 ] ;
5. fo r i:= 2 to n do
6. if A [ i] > R e s u lt t h e n
7. R e s u lt := A [ i] ;
8. re tu r n R e s u lt;
9. }
Issue in the study of algorithm
PRIORI POSTERIORI
1 .D o n e pr ior i t o r u n a lg or it h m 1 .A n alys is a ft e r ru n n in g on a
s pe cific s y s t e m it o n s ys t e m .
2 .Har dw a re in de p e n d e n t 2 .D e p e n de n t o n h a rd w are
3 .Ap p ro x im at e an aly s is 3 .A ct u al s t a tis tic s o f a n
alg o rit h m
4 .D e pe n d e n t on n o of t im e 4 .T h e y d o n o t d o p o s t e rio ri
s ta te m e n ts a re e x e c u te d an aly s is
Problem: Suppose there are 60 students in the class.
How will you calculate the number of absentees in the
class?
P s e u do A p p ro ac h
1 .In it ializ e a va riab le ca lle d as Count t o z e r o, absent t o
z e ro , total to 6 0
2 .FO R E A C H S t u d e n t P R E S E N T DO the follo w in g :
In c re a s e t h e Count b y O n e
3 .Th e n S u b t r act Count fr om total an d s t or e the re s u lt
in absent
4 .D is pla y t h e n u m b e r of a b s e n t s t u de n t s
Problem: Suppose there are 60 students in the class.
How will you calculate the number of absentees in the
class?
A lg o rit h m ic A p p ro ac h :
1 .C o u n t < - 0 , a b s e n t <- 0 , t ot a l < - 6 0
2 .R E P E A T till all s t u d e n ts co u n t e d
Count <- Count + 1
3 .ab s e n t <- t o t al - C o u n t
4 .P rin t " N u m b e r a b s e n t is :" , a b s e n t
Need of Algorithm
1 . T o u n d e rs ta n d t h e b as ic id e a o f t h e pr ob le m .
2 . T o fin d an ap pr o ach t o s olve t h e p ro b le m .
3 . T o im p ro ve t h e e fficie n cy of e x is t in g t e ch n iq u e s .
4 . To u n d e r s t an d t h e b a s ic pr in cip le s o f d e s ig n in g t h e
alg or it h m s . T o c o m pa re t h e p e rfo r m an ce o f t h e a lg or ith m
w it h re s pe c t t o ot h e r t e c h n iq u e s .
6 . It is t h e b e s t m e t h od of d e s c ript io n w it h o u t d e s c rib in g t h e
im p le m e n t at io n d e t ail.
7 . Th e A lg o r ith m g ive s a c le ar d e s c rip tio n o f re q u ir e m e n ts a n d
g oa l o f t h e p r ob le m t o t h e de s ig n e r.
8 . A g oo d de s ig n c an p ro du c e a g oo d s o lu tio n .
9 . T o u n d e rs t an d t h e flo w o f t h e p r ob le m .
Algorithm Design Techniques
▶ An alg o rit h m d e s ig n t e c h n iq u e (o r “ s tr at e g y” or
“ p ara dig m ” ) is a g e n e r al ap p ro ac h to s o lvin g p ro b le m s
alg o rit h m ica lly t h at is a pp licab le to a va rie t y of p ro b le m s
fr om d if fe re n t ar e as o f co m pu t in g .
▶ Learning these techniques is of utmost importance for the
following reasons.
▶ First, they p ro vid e g u id an c e fo r d e s ig n in g alg o rit h m s fo r
n e w pr ob le m s , i.e ., pr ob le m s fo r w h ich t h e re is n o k n o w n
s a t is fac to r y alg o rit h m .
▶ Second, algorithms are t h e c or n e rs to n e o f co m p u te r
s c ie n ce . A lg or it h m d e s ig n t e c h n iq u e s m a k e it p os s ib le t o
cla s s ify alg o rit h m s ac co rd in g t o an u n d e r ly in g d e s ig n
id e a; t h e r e fo re , th e y c an s e r ve as a n a tu r al w ay t o b o t h
ca t e g or iz e an d s tu d y a lg o r ith m s .
Algorithm Design Techniques
▶ P o pu lar A lg o rit h m D e s ig n Te ch n iq u e s
▶ D ivide -a n d- C o n q u e r A pp ro ac h
▶ G re e d y Te c h n iq u e s
▶ D yn a m ic P r o g ram m in g
▶ B r an ch a n d B o u n d
▶ B a ck t ra ck in g A lg o rit h m
▶ R an d o m iz e d A lg o rit h m
Asymptotic Analysis (Growth of Function)
▶ A s ym p t ot ic A n alys is
▶ W h y is A s y m pt o tic A n aly s is Im p or t an t ? A s y m pt o t ic
N o t at ion
▶ B ig O N o t at ion
▶ B ig O m e g a N o t at io n
▶ B ig Th e t a N o t at io n
Brute Force
Description
Br ut e f orc e is a s t ra ig ht f o rw a rd a lg o rit hm ic a p p ro a ch that
sy s t e m a tic a lly e nu m e ra t e s a ll p o s s ib le s olu tio n s to f ind t he co rr ec t
on e . It in v olv e s tr y ing a ll com b in a tio n s w it ho u t us in g a n y o p t im iz a t ion s .
Example Algorithms
▶ Subset Sum Problem
▶ Traveling Salesman Problem (TSP)
▶ String Matching (Naive Approach)
Strengths
▶ Guaranteed Solution: A lw ays fin d s t h e co rr e ct s o lu t ion if o n e
e x is t s .
▶ Simplicity: E a s y t o u n d e rs t an d a n d im p le m e n t.
Weaknesses
▶ Inefficient for Large Inputs: Its t im e co m p le xit y of t e n gr ow s
e xp on e nt ia lly w ith t h e s iz e of t h e in p u t.
▶ Resource-Intensive: C a n be s lo w a nd con s u m e s ign if ica n t
co m p ut a t ion a l r e s ou rce s .
Divide-and-Conquer
▶ Description: B r e ak s a pr ob le m in to s m alle r s u b p ro b le m s ,
s o lve s e a ch re c u rs ive ly, a n d c om b in e s t h e re s u lt s .
▶ Description: S o lv e s p r ob le m s b y b re a k in g t h e m in t o
o ve rla pp in g s u b pr ob le m s a n d s t o rin g s o lu tio n s t o av oid
re co m pu t at io n .
▶ Description: U s e s r an d om n u m b e r s t o m ak e d e cis io n s w it h in
t h e alg o rit h m , in t ro d u cin g ran d o m n e s s t o s o lve pr ob le m s .
Example Algorithm
▶ S e ar ch in g for a s p e cific n u m b e r in a n u n s or t e d lis t .
Strengths
▶ S im plicit y: E a s y t o im p le m e n t a n d u n d e rs ta n d.
▶ N o P r e pr o ce s s in g R e q u ire d : W or k s d ire c tly on u n s or t e d
da ta .
Weaknesses
▶ S low for L a rg e In pu t s : H as a t im e c om p le x it y of
𝑂 (𝑛 )O (n ), as it m ay n e e d t o c h e ck e ve r y e le m e n t .
▶ In e fficie n t : N o t s u it ab le fo r s e a rc h in g in larg e d at as e t s .
PERFORMANCE ANALYSIS
Performance Analysis: A n alg or it h m is s aid to b e e fficie n t an d fas t
if it t a k e le s s tim e t o e x e c u te an d co n s u m e s le s s m e m or y s p ace a t
ru n t im e is c alle d P e r for m an c e A n aly s is .
1. SPACE COMPLEXITY:
Th e s p ac e co m p le x ity o f an a lg o r it h m is th e am o u n t of
M e m or y S p ac e re q u ire d b y an alg o rit h m d u rin g c ou r s e of
e x e cu t io n is calle d s p ac e co m p le x it y .T h e re ar e t h re e t yp e s of
s p ace
a) In s t ru c tio n s p ace :e x e c u ta b le p r og r am
b ) D a t a s p ac e : R e q u ire d t o s t or e all t h e c o n s t an t an d va riab le
da ta s pa ce .
c) E n viro n m e n t : It is re q u ir e d t o s t o re e n viro n m e n t in fo r m at ion
n e e de d t o r e s u m e t h e s u s p e n de d s p ac e .
2. TIME COMPLEXITY:
Th e tim e c om p le x it y of a n a lg or ith m is t h e to t al am o u n t of
tim e re q u ir e d b y a n alg o rit h m t o co m ple t e it s e x e cu t io n .
S p ac e c o m p le x it y
N o w th e r e are t w o t y pe s o f s p ac e co m ple x it y
a) C o n s t an t s p ac e co m ple x it y
b ) L in e ar (v aria b le )s p ac e co m ple x it y
1 .C o n s t an t s pa ce co m ple x it y: A fix e d a m ou n t o f s pa ce for a ll
th e in p u t va lu e s .
E x am p le : in t s q u ar e (in t a)
{
re t u rn a* a;
}
H e re alg o rit h m re q u ir e s fix e d a m ou n t o f s p ace fo r all t h e in p u t
valu e s .
2 .L in e ar s pa ce co m p le x ity : T h e s pa ce n e e d e d fo r a lg or ith m is
b as e d o n s iz e .
▶S iz e o f t h e va riab le ‘ n ’ = 1 w o r d
▶A rra y o f a valu e s = n word
▶L oo p varia b le = 1 w o rd
▶S u m var iab le = 1 w o rd
E x a m ple :
in t s u m (in t A [ ] ,in t n )
{ n
in t s u m = 0 ,i; 1
fo r (i= 0 ;i<n ;i+ + ) 1
S u m = s u m + A [ i] ; 1
R e t u rn s u m ;
} A n s : 1 + n + 1 + 1 = n + 3 w o rd s
E x a m ple s :
1 .A lg o r it h m s u m (a,,b ,c )
{
a= 1 0 ; a-1
b=2 0; b -1
c= a+ b ; c -1
}
s (p )= c + s p
3+0 =3
0 (n )= 3
2 . alg o rit h m s u m (a,n )
{
t o ta l-= 0 ; -1
F or i= 1 t o n do -1 ,1
To t al= t o t al+ a[ i] -- n
R e t u rn t ot a l
DAA
E x am p le : in t s u m (in t a , in t b )
{
re t u rn a+ b ;
}
2 .Lin e a r t im e c o m ple x it y: If t h e in p u t va lu e s ar e in c re a s e d t h e n
th e t im e co m ple x it y w ill c h an g e s .
▶ c om m e n t s = 0 s t e p
▶A s s ig n m e n t s ta t e m e n t = 1 s t e p
▶co n dit io n s t at e m e n t = 1 s t e p
▶ lo o p c on d it ion fo r n t im e s = n + 1 s t e p s
▶ b o dy o f t h e lo o p = n s t e p s
E x a m ple : in t s u m (in t A [ ] ,in t n )
{
in t s u m = 0 ,i;
fo r (i= 0 ;i<n ;i+ + )
s u m = s u m + A [ i] ;
re tu r n s u m ;
co s t re pe t at io n t o ta l
1 1 1
1+1 +1 1 + (n + 1 )+ n 2n+2
2 n 2n
1 1 1
4n+4
TIME COMPLEXITY
Th e t im e T(p) ta k e n b y a p ro g ra m P is t h e s u m of t h e co m p ile t im e
an d t h e r u n tim e (e x e c u t io n t im e )
1 . A lg o rit h m S u m (a,n ) 0 - 0
2.
{ 0 - 0
3. S = 0 .0 ; 1 1 1
4. for i= 1 to n do 1 n+1 n+1
5. s = s + a [ I] ; 1 n n
6. re t u rn s ; 1 1 1
7.} 0 - 0
2n+3
Types of Time Complexity Analysis
W e h ave t h re e t y pe s of a n alys is re la te d t o t im e co m p le x ity ,
w h ic h a re :
1. Worst-case time complexity: F o r 'n ' in p u t s iz e , t h e w o rs t -c as e
tim e c om p le x it y ca n b e de fin e d a s t h e m ax im u m am o u n t of
tim e n e e d e d b y an alg o rit h m to co m p le te it s e x e cu t io n . Th u s ,
it is n o t h in g b u t a fu n ct io n d e fin e d b y t h e m ax im u m n u m b e r
of s t e p s pe rfo rm e d on an in s t an c e h a vin g a n in p u t s iz e o f n .
C o m pu t e r S cie n t is t s ar e m or e in te r e s t e d in t h is .
▶ Th e co m ple x it y ca n b e fo u n d in an y fo rm s u ch as co n s t an t ,
lo g arit h m ic, lin e ar, n * log (n ) , q u ad ra t ic , c u b ic, e x po n e n t ial,
e t c. It is n o t h in g b u t t h e or d e r o f c o n s t an t , lo g ar ith m ic , lin e ar
an d s o on , t h e n u m b e r o f s t e p s e n co u n t e re d fo r t h e co m ple t io n
o f a pa rt icu lar a lg o r it h m . To m a k e it e ve n m or e p re c is e , w e
o ft e n c all t h e c o m ple x it y o f a n alg o rit h m a s "r u n n in g t im e " .
Typical Complexities of an Algorithm
W e t ak e a lo ok a t t h e diffe re n t ty pe s of c om p le x it ie s of a n
alg o rit h m an d on e o r m o re o f ou r a lg or it h m o r p ro g ra m w ill fall
in t o a n y of t h e fo llo w in g cat e g o rie s ;
▶ Constant Complexity
▶ Im p os e s a co m p le x ity o f O (1 ) . It u n d e r g oe s an e x e cu t io n o f a
co n s ta n t n u m b e r o f s t e p s lik e 1 , 5 , 1 0 , e tc . fo r s olvin g a
g ive n pr o b le m . Th e c ou n t o f o p e ra tio n s is in d e pe n de n t o f t h e
in p u t d at a s iz e .
▶ Logarithmic Complexity
▶ Im p os e s a co m p le x ity o f O (log (N ) ). It u n d e rg o e s t h e
e x e c u tio n of t h e o rd e r o f lo g (N ) s te p s . To pe r fo rm
o pe r at io n s on N e le m e n t s , it o ft e n t ak e s t h e log a rit h m ic
b as e as 2 .
▶ F or N = 1 ,0 0 0 ,0 0 0 , a n a lg or it h m t h at h as a co m ple x it y o f
O (lo g (N )) w o u ld u n de rg o 2 0 s t e p s (w it h a c on s t an t
p re c is ion ) . H e r e , t h e log a rit h m ic b as e d oe s n o t h o ld a
n e c e s s a ry co n s e q u e n ce fo r t h e o pe rat io n co u n t o rd e r , s o it
▶ Linear Complexity
▶ Im p os e s a co m p le x it y of O (N ) . It e n co m pa s s e s t h e s a m e
n u m b e r o f s t e p s as t h a t o f t h e t o t al n u m b e r o f e le m e n ts
t o im p le m e n t a n o p e ra tio n o n N e le m e n t s .
▶ F or e x a m ple , if t h e r e e x is t 5 0 0 e le m e n t s , th e n it w ill
t ak e ab o u t 5 0 0 s t e ps . B as ic ally, in lin e a r c om p le x it y, t h e
n u m b e r of e le m e n t s lin e a rly d e p e n d s o n t h e n u m b e r o f
s t e p s . F o r e x a m ple , t h e n u m b e r o f s t e p s for N e le m e n ts
ca n b e N / 2 o r 3 * N .
▶ Quadratic Complexity
▶ It im p o s e s a c o m ple x it y o f O (n 2 ).
▶ F or N in p u t d at a s iz e , it u n de r g oe s th e o rd e r o f N 2 c o u n t o f
o pe r at io n s on N n u m b e r of e le m e n t s for s o lvin g a g ive n
p ro b le m . If N = 1 0 0 , it w ill e n d u re 1 0 ,0 0 0 s t e p s .
▶ In ot h e r w o rd s , w h e n e ve r t h e o rd e r o f op e ra t io n t e n ds t o
h ave a q u ad ra t ic re lat io n w it h t h e in pu t da t a s iz e , it
re s u lt s in q u a dr at ic co m p le x ity . F o r e x am p le , fo r N n u m b e r
o f e le m e n t s , t h e s te ps ar e fou n d to b e in th e o rd e r o f
3*N2 /2.
▶ Cubic Complexity
▶ It im p o s e s a co m p le x ity o f O (n 3 ). Fo r N in pu t da t a s iz e , it
e x e c u te s t h e o rd e r o f N 3 s t e p s o n N e le m e n t s t o s o lve a
g ive n p ro b le m . F o r e x a m p le , if th e re e x is t 1 0 0 e le m e n t s , it
is g o in g t o e x e cu t e 1 ,0 0 0 ,0 0 0 s te p s .
▶ Exponential Complexity
▶ It im p o s e s a co m p le x ity o f O (2 n ) , O (N !) , O (n k ) , … . F o r N
e le m e n t s , it w ill e x e c u t e t h e o rd e r o f co u n t o f o pe r at ion s
t h at is e x po n e n t ially d e pe n da b le o n th e in p u t d at a s iz e . F or
e x am p le , if N = 1 0 , t h e n t h e e x p on e n tia l fu n ct io n 2 N w ill
re s u lt in 1 0 2 4 .
KINDS OF ANALYSIS
1.Worst-case: (u s u ally)
• T(n ) = m a x im u m t im e o f a lg or ith m o n an y in p u t o f
s iz e n .
2.Average-case: (s o m e t im e s )
• T(n ) = e x p e c te d t im e of a lg or ith m o ve r a ll in p u t s o f
s iz e n .
• N e e d as s u m pt io n o f s t a tis tic al dis tr ib u tio n o f in p u t s .
3.Best-case:
• T(n ) = m in im u m t im e o f alg or it h m on a n y in pu t o f
s iz e n .
COMPLEXITY:
C om p le x it y re fe r s t o t h e ra te a t w h ich t h e s t o rag e t im e g ro w s as
a fu n c t io n o f t h e p ro b le m s iz e
Analysis of an Algorithm
D e f in it io n : It is t h e w a y to de s cr ib e t h e b e h avio r o f fu n ct ion s in
th e lim it o r w it h ou t b o u n ds .
A s y m pt o t ic g ro w t h : Th e r at e at w h ic h th e fu n c tio n g ro w s …
“ g ro w th r at e ” is t h e co m ple x it y o f t h e fu n ct io n or th e a m o u n t
of r e s o u rc e it t ak e s u p to co m p u te .
Growth rate Time +memory
Classification of growth
1 .G ro w in g w it h t h e s a m e rat e .
2 . G r o w in g w it h t h e s lo w e r r at e .
3 .G ro w in g w it h th e fa s t e r r at e .
Th e y are 3 a s y m p t o tic n ot at io n s ar e m os tly u s e d t o re p re s e n t
t im e c o m ple x it y o f alg o rit h m .
1 .B ig o h (O ) n ot a tio n
2 .B ig o m e g a (Ω ) n o t at ion
3 .T h e ta (Θ ) n ot at io n
4 .L it tle o h n ot at io n
5 .L it tle o m e g a(Ω ) n o t at io n
1.Big oh (O)notation
1 .B ig o h (O )n o t at io n : A s ym p t o tic “ le s s t h an ” (s lo w e r
rat e ) .T h is n o t at io n m a in ly r e pr e s e n t u pp e r b o u n d o f a lg or ith m
ru n t im e .
B ig o h (O )n o t at io n is u s e fu l t o calc u lat e m ax im u m a m o u n t of
tim e o f e x e cu t io n .
B y u s in g B ig -o h n o ta t io n w e h ave t o c alc u la t e w o r s t cas e tim e
co m ple x it y.
Fo rm u la : f(n )<= c g (n ) n > = n 0 , c >0 ,n 0 > = 1
E x a m ple : f (n ) = 3 n + 2
F or m u la : f(n ) >= c g (n ) n > = n 0 , c> 0 ,n 0 >= 1
f(n )= 3 n + 2
3 n + 2 >= 1 * n , c= 1 p u t t h e valu e o f n = 1
n=1 5 > = 1 t ru e n 0 >= 1 fo r a ll va lu e o f n
It m e an s t h a t f(n ) = Ω g (n ).
3. -Theta notation
Th e ta (Θ ) n o t at aio n : A s ym p t ot ic “ E q u alit y” (s am e r at e ).
It re p r e s e n t ave r ag e b on d o f a lg or it h m r u n n in g t im e .
B y u s in g t h e ta n o ta tio n w e ca n ca lc u lat e ave r ag e a m ou n t o f
t im e . S o it c alle d a ve ra g e cas e tim e c om p le x ity of a lg o r it h m .
For m u la : c 1 g (n ) <= f(n ) <= c 2 g (n )
w h e re c is co n s t an t , n is fu n ct io n
A ve r ag e b o u n d
-Thetanotation
E x a m ple : f(n )= 3 n + 2
F or m u la : c 1 g (n ) <= f(n ) <= c 2 g (n )
f(n )= 2 n + 3
1 * n <= 3 n + 2 < = 4 * n n ow p u t t h e valu e o f n = 1
w e g e t 1 < = 5 < = 4 fals e
n = 2 w e g e t 2 < = 8 < = 8 t ru e
n = 3 w e g e t 3 <= 1 1 <= 1 2 t ru e
N o w all va lu e of n > = 2 it is t r u e a b ov e c o n dit ion is s at is fie d .
4 .L ittle oh n o tatio n
• L itt le o n ot a tio n is u s e d to de s cr ib e an u pp e r b o u n d t h at
ca n n o t b e t ig h t. In ot h e r w o rd s , loo s e u p p e r b o u n d o f f(n ) .
S lo w e r g ro w t h r at e
f(n ) g r ow s s lo w e r t h an g (n )
• L e t f(n ) a n d g (n ) ar e t h e fu n ct io n s t h at m a p p os itive re al
n u m b e rs . W e c an s a y t h at t h e fu n c tio n f(n ) is o (g (n ) ) if for
an y re al p os it ive c o n s t an t c, th e r e e x is t s an in t e g e r c on s t an t
n 0 ≤ 1 s u c h th a t f(n ) > 0 .
U s in g m a th e m a t ic al r e la t ion , w e ca n s ay t h a t f(n ) = o(g (n ))
m e an s ,
if
1 .If f(n ) = n 2 an d g (n ) = n 3 t h e n c h e ck w h e t h e r
f(n ) = o (g (n ) ) o r n o t.
S o l:
Th e re s u lt is 0 , a n d it s at is fie s t h e e q u at io n m e n t io n e d
ab o ve . S o w e can s a y t h at f(n ) = o(g (n )) .
5 .L it t le o m e ga (ω ) n o t a t io n
• A n ot h e r a s ym p t o tic n o t at io n is lit t le om e g a n o t at io n . it is
d e n ot e d b y (ω ).
• L itt le o m e g a (ω ) n o t at ion is u s e d t o d e s c rib e a lo os e lo w e r
b o u n d o f f(n ).
• F as t e r g r o w t h ra t e
• F (n ) g ro w s fas te r th a n g (n )
∞
• If
∞
F or m ally s t a te d as f(n ) = ω g (n )
D ata
S t ru ctu re s
G S Lehal
1
D at a S t ru ctu res
A data structure is a scheme
for organizing data in the
memory of a computer.
Bi nar y Tr e e
Que ue
2
D at a S t ru ctu res
The way in which the data is organized affects the
performance of a program for different tasks.
The choice of data structure and algorithm can
make the difference between a program running in
a few seconds or many days.
3
Selecting a Data Structure
Select a data structure as follows:
1. Analyze the problem to determine the resource constraints
a solution must meet.
2. Determine the basic operations that must be supported.
Quantify the resource constraints for each operation.
3. Select the data structure that best meets these
requirements.
4
Data Structure Philosophy
Each data structure has costs and benefits.
Rarely is one data structure better than another in all
situations.
A data structure requires:
■ space for each data item it stores,
■ time to perform each basic operation,
■ programming effort.
5
Data Structure Philosophy (cont)
Each problem has constraints on available space and time.
Only after a careful analysis of problem characteristics can we
know the best data structure for the task.
Bank example:
■ Start account: a few minutes
■ Transactions: a few seconds
■ Close account: overnight
6
C o m m o n o p e ra tio n s D ata
S tru ctu re s
■ Add
■ Change
■ Delete
■ Traverse
■ Search
7
D at a S t ru ctu re s
■ Example:library
■ is composed of elements (books)
■ Accessing a particular book requires
knowledge of the arrangement of the
books
■ Users access books only through the
librarian
C o m m o n D ata S tru ct u re s
■ Array
■ Stack
■ Queue
■ Linked List
■ Tree
■ Heap
■ Hash Table
■ Priority Queue
9
A rrays
i nt ma r ks 1, mar ks 2, mar k s 3, . . .
i nt mar ks [22]; / * ho l ds 22 i n t e ge r s * /
10
A rrays
v al ue 0 v a l ue 1 v al ue 2 v al ue 3 v al u e 4 . . .
11
A rrays
mar ks
223 256 312 299 329 ...
0 1 2 3 4
12
Th e A rray d at a s tru ctu re
■ An array is an indexed sequence of
components stored in contiguous memory
locations.
■ The length of the array is determined when the array is created, and
cannot be changed
■ Each component of the array has a fixed, unique index
■ Indices range from a lower bound to an upper bound
■ Any component of the array can be
inspected or updated by using its index
■ This is an efficient operation: O(1) = constant time
13
A rray variatio n s
■ The array indices may be integers (C, Java) or other
discrete data types (Pascal, Ada)
14
14
A rrays in C / C ++
15
15
A rrays
i nt ma r k s [5] ;
mar ks ( Al l bo x
c o n t e n t s ar e
i ni t i al l y
u nde f i ne d)
16
A rrays
e l e me n t _ t ype a r r a y_ n a me [s i ze ];
17
A rrays
■ Array indexing:
■ Each element of the array is like an ordinary variable.
■ In order to access that element, you must specify the name of the
array and the index of the element:
a r r a y _ na me [i n de x]
■ Example:
■ Array indexing:
19
A rrays
20
A rray in itializatio n
■ You can initialize some of the elements with non zero values, while
the rest will automatically be initialized to zero.
i nt mar ks [5] = { 219, 320} ;
Equivalent to:
mar ks [0] = 219; mar ks [1]= 320; mar ks [2] = mar ks [3]
= mar ks [4] = 0;
• You can let the compiler figure the size for you:
i nt mar ks [] = { 219, 320, 199, 256, 311} ;
This will set the size of array mar ks equal to the number of entries in
the RHS bracket
21
A rray in itializatio n
22
A rray in itializatio n
i nt mar ks [5];
f o r ( i = 0; i < 5; i + + ) {
pr i nt f ( " Type t o t al ma r ks : " ) ;
s c anf ( " %d" , &mar ks [i ]) ;
}
23
A rray in itializatio n
i nt s c o r e s [5] = { 0} ;
i n t s c o r e s [5] = { 0, 0, 0, 0, 0} ;
24
A rray p it falls
s c o r e s = 18; E R RO R!
i nt s i ze ;
i nt s c o r e s [s i ze ]; E R RO R!
pr i nt f ( " Type ar r ay s i ze : " ) ;
s c anf ( " %d" , &s i ze ) ;
25
A rray p it falls
i nt s c o r e s [5];
s c o r e s [- 1] = 18; CA REFUL !
■ T he pr oce s s to d e te rm in e t he a d d re s s in a m em ory :
a ) Fir st a d d re s s – ba s e a d d res s .
b ) R e la t ive a dd re s s t o ba s e a dd r es s t hro ugh ind e x f unct ion.
E xam p le: ch a r X [ 1 00 ] ;
L et u s es 1 loca t ion s tora ge .
If t he b as e a d d re s s is 1 2 0 0 t he n t he n ex t e lem e nt is in 1 20 1 .
Ind e x F un ct ion is w ritt e n as :
Loc (X[i]) = Loc(X[0]) + i , is s u bs crip t a nd LB = 0
12 0 0 12 0 1 1 2 02 12 0 3
] ] ]
R e p r e s e n ta tio n o f A rra y in a M e m o ry
■ In g e n e ra l, in d e x fu n ct ion :
L o c (X [ i] ) = L o c(X [ 0 ] ) + w *(i);
w h e r e w is le n g t h o f m e m o ry lo ca t ion r eq u ire d .
F o r re al n u m b e r: 4 b yt e , in t e g e r: 2 b yt e an d c h ara ct e r : 1
b yt e .
■ E x a m p le :
If L o c(X [ 0 ] ) = 1 2 0 0 , a n d w = 4 , fin d L oc (X[ 8 ] ) ?
L o c(X [ 8 ] ) = L o c(X [ 0 ] ) + 4 * (8 )
= 1232
28
In s e rtio n in to A rray
29
In s e rt in g a n e le m e n t in to an array
■ Suppose we want to insert the value 8 into this sorted array
(while keeping the array sorted)
1 3 3 7 12 14 17 19 22 30
■ We can do this by shifting all the elements after the mark
right by one location
■ Of course, we have to discard the 30 when we do this
1 3 3 7 8 12 14 17 19 22
• M o vin g a ll t h o s e elem en t s m a k es t h is a
o p er a t io n (lin ea r in t h e s iz e o f t h e a rr a y)
30
In s ert in g an e lem e n t in t o an
■
array
Suppose we want to insert the value 8 into this sorted array
(while keeping the array sorted)
1 3 3 7 12 14 17 19 22 30
■ We can do this by shifting all the elements after the mark
right by one location
■ Of course, we have to discard the 30 when we do this
1 3 3 7 8 12 14 17 19 22
• M o vin g a ll t h o s e elem en t s m a k es t h is a
o p er a t io n (lin ea r in t h e s iz e o f t h e a rr a y)
31
R e m o val f ro m A rrays
32
D ele t in g an e le m e n t f ro m an array
■ Deleting an element is similar--again, we have to
move all the elements after it
1 3 3 7 8 12 14 17 19 22
1 3 3 7 12 14 17 19 22 ?
34 4
P a s s in g A rra ys a s P a ra m e te rs in
F u n ctio n s
34
S m alle s t V alu e
■ Problem
■ Find the smallest value in an array of integers
■ Input
■ An array of integers and a value indicating the
number of integers
■ Output
■ Smallest value in the array
■ Note
■ Array remains unchanged after finding the
smallest value!
35
P a s s in g A n A rray
No t i c e e mpt y br ac ke t s
i nt L i s t Mi ni mum( i n t Ar [], i n t as i z e ) {
i nt Smal l e s t Val ue So Far = Ar [0];
f o r ( i nt i = 1; i < as i ze ; + + i ) { Co ul d we
i f ( Ar [i ] < Smal l e s t Val ue So Far ) {
jus t
Sma l l e s t Val ue So Far = Ar [i ]; as s i gn a
} 0 and
} h av e i t
r e t ur n Smal l e s t Val ue So Far ; wo r k?
}
36
R e ver s e fu n c tion
v o i d r e v e r s e ( i n t l i s t [], i nt n e wL i s t [], i n t s i ze )
{
f o r ( i n t i = 0, j = s i z e - 1; i < s i z e ; i + + , j - - )
{
ne wL i s t [j] = l i s t [i ];
}
}
i nt mai n( ) {
i nt l i s t 1[5] = { 1, 2 , 4, 5, 6} ;
i nt l i s t 2[5] ; r e v e r s e ( l i s t 1, l i s t 2 , 5) ;
r e t ur n 0;
}
1 2 3 4 5
lis
6
t
n e w Lis t 6 5 4 3 2 1
37
A f ew e xe rcis es on arrays
38
2 D A rrays
■ Arrays are not limited to type int; we can have arrays of...
any other type including float, char , double and even
arrays !!!
■ An array of arrays is called a two-dimensional array and
can be regarded as a table with a number of rows and
columns:
39
2 D A rrays
50 50 50 60
i s a 5 × 4 ar r ay : 80 80 88 72
5 r o ws , 4 c o l umns 49 50 40 60
60 60 60 76
75 80 75 81
40
D e clarin g 2 D A rrays
50 50 50 60
80 80 88 72
49 50 40 60
60 60 60 76
75 80 75 81
■ This is
an array of size 5 marks[5
whose elements are arrays of size 4 ]
whose elements are integer [4]
■ Declare it like this: in
i nt ma r k s [5] [4];
t
i nt mar ks [5][4] = {
{ 50, 50, 50, 60} ,
{ 80, 80, 88, 72} ,
{ 49, 50, 40, 60} ,
{ 60, 60, 60, 76} ,
{ 75, 80, 75, 81}
};
42
In it ializin g 2 D arrays
#de f i ne ROWS 5
#de f i ne COL S 4
i nt mai n ( ) {
i nt i , j;
i nt mar ks [5][4];
/* i ni t i al i ze al l e l e me nt s t o ze r o e s : * /
f o r ( i = 0; i < ROWS; i + + )
f o r ( j= 0; j< COL S; j+ + ) mar ks [i ][j]
= 0;
}
43
In it ializin g 2 D arrays
#de f i ne ROWS 5
#de f i ne COL S 4
i nt mai n ( ) {
i nt i , j;
i nt mar ks [5][4];
f o r ( i = 0; i < ROWS; i + + )
f o r ( j= 0; j< COL S; j+ + )
s c anf ( " %d" , &mar ks [i ][j]) ;
}
44
In it ializin g 2 D arrays
45
U s in g 2 D arrays
46
Storing Two-Dimensional Array in memory
■ A t w o -d im en s io n al a r r a y c a n be s t o r ed in
t h e m em o ry in t w o w ay s :
■ R ow - m a jo r im ple m e nt at ion
■ C olum n-m a j or im p le m e nt at ion
47
Row-major Implementation
■ R o w -m a j o r im p le m en t a t io n is a lin ea r iz a tio n
t e ch n iq u e in w h ic h elem e n t s o f a r r a y a re s t o re d
r o w -w ise t h a t m ea n s t h e co m p let e fir s t ro w is
s t o r ed t h e n t h e c o m p let e s ec o n d ro w is s t o re d
and so on.
■ F o r ex a m p le an a r r a y [ 3 ] [ 3 ] is s t o re d in th e
m e m o r y a s s h o w be llo w :
Ro w Ro w Ro w 4
1 2 3
9
Row-major Implementation
■ T h e s t o ra g e c an b e c le arly u n de r s t o od b y a rr an g in g
a rray a s m a tr ix as s h o w b e llow :
49
Row-major Implementation
■ In c o lu m n m a j o r im p lem en t a tio n m em o r y
a llo ca t io n is d o n e c o lu m n b y c o lu m n t h a t
m e an s f ir s t t h e ele m en t s o f t h e c o m p let e fir s t
c o lu m n is s t o r ed t h en ele m en t s o f c o m p let e
s e co n d co lu m n is s t o r ed a n d s o o n .
■ F o r ex a m p le an a r r a y [ 3 ] [ 3 ] is s t o re d in th e
m e m o r y a s s h o w be llo w :
Co l Co l Co l 5
1 2 3
3
Column-major Implementation
■ T h e s t o ra g e c an b e c le arly u n de r s t o od b y a rr an g in g
a rray a s m a tr ix as s h o w b e llow :
Co l 1Co l 2Co l 3
53
Column-major Implementation
i s a po i nt e r o nl y t o t he f i r s t
e l e me nt — no t t h e who l e ar r ay
56
A rr a y s a n d
P■ oIfin
a istters
h e n am e o f a n ar ra y, t h e e xp re s s io n
a is eq uiv a le nt to & a [ 0 ]. a [ 0 ] is eq uiv a len t to *a .
a [ i ] is e quiv a le nt t o *(a + i).
57
A rr a ys an d P o in te rs
T o ac c e s s an ar r ay, a ny po i nt e r t o
t he f i r s t e l e me nt c an be us e d
i ns t e a d o f t he name o f t he a r r ay.
58
A rray E xa m p le U s in g a P o in te r
in t x[ 4 ] = {1 2 , 2 0 , 3 9 , 4 3 },
*y
y =; & x[ 0] ; / / y p oin ts t o t he be ginn ing o f t he
p rin tf ( "% d \n ", x[ 0 ]); a rra y
p rin tf ( "% d \n ", *y ); / / out p ut s 1 2
p rin tf ( "% d \n ", *y + 1 ); / / als o ou tp u ts 1 2
p rin tf ( "% d \n ", / / out p ut s 1 3 ( 1 2 + 1 )
( *yint
pr )+1f();
"% d \n ", /
/// als
outop ut
oustpx[
u ts
1 ]1or
3 20
*(y +1 )); y+ = 2 ; / / y no w p oint s t o x[ 2 ]
pr int f( "% d \n ", *y ); / / pr int s out 39
*y = 38 ; pr int f( "% d \ n", / / cha ng es x [ 2 ] t o 38
* y- 1 ); / / pr int s out x[ 2 ] - 1 or 3 7
*y+ + ; p r int f( "% d \n ", / / se t s y to p o int a t t he n e xt a rra y e le m e nt
*y); (*y )+ +; // out p ut s x[ 3 ] ( 4 3)
pr int f( "% d \n ", *y ); / / s e ts w h at y p oin ts t o t o be 1 g rea te r
/ / ou tp u ts t he ne w v al ue of x[ 3 ]
( 4 4)
59
A rr a ys an d P o in te rs
What wi l l be t h e v al u e o f
e l e me n t s of ar r ay a ?
mai n( )
{
i nt * p, * p2, x = 50;
i nt a[5] = { 1, 2, 3, 4, 5} ; p : 4000 4028
4008
?
p= &x ; p2= a;
p2 : 4004 4012
?
* ( p2+ 1) = * p;
* p= * p2 + * ( p2+ 2) ; x : 4008 3
50
* p2 = 10; p= p2+ 4; a a [0] : 4012 1
p[- 2] = 60;
p2[3] = x; p2[4] = a [1] : 4016 50
2
p[- 1]; a [2] : 4020 60
3
a[3] : 4024 3
4
} a[4] : 4028 3
5
60
A rrays an d
P■ oin t ers
Consider the declaration:
■ int b[10], *bPtr=b;
■ Element b[n] same as
■ *(b+n)
■ *( bPtr + n )
■ bPtr[n]
■ n[b] (Why?)
■ b[n] = *(b+n) = *(n+b) = n[b]
61
A rrays a n d P o in te rs
i nt i; i n t * p;
i nt a r r a y [10]; i nt a r r a y [10];
62
A rra ys an d
P oin
W h a t w ill be t h e oter
utps
u t?
vo id m ain ()
{
in t b [ 1 0 ] , * b P t r = b , i;
fo r (i= 0 ; i< 1 0 ;
p
i+rin
+ )bt[f("% d \n",
i] = i; 5
* (bP t r+ 5 ) ); 5
p r in t f(" % d \ n " , bP t r[ 5 ] ); 5
p r in t f(" % d \ n " , 5 [ b P tr ] ); 5
} p r in t f(" % d \ n " , 5 [ b ] );
63
A rra ys an d
W ha t w ill be th ePv aoin
lue ofter
el em esnt s o f a rra y a ?
m a in()
{
in t *p , *p 2 , x= 5 0 , i=2 ; int a [ 5 ] ={1 ,2 ,
3 ,4,5 };
p = & x; p2 = a ;
*( p 2 +1 )=*p ;
* p= *p 2 + *(p 2 + 2 );
* p2 = 1 0 ; p= p 2 +4 ; p [ - 2 ] = 6 0;
p 2 [ 3 ] = *p2 * i;
p 2 [ 4 ] = p[ - 1 ] + * (p - 1 );
}
6
5
A rra ys an d
W ha t w ill be th ePv aoin
lue ofter
el em esnt s o f a rra y a ?
m a in()
{
in t *p , *p 2 , x= 5 0 , i=2 ; int a [ 5 ] ={1 ,2 ,
3 ,4,5 };
p = & x; p2 = a ;
*( p 2 +1 )=*p ;
* p= *p 2 + *(p 2 + 2 );
* p2 = 1 0 ; p= p 2 +4 ; p [ - 2 ] = 6 0;
p 2 [ 3 ] = *p2 * i;
p 2 [ 4 ] = p[ - 1 ] + * (p - 1 );
} 10, 50, 60, 20, 40
6
6
A rra ys an d
W h at w ill b e t hePv aoin
lue ofter s ts of a rr ay x ?
e le m en
m ain( )
{
int i, x [ 1 0 ], * p , *q; f or( i= 0 ; i< 1 0 ;
i++ )
x[ i]= i+ 1 ;
p = x+9 ;
q = p-8;
f or( i=3 ; i>0 ; i- - , p -- )
*p + = 1 0 ;
f or( i=1 ; i< 5 ; i+ + , q ++)
*q += i;
}
66
A rra ys an d
W h at w ill b e t hePv aoin
lue ofter s ts of a rr ay x ?
e le m en
m ain( )
{
int i, n = 8 , x[ 1 0 ], * p, * q, j= 1 0 ;
f or( i=0 ; i< 1 0 ; i++ )
x[ i]= i+ 1 ;
p = x+9 ;
q = p-8;
f or( i=3 ; i>0 ; i- - , p -- )
*p + = 1 0 ;
f or( i=1 ; i< 5 ; i+ + , q ++)
*q += i;
} 1, 3 , 5 , 7 , 9 , 6 , 7 , 1 8 , 19 , 2 0
67
P a s s in g Tw o-D im e n s io n a l A rr a ys to
F u n ctio n s
68
P as s in g T w o -D im en s ion al A rra ys to
F u n ction s
■ Good things:
■ Fast, random access of elements
■ Very memory efficient, very little memory is required other
than that needed to store the contents (but see bellow)
■ Bad things:
■ Slow deletion and insertion of elements
■ Size must be known when the array is created and is fixed
(static)
70