19 algorithms-and-complexity-110627100203-phpapp02

Algorithms Complexity andAlgorithms Complexity and
Data Structures EfficiencyData Structures Efficiency
Computational Complexity, Choosing Data StructuresComputational Complexity, Choosing Data Structures
Svetlin NakovSvetlin Nakov
Telerik CorporationTelerik Corporation
www.telerik.comwww.telerik.com

Table of ContentsTable of Contents
1.1. Algorithms Complexity and AsymptoticAlgorithms Complexity and Asymptotic
NotationNotation
 Time and Memory ComplexityTime and Memory Complexity
 Mean, Average and Worst CaseMean, Average and Worst Case
1.1. Fundamental Data Structures – ComparisonFundamental Data Structures – Comparison
 Arrays vs. Lists vs. Trees vs. Hash-TablesArrays vs. Lists vs. Trees vs. Hash-Tables
1.1. Choosing Proper Data StructureChoosing Proper Data Structure
2

Why Data Structures areWhy Data Structures are
Important?Important?
 Data structuresData structures andand algorithmsalgorithms are theare the
foundation of computer programmingfoundation of computer programming
 Algorithmic thinking, problem solving andAlgorithmic thinking, problem solving and
data structures are vital for software engineersdata structures are vital for software engineers
All .NET developers should know when to useAll .NET developers should know when to use
T[]T[],, LinkedList<T>LinkedList<T>,, List<T>List<T>,, Stack<T>Stack<T>,,
Queue<T>Queue<T>,, Dictionary<K,T>Dictionary<K,T>,, HashSet<T>HashSet<T>,,
SortedDictionary<K,T>SortedDictionary<K,T> andand SortedSet<T>SortedSet<T>
 Computational complexity is important forComputational complexity is important for
algorithm design and efficient programmingalgorithm design and efficient programming
3

Algorithms ComplexityAlgorithms Complexity
Asymtotic NotationAsymtotic Notation

Algorithm AnalysisAlgorithm Analysis
 Why we should analyze algorithms?Why we should analyze algorithms?
Predict the resources that the algorithmPredict the resources that the algorithm
requiresrequires
 Computational time (CPU consumption)Computational time (CPU consumption)
 Memory space (RAM consumption)Memory space (RAM consumption)
 Communication bandwidth consumptionCommunication bandwidth consumption
TheThe running timerunning time of an algorithm is:of an algorithm is:
 The total number of primitive operationsThe total number of primitive operations
executed (machine independent steps)executed (machine independent steps)
 Also known asAlso known as algorithm complexityalgorithm complexity
5

Algorithmic ComplexityAlgorithmic Complexity
 What to measure?What to measure?
MemoryMemory
TimeTime
Number of stepsNumber of steps
Number of particular operationsNumber of particular operations
 Number of disk operationsNumber of disk operations
 Number of network packetsNumber of network packets
Asymptotic complexityAsymptotic complexity
6

Time ComplexityTime Complexity
 Worst-caseWorst-case
An upper bound on the running time for anyAn upper bound on the running time for any
input of given sizeinput of given size
 Average-caseAverage-case
Assume all inputs of a given size are equallyAssume all inputs of a given size are equally
likelylikely
 Best-caseBest-case
The lower bound on the running timeThe lower bound on the running time
7

Time Complexity – ExampleTime Complexity – Example
 Sequential search in a list of size nSequential search in a list of size n
Worst-case:Worst-case:
 nn comparisonscomparisons
Best-case:Best-case:
 11 comparisoncomparison
Average-case:Average-case:
 n/2n/2 comparisonscomparisons
 The algorithm runs inThe algorithm runs in linear timelinear time
Linear number of operationsLinear number of operations
…… …… …… …… …… …… ……
nn
8

Algorithms ComplexityAlgorithms Complexity
 Algorithm complexityAlgorithm complexity is rough estimation of theis rough estimation of the
number of steps performed by given computationnumber of steps performed by given computation
depending on the size of the input datadepending on the size of the input data
 Measured through asymptotic notationMeasured through asymptotic notation
 O(g)O(g) wherewhere gg is a function of the input data sizeis a function of the input data size
Examples:Examples:
 Linear complexityLinear complexity O(n)O(n) – all elements are– all elements are
processed once (or constant number of times)processed once (or constant number of times)
 Quadratic complexityQuadratic complexity O(nO(n22
)) – each of the– each of the
elements is processedelements is processed nn timestimes
9

Asymptotic Notation: DefinitionAsymptotic Notation: Definition
 Asymptotic upper boundAsymptotic upper bound
 O-notation (Big O notation)O-notation (Big O notation)
 For given functionFor given function g(n)g(n), we denote by, we denote by O(g(n))O(g(n))
the set of functions that are different thanthe set of functions that are different than g(n)g(n)
by a constantby a constant
 Examples:Examples:
 33 ** nn22
++ n/2n/2 ++ 1212 ∈∈ O(nO(n22
))
 4*n*log4*n*log22(3*n+1)(3*n+1) ++ 2*n-12*n-1 ∈∈ O(nO(n ** loglog n)n)
O(g(n))O(g(n)) == {{f(n)f(n): there exist positive constants: there exist positive constants cc
andand nn00 such thatsuch that f(n)f(n) <=<= c*g(n)c*g(n) for allfor all nn >=>= nn00}}
10

Typical ComplexitiesTypical Complexities
11
ComplexityComplexity NotationNotation DescriptionDescription
constantconstant O(1)O(1)
Constant number ofConstant number of
operations, not depending onoperations, not depending on
the input data size, e.g.the input data size, e.g.
n = 1 000 000n = 1 000 000  1-21-2
operationsoperations
logarithmiclogarithmic O(logO(log n)n)
Number of operationsNumber of operations propor-propor-
tionaltional of logof log22(n) where n is the(n) where n is the
size of the input data, e.g. n =size of the input data, e.g. n =
1 000 000 0001 000 000 000  30 operations30 operations
linearlinear O(n)O(n)
Number of operationsNumber of operations
proportional to the input dataproportional to the input data
size, e.g. n = 10 000size, e.g. n = 10 000  5 0005 000
operationsoperations

Typical Complexities (2)Typical Complexities (2)
12
ComplexityComplexity NotationNotation DescriptionDescription
quadraticquadratic O(nO(n22
))
Number of operationsNumber of operations
proportional to the square ofproportional to the square of
the size of the input data, e.g.the size of the input data, e.g.
n = 500n = 500  250 000 operations250 000 operations
cubiccubic O(nO(n33
))
Number of operationsNumber of operations propor-propor-
tionaltional to the cube of the sizeto the cube of the size
of the input data, e.g. n =of the input data, e.g. n =
200200  8 000 000 operations8 000 000 operations
exponentialexponential
O(2O(2nn
)),,
O(O(kknn
)),,
O(n!)O(n!)
Exponential number ofExponential number of
operations, fast growing, e.g.operations, fast growing, e.g.
n = 20n = 20  1 048 576 operations1 048 576 operations

Time Complexity and SpeedTime Complexity and Speed
13
ComplexityComplexity 1010 2020 5050 100100 11 000000 1010 000000 100100 000000
O(1)O(1) << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss
O(log(n))O(log(n)) << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss
O(n)O(n) << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss
O(n*log(n))O(n*log(n)) << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss
O(nO(n22
)) << 11 ss << 11 ss << 11 ss << 11 ss << 11 ss 22 ss 33--44 minmin
O(nO(n33
)) << 11 ss << 11 ss << 11 ss << 11 ss 2020 ss 55 hourshours 231231 daysdays
O(2O(2nn
)) << 11 ss << 11 ss
260260
daysdays
hangshangs hangshangs hangshangs hangshangs
O(n!)O(n!) << 11 ss hangshangs hangshangs hangshangs hangshangs hangshangs hangshangs
O(nO(nnn
)) 33--44 minmin hangshangs hangshangs hangshangs hangshangs hangshangs hangshangs

Time and Memory ComplexityTime and Memory Complexity
 Complexity can be expressed as formula onComplexity can be expressed as formula on
multiple variables, e.g.multiple variables, e.g.
 Algorithm filling a matrix of sizeAlgorithm filling a matrix of size nn ** mm with naturalwith natural
numbersnumbers 11,, 22, … will run in, … will run in O(n*m)O(n*m)
 DFS traversal of graph withDFS traversal of graph with nn vertices andvertices and mm edgesedges
will run inwill run in O(nO(n ++ m)m)
 Memory consumption should also be considered,Memory consumption should also be considered,
for example:for example:
 Running timeRunning time O(n)O(n), memory requirement, memory requirement O(nO(n22
))
 n = 50 000n = 50 000  OutOfMemoryExceptionOutOfMemoryException
14

Polynomial AlgorithmsPolynomial Algorithms
 AA polynomial-time algorithmpolynomial-time algorithm is one whoseis one whose
worst-case time complexity is bounded aboveworst-case time complexity is bounded above
by a polynomial function of its input sizeby a polynomial function of its input size
 Example of worst-case time complexityExample of worst-case time complexity
Polynomial-time:Polynomial-time: loglog nn,, 2n2n,, 3n3n33
++ 4n4n,, 22 ** nn loglog nn
Non polynomial-time :Non polynomial-time : 22nn
,, 33nn
,, nnkk
,, n!n!
 Non-polynomial algorithms don't work forNon-polynomial algorithms don't work for
large input data setslarge input data sets
W(n)W(n) ∈ O(p(n))O(p(n))
15

Analyzing ComplexityAnalyzing Complexity
of Algorithmsof Algorithms
ExamplesExamples

Complexity ExamplesComplexity Examples
 Runs inRuns in O(n)O(n) wherewhere nn is the size of the arrayis the size of the array
 The number of elementary steps isThe number of elementary steps is ~~ nn
int FindMaxElement(int[] array)int FindMaxElement(int[] array)
{{
int max = array[0];int max = array[0];
for (int i=0; i<array.length; i++)for (int i=0; i<array.length; i++)
{{
if (array[i] > max)if (array[i] > max)
{{
max = array[i];max = array[i];
}}
}}
return max;return max;
}}

Complexity Examples (2)Complexity Examples (2)
 Runs inRuns in O(nO(n22
)) wherewhere nn is the size of the arrayis the size of the array
 The number of elementary steps isThe number of elementary steps is
~~ n*(n+1)n*(n+1) // 22
long FindInversions(int[] array)long FindInversions(int[] array)
{{
long inversions = 0;long inversions = 0;
for (int i=0; i<array.Length; i++)for (int i=0; i<array.Length; i++)
for (int j = i+1; j<array.Length; i++)for (int j = i+1; j<array.Length; i++)
if (array[i] > array[j])if (array[i] > array[j])
inversions++;inversions++;
return inversions;return inversions;
}}

 Runs in cubic timeRuns in cubic time O(nO(n33
))
 The number of elementary steps isThe number of elementary steps is ~~ nn33
decimal Sum3(int n)decimal Sum3(int n)
{{
decimal sum = 0;decimal sum = 0;
for (int a=0; a<n; a++)for (int a=0; a<n; a++)
for (int b=0; b<n; b++)for (int b=0; b<n; b++)
for (int c=0; c<n; c++)for (int c=0; c<n; c++)
sum += a*b*c;sum += a*b*c;
return sum;return sum;
}}

 Runs in quadratic timeRuns in quadratic time O(n*m)O(n*m)
 The number of elementary steps isThe number of elementary steps is ~~ n*mn*m
long SumMN(int n, int m)long SumMN(int n, int m)
{{
long sum = 0;long sum = 0;
for (int x=0; x<n; x++)for (int x=0; x<n; x++)
for (int y=0; y<m; y++)for (int y=0; y<m; y++)
sum += x*y;sum += x*y;
}}

 Runs in quadratic timeRuns in quadratic time O(n*m)O(n*m)
~~ n*mn*m ++ min(m,n)*nmin(m,n)*n
long SumMN(int n, int m)long SumMN(int n, int m)
{{
long sum = 0;long sum = 0;
for (int x=0; x<n; x++)for (int x=0; x<n; x++)
for (int y=0; y<m; y++)for (int y=0; y<m; y++)
if (x==y)if (x==y)
for (int i=0; i<n; i++)for (int i=0; i<n; i++)
sum += i*x*y;sum += i*x*y;
}}

 Runs in exponential timeRuns in exponential time O(2O(2nn
))
 The number of elementary steps isThe number of elementary steps is ~~ 22nn
decimal Calculation(int n)decimal Calculation(int n)
{{
decimal result = 0;decimal result = 0;
for (int i = 0; i < (1<<n); i++)for (int i = 0; i < (1<<n); i++)
result += i;result += i;
return result;return result;
}}

 Runs in linear timeRuns in linear time O(n)O(n)
 The number of elementary steps isThe number of elementary steps is ~~ nn
decimal Factorial(int n)decimal Factorial(int n)
{{
if (n==0)if (n==0)
return 1;return 1;
elseelse
return n * Factorial(n-1);return n * Factorial(n-1);
}}

 Runs inRuns in exponential timeexponential time O(2O(2nn
))
~~ Fib(n+1)Fib(n+1) wwherehere Fib(k)Fib(k) is theis the kk-th-th
FibFiboonacci's numbernacci's number
decimal Fibonacci(int n)decimal Fibonacci(int n)
{{
if (n == 0)if (n == 0)
return 1;return 1;
else if (n == 1)else if (n == 1)
return 1;return 1;
elseelse
return Fibonacci(n-1) + Fibonacci(n-2);return Fibonacci(n-1) + Fibonacci(n-2);
}}

Comparing Data StructuresComparing Data Structures
ExamplesExamples

26
Data StructureData Structure AddAdd FindFind DeleteDelete
Get-by-Get-by-
indexindex
Array (Array (T[]T[])) O(n)O(n) O(n)O(n) O(n)O(n) O(1)O(1)
Linked listLinked list
((LinkedList<T>LinkedList<T>))
O(1)O(1) O(n)O(n) O(n)O(n) O(n)O(n)
Resizable array listResizable array list
((List<T>List<T>))
O(1)O(1) O(n)O(n) O(n)O(n) O(1)O(1)
Stack (Stack (Stack<T>Stack<T>)) O(1)O(1) -- O(1)O(1) --
Queue (Queue (Queue<T>Queue<T>)) O(1)O(1) -- O(1)O(1) --

Data Structures EfficiencyData Structures Efficiency (2)(2)
27
Data StructureData Structure AddAdd FindFind DeleteDelete
Get-by-Get-by-
indexindex
Hash tableHash table
((Dictionary<K,T>Dictionary<K,T>))
O(1)O(1) O(1)O(1) O(1)O(1) --
Tree-basedTree-based
dictionary (dictionary (SortedSorted
Dictionary<K,T>Dictionary<K,T>))
O(logO(log n)n) O(logO(log n)n) O(logO(log n)n) --
Hash table basedHash table based
set (set (HashSet<T>HashSet<T>))
O(1)O(1) O(1)O(1) O(1)O(1) --
Tree based setTree based set
((SortedSet<T>SortedSet<T>))
O(logO(log n)n) O(logO(log n)n) O(logO(log n)n) --

Choosing Data StructureChoosing Data Structure
 Arrays (Arrays (T[]T[]))
Use when fixed number of elements should beUse when fixed number of elements should be
processed by indexprocessed by index
 Resizable array lists (Resizable array lists (List<T>List<T>))
Use when elements should be added andUse when elements should be added and
processed by indexprocessed by index
 Linked lists (Linked lists (LinkedList<T>LinkedList<T>))
Use when elements should be added at theUse when elements should be added at the
both sides of the listboth sides of the list
Otherwise use resizable array list (Otherwise use resizable array list (List<T>List<T>))
28

Choosing Data Structure (2)Choosing Data Structure (2)
 Stacks (Stacks (Stack<T>Stack<T>))
 Use to implement LIFO (last-in-first-out) behaviorUse to implement LIFO (last-in-first-out) behavior
 List<T>List<T> could also work wellcould also work well
 Queues (Queues (Queue<T>Queue<T>))
 Use to implement FIFO (first-in-first-out) behaviorUse to implement FIFO (first-in-first-out) behavior
 LinkedList<T>LinkedList<T> could also work wellcould also work well
 Hash table based dictionary (Hash table based dictionary (Dictionary<K,T>Dictionary<K,T>))
 Use when key-value pairs should be added fast andUse when key-value pairs should be added fast and
searched fast by keysearched fast by key
 Elements in a hash table have no particular orderElements in a hash table have no particular order
29

Choosing Data Structure (3)Choosing Data Structure (3)
 Balanced search tree based dictionaryBalanced search tree based dictionary
((SortedDictionary<K,T>SortedDictionary<K,T>))
 Use when key-value pairs should be added fast,Use when key-value pairs should be added fast,
searched fast by key and enumerated sorted by keysearched fast by key and enumerated sorted by key
 Hash table based set (Hash table based set (HashSet<T>HashSet<T>))
Use to keep a group of unique values, to addUse to keep a group of unique values, to add
and check belonging to the set fastand check belonging to the set fast
Elements are in no particular orderElements are in no particular order
 Search tree based set (Search tree based set (SortedSet<T>SortedSet<T>))
Use to keep a group of ordered unique valuesUse to keep a group of ordered unique values
30

SummarySummary
 Algorithm complexityAlgorithm complexity is rough estimation of theis rough estimation of the
number of steps performed by given computationnumber of steps performed by given computation
 Complexity can be logarithmic, linear, n log n,Complexity can be logarithmic, linear, n log n,
square, cubic, exponential, etc.square, cubic, exponential, etc.
 Allows to estimating the speed of given codeAllows to estimating the speed of given code
before its executionbefore its execution
 Different data structures have differentDifferent data structures have different
efficiency on different operationsefficiency on different operations
The fastest add / find / delete structure is theThe fastest add / find / delete structure is the
hash table –hash table – O(1)O(1) for all these operationsfor all these operations
31

Algorithms Complexity andAlgorithms Complexity and
Questions?Questions?
http://academy.telerik.com

Exercises (2)Exercises (2)
2.2. A large trade company has millions of articles, eachA large trade company has millions of articles, each
described by barcode, vendor, title and price.described by barcode, vendor, title and price.
Implement a data structure to store them thatImplement a data structure to store them that
allows fast retrieval of all articles in given price rangeallows fast retrieval of all articles in given price range
[x…y][x…y]. Hint: use. Hint: use OrderedMultiDictionary<K,T>OrderedMultiDictionary<K,T>
fromfrom Wintellect's Power Collections for .NET.Wintellect's Power Collections for .NET.
3.3. Implement a data structureImplement a data structure PriorityQueue<T>PriorityQueue<T>
that provides a fast way to execute the followingthat provides a fast way to execute the following
operations:operations: add element; extract the smallest element.add element; extract the smallest element.
4.4. Implement a classImplement a class BiDictionary<K1,K2,T>BiDictionary<K1,K2,T> thatthat
allows adding triplesallows adding triples {key1,{key1, key2,key2, value}value} and fastand fast
search bysearch by key1key1,, key2key2 or by bothor by both key1key1 andand key2key2..
Note: multiple values can be stored for given key.Note: multiple values can be stored for given key.
34

Exercises (3)Exercises (3)
5.5. A text fileA text file phones.txtphones.txt holds information aboutholds information about
people, their town and phone number:people, their town and phone number:
Duplicates can occur in people names, towns andDuplicates can occur in people names, towns and
phone numbers. Write a program to execute aphone numbers. Write a program to execute a
sequence of commands from a filesequence of commands from a file commands.txtcommands.txt::
 find(name)find(name) – display all matching records by given– display all matching records by given
name (first, middle, last or nickname)name (first, middle, last or nickname)
 find(name,find(name, town)town) – display all matching records by– display all matching records by
given name and towngiven name and town
35
Mimi Shmatkata | Plovdiv | 0888 12 34 56Mimi Shmatkata | Plovdiv | 0888 12 34 56
Kireto | Varna | 052 23 45 67Kireto | Varna | 052 23 45 67
Daniela Ivanova Petrova | Karnobat | 0899 999 888Daniela Ivanova Petrova | Karnobat | 0899 999 888
Bat Gancho | Sofia | 02 946 946 946Bat Gancho | Sofia | 02 946 946 946

19 algorithms-and-complexity-110627100203-phpapp02

Recommended

More Related Content

What's hot (19)

Viewers also liked (18)

Similar to 19 algorithms-and-complexity-110627100203-phpapp02 (20)

Recently uploaded (20)

19 algorithms-and-complexity-110627100203-phpapp02