19. Java data structures algorithms and complexity

SoftUni Team
Technical Trainers
Software University
http://softuni.bg
Data Structures,
Algorithms and Complexity
Analyzing Algorithm Complexity.
Asymptotic Notation

Table of Contents
1. Data Structures
 Linear Structures, Trees, Hash Tables, Others
2. Algorithms
 Sorting and Searching, Combinatorics,
Dynamic Programming, Graphs, Others
3. Complexity of Algorithms
 Time and Space Complexity
 Mean, Average and Worst Case
 Asymptotic Notation O(g)
2

6
 Examples of data structures:
 Person structure (first name + last name + age)
 Array of integers – int[]
 List of strings – List<string>
 Queue of people – Queue<Person>
What is a Data Structure?
“In computer science, a data structure is a particular
way of storing and organizing data in a computer so
that it can be used efficiently.” -- Wikipedia

7
Student Data Structure
struct Student {
string Name { get; set; }
short Age { get; set; } // Student age (< 128)
Gender Gender { get; set; }
int FacultyNumber { get; set; }
};
enum Gender
{
Male,
Female,
Other
}
Student
Name Maria Smith
Age 23
Gender Female
FacultyNumber SU2007333

9
Stack
class Stack<T>
{
// Pushes an elements at the top of the stack
void Push(T data);
// Extracts the element at the top of the stack
T Pop();
// Checks for empty stack
bool IsEmpty();
}
// The type T can be any data structure like
// int, string, DateTime, Student

10
Stack
data
data
data
data
data
data
data
top of the stack

15
Queue
class Queue<T>
{
// Appends an elements at the end of the queue
void Enqueue(T data);
// Extracts the element from the start of the queue
T Dequeue();
// Checks for empty queue
bool IsEmpty();
}
// The type T can be any data structure like
// int, string, DateTime, Student

16
Queue
data
data
data
data
data
data
data
Start of the queue: elements
are excluded from this position
End of the queue: new
elements are inserted here

17
Linked List
2
next
7
next
head
(list start)
4
next
5
next
null

19
Trees
17
15149
6 5 8
Project
Manager
Team
Leader
De-
signer
QA Team
Leader
Developer
1
Developer
2
Tester 1
Developer
3
Tester
2

20
Binary Tree
data
left link 
.......
NULL
tree root
right link
data
left link 
right link
data
left link 
right link
data
left link 
right link
data
left link 
right link
.......
NULL
NULL
.......
.......

23
Graphs 7
19
21
14
1
12
31
4
11
G
J
F
D
A
E C H
GA
H N
K

25
Hash-Table: Structure
Key1
Element1
…
Hash-Function Slot 0
……
Slot 1
Slot 2
Slot
Key 2
Element 2
Key m
Element m
n-1

26
Hash-Table: Chaining on Collision
NULL
234 
0 
1 
2 
3 
8 
9 
4 
5 
6 
7 
235 
567 
647 
534 
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
123  NULL

27
 Data structures and algorithms are the foundation of computer
programming
 Algorithmic thinking, problem solving and data structures are
vital for software engineers
 C# developers should know when to use T[], LinkedList<T>,
List<T>, Stack<T>, Queue<T>, Dictionary<K, T>,
HashSet<T>, SortedDictionary<K, T> and SortedSet<T>
 Computational complexity is important for algorithm design and
efficient programming
Why Are Data Structures So Important?

 Primitive data types
 Numbers: int, float, double, decimal, …
 Text data: char, string, …
Simple structures
 A group of primitive fields stored together
 E.g. DateTime, Point, Rectangle, Color, …
 Collections
 A set / sequence of elements (of the same type)
 E.g. array, list, stack, queue, tree, hashtable, bag, …
Primitive Types and Collections

 An Abstract Data Type (ADT) is
 A data type together with the operations, whose properties are
specified independently of any particular implementation
 ADT is a set of definitions of operations
 Like the interfaces in C# / Java
 Defines what we can do with the structure
 ADT can have several different implementations
 Different implementations can have different efficiency, inner
logic and resource needs
Abstract Data Types (ADT)

 Linear structures
 Lists: fixed size and variable size sequences
 Stacks: LIFO (Last In First Out) structures
 Queues: FIFO (First In First Out) structures
 Trees and tree-like structures
 Binary, ordered search trees, balanced trees, etc.
 Dictionaries (maps, associative arrays)
 Hold pairs (key  value)
 Hash tables: use hash functions to search / insert
Basic Data Structures

31
 Sets, multi-sets and bags
 Set – collection of unique elements
 Bag – collection of non-unique elements
 Ordered sets and dictionaries
 Priority queues / heaps
 Special tree structures
 Suffix tree, interval tree, index tree, trie, rope, …
 Graphs
 Directed / undirected, weighted / unweighted,
connected / non-connected, cyclic / acyclic, …
Basic Data Structures (2)

33
 The term "algorithm" means "a sequence of steps"
 Derived from Muḥammad Al-Khwārizmī', a Persian mathematician
and astronomer
 He described an algorithm for solving quadratic equations in 825
What is an Algorithm?
“In mathematics and computer science, an algorithm is
a step-by-step procedure for calculations. An algorithm
is an effective method expressed as a finite list of well-
defined instructions for calculating a function.”
-- Wikipedia

34
 Algorithms are fundamental in programming
 Imperative (traditional, algorithmic) programming means to
describe in formal steps how to do something
 Algorithm == sequence of operations (steps)
 Can include branches (conditional blocks) and repeated logic (loops)
 Algorithmic thinking (mathematical thinking, logical thinking,
engineering thinking)
 Ability to decompose the problems into formal sequences of steps
(algorithms)
Algorithms in Computer Science

35
 Algorithms can be expressed as pseudocode, through flowcharts or
program code
Pseudocode and Flowcharts
BFS(node)
{
queue  node
while queue not empty
v  queue
print v
for each child c of v
queue  c
}
Pseudo-code Flowchart
public void DFS(Node node)
{
Print(node.Name);
for (int i = 0; i < node.
Children.Count; i++)
{
if (!visited[node.Id])
DFS(node.Children[i]);
}
visited[node.Id] = true;
}
Source code

 Sorting and searching
 Dynamic programming
 Graph algorithms
 DFS and BFS traversals
 Combinatorial algorithms
 Recursive algorithms
 Other algorithms
 Greedy algorithms, computational geometry, randomized
algorithms, parallel algorithms, genetic algorithms
36
Some Algorithms in Programming

Algorithm Complexity
Asymptotic Notation

39
 Why should we analyze algorithms?
 Predict the resources the algorithm will need
 Computational time (CPU consumption)
 Memory space (RAM consumption)
 Communication bandwidth consumption
 The expected running time of an algorithm is:
 The total number of primitive operations executed
(machine independent steps)
 Also known as algorithm complexity
Algorithm Analysis

40
 What to measure?
 CPU time
 Memory consumption
 Number of steps
 Number of particular operations
 Number of disk operations
 Number of network packets
 Asymptotic complexity
Algorithmic Complexity

41
 Worst-case
 An upper bound on the running time for any input of given size
 Typically algorithms performance is measured for their worst case
 Average-case
 The running time averaged over all possible inputs
 Used to measure algorithms that are repeated many times
 Best-case
 The lower bound on the running time (the optimal case)
Time Complexity

 Sequential search in a list of size n
 Worst-case:
 n comparisons
 Best-case:
 1 comparison
 Average-case:
 n/2 comparisons
 The algorithm runs in linear time
 Linear number of operations
42
Time Complexity: Example
… … … … … … …
n

45
 Algorithm complexity is a rough estimation of the number of steps
performed by given computation, depending on the size of the input
 Measured with asymptotic notation
 O(g) where g is a function of the size of the input data
 Examples:
 Linear complexity O(n)
 All elements are processed once (or constant number of times)
 Quadratic complexity O(n2)
 Each of the elements is processed n times
Algorithms Complexity

46
 Asymptotic upper bound
 O-notation (Big O notation)
 For a given function g(n), we denote by O(g(n)) the set of functions
that are different than g(n) by a constant
 Examples:
 3 * n2 + n/2 + 12 ∈ O(n2)
 4*n*log2(3*n+1) + 2*n-1 ∈ O(n * log n)
Asymptotic Notation: Definition
O(g(n)) = {f(n): there exist positive constants c and n0
such that f(n) <= c*g(n) for all n >= n0}

47
Positive examples:
Examples
Negative examples:

49
Complexity Notation Description
constant O(1)
Constant number of operations, not
depending on the input data size, e.g.
n = 1 000 000  1-2 operations
logarithmic O(log n)
Number of operations proportional to
log2(n) where n is the size of the input
data, e.g.
n = 1 000 000 000  30 operations
linear O(n)
Number of operations proportional to the
input data size, e.g. n = 10 000  5 000
operations
Typical Complexities

50
Complexity Notation Description
quadratic O(n2)
Number of operations proportional to the
square of the size of the input data, e.g.
n = 500  250 000 operations
cubic O(n3)
Number of operations propor-tional to
the cube of the size of the input data, e.g.
n = 200  8 000 000 operations
exponential
O(2n),
O(kn),
O(n!)
Exponential number of operations, fast
growing, e.g. n = 20  1 048 576
operations
Typical Complexities (2)

52
Time Complexity and Program Speed
Complexity 10 20 50 100 1 000 10 000 100 000
O(1) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(log(n)) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(n) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(n*log(n)) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(n2) < 1 s < 1 s < 1 s < 1 s < 1 s 2 s 3-4 min
O(n3) < 1 s < 1 s < 1 s < 1 s 20 s 5 hours 231 days
O(2n) < 1 s < 1 s 260 days hangs hangs hangs hangs
O(n!) < 1 s hangs hangs hangs hangs hangs hangs
O(nn) 3-4 min hangs hangs hangs hangs hangs hangs

54
 Complexity can be expressed as a formula of multiple variables, e.g.
 Algorithm filling a matrix of size n * m with the natural numbers 1,
2, … n*m will run in O(n*m)
 Algorithms to traverse a graph with n vertices and m edges will
take O(n + m) steps
 Memory consumption should also be considered, for example:
 Running time O(n) & memory requirement O(n2)
 n = 50 000  OutOfMemoryException
Time and Memory Complexity
BTW: Is it possible in
practice? In theory?

56
 A linear algorithm could be slower than a quadratic algorithm
 The hidden constant could be significant
 Example:
 Algorithm A performs 100*n steps  O(n)
 Algorithm B performs n*(n-1)/2 steps  O(n2)
 For n < 200, algorithm B is faster
 Real-world example:
 The "insertion sort" is faster than "quicksort" for n < 9
The Hidden Constant

57
Amortized Analysis
 Worst case: O(n2)
 Average case: O(n) – Why?
void AddOne(char[] chars, int m)
{
for (int i = 0; 1 != chars[i] && i < m; i++)
{
c[i] = 1 - c[i];
}
}

58
 Complexity: O(n3)
Using Math
sum = 0;
for (i = 1; i <= n; i++)
{
for (j = 1; j <= i*i; j++)
{
sum++;
}
}

59
 Just calculate and print sum = n * (n + 1) * (2n + 1) / 6
Using a Barometer
uint sum = 0;
for (i = 0; i < n; i++)
{
for (j = 0; j < i*i; j++)
{
sum++;
}
}
Console.WriteLine(sum);

60
 A polynomial-time algorithm has worst-case time complexity
bounded above by a polynomial function of its input size
 Examples:
 Polynomial time: 2n, 3n3 + 4n
 Exponential time: 2n, 3n, nk, n!
 Exponential algorithms hang for large input data sets
Polynomial Algorithms
W(n) ∈ O(p(n))

61
 Computational complexity theory divides the computational
problems into several classes:
Computational Classes

62
P vs. NP: Problem #1 in Computer Science Today

Analyzing the Complexity of Algorithms
Examples

 Runs in O(n) where n is the size of the array
 The number of elementary steps is ~ n
Complexity Examples
int FindMaxElement(int[] array)
{
int max = array[0];
for (int i=1; i<array.length; i++)
{
if (array[i] > max)
{
max = array[i];
}
}
return max;
}

 Runs in O(n2) where n is the size of the array
 The number of elementary steps is ~ n * (n+1) / 2
Complexity Examples (2)
long FindInversions(int[] array)
{
long inversions = 0;
for (int i=0; i<array.Length; i++)
for (int j = i+1; j<array.Length; i++)
if (array[i] > array[j])
inversions++;
return inversions;
}

 Runs in cubic time O(n3)
 The number of elementary steps is ~ n3
decimal Sum3(int n)
{
decimal sum = 0;
for (int a=0; a<n; a++)
for (int b=0; b<n; b++)
for (int c=0; c<n; c++)
sum += a*b*c;
return sum;
}

 Runs in quadratic time O(n*m)
 The number of elementary steps is ~ n*m
long SumMN(int n, int m)
{
long sum = 0;
for (int x=0; x<n; x++)
for (int y=0; y<m; y++)
sum += x*y;
return sum;
}

 Runs in quadratic time O(n*m)
 The number of elementary steps is ~ n*m + min(m,n)*n
long SumMN(int n, int m)
{
long sum = 0;
for (int x=0; x<n; x++)
for (int y=0; y<m; y++)
if (x==y)
for (int i=0; i<n; i++)
sum += i*x*y;
return sum;
}

 Runs in exponential time O(2n)
 The number of elementary steps is ~ 2n
decimal Calculation(int n)
{
decimal result = 0;
for (int i = 0; i < (1<<n); i++)
result += i;
return result;
}

 Runs in linear time O(n)
 The number of elementary steps is ~ n
decimal Factorial(int n)
{
if (n==0)
return 1;
else
return n * Factorial(n-1);
}

 Runs in exponential time O(2n)
 The number of elementary steps is ~ Fib(n+1)
where Fib(k) is the kth Fibonacci's number
decimal Fibonacci(int n)
{
if (n == 0)
return 1;
else if (n == 1)
return 1;
else
return Fibonacci(n-1) + Fibonacci(n-2);
}

Lab Exercise
Calculating Complexity of Existing Code

78
 Data structures organize data in computer systems for efficient use
 Abstract data types (ADT) describe a set of operations
 Collections hold a group of elements
 Algorithms are sequences of steps for calculating / doing something
 Algorithm complexity is a rough estimation of the number of steps
performed by given computation
 Can be logarithmic, linear, n log n, square, cubic, exponential, etc.
 Complexity predicts the speed of given code before its execution
Summary

?
https://softuni.bg/trainings/1147/Data-Structures-June-2015
Data Structures, Algorithms and Complexity

License
 This course (slides, examples, labs, videos, homework, etc.)
is licensed under the "Creative Commons Attribution-
NonCommercial-ShareAlike 4.0 International" license
81
 Attribution: this work may contain portions from
 "Fundamentals of Computer Programming with C#" book by Svetlin Nakov & Co. under CC-BY-SA license
 "Data Structures and Algorithms" course by Telerik Academy under CC-BY-NC-SA license

Free Trainings @ Software University
 Software University Foundation – softuni.org
 Software University – High-Quality Education,
Profession and Job for Software Developers
 softuni.bg
 Software University @ Facebook
 facebook.com/SoftwareUniversity
 Software University @ YouTube
 youtube.com/SoftwareUniversity
 Software University Forums – forum.softuni.bg

19. Java data structures algorithms and complexity

Recommended

More Related Content

What's hot (20)

Similar to 19. Java data structures algorithms and complexity (20)

More from Intro C# Book (15)

Recently uploaded (19)

19. Java data structures algorithms and complexity

Editor's Notes