Natural Language Processing

Data Structures

Dr. Narjes Nikzad Khasmakhi

[email protected]
 Who I am :)

Narjes Nikzad Khasmakhi

● Professor at Gisma
● A special focus on text and graph data.
● Recent advances in large language models and multimodal deep
learning inspire me.
● Currently, I work on conversational systems, vision-language
models, and knowledge representation.

3
Who I am :)

4
 Who I am :)

❏ Collaboration with:

5
 Who You are :)

❖ Name?
❖ Background?
❖ Expectations?
❖ Familiarity with the topics?
● Algorithms
● Programing

6
 Course organization

The lecture will be ….

8
 Course organization

You can tap into slides, material and further information on Canvas.

9
Syllabus

10
Assessment Summary

11
Anything else?

12
Introduction

13
 Questions

❖ Why Computers and What are computer features?

❖ Similarity of human and computers?
❖ What are Computer Components?
❖ What is Data (Definition of Data at Hardware Level)?
❖ What is Data Structure?
❖ What is Algorithm?
❖ What is Time and Space Complexity?
14
 Why Computers?

A computer is a device that processes data, does calculations, and

follows instructions to solve problems, automate tasks, and help people
communicate.

15
https://www.pinayhomeschoolershop.com/products/history-of-computers-timeline-and-ordering-activities
 Features of Computers?

❏ Speed: Computers can perform tasks

and process data at extremely high
speeds, far faster than humans.
❏ Reliability: Computers are
dependable and can operate
consistently over long periods
without error.
❏ Remembrance Power: Computers
can store vast amounts of data and
recall it instantly when needed.
16
 Features of Computers?

❏ Automation: Computers can carry out

a series of tasks automatically once
programmed, without further human
intervention.
❏ Accuracy: Computers perform tasks
with a high degree of precision,
minimizing the chances of errors.
❏ Consistency: Computers provide the
same level of output and performance
repeatedly without deviation.
17
 Features of Computers?

❏ Logical: Computers follow logical

operations and decision-making
based on predefined instructions.
❏ Versatility: Computers are capable of
performing a wide variety of tasks,
from simple calculations to complex
simulations.

18
Similarity of human and computers

19
Computer Components

20
https://www.facebook.com/pg/Hardware-Software-267286954079644/groups/?ref=page_internal
Computer Hardware

https://www.tutorialspoint.com/computer_concepts/computer_concepts_components_of_computer_system.htm
21
 CPU(Central Processing Unit)

https://medium.com/@nateshganesh/no-the-brain-is-not-a-computer-1c566d99318c

22
Arithmetic Logic Unit (ALU)

23
Control Unit

The brain communicates with the rest of the body via the
nervous system.

24
Memory (RAM) and Short-Term Memory

https://study.com/learn/lesson/video/random-access-memory-ram-concept-types-uses.html

https://app.uxcel.com/courses/ux-design-psychology/how-human-
memory-works-226/short-term-memory-3031

25
 Software

❏ Software is a set of instructions, programs, or code that tells a

computer how to perform tasks.

26
https://xtendedview.com/technology/physical-storage-vs-digital-document/2146/
 Software

❏ Software acts on data and transforms it from one form to another.

❏ Software follows an Input-Process-Output (IPO) cycle.

https://www.researchgate.net/figure/Schematic-description-of-the-IPO-concept-
Through-defined-inputs-a-process-is_fig1_363819397
27
 Fitness APP

● Imagine you’re working at a

company that develops an
app for tracking user
activities like steps
walked, calories burned,
and hours slept.
● This app collects data
from millions of users every
day. 28
 What is Data?

❖ Data is the raw

information being
collected from each user
(steps, calories, hours,
etc.).

29
 Data at the Hardware Level

❖ In a computer, data is represented

in binary form, using only two
states: 0 and 1. These states
correspond to electrical voltages
in the computer's hardware:
● 0 typically represents a low
voltage (near zero volts).
● 1 represents a high voltage (often
a positive voltage, like 5V or
30
3.3V, depending on the system).
 Data at the Hardware Level

❖ Circuits in computers use transistors to create and control these

voltage levels, allowing them to process data by performing operations
on binary values.

31
 Importance of Data

★ Data gives insights into patterns and trends, helping improve

services, make recommendations, provide personalized experiences
and enabling decision-making in software, AI, business, and beyond.

32
https://stratoflow.com/amazon-recommendation-system/
Example of Data

33
 Structured Data vs Unstructured Data

Imagine you are setting up a

library.
You have thousands of books but
no system for organizing them.
You could pile them on the floor,
but finding a book would be a
nightmare.

34
 Structured Data vs Unstructured Data

Raw data can contain valuable information, it is often unstructured and

difficult to interpret.
To extract meaningful insights, it needs to be transformed into a
structured format and processed using appropriate techniques.

35
 What is a Data Structure?

❏ Definition:
○ A data structure is a way to organize and store data so that it is
easy to access and manipulate.
❏ Why Data Structures are Important
○ Just like bookshelves and categories help organize a library, data
structures help organize digital data in a meaningful way.
○ In Industry, efficient data structures speed up processes, save
storage, and improve performance in apps and software.
36
Types of Data Structure: simple

37
 Types of Data Structure: complex

❏ A digital image image contains discrete number of pixels

− Pixel value: shows an intensity value
• “grayscale”

38
 Types of Data Structure: complex

❏ A digital image image contains discrete number of pixels

− Pixel value: shows an intensity value
• “grayscale”
• “color”

39
 Types of Data Structure: complex

Binary image

Grayscale image
40
 Exercise

❏ Read a grayscale image as a matrix

41
 Types of Data Structure: complex

Video as a Sequence of Images

42
Types of Data Structure

43
https://simplerize.com/data-structures/data-structure-introduction
 Exercise

❏ Primitive data types

44
 Common operations on Data Structure

rearrange elements in an orde

adding an element
Traversing all elements
Looking for an element
removing an element
45
https://findtodaysnotes.wordpress.com/operations-of-data-structures/
 What is Algorithm?

Imagine you want to

bake a cake. You follow
a recipe with specific
steps: measure
ingredients, mix, bake,
etc.

46
 What is Algorithm?

● Definition:
❏ An algorithm is a sequence of instructions to complete a task.

● Importance of Algorithms in Industry:

❏ Algorithms power everything from recommendation engines to
image recognition in autonomous vehicles.

47
 Why Data Structures Need Algorithms

Example – Searching for a Book

in a Library:
Returning to the library example:
You want a specific book. You could
search each book one by one (linear
search) or go directly to the relevant
shelf and section (binary search).

48
 Why Data Structures Need Algorithms

Key Idea: The data structure (shelf

organization) determines the search strategy
(algorithm).
● Industry Application: Faster search
and retrieval methods save time and
resources, especially in systems
handling vast amounts of data, like e-
commerce databases and search
engines. 49
 Real-World Example – Building a Ride-Sharing App

Imagine you are tasked with building a ride-sharing app.

Data Collected: User locations, ride times, driver availability.

Data Structures Used:

● Hash Maps: For quick lookups of drivers and locations.
● Priority Queues: To match the closest available driver to
a user.

50
 Real-World Example – Building a Ride-Sharing App

Imagine you are tasked with

building a ride-sharing app.
Algorithms Applied:
● Shortest path algorithms
to find the fastest route.
● Matching algorithms to pair
drivers with riders efficiently.

51
Ways of algorithms expression

Flowchart

Pseudo Code
52
 Flowchart

● The American National Standards Institute (ANSI) set

standards for flowcharts and their symbols in the
1960s.
● The International Organization for Standardization
(ISO) adopted the ANSI symbols in 1970.
● The current standard, ISO 5807, was revised in 1985.
● Generally, flowcharts flow from top to bottom and left
to right.
● Flowchart is a graphical representations of steps of an
53
Algorithm and programming logic.
Flowchart SYMBOLS

54
 Example

Consider the example to add three numbers and print the sum

55
 Example

Finding the average of N numbers

56
 Example

Find the Maximum of Three Numbers

57
 PSEUDO CODE

● A simple way of writing programming code in English

● Simply an implementation of an algorithm in the form of annotations
and informative text written in plain English.
● Not actual programming language.
● While underrating pseudo code is usually not difficult, writing it can be
a challenge.

58
 WHY USE PSEUDO CODE AT ALL?

❖ If we write an algorithm in English:

➢ The description may be at so high-level that it is difficult to analyze
the algorithm and to transform it into code. Not actual
programming language.
❖ If instead we write the algorithm in code:
➢ We have invested a lot of time in determining the details of an
algorithm we may not choose to implement?
❖ GOAL:
➢ Provide a high-level description of an algorithm which facilitates
59
analysis and eventual coding
PSEUDO CODE: SOME CONVENTIONS

60
 The Main Constructs of Pseudocode

● SEQUENCE represents linear tasks sequentially performed one after the

other.
● WHILE is a loop with a condition at its beginning.
● REPEAT-UNTIL is a loop with a condition at the bottom.
● FOR is another way of looping.
● IF-THEN-ELSE is a conditional statement changing the flow of the
algorithm.
● CASE is the generalization form of IF-THEN-ELSE.
● Invoking classes or calling functions (using the CALL keyword).
62
● Handling exceptions (using EXCEPTION, WHEN keywords).
The Main Constructs of Pseudocode

https://builtin.com/data-science/pseudocode 63
 The Main Constructs of Pseudocode

https://builtin.com/data-science/pseudocode

● Exception handling is the process of responding to unwanted or unexpected events when a computer
program runs. Exception handling deals with these events to avoid the program or system crashing,
and without this process, exceptions would disrupt the normal operation of a program.
● Exceptions occur for numerous reasons, including invalid user input, code errors, device failure, the 64
loss of a network connection, insufficient memory to run an application, and so on.
 Difference between CASE and IF

IF Statement:
● Primarily used when you need to evaluate a series of conditions that
may not be based on a single expression.

CASE Statement:
● Used when there is a single variable or expression with multiple
specific, distinct values.
65
PSEUDO CODE: IF-THE-ELSE

66
PSEUDO CODE: WHILE

67
 PSEUDO CODE: CASE

Conditional branching

68
PSEUDO CODE: For

69
 Example

Consider the example to add three numbers and print the sum
Step 1 : Start
Step 2 : Read A,B
Step 3 : Sum = A + B
Step 4 : Print Sum
Step 5: Stop

70
Example

71
Example

72
Example

73
Example

IF number 1 > number 2 and number 2> number 3

74
 Time and Space Complexity

❖ When it comes to solving

computational problems, choosing the
right algorithm is key to getting the job
done efficiently and effectively.
❖ That is where the concept of algorithm https://media4.giphy.com/media/EbeeDkvlC3fFRGJ6Om/200w.gif?cid=

complexity comes in. 6c09b952okngn7awcrabys3p99w07sp0cewxxenwp7qcyajk&ep=v1_gifs

_search&rid=200w.gif&ct=g

❖ Algorithm complexity refers to how

efficient an algorithm is in terms of
time and space required to solve a 75
 Problem: Finding the sum of numbers from 1 to 𝑛

Naive Approach: This solution

has a time complexity of O(n). it
requires n iterations to compute
the sum. For large n, it will take
significantly longer.
Optimized Approach: This
❖ Which code performs faster? approach has a time complexity
❖ Why is one code better? of O(1), as it computes the sum
in a single step regardless of n.

76
Problem: Calculate the Average Grade of Students

77
Problem: Calculate the Average Grade of Students

❏ We use multiple nested loops

unnecessarily, which increases
time complexity.
❏ We store redundant data,
increasing memory usage.
❏ We repeatedly calculate sums in
an inefficient way.

Time Complexity: O(n^2), due to the unnecessary nested loop.

Space Complexity: O(n), storing all student data in dictionaries

78
Problem: Calculate the Average Grade of Students

❖ We avoid unnecessary loops.

❖ We directly access the grade
values, reducing time complexity to
O(n).
❖ We only keep the data necessary
to calculate the average, reducing
memory usage.
Time Complexity: O(n) because we iterate over the list of students once.
Space Complexity: O(1)

79
 What is Time Complexity?

● A measure of the amount of time an algorithm takes to run as a

function of the size of its input.

80
 Best, Worst, and Average case

❖ Best case scenario: (Very rarely used)

➢ This scenario describes the best-performing version of an
algorithm.
➢ This scenario occurs when the input data is particularly well-suited
for the algorithm, and the algorithm is able to solve the problem with
minimal time and space complexity.
❖ Average case scenario: (Rarely used)
➢ This scenario describes the average-performing version of an
algorithm.
➢ This scenario occurs when the input data is typical for the algorithm,
and it takes an average amount of time and memory to solve the
81
problem.
 Best, Worst, and Average case

❖ Worst case scenario: (Mostly used)

➢ This scenario describes the worst-performing version of an
algorithm.
➢ This scenario occurs when the input data is particularly
challenging for the algorithm, and the algorithm takes a long time
and requires a large amount of memory to solve the problem.

82
Notations

83
 Notations

❖ O(n) — Big-O Notation (Upper Bound)

● Definition: O(n) describes an upper bound on the time complexity
of an algorithm. It represents the worst-case scenario, where the
runtime grows no faster than linearly with the size of the input.
● Example: If an algorithm is O(n), it means that as n grows, the
time taken by the algorithm will not exceed a linear rate in n.

84
 Notations

❖ Ω(n) — Big Omega Notation (Lower Bound)

● Definition: Ω(n) represents a lower bound on the time complexity.
It’s the best-case time complexity, meaning that the runtime will
grow at least linearly with the size of the input.
● Example: If an algorithm is Ω(n), there is no input size for which it
will take less than linear time.

85
 Notations

❖ Θ(n) — Big Theta Notation (Tight Bound)

● Definition: Θ(n) describes both an upper and lower bound on the
time complexity. This indicates that the algorithm’s time complexity
grows linearly with the input size in the average case as well as in
the worst and best cases.
● Example: If an algorithm has Θ(n) complexity, the time taken
grows linearly with n in all cases, making it both the upper and
lower bound.

86
 Notations

❖ o(n)— Little-O Notation (Strict Upper Bound)

● Definition: o(n) denotes a strict upper bound on the growth rate of
the time complexity, meaning the algorithm’s runtime grows more
slowly than linear time for very large n.
● Example: If an algorithm has a time complexity of o(n), it will
eventually take less than linear time for large input sizes, making it
strictly better than O(n).

87
 Notations

❖ ω(n) — Little Omega Notation (Strict Lower Bound)

● Definition: ω(n) represents a strict lower bound, meaning that the
algorithm’s runtime will grow faster than linear time for large input
sizes.
● Example: If an algorithm has ω(n) complexity, its runtime increases at
a rate faster than linear time as nnn grows.

88
 big O notation

The big O notation can be used to express time complexity in several

different ways:
O(1): Constant time complexity (Excellent/Best).
O(log n): Logarithmic time complexity (Good).
O(n): Linear time complexity (Fair).
O(n log n): Log linear time complexity (Bad).
O(n^2): Quadratic time complexity (worse).
O(n^3): Cubic time complexity (worse).
O(2^n): Exponential time complexity (worse).

89
big O notation

90
 Exercise

What is the time complexity of this code? (Single Loop)

1.Identify input size n.

2.Count significant operations (especially
in loops and recursion).
3.Express the complexity using Big-O.
Answer: O(n) 4.Drop constants and lower-order terms
Explanation: The loop runs n for simplification.
times, so the time complexity is
directly proportional to n.
91
 Exercise

What is the time complexity of this code? (Nested Loop)

Answer: O(n^2)
Explanation: The outer loop runs n times, and for each iteration of the
outer loop, the inner loop also runs n times. This results in n * n
= n^2 total operations.
92
 Exercise

What is the time complexity of this code? (Variable Loop)

O(1): Constant time complexity
(Excellent/Best).
O(log n): Logarithmic time complexity (Good).
O(n): Linear time complexity (Fair).
O(n log n): Log linear time complexity (Bad).
O(n^2): Quadratic time complexity (worse).
O(n^3): Cubic time complexity (worse).
O(2^n): Exponential time complexity (worse).

Answer: O(n^2)
Explanation: The inner loop's number of iterations decreases as i
increases, but overall, it still sums to approximately n * (n - 1)
/ 2, which simplifies to O(n^2).
93
 Exercise

What is the time complexity of this code? (Loop with Condition)

For i in range(n):
if i%2 == 0
print(i)
i = i /2

Answer: O(n)
Explanation: Although only half the values of i satisfy the condition i % 2 ==
0, the if check itself takes constant time. So, the overall complexity remains
O(n).
n+n/2 or 2n, it simplifies to O(n)
94
 Exercise

What is the time complexity of this code? (If-Else Blocks with Loops)

Answer: O(n^2)

95
 Exercise

What is the time complexity of this code? (If-Else Blocks with Loops)
Answer: O(n^2)

96
 Exercise

What is the time complexity of this code? (If-Else Blocks with Loops)
Answer: O(n^2)

97
 Exercise

What is the time complexity of this code? (If-Else Blocks with Loops)
Answer: O(n^2)

98
 Exercise

What is the time complexity of this code? (Dividing in a Loop)

Answer: O(log n)

99
 What is Space Complexity?

● A measure of the amount of memory an algorithm requires to run.

100
 Exercise

What is the Space complexity of this code?

Answer: O(1)
Explanation: The function only
creates a single integer variable x,
which requires constant space. The
space complexity is independent of n.

101
 Exercise

What is the Space complexity of this code?

Answer: O(n)
Explanation: The function creates an
array of size n, requiring space
proportional to n.

102
 Exercise

What is the Space complexity of this code?

Answer: O(n)

103
 Answers

❖ Why Computers and What are

computer features?
✔ Computers follow instructions to
solve problems, automate tasks,
and help people communicate.
✔ Features of computers include:

104
 Answers

❖ Similarity of human and computers?

✔ CPU
✔ RAM
✔ …

105
 Answers

❖ What are Computer Components?

106
 Answers

❖ What is Data (Definition of Data at Hardware Level)?

✔ Data is the raw information.
✔ In a computer, data is represented in binary form, using only two
states: 0 and 1. These states correspond to electrical voltages in the
computer's hardware.

107
 Answers

❖ What is Data Structure?

✔ A data structure is a way to organize and store data so that it is easy
to access and manipulate.

108
 Answers

❖ What is Algorithm?
✔ An algorithm is a sequence of instructions to complete a task.

109
 Answers

❖ What is Time and Space Complexity?

✔ Time complexity is a measure of the amount of time an algorithm
takes to run as a function of the size of its input.
✔ Space complexity is a measure of the amount of memory an
algorithm requires to run.

110
 References

❑ [Agarwal, 2022] Agarwal, B. (2022). Hands-on data structures and

algorithms with Python: Store, manipulate, and access data
effectively and boost the performance of your applications. Packt
Publishing Ltd.

111
THANKS!

112
113