Data Structures & Algorithms

Lecture 1: Introduction

Dr. Syed Imran Ali

[email protected]

Department Of Computing (DOC),

School of Electrical Engineering & Computer Science (SEECS),
National University of Sciences & Technology (NUST)

Courtesy: Dr. Arsalan Ahmad and Dr. Faheem

 Contact Details

▪ Office A-301, Department of Computing S E E C S

▪ Email: [email protected]

▪ Consultation hours: Wednesday (11am ~ 1pm)

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 2

 What to expect?

▪ Objectives:

► Learn common data structures i.e., linked lists, queues, trees

and graphs

► Study routinely used algorithms in computer science including

sorting, search trees, hash-tables, heaps, graph theory and
recursion

► Evaluation of these algorithms in terms of running time

complexity

- Introduction to techniques like Big-Oh notation, Recurrences and

basic probabilistic analysis of random algorithms

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 3

 Course content
▪ Introduction to data structures & algorithms
▪ Pointers, structures & arrays
▪ Linked Lists (single, double & circular)
▪ Stacks (and its applications)
▪ Queues
▪ Algorithm (with asympototic analysis)
▪ Sorting algorithms (insertion-, merge-, quick-sort)
▪ Recurrsion
▪ Trees (Binary tree, B-tree, etc)
▪ Heapsort
▪ Hash tables
▪ Graphs
▪ Search algorithms (Depth First Search)
▪ Shortest path algorithm

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 4

 Recommended literature

▪ Text books:

► Adam Drozdek, Data Structures and Algorithms in C++

(Fourth edition)

► T. H. Cormen, Charles E. Leiserson, R. L. Rivest, Clifford S.

Introduction to Algorithms, (Third Edition)

▪ Reference books:

► Steven S Skiena. The Algorithm Design Manual, Second

Edition (2008)

► Mark A. Weiss, Data Structures and Algorithm Analysis in

C++, Third Edition (2006)
Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 5
 Class ethics

▪ Attendance (75% mandatory)

▪ Be disciplined & give respect to all
▪ Quizzes
► Anytime
▪ “Don’t cheat”
► You may try BUT if caught NO MERCY!
► Will NOT accept phrases like:
- “… by chance we had the same variable name”
▪ Assignments
▪ Plagiarism
► No copying

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 6

 Marks Distribution

▪ Assignments : 10%
▪ Quizzes : 10%
▪ Mid Term : 30 % 75%
▪ Final : 40%
▪ Project Doc. : 10%
▪ Lab Tasks : 70%
25%
▪ Final Lab : 10%
▪ Project : 20%

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 7

 What is a Data Structure?

▪ Data Structure => Data StructurING

► How do we organize information so that we can find, update,

add, and delete portions of it efficiently?

► Data Structure is a systematic way to organize data in order to

use it efficiently

► Data Structure Categorization:

- In terms of Types: Primitive vs User-Defined
- in terms of memory allocation: static vs dynamic

► Non-primitive or User-Defined:
- Linear vs Non-linear

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 8

 Why to perform Data Structuring?

▪ How many cities with more than 250,000 people lie within
500 km of Islamabad?

▪ How many people in a company make over Rs.1,000,000

per year?

▪ Can we connect all of our telephone customers with less

than 1,000 kms of cable?

To answer questions like these, it is not enough to have the

necessary information
We must organize that information in a way that allows
us to find the answers in time to satisfy our needs

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 9

 What is a Data Structure?

▪ It’s an agreement about:

► How to store a collection of objects in memory,

► What operations we can perform on that data,

► The algorithms for those operations, and

► How time and space efficient those algorithms are

▪ A collection of data elements whose organization is

managed by the operations that are used to store and
retrieve the individual data elements

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 10

 What is a Data Structure Anyway?

▪ It’s an agreement about:

► how to store a collection of objects in memory,
► what operations we can perform on that data,
► the algorithms for those operations, and
► how time and space efficient those algorithms are.

▪ Ex. List data structure

► A collection of homogenous elements with linear relationship b/w
them e.g. list of students registered in this class
► Operations: Can access, change, insert or delete elements
► Algorithms: Searching → linear vs binary search
► Time Complexity: linear search O(n), binary search in array-list
O(log2n), More operations later.

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction

 Why is Data StructurING necessay?

Three common problems:

▪ Data Search
► E.g., an inventory of 1 million(106) items of a store

▪ Processor Speed
► Processor speed although being very high, falls limited if the
data grows to billion records

▪ Multiple Requests
► As thousands of users can search data simultaneously on a
web server, even the fast server fails while searching the data

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 12

 Foundation terms

Each data structure has an

▪ Interface
► Represents the set of operations that a data structure
supports &
► Only provides the list of supported operations, type of
parameters they can accept and the return type of these
operations

▪ Implementation
► Provides the internal representation of a data structure &
► The definition of the algorithms used in the operations of the
data structure

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 13

 Example

▪ Data structure for storing data of students:-

► Arrays
► Linked Lists

▪ Issues
► Space needed
► Operations efficiency (Time required to complete operations)
- Retrieval
- Insertion
- Deletion

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 14

 Which data structure to use?

▪ Data structures let the input and output be represented in a

way that can be handled efficiently and effectively

array

Linked list

queue
tree stack

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 15

 Introduction to algorithms

▪ Algorithm

► Named after a Muslim mathematician, Al-Khawarzmi

► Definition

- An algorithm is a definite procedure for solving a problem in

finite number of steps

- Algorithm is a well-defined computational procedure that takes

some value(s) as input, and produces some value(s) as output

- Algorithm is finite number of computational statements that

transform input into the output

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 16

 What is an algorithm?

▪ An algorithm is an effective method for solving a

problem using a finite sequence of instructions

▪ Examples
► sort a list of numbers
► find a route from one place to another (cars, packet routing,
phone routing, ...)
► find the longest common substring between two strings
► microchip wiring/design (VLSI)
► solving sudoku
► cryptography
► compression (file, audio, video)
► pagerank
► classify a web page

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 17

 Properties of interest

▪ What properties of algorithms are we interested in?

► Does it terminate?

► Is it correct, i.e. does it do what we think it’s supposed to do?

► What are the computational costs?

► What are the memory/space costs?

► What happens to the above with different inputs?

► How difficult is it to implement and implementing it correctly?

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 18

 Data structures & algorithms

▪ Operations are associated with data structure

▪ To perform these operations, algorithms are needed

Data structures +Algorithms = Programs

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 19

 Data type

▪ A data type is a type together with a collection of operations

to manipulate the type

► E.g., an integer variable is a member of the integer data type

while addition is an example of an operation on the integer
data type

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 20

 Data types

▪ Primitive Data Types

► Bool, Int, float etc.

▪ User-Defined Data Types (UDTs)

► Aggregate data types e.g., structures, array-of-integers etc.

▪ Abstract Data Types (ADTs)

► Does not specify how the data type is implemented

► These implementation details are hidden from the user of the

ADT and protected from outside access, a concept referred to
as encapsulation

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 21

 Abstract data types (ADT)

▪ Different implementations possible for same ADT

▪ In an object-oriented language such as C++, an ADT

and its implementation together make up a class

Much of our concern in this course is how to implement data

structures as ADTs

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 22

 Abstract Data Types (ADTs)

▪ Data storage & operations encapsulated by an ADT.

▪ ADT specifies permitted operations as well as time and space
guarantees.
▪ User unconcerned with how it’s implemented
► but we are concerned with implementation in this class
▪ ADT is a concept or convention:
► not something that directly appears in your code
► programming language may provide support for communicating ADT
to users
► (e.g. classes in Java & C++)
Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 23
 Data Organizing Principles

▪ Ordering
► Put keys into some order so that we know something about where
each key is relative to the other keys.
► Phone books are easier to search because they are alphabetized.
▪ Linking
► Add pointers to each record so that we can find related records
quickly.
► E.g. The index in the back of book provides links from words to the
pages on which they appear.
▪ Partitioning:
► Divide the records into 2 or more groups, each group sharing a
particular property.
► E.g. Multi-volume encyclopedias (Aa-Be, W-Z)
► E.g. Folders on your hard drive

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 24

 Ordering

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 25

 Linking

▪ Records located any where in memory

▪ Green pointers give “next” element
▪ Red pointers give “previous” element
▪ Insertion & deletion easy if you have a pointer to the middle of the
list
▪ Don’t have to know size of data at start
▪ Pointers let us express relationships between pieces of
information.

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 26

 Partitioning

▪ Ordering implicitly gives a partitioning based on the “<“ relation.

▪ Partitioning usually combined with linking to point to the two
halves.
▪ Prototypical example is the Binary Search Tree:

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 27

 So,

▪ Much of programming (and thinking about programming) involves

deciding
▪ how to arrange information in memory. [Aka data structures.]
▪ Choice of data structures can make a big speed difference.
► Sequential search vs. Binary Search means O(n) vs. O(log n).
► [log (1 billion) < 40].
▪ Abstract Data Types are a way to encapsulate and hide the
implementation of a data structure, while presenting a clean
interface to other programmers.
▪ Data structuring principles:
► Ordering
► Linking
► (Balanced) partitioning

Imran Ali: Data Structures & Algorithms Lecture 1: Introduction 28