Data Structures

<Lecture 14: Heap>

Sungmin Cha
New York University

10.23.2024
 Outline
• Notice

• Review the previous lecture

• Heap
– Priority queue
– Operations of heap
– Heap Sort

• Summary for the midterm exam

2
 Outline
• Notice

• Review the previous lecture

• Heap
– Priority queue
– Operations of heap
– Heap Sort

• Summary for the midterm exam

3
 Notice
• Midterm exam
– 12:30 PM on Monday, October 28th
– Duration: 1 hour 15mins
– Location: 60 Fifth Ave, 110
– The Midterm exam covers Lecture 1-14
– Offline exam and closed book (no cheat sheets allowed)
– Include True/False and multiple-choice questions, subjective
questions, and filling in appropriate pseudo code for blanks
 All test questions will be created based on lecture slides and HW1-2
(Labs are not considered)
 There will be 1-2 challenging questions
 In total, there will be 10-12 questions (excluding T/F questions)

4
 Outline
• Notice

• Review the previous lecture

• Heap
– Priority queue
– Operations of heap
– Heap Sort

• Summary for the midterm exam

5
 Quick Sort
• Quick sort
– A divide-and-conquer algorithm devised by Charles Antony
Richard Hoare

– Three steps of quick sort

1. Divide: split the input array into two uneven parts based on the
pivot (left of the pivot: elements smaller than the pivot, right of the
pivot: elements larger than the pivot)
2. Conquer: Sort the subarrays. If the size of a subarray is not small
enough, apply the divide-and-conquer method again using recursive calls.
3. Combine: Merge the sorted subarrays into a single array

6
 Quick Sort
• Pseudo code of quick sort

7
 Quick Sort
• Time complexity of quick sort (Big-O, worst case)
– When is the worst case of quick sort?
 In case of each subarray being unevenly divided
(when the array is already sorted)

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8
…

1 2

1 2 8
 Quick Sort
• Time complexity of quick sort (Big-O, best case)
– When is the best case of quick sort?
 In case of each subarray being evenly divided

n/2 n/2

n/4 n/4 n/4 n/4

1 1 1 1 1 1 1 1

9
 Quick Sort
• Pros and Cons of quick sort
– Pros
 Fastest sorting algorithm on average (empirically)
 Do not require additional memory

– Cons
 time complexity of O(n^2) in case of the worst case

10
 Quick Sort
• Merge vs Quick sort
– They are based on divide-and-conquer but
 Merge sort: the given array is divided into smaller units until it cannot
be divided further, and sorting is performed during the merging process.
 Quick sort: sorting is performed during the process of dividing the
given array.
– Additional memory
 Merge sort requires an additional memory but quick sort is not

11
 Tree
• What is Tree?
– A non-linear hierarchical data structure where nodes are
connected like branches of a tree

12
 Tree
• Usage of Tree
1. It can be used to implement a Head (priority queue)
 Sorting an array using Heap => Heap sort
2. It can be used to store a dataset in a tree-structured format
 Example: directory structure and organizational chart

13
 Tree
• Binary tree
– A tree where every node has at most two child nodes
• Full binary tree
– A binary tree that has exactly two child nodes and is fully
occupied
– Example: 9

8 8

4 7 3 5

2 1 3 4 9 1 2 7

Every leaf node is located at the bottommost level 14

 Tree
• Complete binary tree
– When a binary tree is not fully occupied, a complete binary
tree tree is the closest approximation to a full binary tree
 Since it’s not possible to fill all leaf nodes, they are filled sequentially
starting from the left

8 8

4 7 3 5

2 1 3
15
 Outline
• Notice

• Review the previous lecture

• Heap
– Priority queue
– Operations of heap
– Heap Sort

• Summary for the midterm exam

16
 Heap
• Priority queue
– The data structure that returns and deletes the data with the
highest priority, regardless of the order of data addition.
– Example: managing messages with priority

– Heap is the representative priority queue

17
 Heap
• Heap(max heap) must satisfy the following two conditions
1. Complete binary tree
2. Heap property
 Every node has a value and the value of each node is greater than or
equal to the value of its child nodes
– Example of Heap
The root node always has
9 the maximum value

8 8

4 7 3 5

2 1 3 18
 Heap
• Heap and its corresponding array
– Heap is generally implemented using an array
 Because we can easily get the index of a parent and child node of a node

9
The child nodes of A[k]
are A[2k+1] and A[2k+2]
8 A[1] 8

A[3] 4 7 A[4] 3 5

2 1 3

9 8 8 4 7 3 5 2 1 3

19
 Heap
• Heap and its corresponding array
– Heap is generally implemented using an array
 Because we can easily get the index of a parent and child node of a node
𝑘−1
9 The parent node of A[k] is A[⌊ ⌋]
2

8 8 A[2]

4 7 3 5 A[6]

2 1 3

9 8 8 4 7 3 5 2 1 3

20
 Heap
• ADT of Heap

– Insert() and deleteMax()

 After adding and removing an element from the heap, the heap must
retain its properties
– buildHeap()
 For the given array, it builds a heap which satisfies its properties
21
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))

7 8

4 3 3 5

2 1
A[0]
9 7 8 4 3 3 5 2 1

22
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 1. insert ‘8’ to the end of A[0, …, n-1]

7 8

4 3 3 5

2 1 8
A[0]
9 7 8 4 3 3 5 2 1 8

23
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 2. compare A[9]=8 with its parent node (A[4] = 3)

7 8

4 3 3 5

A[9]=8 > A[4]=3

2 1 8 : it does not satisfy the property of Heap
A[0] A[4] A[9]
9 7 8 4 3 3 5 2 1 8

24
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 2. compare A[9]=8 with its parent node (A[4] = 3)

7 8

4 8 3 5

2 1 3 Swap A[4] and A[9]!

A[0] A[4] A[9]

9 7 8 4 8 3 5 2 1 3

25
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 2. compare A[4]=8 with its parent node (A[1] = 7)

7 8

4 8 3 5

2 1 3
A[0] A[1] A[4]
9 7 8 4 8 3 5 2 1 3

26
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 2. compare A[4]=8 with its parent node (A[1] = 8)

A[4]=8 > A[1]=7

7 8

4 8 3 5

2 1 3
A[0] A[1] A[4]
9 7 8 4 8 3 5 2 1 3

27
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 2. compare A[4]=8 with its parent node (A[1] = 8)

8 8

4 7 3 5

2 1 3 Swap A[4] and A[1]!

A[0] A[1] A[4]

9 8 8 4 7 3 5 2 1 3

28
 Insert Operation of Heap
• Implementation of insert(x) function
– Example: Insert ‘8’ to the heap A[0, …, n-1] (insert(8))
 3. compare A[1]=8 with its parent node (A[0] = 9)

9
A[1]=8 < A[0]=9
: satisfy the property
8 8

4 7 3 5

2 1 3
A[0] A[1]
9 8 8 4 7 3 5 2 1 3

29
 Insert Operation of Heap
• Implementation of insert(x) function
– PercolateUp operation
1. Compare node A[i] with its parent’s value and
swap them if they break the heap property
2. Compare the changed parent with its parent again and
if it doesn’t satisfy the heap property
3. Continue this process until reaching
o the first point where the heap property is satisfied
o or until it cannot go further up

– It can be implemented by a recursive fuction!

30
 Insert Operation of Heap
• Pseudo algorithm of insert(x) function

• Time complexity of insert(x)

– Best case: O(1), worst case: O(logn) 31
 Delete Operation of Heap
• Implementation of deleteMax() function
– Priority queue deletes the value which has the highest priority
– In Heap, the root node should be removed and return its value
9

8 8

4 7 3 5

2 1 3
A[0]
9 8 8 4 7 3 5 2 1 3

32
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 1. delete and return the value of the root node

8 8

4 7 3 5

2 1 3
A[0]
9 8 8 4 7 3 5 2 1 3

33
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 1. delete and return the value of the root node

It does not satisfy the

property of Heap
8 8

4 7 3 5

2 1 3
A[0]
8 8 4 7 3 5 2 1 3

34
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 2. Move the last node to the root node

8 8

4 7 3 5

2 1 3
A[0]
8 8 4 7 3 5 2 1 3

35
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 2. Move the last node to the root node

8 8

4 7 3 5

2 1
A[0]
3 8 8 4 7 3 5 2 1

36
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 3. compare A[0]=3 with its largest child node (A[1] = 7)

3
A[0]=3 < A[1]=8

8 8

4 7 3 5

2 1
A[0] A[1]
3 8 8 4 7 3 5 2 1

37
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 3. compare A[0]=3 with its largest child node (A[1] = 7)

3 8

4 7 3 5

2 1 Swap A[0] and A[1]!

A[0] A[1]
8 3 8 4 7 3 5 2 1

38
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 4. compare A[1]=3 with its largest child node (A[1] = 7)

3 8

4 7 3 5

2 1 A[1]=3 < A[4]=7

A[1] A[4]
8 3 8 4 7 3 5 2 1

39
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 4. compare A[1]=3 with its largest child node (A[1] = 7)

7 8

4 3 3 5

2 1 Swap A[1] and A[4]!

A[1] A[4]
8 7 8 4 3 3 5 2 1

40
 Delete Operation of Heap
• Implementation of deleteMax() function
– Example: deleteMax()
 5. A[4]=3 is the leaf node

7 8

4 3 3 5

2 1 Swap A[1] and A[4]!

A[1] A[4]
8 7 8 4 3 3 5 2 1

41
 Delete Operation of Heap
• Implementation of deleteMax() function
– PercolateDown operation
1. Return the value of the root node
and move the last node to the root node position
2. Compare the changed root node with its child nodes
and swap if it doesn’t satisfy the heap property
3. Compare the changed node with its child nodes
and swap if it doesn’t satisfy the heap property
4. Repeat step 3 until the heap property is satisfied
or until reaching a leaf node

– Similarly, this is a recursive operation!

42
 Delete Operation of Heap
• Pseudo algorithm of deleteMax() function

• Time complexity of deleteMax()

– Best case: O(1), worst case: O(logn) 43
 Build Operation of Heap
• Implementation of buildHeap() function
– A heap can be initialized from an empty array or starting from a
given array
– In the latter case, we first need to convert it into a heap
– Example: the given array

A[0]
4 1 8 7 3 3 5 2 9

Heap

44
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 1. get an array and interpret it as a complete binary tree

How can we modify this tree

to satisfy the properties of heap? 4

1 8

7 3 3 5

2 9
A[0]
4 1 8 7 3 3 5 2 9
45
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 1. get an array and interpret it as a complete binary tree
(4) Sequentially make each subtree
satisfy the heap property
4 starting from the smallest one

1 8

7 3 3 5
(1) (2)
2 9 (3)
A[0]
4 1 8 7 3 3 5 2 9
46
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 2. compare A[3]=7 with its largest child node (A[8] = 9)

1 8

7 3 3 5
A[3]=7 < A[8]=9

2 9

A[0] A[3] A[8]

4 1 8 7 3 3 5 2 9
47
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 2. compare A[3]=7 with its largest child node (A[8] = 9)

1 8

9 3 3 5

2 7 Swap A[8] and A[3]!

A[0] A[3] A[7]

4 1 8 9 3 3 5 2 7
48
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 3. compare A[2]=8 with its largest child node (A[6] = 5)

1 8
A[2]=8 > A[6]=5
9 3 3 5

2 7

A[0] A[2] A[6]

4 1 8 9 3 3 5 2 7
49
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 3. compare A[1]=1 with its largest child node (A[3] = 9)

1 8
A[1]=1 < A[3]=9

9 3 3 5

2 7

A[0] A[1] A[3]

4 1 8 9 3 3 5 2 7
50
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 3. compare A[1]=1 with its largest child node (A[3] = 9)

9 8

1 3 3 5

2 7 Swap A[1] and A[3]!

A[0] A[1] A[3]

4 9 8 1 3 3 5 2 7
51
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 4. compare A[3]=1 with its largest child node (A[8] = 7)

9 8

1 3 3 5
A[3]=1 < A[8]=7

2 7

A[0] A[3] A[8]

4 9 8 1 3 3 5 2 7
52
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 4. compare A[3]=1 with its largest child node (A[8] = 7)

9 8

7 3 3 5

2 1 Swap A[3] and A[8]!

A[0] A[3] A[8]

4 9 8 7 3 3 5 2 1
53
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 5. compare A[0]=4 with its largest child node (A[1] = 9)

A[0]=4 < A[1]=9 4

9 8

7 3 3 5

2 1

A[0] A[1]
4 9 8 7 3 3 5 2 1
54
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 5. compare A[0]=4 with its largest child node (A[1] = 9)

4 8

7 3 3 5

2 1

A[0] A[1] Swap A[0] and A[1]!

9 4 8 7 3 3 5 2 1
55
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 6. compare A[1]=4 with its largest child node (A[3] = 7)

4 8
A[1]=4 < A[3]=7

7 3 3 5

2 1

A[1] A[3]
9 4 8 7 3 3 5 2 1
56
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 6. compare A[1]=4 with its largest child node (A[3] = 7)

7 8

3 3 3 5

2 1

A[1] A[3] Swap A[1] and A[3]!

9 7 8 3 3 3 5 2 1
57
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 7. compare A[3]=4 with its largest child node (A[7] = 2)

7 8

3 3 3 5
A[3]=4 > A[7]=2

2 1

A[3] A[7]
9 7 8 3 3 3 5 2 1
58
 Build Operation of Heap
• Implementation of buildHeap() function
– Example: buildHeap()
 8. Checked all subtrees!

7 8

3 3 3 5

2 1

A[3] A[7]
9 7 8 3 3 3 5 2 1
59
 Build Operation of Heap
• Implementation of buildHeap() function
– The overall idea of buildHeap()
 Sequential access to subtrees

9 A[0]

A[1] 7 8 A[2]

A[3] 3 3 3 5

2 1

A[0] A[1] A[2] A[3]

9 7 8 3 3 3 5 2 1
60
 Build Operation of Heap
• Implementation of buildHeap() function
– The overall idea of buildHeap()
 Confirm whether satisfies the property => percolateDown()

9 A[0]

A[1] 7 8 A[2]

A[3] 3 3 3 5

2 1

A[0] A[1] A[2] A[3]

9 7 8 3 3 3 5 2 1
61
 Delete Operation of Heap
• Pseudo algorithm of buildHeap() function

• Time complexity of buildHeap()

– O(n)
 [for each percolateDown: one comparison + one movement]
X [(n-1)/2 numbers of percolateDown calls]
= O(n-1) = O(n)
62
 Heap Sort?
• Pseudo algorithm of heapSort() function

• Time complexity of heapSort()

– O(nlogn)
 [n iterations] X [time complexity of deleteMax()]
= O(n x logn) = O(nlogn)

63
 Review for the midterm exam
• What we have learned so far
– JAVA’s variables and memory allocation of them
– Recursion and time complexity
– Array and Linked List
– Stack and Queue
 Implemented by an array and linked list
– Sorting algorithm
 Bubble, selection, insertion
 Merge, quick

64
 Review for the midterm exam
• JAVA’s variables and memory allocation
– Primitive and reference variable
– Memory allocation in Stack and Heap memory

• Recursion and time complexity

– Calculating the time complexity of algorithm (Big-O)
– What is recursion? How to implement the recursive function?

• Array and Linked List

– Operations to maintain each list
 How to implement them?
– Time complexity of each operation
65
 Review for the midterm exam
• Stack and Queue
– Operations to maintain Stack and Queue
 FIFO, LIFO
 How to implement them?
o LinkedQueue and Stack
o ArrayQueue and Stack
 Time complexity of each operation

• Sorting
– Why do we need to sort the data?
– How to implement sorting algorithms?
– Time complexity of them?
 What is the time complexity in the best and worst cases?
66
 Review for the midterm exam
• Tree and Heap
– Terminology
– Binary tree
– Heap and its property and operations
– Heap sort

67
 Thank you!

E-mail: [email protected]

68