Introduction to Algorithms

Chapters: 1 & 2 Lecture 1 Outline

Pages: 1 …28  Course information and
Policies
 Introduction to Algorithms
 Insertion Sort
 Algorithm
 Example
 Analysis
 Loop Invariants

H. Frigui CECS 419/619: Lecture 1 1

 Instructor
 Dr. Hichem Frigui
204 Duthie Center
[email protected]
852 – 2009

Office Hours: M. W. 10:00 – 10:55am

H. Frigui CECS 419/619: Lecture 1 2

 Course Information
 CECS-419 and CECS-619
 Same material for both courses
 Additional reading/hwk/exam questions/project for CECS-619

 Course Prerequisites:
 Good working knowledge of data structures such as linked lists, trees, and
dynamically allocated structures is required.
 CECS 302 Data Structures
 CECS 310 Discrete Structures
 Text Book:
“Introduction to Algorithms”, 3rd Edition.
Thomas H. Cormen, Charles E. Leiserson,
Ronald L. Rivest, Clifford Stein. (2009). MIT Press.

H. Frigui CECS 419/619: Lecture 1 3

 Course Grading:
 1 mid-term exam (15%)
 1 final exam (20%)
 Quizzes (40%):
 Pre-Subject Quizzes (15%)
 Post-Subject Quizzes (25%).
 Homework (15%):
 Several homework sets will be assigned.
 Not all assignments are equally weighted, and some of
them would involve programming.
 Project (10%)
H. Frigui CECS 419/619: Lecture 1 4
 Course Schedule
Week Day Topics Notes
Week 1 M 1) Introduction; Insertion Sort
8/22 -- 8/26 W 2) Asymptotic Analysis/Notation
F 3) Recursion; Recurrences; Merge Sort
Week 2 M 4) Solving Recurrences
8/29 -- 9/2 W 5) Divide&Conquer; examples; Strassen's Alg.
F 6) Quicksort; Randomized Quicksort; Order Stat Assign Hwk 1
Week 3 M No Class: Labor day
9/5 -- 9/9 W 7) Review Session 1 Hwk 1 due (by 10am)
F 8) Order Statistics; Quiz 1
Week 4 M 9) Randomized divide & conquer
9/12 -- 9/16 W 10) Linear time sorting:Counting &Radix sort
F 11) Trees; Heap; Heapsort Assign Hwk 2
Week 5 M 12) Priority Queue; Assign Project 1
9/19 -- 9/23 W 13) Review Session 2 Hwk 2 due (by 10am)
F 14) Assign Project 1 Quiz 2
Week 6 M 15) Hash Tables; Hash Functions (1)
9/26 -- 9/30 W 16) Hash Tables; Hash Functions (2)
F 17) Hashing with Chaining; Open Addressing Assign Hwk 3
Week 7 M No Class: Mid-term break
10/3 -- 10/7 W 18) Review Session 3 Hwk 3 due (by 10am)
F 19) Red-Black Trees(1) Quiz 3
Week 8 M 20) Red-Black Trees(2) Project 1 due
10/10 -- 10/14 W 21) Dynamic Programming (1)
F 22) Dynamic Programming (2) Assign Hwk 4
Week 9 M 23) Greedy algorithms
10/17 -- 10/21 W 24) Review Session 4 Hwk 4 due (by 10am)
F 25) Mid-Term Exam

H. Frigui CECS 419/619: Lecture 1 5

H. Frigui CECS 419/619: Lecture 1 6
 Course Policies
 Lecture slides, homework assignments, and other useful
information will be posted on the course web page.
 Regular attendance is highly recommended. If you miss a
lecture, you are responsible for all material covered or
assigned in class.
 2 TA’s will be available for this course. You are encouraged
to seek their help (their office hours will be announced soon)
 You should carefully read the Speed School section on Academic
Dishonesty Policy and Procedure (can be found online at
http://louisville.edu/speed/academics/academicDishonesty )
 You should carefully read the UofL Title IX/Clery Act Notification
(sexual misconduct). See
http://louisville.edu/hr/employeerelations/sexual-misconduct-brochure

H. Frigui CECS 419/619: Lecture 1 7

 Analysis of Algorithms
 The theoretical study of computer-program
performance and resource usage.
 What is more important than performance?
 Correctness
 Reliability
 Extensibility
 User-friendliness …

H. Frigui CECS 419/619: Lecture 1 8

 Why we study algorithms?
 Performance.
 Performance is the currency of computing
 Performance often draws the line between what is
feasible and what is impossible.
 Wise use of resources.
 Efficiency (time and space)
 Algorithms help us to understand scalability.
 We should consider algorithms, like computer
hardware, as a technology.
H. Frigui CECS 419/619: Lecture 1 9
 The Sorting Problem


H. Frigui CECS 419/619: Lecture 1 10

 Structure of Data
 Typically, the numbers to be sorted are part of a
collection of data called a record.
 Each record contains a key = value to be sorted

Record
Key Other data

 When the keys are rearranged, the records associated

with the keys need to be rearranged too.

H. Frigui CECS 419/619: Lecture 1 11

Insertion Sort: The Idea
 Start with empty hand and
cards on table face down
 Repeat
1. Pick one card at a time
2. Insert it in correct position in hand

To insert 12, we need to make

room for it by moving first 36
and then 24.

H. Frigui CECS 419/619: Lecture 1 12

Insertion Sort: The Idea
 Start with empty hand and
cards on table face down
 Repeat
1. Pick one card at a time
2. Insert it in correct position in hand

To insert 12, we need to make

room for it by moving first 36
and then 24.

H. Frigui CECS 419/619: Lecture 1 13

Insertion Sort: The Idea
 Start with empty hand and
cards on table face down
 Repeat
1. Pick one card at a time
2. Insert it in correct position in hand

To insert 12, we need to make

room for it by moving first 36
and then 24.

H. Frigui CECS 419/619: Lecture 1 14

Insertion Sort: The Idea
 Start with empty hand and
cards on table face down
 Repeat
1. Pick one card at a time
2. Insert it in correct position in hand

i j n

sorted
Key

H. Frigui CECS 419/619: Lecture 1 15

 Insertion Sort: The Algorithm
INSERTION-SORT (A,n)
j: “current card” being
inserted into hand for j=2 to n
key = A[j]
A[1 … j-1] : sorted hand i = j-1
“pseudocode”
while i>0 and A[i]>key
A[j+1 … n] : cards still on table
A[i+1] = A[i]
i = i-1
A[i+1] = key

Insertion sort sorts the elements in place

H. Frigui CECS 419/619: Lecture 1 16

 Insertion Sort

input array A 5 2 4 6 1 3

at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted

H. Frigui CECS 419/619:

17 Lecture 1
Insertion Sort: Example

H. Frigui CECS 419/619: Lecture 1 18

Insertion Sort: Example

H. Frigui CECS 419/619: Lecture 1 19

 Loop Invariants
 Proving loop invariants works like induction
1. Initialization (base case):
 It is true prior to the first iteration of the loop
2. Maintenance (inductive step):
 It is true before an iteration of the loop, and
 It remains true before the next iteration
3. Termination:
 When the loop terminates, the invariant gives us a useful property that
helps show that the algorithm is correct
 Stop the induction when the loop terminates

H. Frigui CECS 419/619: Lecture 1 20

 Loop Invariant: Insertion Sort
INSERTION-SORT (A,n)
for j=2 to n
1. Initialization
key = A[j] Just before the first iteration, j = 2:
i = j-1 the subarray A[1 . . j-1] = A[1],
while i>0 and A[i]>key
A[i+1] = A[i] (the element originally in A[1]) – is
I = i-1 sorted
A[i+1] = key

H. Frigui CECS 419/619: Lecture 1 21

 Loop Invariant: Insertion Sort
INSERTION-SORT (A,n)
for j=2 to n
2. Maintenance
key = A[j] • The loop moves A[j -1], A[j -2], A[j -
i = j-1 3], etc., by one position to the right
while i>0 and A[i]>key until the proper position for key is
A[i+1] = A[i]
I = i-1
found
A[i+1] = key • Then, the value of key is placed into
this position.

True before True after loop

j=3 j=4
loop

H. Frigui CECS 419/619: Lecture 1 22

 Loop Invariant: Insertion Sort
INSERTION-SORT (A,n)
for j=2 to n
3. Termination
key = A[j] Loop ends when j = n + 1  j-1 = n
i = j-1 Replace n with j-1 in the loop invariant:
while i>0 and A[i]>key
A[i+1] = A[i]
I = i-1
Loop invariant: at the start of the for
A[i+1] = key loop the elements in A[1 . . j-1] are in
sorted order

The entire array is sorted!

j=6 j-1 j

H. Frigui CECS 419/619: Lecture 1 23

 Analyzing Algorithms
 An algorithm is a finite set of precise instructions for
performing a computation or for solving a problem.
 What is the goal of analysis of algorithms?
 To predicting resources that it requires: memory,
Communication bandwidth, Computer hardware,
and Computational time (Running time)
 What do we mean by running time analysis?
 Determine how running time increases as the size
of the problem increases.

H. Frigui CECS 419/619: Lecture 1 24

 Input Size

 Input size (number of elements in the input)

 size of an array

 polynomial degree

 # of elements in a matrix

 # of bits in the binary representation of the input

 vertices and edges in a graph

25
 How do we compare algorithms?
 We need to define objective measures.
1. Compare execution times?
Not good: times are specific to a particular
computer !!
2. Count the number of statements executed?
Not good: number of statements vary with
the programming language as well as the
style of the individual programmer.

26
 Ideal Solution
 Express running time as a function of the
input size n (i.e., f(n)).
 Compare different functions corresponding to
running times.
 Such an analysis is independent of machine
time, programming style, etc.

27
 Machine-independent time
What is insertion sort worst case time?
 Depends on the speed of the computer
 Relative speed (on the same machine)
 Absolute speed (on different machines)

H. Frigui CECS 419/619: Lecture 1 28

 Analysis of Insertion Sort
INSERTION-SORT (A,n) Cost Times
for j=2 to n c1 n

key = A[j] c2 n-1

i = j-1 c3 n-1

while i>0 and A[i]>key c4

A[i+1] = A[i] c5
i = i-1 c6
A[i+1] = key c7 n-1

tj= number of times while loop is executed

H. Frigui CECS 419/619: Lecture 1 29
Running Time of Insertion Sort
Cost Times
c1 n
c2 n-1

c3 n-1

c4

c5

c6
c7 n-1

H. Frigui CECS 419/619: Lecture 1 30

 The running time depends on:
1. The number of elements to be sorted
Parameterize the running time by the size of the
input.

2. The input: an already sorted sequence is easier to sort

 Generally, we seek upper bounds on the
running time

H. Frigui CECS 419/619: Lecture 1 31

 Insertion Sort: Best Case

 Best Case: Array is already sorted: tj=1

A B

Linear

H. Frigui CECS 419/619: Lecture 1 32

 Insertion Sort: Worst-Case

 Worst-Case: Array is sorted in reverse order

 We must compare each element: tj=j

B C
A

Quadratic
H. Frigui CECS 419/619: Lecture 1 33