Scribd
Upload
32 views

Deterministic Selection and Sorting:: Problem P

This document discusses algorithms for sorting and selection problems. It begins by introducing the divide and conquer approach for solving problems recursively by dividing them into subproblems. It then provides examples of sorting algorithms like merge sorting and discusses their time complexities. Lower bounds for sorting are proved using a decision tree model. Selection algorithms for finding the kth best element are presented, along with lower bounds proofs. Other sorting algorithms discussed include Shellsort and Radixsort. Open problems in sorting complexity are noted.

Uploaded by

raw.junk
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
32 views

Deterministic Selection and Sorting:: Problem P

This document discusses algorithms for sorting and selection problems. It begins by introducing the divide and conquer approach for solving problems recursively by dividing them into subproblems. It then provides examples of sorting algorithms like merge sorting and discusses their time complexities. Lower bounds for sorting are proved using a decision tree model. Selection algorithms for finding the kth best element are presented, along with lower bounds proofs. Other sorting algorithms discussed include Shellsort and Radixsort. Open problems in sorting complexity are noted.

Uploaded by

raw.junk
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

size n

Problem P
Algorithms
Professor John Reif

fi divide into

subproblems
size n
solve these and "glue" together
solutions
k
T(n) = T(ni ) + g(n)
i=1

ALG 1.3
Deterministic Selection
and Sorting:

,...,n

time to combine solutions

(a) Selection Algorithms and Lower


Bounds
(b) Sorting Algorithms and Lower
Bounds

Examples:
1s t lecture' s mult

fas t fourier trans form

Main Reading Selections:


CLR, Chapters 7, 9, 10
Auxillary Reading Selections:
AHU-Design, Chapters 2 and 3
AHU-Data, Chapter 8
BB, Sections 4.4, 4.6 and 10.1
1

M(n) = 3 M

F(n) = 2 F

+ q(n)

( )

+ q(n)

B(n) = B

merge s orting

S (n) = 2 S

n
2

n
2

( )

binary search

n
2

n
2

+ q(1)

( )

+ q(n)
2

Examples

Selection, and Sorting


Decision Tree Model

Time =

on

( #oncomparisons
longest path )

input a,b,c
Binary tree
a<b?
root

Y
b<c?

Y
a,b,c

facts:

a<c?

b,a,c

a<c?
Y

b<c?

N
Y

a,c,b

with L Leaves
(1) has = L-1 internal nodes
(2) max height
log L

c<a<b

b,c,a

N
c,b,a

Merging

Algorithm Insert
input
( X1 < X 2

2 lists with total of n keys


X1 < X2 < ... < X k
input

< ... < X ) , (Y 1 )


k

and

Case k=n-1

Y1 < Y2 < ... < Y n-k


output
ordered merge of two key lists

Algorithm : Binary Search


by Divide-and-Conquer

[1] Compare Y1 w ith X k

goal
provably asymptotically optimal
algorithm in

[2] i f Y1 > X

Decision Tree Model

use this Model because it

k + 1

(
5

< . . . < Xk

else Y1 X

of lower bounds
allows simple proofs
so easy
time = # comparisons
to bound time costs

insert Y 1 into

k
2

k
2

and insert Y1 into

X1 < . . . < X k
2

Case: Merging equal length lists

(X < X < ... < X )

Input

(Y < Y < ... < Y


1

Total Comparison Cost:

log (k+1)

log (n)

w h e re k =

n- k

n
2

Since a binary tree with n=k+1 leaves

has depth > log(n) , this is optimal!

Algorithm
[1] i 1, j 1
[2] while

i k and j

k do

i f Xi < Yj t h e n ou t p u t ( Xi ) an d s e t i i +1
e l s e ou t p u t ( Yj ) an d s e t jj+1

[3] output

remaining elements

Algorithm clearly uses 2k-1 =


7

n-1 comparisons
8

Sorting by Divide-and-Conquer
Algorithm

Lower bound:

input

consider case X
1 < Y1 < X 2 < Y2 <...< X k < Yk
any merge algorithm must compare

Merge Sort
set S of n keys

claim:
n
[1] partition S into s et X of
2
n
keys
and s et Y of
2

(1) X i with Y i for i=1 ,..., k


(2) Y i with X i+1 for j=1 ,..., k-1
(otherwise we could
fi so requires

flip

Y i < Xi with no change)

keys

[2] Recursively compute


Merge Sort (X) = (X 1 , X 2 , . . . , X

2k-1 = n-1 comparisons!

n
2

Merge Sort (Y) = (Y 1 , Y 2 , . . . , Y

n
2

[3] merge above sequences


using n-1 comparisons
9

[4] output

merged sequence

10

Time Analysis

( ) ( )

T(n) = T

n
2

+ T

T(l) = 0
fi T(n) = n

log n

n
2

-2

log n

Lower Bounds on Sorting


(on decision tree model)

+ n-1

+ 1
d epth log (n !)

= q (n log n)
.
...

n! distinct leaves

11

12

Easy Approximation

(via Integration)

log(n!) = log(n) + log(n-1) + . . . + log(2) + log(1)

n -1

log x dx + ... + log x dx


1

log x dx ( Since log k


1

k -1

Selection Problems

log x dx )
input

n log n - n log e + log e

output
Better bound
using

pn
2p

() (
n
e

e {1,...,n}

x (k) = the k'th best

Sterling Approximation
n

n!

X1 , X 2 ,..., X n
and index k

1+

1
n
12

fi log (n!) n log n - n log e +

1
pn)
log (2p
2

13

14

History:

Selection of the champion

Rev C.L. Dodge (Lewis Carol)

- X (1) is easily determined in

wrote article on lawn tennis


tournament
in James Gazett, 1883

felt prizes unjust

because:
..

(1)
- although winner X
always gets lst prize
- second X (2) may not get 2nd prize

X (1)

X(1)

X(2)

n-1

- X (1) requires

X (2)

n-1 comparisons

proof
everyone except the champion X(1)
must lose at least once!

note: X (2) not


declared 2nd best
if it is
left branch

Carol proposed his own (nonoptimal) tournament....

n-1 comparison

...

X(1)

X (1)

15

16

Selection of the second best


using

n-2 +

logn

X(2)

[2] Let Y be the set of players

comparisons

knocked out by champion

|Y| logn

X(1)

Algorithm
[1] form a balanced binary tree
for tournament to find X
using n-1 comparisons

[3] Play a tournament among the Y's


(1)

[4] output

x
x
x

using

(1)

X (2) = champion of the Y's

logn

-1 more comparisons

Y
(1)

log n

height

(1)

x (1)
17

18

Lower Bounds on finding


requires

n-2 +

X (2)

logn comparisons

Claim m 1 n-1, since at end we


must know X (1) as well as X

(2)

proof
#comparison
where m

Claim m 2 (#who lost to X


)-1
(1)
since everyone (except X
(2) ) who
lost to X (1) must also have
lost one more time.

m 1 + m 2 + ...

= #players who lost i or


more matches

19

20

declare

lemma
(#who lost to X

(1)

Xi > Xj if

) logn

in worst case
(a) X previously undefeated and
i
X j lost at least once
(b) both undefeated but Xi played
more matches

proof
Use oracle who "fixes" results
of games so that champion X
plays logn matches

(c) otherwise, decide consistently


with previous decisions
(1)

fi forces path from X


to have

21

(1)

to root

length logn

22

[1] if n < c o then output X


and halt

Selection by Divide-and-Conquer

(k)

by sorting X

[ 2] divide X into

Algorithm

Select

n sequences
d
of d elements each (with < d leftover),
and sort each sequence

(X)

[3] let M be the


medians
these sequences
[4] m Select

input

set X of n keys and index k

[5] let X
let X

|M|
2

of each of

(M)

= {x e X | x < m}
= {x e X | x > m}

| k then output Select


+
else if n - |X | = k then output

[6] if |X

else
23

output

Select

k-(n - |X |)

(X )

m
+
(X )
24

Columns

smallest

sequences

n
d

n ot X

M in sorted order
d

n ot X

(assuming d is constant)

If say d=5,
largest

d+1
Propos ition |X |, |X | each n - 2

T(n)

c1

for a sufficiently large constant

c1
T

T(n)

20c n = O(n)
1

n
2d 3 n

if n < co

( )
n
d

+ T

( )
3
n
4

+ c1n
25

26

proof
Fix a path p from root to leaf
The comparisons done on p define a
relation R p
+

Let R p = trans itiv e clos ure of Rp

Lower Bounds for Selecting X


input
Theorem

X = {x

,..., x

n },

Lemma
(k)

index k

proof

m either x i Rp x m or x m Rp x i
then for all i

Every leaf of Decision Tree has


depth

If path p determines X m = X (k)

Suppose x i is un related to x m by Rp

Then can replace x i in linear order either


before or after x m to violate x m = x (k)

n-1

Let the "key" comparison


x i is compared with x

for x i be when
j where either

(1) j=m
+

(2) x i Rp x j and x j Rp x m , or
+

(3) x j Rp x i and x m Rp x j
Fact

27

x i has unique "key" comparison determining


+
+
either x i Rp x m or x m Rp x i

fi So there are n-1 "key" comparisons, each distinct!

28

n
(1) D 2

(2) w hile D > 0 do


for i = D +1 to n do

A hard to analyze sort:


SHELLSORT
input keys

X1 ,..., X n

increment
sort

begin
j i- D
while j>0 do
if x j > x

j+ D

then

begin
SWAP (x j , x
j j- D

j+ D)

end
else j 0
end

D D/2

AHU Data Structures & Alg., pp. 290-291

29

30

procedure
increment sort (Y

passes of SHELLSORT:
1

increment s ort

increment s ort

(
(

Xk , X

n
+ k
2

Xk , X n
4

+k

for k=1, . . . ,

, Xn
2

+k

, X3
4

n
2

n+ k

facts

,..., Y

for i = z by 1 until i>n or X i-1 < X i


do for j=1 by -1 until 1 do

if X j-1 > X j then s w ap X j-1 , X j


(1) if X i , X n
p

n
for k=1, . . . ,
4

sorted in pass p
2

+ 1

fi they remain sorted in later passes


(2) dis tance betw een comparis ons diminis h
n n
n
, ....
as , , . . . ,
2 4
p2
(3) The bes t know n time bound is 0

31

(n )
1. 5

32

procedure RADIXSORT

input
[1]

-- Costs O(n) time

X1 , ... , X n e {1 ,..., n}

-- avoids

for j=1 ,..., n


do
initialize B[j] to be the empty list

[2]

for i=1 ,..., n


do
add i to B[X i ]

[3]

let L = (i 1 , i 2 ,..., i n ) be the


concatenation of B[1] ,..., B[n]

[4]

output

X i 1 X i 2 ...

on unit cost RAM

W(nlogn) lower bound on SORT

by avoiding comparisons
instead uses indexing of RAM
-- generalizes (in c passes) to key
domains {1 ,..., n c }

X in

33

34

open problems in sorting

(1)

Complexity of SHELLSORT
-- very good in practice
claims Sedgewick
-- Is it

(2)

q(n 1.5 )?

Complexity of variable length


-- sort on multitape TM or RAM
-- Is it W(n log n)?
35

You might also like