Database Applications (15-

415)

DBMS Internals- Part V

Lecture 14, Oct 18, 2016
Mohammad Hammoud
 Today…
 Last Session:
 DBMS Internals- Part IV
 Tree-based (i.e., B+ Tree– cont’d)

 Today’s Session:
 DBMS Internals- Part V
 Hash-based indexes and External Sorting

 Announcements:
 Mid-semester grades are out
 PS3 is out. It is due on Nov 1st
 Project 2 is due on Oct 27
 DBMS Layers
Queries

Query Optimization Continue…

and Execution

Relational Operators
Transaction Files and Access Methods
Manager
Recovery
Buffer Management Manager
Lock
Manager Disk Space Management

DB
 Outline

B+ Trees with Duplicates

B+ Trees with Key Compression

Bulk Loading of a B+ Tree

A Primer on Hash-Based Indexing 

Static Hashing

Extendible Hashing
 Hash-Based Indexing
 What indexing technique can we use to support range
searches (e.g., “Find s_name where gpa >= 3.0)?
 Tree-Based Indexing

 What about equality selections (e.g., “Find s_name

where sid = 102”?
 Tree-Based Indexing
 Hash-Based Indexing (cannot support range searches!)

 Hash-based indexing, however, proves to be very useful

in implementing relational operators (e.g., joins)
 Outline

B+ Trees with Duplicates

B+ Trees with Key Compression

Bulk Loading of a B+ Tree

A Primer on Hash-Based Indexing

Static Hashing 
Extendible Hashing
 Static Hashing
 A hash structure (or table or file) is a generalization of
the simpler notion of an ordinary array
 In an array, an arbitrary position can be examined in O(1)

 A hash function h is used to map keys into a range of

bucket numbers
0
h(key) mod N 2
key
h With Static Hashing,
allocated (as needed)
With Static Hashing, when corresponding
allocated sequentially buckets become full
and never de-allocated N-1
Primary bucket pages Overflow pages
 Static Hashing
 Data entries can be any of the three alternatives (A (1), A (2) or A
(3)- see previous lecture)

 Data entries can be sorted in buckets to speed up searches

 The hash function h is used to identify the bucket to which a

given key belongs and subsequently insert, delete or locate a
respective data record
 A hash function of the form h(key) = (a * key + b) works well in
practice

 A search ideally requires 1 disk I/O, while an insertion or a

deletion necessitates 2 disk I/Os
 Static Hashing: Some Issues
 Similar to ISAM, the number of buckets is fixed!
 Cannot deal with insertions and deletions gracefully

 Long overflow chains can develop easily and degrade

performance!
 Pages can be initially kept only 80% full

 Dynamic hashing techniques can be used to fix

the problem
 Extendible Hashing (EH)
 Liner Hashing (LH)
 Outline

B+ Trees with Duplicates

B+ Trees with Key Compression

Bulk Loading of a B+ Tree

A Primer on Hash-Based Indexing

Static Hashing

Extendible Hashing 
 Directory of Pointers
 How else (as opposed to overflow pages) can we add a
data record to a full bucket in a static hash file?
 Reorganize the table (e.g., by doubling the number of
buckets and redistributing the entries across the new
set of buckets)
 But, reading and writing all pages is expensive!

 In contrast, we can use a directory of pointers to buckets

 Buckets number can be doubled by doubling just the
directory and splitting “only” the bucket that overflowed
 The trick lies on how the hash function can be adjusted!
 Extendible Hashing
 Extendible Hashing uses a directory of pointers to buckets
GLOBAL DEPTH
4* 12* 32*16* Bucket A
 The result of applying a hash
2
function h is treated as a 00 1* 5* 21* Bucket B
binary number and 01
the last d bits are 10
10* Bucket C
interpreted as an 11

offset into the directory DIRECTORY 15* 7* 19* Bucket D

 d is referred to as the global depth DATA PAGES

of the hash file and is kept as part

of the header of the file
Extendible Hashing: Searching for Entries
 To search for a data entry, apply a hash function h to the
key and take the last d bits of its binary representation to
get the bucket number

 Example: search for 5*

2

00 Bucket A
4* 12* 32*16*
01 Bucket B
5 = 101 1* 5* 21*
10
Bucket C
10*
11
Bucket D
15* 7* 19*
DIRECTORY
DATA PAGES
Extendible Hashing: Inserting Entries
 An entry can be inserted as follows:
 Find the appropriate bucket (as in search)

 Split the bucket if full and redistribute contents

(including the new entry to be inserted) across
the old bucket and its “split image”

 Double the directory if necessary

 Insert the given entry

Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry

 Example: insert 13*

2

00 Bucket A
4* 12* 32*16*
01 Bucket B
13 = 1101 1* 5* 21* 13*
10
Bucket C
10*
11
Bucket D
15* 7* 19*
DIRECTORY
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry

 Example: insert 20*

FULL, hence, split and redistribute!
2

00 Bucket A
4* 12* 32*16*
01 Bucket B
20 = 10100 1* 5* 21* 13*
10
Bucket C
10*
11
Bucket D
15* 7* 19*
DIRECTORY
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry
Bucket A
32*16*
2
 Example: insert 20* 00
Bucket B
1* 5* 21* 13*
01
10
20 = 10100 Bucket C
11 10*

DIRECTORY
Bucket D
15* 7* 19*

Bucket A2
Is this enough? 4* 12*20*
(`split image'
of Bucket A)
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry
Bucket A
32*16*
2
 Example: insert 20* 00
Bucket B
1* 5* 21* 13*
01
10
20 = 10100 Bucket C
11 10*

DIRECTORY
Bucket D
15* 7* 19*

Double the directory and

Bucket A2
increase the global depth 4* 12*20*
(`split image'
of Bucket A)
 Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry GLOBAL DEPTH
32*16* Bucket A

3
 Example: insert 20* 0 00 1* 5* 21*13* Bucket B
001
These two bits indicate a data entry that
010
belongs to one of these two buckets
011 10* Bucket C
1 00
101
The third bit distinguishes between these
110 15*7* 19* Bucket D
two buckets!
111

But, is it necessary always to DIRECTORY 4* 12*20* Bucket A2

double the directory? (`split image'
of Bucket A)
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry GLOBAL DEPTH 32*16* Bucket A

3 FULL, hence, split!

 Example: insert 9* 000 1* 5* 21*13* Bucket B
001
010
9 = 1001 011 10* Bucket C
100
101
110 15*7* 19* Bucket D
111

DIRECTORY 4* 12*20* Bucket A2

(`split image'
of Bucket A)
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry GLOBAL DEPTH 32*16* Bucket A

3
 Example: insert 9* 000 1* 9* Bucket B
001
010
10* Bucket C
9 = 1001 011
100
101 15*7* 19* Bucket D
110
Almost there… 111
4* 12*20* Bucket A2
(`split image‘ of A)
DIRECTORY
5* 21*13* Bucket B2
(`split image‘ of B)
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry GLOBAL DEPTH 32*16* Bucket A

3
 Example: insert 9* 000 1* 9* Bucket B
001
010
10* Bucket C
9 = 1001 011
100
There was no need to 101 15*7* 19* Bucket D
double the directory! 110
111
4* 12*20* Bucket A2
(`split image‘ of A)
When NOT to double the DIRECTORY
directory? 5* 21*13* Bucket A2
(`split image‘ of A)
Extendible Hashing: Inserting Entries
 Find the appropriate bucket (as in search), split the bucket
if full, double the directory if necessary and insert the
given entry LOCAL DEPTH 3
GLOBAL DEPTH 32*16* Bucket A

3 3
 Example: insert 9* 000 1* 9* Bucket B
001 2
010
10* Bucket C
9 = 1001 011
100 2
101 15*7* 19* Bucket D
If a bucket whose local depth 110 3
equals to the global depth is 111
4* 12*20* Bucket A2
split, the directory must be (`split image‘ of A)
doubled DIRECTORY 3
5* 21*13* Bucket A2
(`split image‘ of A)
Extendible Hashing: Inserting Entries
 Example: insert 9*
LOCAL DEPTH 3
Repeat… 32*16* Bucket A
GLOBAL DEPTH

2
FULL, hence, split!
3
000 1* 5* 21*13* Bucket B
001
010 2

9 = 1001 011 10* Bucket C

100
101 2
Because the local depth 110 15*7* 19* Bucket D
(i.e., 2) is less than the 111
global depth (i.e., 3), NO 3
need to double the DIRECTORY 4* 12*20* Bucket A2
directory (`split image'
of Bucket A)
Extendible Hashing: Inserting Entries
 Example: insert 9*
LOCAL DEPTH 3
Repeat… 32*16* Bucket A
GLOBAL DEPTH

3 3
000 1* 9* Bucket B
001 2
010
10* Bucket C
9 = 1001 011
100 2
101 15*7* 19* Bucket D
110 3
111
4* 12*20* Bucket A2
(`split image‘ of A)
DIRECTORY 3
5* 21*13* Bucket B2
(`split image‘ of B)
Extendible Hashing: Inserting Entries
 Example: insert 9*
LOCAL DEPTH 3
Repeat… 32*16* Bucket A
GLOBAL DEPTH

3 3
000 1* 9* Bucket B
001 2
010
10* Bucket C
9 = 1001 011
100 2
101 15*7* 19* Bucket D
FINAL STATE! 110 3
111
4* 12*20* Bucket A2
(`split image‘ of A)
DIRECTORY 3
5* 21*13* Bucket B2
(`split image‘ of B)
Extendible Hashing: Inserting Entries
 Example: insert 20*
FULL, hence, split!
Repeat… LOCAL DEPTH 2
Bucket A
GLOBAL DEPTH 4* 12* 32*16*

2 2
Bucket B
00 1* 5* 21* 13*

01
20 = 10100
10 2
Bucket C
11 10*
Because the local depth
and the global depth are
DIRECTORY 2
both 2, we should double Bucket D
the directory! 15* 7* 19*

DATA PAGES
Extendible Hashing: Inserting Entries
 Example: insert 20*
Repeat… LOCAL DEPTH 2
Bucket A
GLOBAL DEPTH 32*16*

2 2

00 1* 5* 21*13*Bucket B
01
10 2
20 = 10100 11 10* Bucket C

2
DIRECTORY Bucket D
15*7* 19*
Is this enough?
2
4* 12*20* Bucket A2
(`split image'
of Bucket A)
Extendible Hashing: Inserting Entries
 Example: insert 20*
2
Repeat… LOCAL DEPTH
32*16* Bucket A
GLOBAL DEPTH

3 2
000 1* 5* 21*13* Bucket B
001
010 2

011 10* Bucket C

100
101 2

110 15*7* 19* Bucket D

Is this enough? 111
2
DIRECTORY 4* 12*20* Bucket A2
(`split image'
of Bucket A)
Extendible Hashing: Inserting Entries
 Example: insert 20*
3
Repeat… LOCAL DEPTH
32*16* Bucket A
GLOBAL DEPTH

3 2
000 1* 5* 21*13* Bucket B
001
FINAL STATE! 010 2

011 10* Bucket C

100
101 2

110 15*7* 19* Bucket D

111
3
DIRECTORY 4* 12*20* Bucket A2
(`split image'
of Bucket A)
 Outline
Linear Hashing 
Why Sorting?

In-Memory vs. External Sorting

A Simple 2-Way External Merge Sorting

General External Merge Sorting

Optimizations: Replacement Sorting, Blocked I/O and Double Buffering

 Linear Hashing
 Another way of adapting gracefully to insertions and
deletions (i.e., pursuing dynamic hashing) is to use
Linear Hashing (LH)

 In contrast to Extendible Hashing, LH

 Does not require a directory
 Deals naturally with collisions
 Offers a lot of flexibility w.r.t the timing of bucket split
(allowing trading off greater overflow chains for higher
average space utilization)
 How Linear Hashing Works?
 LH uses a family of hash functions h0, h1, h2, ...
 hi(key) = h(key) mod(2iN); N = initial # buckets

 h is some hash function (range is not 0 to N-1)

 If N = 2d0, for some d0, hi consists of applying h and

looking at the last di bits, where di = d0 + i

 hi+1 doubles the range of hi (similar to directory

doubling)
How Linear Hashing Works? (Cont’d)
 LH uses overflow pages, and chooses buckets to split in
a round-robin fashion
Buckets split
in this round

 Splitting proceeds in “rounds” Next

 A round ends when all NR
Buckets that existed at the
(for round R) initial beginning of this round:
buckets are split this is the range of
 Buckets 0 to Next-1 h
Level
have been split;
Next to NR yet to be split
‘split image’
 Current round number buckets created
is referred to as Level in this round
Linear Hashing: Searching For Entries
 To find a bucket for data entry r, find hLevel(r):
 If hLevel(r) in range `Next to NR’ , r belongs there
 Else, r could belong to bucket hLevel(r) or bucket
hLevel(r) + NR; must apply hLevel+1(r) to find out
Level=0, N=4
h h PRIMARY
 Example: search for 5* 1 0 Next=0 PAGES

32*44* 36*
Level = 0  h0 000 00

5* = 101  01 9* 25* 5*
Data entry r
001 01 with h(r)=5

14* 18*10*30*
010 10 Primary
bucket page
31*35* 7* 11*
011 11
 Linear Hashing: Inserting Entries
 Find bucket as in search
 If the bucket to insert the data entry into is full:
 Add an overflow page and insert data entry
 (Maybe) Split Next bucket and increment Next

 Some points to Keep in mind:

 Unlike Extendible Hashing, when an insert triggers a split, the bucket
into which the data entry is inserted is not necessarily the bucket that
is split

 As in Static Hashing, an overflow page is added to store the newly

inserted data entry

 However, since the bucket to split is chosen in a round-robin fashion,

eventually all buckets will be split
 Linear Hashing: Inserting Entries
 Example: insert 43*

Level = 0  h0 Level=0, N=4

43* = 101011  11
h h PRIMARY
1 0 Next=0 PAGES

32*44* 36*
000 00

001 01 9* 25* 5*

14* 18*10*30*
010 10

31*35* 7* 11*
011 11
Add an overflow page and
insert data entry
 Linear Hashing: Inserting Entries
 Example: insert 43*

Level = 0  h0 Level=0, N=4

43* = 101011  11
h h PRIMARY OVERFLOW
1 0 Next=0 PAGES PAGES

32*44* 36*
000 00

001 01 9* 25* 5*

14* 18*10*30*
010 10

Split Next bucket and 011

31*35* 7* 11*
43*
11
increment Next
 Linear Hashing: Inserting Entries
 Example: insert 43*

Level = 0  h0 Level=0, N=4

43* = 101011  11
PRIMARY OVERFLOW
h h
1 0 Next=0 PAGES PAGES

000 00 32*

001 01 9* 25* 5*

010 10 14* 18*10*30*

011 11 31*35* 7* 11*

Almost there… 43*

100 00 44* 36*

 Linear Hashing: Inserting Entries
 Example: insert 43*

Level = 0  h0 Level=0, N=4

43* = 101011  11
PRIMARY OVERFLOW
h h
PAGES PAGES
1 0

000 00 32*
Next=1
001 01 9* 25* 5*

010 10 14* 18*10*30*

011 11 31*35* 7* 11*

FINAL STATE! 43*

100 00 44* 36*

 Linear Hashing: Inserting Entries
 Another Example: insert 50*
Level=0, N= 4
PRIMARY OVERFLOW
Level = 0  h0 h1 h0 PAGES PAGES
50* = 110010  10
000 00 32*

001 01 9* 25*

010 10 66*18* 10* 34*

Next=3
011 11 31*35* 7* 11* 43*

100 00 44*36*

101 01 5* 37*29*
Add an overflow page and
insert data entry 110 10 14*30*22*
 Linear Hashing: Inserting Entries
 Another Example: insert 50*
Level=0, N= 4
PRIMARY OVERFLOW
Level = 0  h0 h1 h0 PAGES PAGES
50* = 110010  10
000 00 32*

001 01 9* 25*

010 10 66*18* 10* 34* 50*

Next=3
011 11 31*35* 7* 11* 43*

100 00 44*36*

Split Next bucket and 101 01 5* 37*29*

increment Next
110 10 14*30*22*
 Linear Hashing: Inserting Entries
 Another Example: insert 50*
Level=0
PRIMARY OVERFLOW
h1 h0 PAGES PAGES
Level = 0  h0 000 00 32*
50* = 110010  10
001 01 9* 25*

010 10 66* 18* 10* 34* 50*

Next=3
011 11 43* 35* 11*

100 00 44* 36*

101 11 5* 37* 29*

Almost there…
110 10 14* 30* 22*

111 11 31*7*
 Linear Hashing: Inserting Entries
 Another Example: insert 50*
Level=0
PRIMARY OVERFLOW
h1 h0 PAGES PAGES
Next=0
Level = 0  h0 000 00 32*
50* = 110010  10
001 01 9* 25*

010 10 66* 18* 10* 34* 50*

011 11 43* 35* 11*

100 00 44* 36*

101 11 5* 37* 29*

Almost there…
110 10 14* 30* 22*

111 11 31*7*
 Linear Hashing: Inserting Entries
 Another Example: insert 50*
Level=1
PRIMARY OVERFLOW
h1 h0 PAGES PAGES
Next=0
Level = 0  h0 000 00 32*
50* = 110010  10
001 01 9* 25*

010 10 66* 18* 10* 34* 50*

011 11 43* 35* 11*

100 00 44* 36*

101 11 5* 37* 29*

FINAL STATE!
110 10 14* 30* 22*

111 11 31*7*
 Linear Hashing: Deleting Entries
 Deletion is essentially the inverse of insertion

 If the last bucket in the file is empty, it can be removed and

Next can be decremented

 If Next is zero and the last bucket becomes empty

 Next is made to point to bucket M/2 -1 (where M is the current
number of buckets)
 Level is decremented
 The empty bucket is removed

 The insertion examples can be worked out backwards as

examples of deletions!
 DBMS Layers
Queries

Query Optimization But, before we will

and Execution discuss “Sorting”
Relational Operators
Transaction Files and Access Methods
Manager
Recovery
Buffer Management Manager
Lock
Manager Disk Space Management

DB
 Outline
Linear Hashing

Why Sorting?

In-Memory vs. External Sorting

A Simple 2-Way External Merge Sorting

General External Merge Sorting

Optimizations: Replacement Sorting, Blocked I/O and Double Buffering

 When Does A DBMS Sort Data?
 Users may want answers in some order
 SELECT FROM student ORDER BY name
 SELECT S.rating, MIN (S.age) FROM Sailors S GROUP BY S.rating

 Bulk loading a B+ tree index involves sorting

 Sorting is useful in eliminating duplicates records

 The Sort-Merge Join algorithm involves sorting

(next session!)
 Outline
Linear Hashing

Why Sorting?

In-Memory vs. External Sorting


A Simple 2-Way External Merge Sorting

General External Merge Sorting

Optimizations: Replacement Sorting, Blocked I/O and Double Buffering

 In-Memory vs. External Sorting
 Assume we want to sort 60GB of data on a machine
with only 8GB of RAM
 In-Memory Sort (e.g., Quicksort) ?
 Yes, but data do not fit in memory
 What about relying on virtual memory?

 In this case, external sorting is needed

 In-memory sorting is orthogonal to external sorting!
 Outline
Linear Hashing

Why Sorting?

In-Memory vs. External Sorting

A Simple 2-Way External Merge Sorting


General External Merge Sorting

Optimizations: Replacement Sorting, Blocked I/O and Double Buffering

 A Simple Two-Way Merge Sort
 IDEA: Sort sub-files that can fit in memory and merge

 Let us refer to each sorted sub-file as a run

 Algorithm:
 Pass 1: Read a page into memory, sort it, write it
 1-page runs are produced
 Passes 2, 3, etc.,: Merge pairs (hence, 2-way) of runs
to produce longer runs until only one run is left
 A Simple Two-Way Merge Sort
 Algorithm:
 Pass 1: Read a page into memory, sort it, write it
 How many buffer pages are needed? ONE
 Passes 2, 3, etc.,: Merge pairs (hence, 2-way) of runs to
produce longer runs until only one run is left
 How many buffer pages are needed? THREE

INPUT 1

OUTPUT
INPUT 2

Main memory buffers Disk

Disk
2-Way Merge Sort: An Example
3,4 6,2 9,4 8,7 5,6 3,1 2 Input File
PASS 0
3,4 2,6 4,9 7,8 5,6 1,3 2 1-Page Runs
PASS 1
2,3 4,7 1,3
2-Page Runs
4,6 8,9 5,6 2
PASS 2
2,3
4,4 1,2 4-Page Runs
6,7 3,5
8,9 6
PASS 3

1,2
2,3
3,4 8-Page Runs
4,5
6,6
7,8
9
2-Way Merge Sort: I/O Cost Analysis
 If the number of pages in the input file is 2 k
 How many runs are produced in pass 0 and of what size?
 2k 1-page runs
 How many runs are produced in pass 1 and of what size?
 2k-1 2-page runs
 How many runs are produced in pass 2 and of what size?
 2k-2 4-page runs
 How many runs are produced in pass k and of what size?
 2k-k 2k-page runs (or 1 run of size 2k)
 For N log
numberN of
1 pages, how many passes are incurred?
 2 

 How many pages do we read and write in each pass?
 2N
 What is the overall cost?
2 N  ( log 2 N   1)
 2-Way Merge Sort: An Example
3,4 6,2 9,4 8,7 5,6 3,1 2 Input File
PASS 0
3,4 2,6 4,9 7,8 5,6 1,3 2 1-Page Runs
PASS 1
2,3 4,7 1,3
2-Page Runs
8,9 5,6 2
log 2 8  1 PASS 2
4,6

= 4 passes 2,3
4,4 1,2 4-Page Runs
6,7 3,5
8,9 6
PASS 3

1,2
Formula Check: 2,3
2 N  ( log 2 N   1) 3,4 8-Page Runs
4,5
= (2 × 8) × (3 + 1) = 64 I/Os 6,6
Correct! 7,8
9
 Outline
Linear Hashing

Why Sorting?

In-Memory vs. External Sorting

A Simple 2-Way External Merge Sorting

General External Merge Sorting


Optimizations: Replacement Sorting, Blocked I/O and Double Buffering
 B-Way Merge Sort
 How can we sort a file with N pages using B buffer pages?
 Pass 0: use B buffer pages and sort internally
 This will produce  N / B sorted B-page runs
 Passes 1, 2, …: use B – 1 buffer pages for input and the
remaining page for output; do (B-1)-way merge in each run

INPUT 1

... ...
INPUT 2
... OUTPUT

INPUT B-1
Disk Disk
B Main memory buffers
B-Way Merge Sort: I/O Cost Analysis
 I/O cost = 2N × Number of passes

 Number of passes = 1   log B 1  N / B  

 Assume the previous example (i.e., 8 pages), but using

5 buffer pages (instead of 2)
 I/O cost = 32 (as opposed to 64)

 Therefore, increasing the number of buffer pages

minimizes the number of passes and accordingly the
I/O cost!
 Number of Passes of B-Way Sort
N B=3 B=5 B=9 B=17 B=129 B=257
100 7 4 3 2 1 1
1,000 10 5 4 3 2 2
10,000 13 7 5 4 2 2
100,000 17 9 6 5 3 3
1,000,000 20 10 7 5 3 3
10,000,000 23 12 8 6 4 3
100,000,000 26 14 9 7 4 4
1,000,000,000 30 15 10 8 5 4
High Fan-in during merging is crucial!

How else can we minimize I/O cost?

 Outline
Linear Hashing

Why Sorting?

In-Memory vs. External Sorting

A Simple 2-Way External Merge Sorting

General External Merge Sorting

Optimizations: Replacement Sorting, Blocked I/O and Double Buffering


 Replacement Sort
 With a more aggressive implementation of B-way sort,
we can write out runs of 2×B (on average) internally
sorted pages
 This is referred to as replacement sort

12 8 3

4 10 5

INPUT CURRENT SET OUTPUT

IDEA: Pick the tuple in the current set with the smallest value that is greater than the
largest value in the output buffer and append it to the output buffer
 Replacement Sort
 With a more aggressive implementation of B-way sort,
we can write out runs of 2×B (on average) internally
sorted pages
 This is referred to as replacement sort

12 8 3

4 10 5

INPUT CURRENT SET OUTPUT

When do we terminate the current run and start a new one?

 Blocked I/O and Double Buffering
 So far, we assumed random disk accesses

 Would cost change if we assume that reads and writes

are done sequentially?
 Yes

 How can we incorporate this fact into our

cost model?
 Use bigger units (this is referred to as Blocked I/O)
 Mask I/O delays through pre-fetching (this is
referred to as double buffering)
 Blocked I/O
 Normally, we go with‘B’ buffers of size (say) 1 page

INPUT 1

... ...
INPUT 2
... OUTPUT

INPUT 5
Disk Disk
 Blocked I/O
 Normally, we go with‘B’ buffers of size (say) 1 page
 INSTEAD: let us go with B/b buffers, of size ‘b’ pages

INPUT 1

OUTPUT

INPUT 2
... ...
Disk Disk
3 Main memory buffers
 Blocked I/O
 Normally, we go with‘B’ buffers of size (say) 1 page
 INSTEAD: let us go with B/b buffers, of size ‘b’ pages

 What is the main advantage?

 Fewer random accesses (as some of the pages will be
arranged sequentially!)

 What is the main disadvantage?

 Smaller fan-in and accordingly larger number of passes!
 Double Buffering
 Normally, when, say ‘INPUT1’ is exhausted
 We issue a ‘read’ request and
 We wait …

INPUT 1

... ...
INPUT 2
... OUTPUT

INPUT B-1
Disk Disk
B Main memory buffers
 Double Buffering
 INSTEAD: pre-fetch INPUT1’ into a `shadow block’
 When INPUT1 is exhausted, issue a ‘read’
 BUT, also proceed with INPUT1’
 Thus, the CPU can never go idle!

INPUT 1

INPUT 1'

INPUT 2
OUTPUT
INPUT 2'
OUTPUT'

b
block size
Disk INPUT k
Disk
INPUT k'

B main memory buffers, k-way merge

 Next Class
Queries

Query Optimization
and Execution
Continue…
Relational Operators
Transaction Files and Access Methods
Manager
Recovery
Buffer Management Manager
Lock
Manager Disk Space Management

DB