Scribd
Upload
33 views

File Organizations and Indexing: R&G Chapter 8

File organizations include heap files, sorted files, and clustered files with indexes. Heap files are suitable for full scans but poor for searches. Sorted files and clustered files with indexes support efficient searches through tree-based indexing structures like B-trees. Indexes speed up selections by maintaining a data structure that maps keys to record identifiers. The data structure can be hash-based or tree-based depending on the needed query capabilities. Indexes are classified as clustered or unclustered depending on whether the indexed records are stored in key order.

Uploaded by

raw.junk
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
33 views

File Organizations and Indexing: R&G Chapter 8

File organizations include heap files, sorted files, and clustered files with indexes. Heap files are suitable for full scans but poor for searches. Sorted files and clustered files with indexes support efficient searches through tree-based indexing structures like B-trees. Indexes speed up selections by maintaining a data structure that maps keys to record identifiers. The data structure can be hash-based or tree-based depending on the needed query capabilities. Indexes are classified as clustered or unclustered depending on whether the indexed records are stored in key order.

Uploaded by

raw.junk
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 26

File Organizations and Indexing

R&G Chapter 8

"If you don't find it in the index, look very


carefully through the entire catalogue."
-- Sears, Roebuck, and Co.,
Consumer's Guide, 1897

Context

Query Optimization
and Execution
Relational Operators
Files and Access Methods
Buffer Management
Disk Space Management

DB

Alternative File Organizations


Many alternatives exist, each good for
some situations, and not so good in
others:
Heap files: Suitable when typical
access is a file scan retrieving all
records.
Sorted Files: Best for retrieval in search
key order, or only a `range of records is
needed.
Clustered Files (with Indexes): Coming
soon

Cost Model for Analysis


We ignore CPU costs, for simplicity:

B: The number of data blocks


R: Number of records per block
D: (Average) time to read or write disk block
Measuring number of block I/Os ignores
gains of pre-fetching and sequential access;
thus, even I/O cost is only loosely
approximated.
Average-case analysis; based on several
simplistic assumptions.
Good enough to show the overall trends

Some Assumptions in the


Analysis
Single record insert and delete.
Equality selection - exactly one
match (what if more or less???).
Heap Files:
Insert always appends to end of file.

Sorted Files:
Files compacted after deletions.
Selections on search key.

Cost of
Operations
Heap File
Scan all
records
Equality
Search
Range
Search
Insert
Delete

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Sorted File

Clustered File

Cost of
Operations

Scan all
records
Equality
Search
Range
Search
Insert
Delete

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Heap File

Sorted File

BD

BD

Clustered File

Cost of
Operations

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Heap File

Sorted File

Scan all
records

BD

BD

Equality
Search

0.5 BD

(log2 B) * D

Range
Search
Insert
Delete

Clustered File

Cost of
Operations

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Heap File

Sorted File

Scan all
records

BD

BD

Equality
Search

0.5 BD

(log2 B) * D

Range
Search

BD

[(log2 B) +
#match pg]*D

Insert
Delete

Clustered File

Cost of
Operations

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Heap File

Sorted File

Scan all
records

BD

BD

Equality
Search

0.5 BD

(log2 B) * D

Range
Search

BD

[(log2 B) +
#match pg]*D

Insert

2D

((log2B)+B)D
(because R,W 0.5)

Delete

Clustered File

Cost of
Operations

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Heap File

Sorted File

Scan all
records

BD

BD

Equality
Search

0.5 BD

(log2 B) * D

Range
Search

BD

[(log2 B) +
#match pg]*D

Insert

2D

((log2B)+B)D

Delete

0.5BD + D

((log2B)+B)D
(because R,W 0.5)

Clustered File

Indexes
Sometimes, we want to retrieve records by
specifying the values in one or more fields, e.g.,
Find all students in the CS department
Find all students with a gpa > 3
An index on a file is a disk-based data structure
that speeds up selections on the search key
fields for the index.
Any subset of the fields of a relation can be
the search key for an index on the relation.
Search key is not the same as key (e.g.
doesnt have to be unique ID).
Can have multiple (different) indexes per file.
E.g. sort by age, with an index on salary and
name.

Index Breakdown
Index Data Structure
Tree-based, hash-based, other
What can the index speed up, and how much?

Associating index entries with records


primary vs. secondary indexes, handling
duplicates
what kind of info is the index actually storing?

Clustered vs. Unclustered Indexes


Single Part vs. Multi-Part Keys
Multi-part key = Composite Indexes

Data structures
What kinds of selections do they support?
Selections of form field <op> constant
Equality selections (op is =)
Range selections (op is one of <, >, <=, >=,
BETWEEN)

Hash-based structures (how to


grow/shrink)
Key problem on disk is handling growth
Extendible/Linear Hashing (Chap 11)

Tree based structures


Why not binary tree? Estimate log 2(1M) * D
B-Tree, B+-Tree (Chap 10)

Wide World of Index


Structures
2-dimensional ranges (east of Berkeley and west of
Truckee and North of Fresno and South of Eureka)
Or distances (within 2 miles of Soda Hall)
Or n-dimensional
One common n-dimensional index: R-tree
Supported in Oracle and Informix

See http://gist.cs.berkeley.edu for research on this topic

Nearest neighbor (closest BMW dealer)


Ranking queries (10 best Berkeley Thai restaurants on
price and atmosphere)
these are hard to support!

Regular expression matches


Suffix Trees

XML path matches


DataGuide, 1-Index

Primary vs. Secondary Index


Primary index search key must contain
a real key, usually primary key
e.g., social security #, ISBN, etc.
No duplicate support
Store record in the index?

Secondary index
e.g., eye color, year of birth, etc.
Duplicate support required
Use RID or primary key to refer to record?

Alternatives for Data Entry k* in


Index
Three alternatives:
Actual data record (primary index

only)
<k, rid of matching data record>
<k, list of rids of matching data
records>

Choice is orthogonal to the indexing


technique.

Alternatives for Data Entries


(Contd.)
Alternative 1:

Actual data record


Use index structure as the file structure
Saves pointer lookups for primary index
searches
Adds a primary index lookup for secondary
index access!
Index nodes have
all the issues of
record
management

Alternatives for Data Entries


(Contd.)
Alternative 2

<k, rid of matching data record>


and Alternative 3

<k, list of rids of matching data records>


If heap file is used (no alt 1 indexes), then physical rid
can be used instead of primary key to refer to records
Alternative 3 more compact than Alternative 2, but
leads to variable sized data entries even if search keys
are of fixed length.
Even worse, for large rid lists the data entry would
have to span multiple blocks! (how many?)
Typical solution: add primary key or rid to end of
secondary keys, and use Alternative 2!

Index Classification

Clustered vs. unclustered: If order of data


records is the same as, or `close to, order
of index data entries, then called clustered
index.
A file can be clustered on at most one search key.
Cost of range scans through index varies greatly
based on whether index is clustered or not!
Alternative 1-light
Alternative 1 implies clustered, but not vice-versa.
Use Physical RID in secondary index (why is this good?)

Clustered vs. Unclustered Index


Suppose that Alternative (2) is used for data entries,
and that the data records are stored in a Heap file.
To build clustered index, first sort the Heap file (with some free
space on each block for future inserts).
Overflow blocks may be needed for inserts. (Thus, order of
data recs is `close to, but not identical to, the sort order.)

CLUSTERED

Index entries
direct search for
data entries

Data entries

UNCLUSTERED

Data entries
(Index File)
(Data file)

Data Records

Data Records

Unclustered vs. Clustered


Indexes
What are the tradeoffs????
Clustered Pros
Efficient for range searches
May be able to do some types of
compression
Possible locality benefits (related data?)
???

Clustered Cons
Expensive to maintain (on the fly or
sloppy with reorganization)
Pages tend to be only 2/3 full!

Cost of
Operations

B: The number of data pages


R: Number of records per page
D: (Average) time to read or write disk p

Heap File

Sorted File

Clustered File

Scan all
records

BD

BD

1.5 BD

Equality
Search

0.5 BD

(log2 B) * D

(logF 1.5B) * D

Range
Search

BD

[(log2 B) +
#match pg]*D

[(logF 1.5B) +
#match pg]*D

Insert

2D

((log2B)+B)D

((logF 1.5B)+1) *
D

Delete

0.5BD + D

((log2B)+B)D
(because R,W 0.5)

((logF 1.5B)+1) *
D

Composite Search Keys


Search on a combination of
fields.
Equality query: Every field value
is equal to a constant value. E.g.
wrt <age,sal> index:
age=20 and sal =75

Range query: Some field value is


not a constant. E.g.:
age > 20; or age=20 and sal >
10

Data entries in index sorted


by search key to support
range queries.
Lexicographic order
Like the dictionary, but on fields,
not letters!

Examples of composite key


indexes using lexicographic order.
11,80

11

12,10

12

12,20
13,75
<age, sal>
10,12
20,12
75,13

name age sal


bob 12

10

cal 11

80

joe 12

20

sue 13

75

12
13
<age>
10

Data records
sorted by name

80,11
<sal, age>

Data entries in index


sorted by <sal,age>

20
75
80
<sal>

Data entries
sorted by <sal>

Summary
File Layer manages access to records in pages.
Record and page formats depend on fixed vs. variablelength.
Free space management an important issue.
Slotted page format supports variable length records
and allows records to move on page.

Many alternative file organizations exist, each


appropriate in some situation.
If selection queries are frequent, sorting the file
or building an index is important.
Hash-based indexes only good for equality search.
Sorted files and tree-based indexes best for range
search; also good for equality search. (Files rarely
kept sorted in practice; B+ tree index is better.)

Index is a collection of data entries plus a way


to quickly find entries with given key values.

Summary (Contd.)
Data entries in index can be actual data records, <key,
rid> pairs, or <key, rid-list> pairs.
Choice orthogonal to indexing structure (i.e., tree, hash,
etc.).

Usually have several indexes on a given file of data


records, each with a different search key.
Indexes can be classified as clustered vs. unclustered
Differences have important consequences for
utility/performance.
Catalog relations store information about relations,
indexes and views.

You might also like