BITS Pilani

BITS Pilani
Hyderabad Campus
Pilani Campus
Storage and File Structure

BITS Pilani,Pilani
Hyderabad Campus
Campus
Storage and File Structure

Overview of Physical Storage Media

Magnetic Disks
RAID
Tertiary Storage
Storage Access
File Organization
Organization of Records in Files
Data-Dictionary Storage

BITS Pilani,Pilani
Hyderabad Campus
Campus
Classification of Physical Storage Media

Speed with which data can be accessed

Cost per unit of data
Reliability
– data loss on power failure or system crash
– physical failure of the storage device
Can differentiate storage into:
– volatile storage: loses contents when power is switched off
– non-volatile storage:
• Contents persist even when power is switched off.
• Includes secondary and tertiary storage.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Storage Hierarchy

BITS Pilani,Pilani
Hyderabad Campus
Campus
Storage Hierarchy (Cont.)

primary storage: Fastest media but volatile (cache, main memory).

secondary storage: next level in hierarchy, non-volatile, moderately fast access time
– also called on-line storage
– E.g. flash memory, magnetic disks
tertiary storage: lowest level in hierarchy, non-volatile, slow access time
– also called off-line storage
– E.g. magnetic tape, optical storage

BITS Pilani,Pilani
Hyderabad Campus
Campus
Magnetic Hard Disk Mechanism

NOTE: Diagram is schematic, and simplifies the structure of actual disk drives

BITS Pilani,Pilani
Hyderabad Campus
Campus
Magnetic Disks

Read-write head
– Positioned very close to the platter surface (almost touching it)
– Reads or writes magnetically encoded information.
Surface of platter divided into circular tracks
– Over 50K-100K tracks per platter on typical hard disks
Each track is divided into sectors.
To read/write a sector
– disk arm swings to position head on right track
– platter spins continually; data is read/written as sector passes under head
Cylinder i consists of ith track of all the platters

BITS Pilani,Pilani
Hyderabad Campus
Campus
Disk Subsystem

Disks usually connected directly to computer system

In Storage Area Networks (SAN), a large number of disks are connected by a high-speed
network to a number of servers
In Network Attached Storage (NAS) networked storage provides a file system interface using
networked file system protocol, instead of providing a disk system interface

BITS Pilani,Pilani
Hyderabad Campus
Campus
Performance Measures of Disks

Access time – the time it takes from when a read or write request is issued to when data
transfer begins. Consists of:
– Seek time – time it takes to reposition the arm over the correct track.
• 4 to 10 milliseconds on typical disks
– Rotational latency – time it takes for the sector to be accessed to appear under the head.
• 4 to 11 milliseconds on typical disks (5400 to 15000 r.p.m.)
Data-transfer rate – the rate at which data can be retrieved from or stored to the disk.
– 25 to 100 MB per second max rate, lower for inner tracks
– Multiple disks may share a controller, so rate that controller can handle is also important
• E.g. SATA: 150 MB/sec, SATA-II 3Gb (300 MB/sec)
• Ultra 320 SCSI: 320 MB/s, SAS (3 to 6 Gb/sec)
• Fiber Channel (FC2Gb or 4Gb): 256 to 512 MB/s

BITS Pilani,Pilani
Hyderabad Campus
Campus
Performance Measures (Cont.)

Mean time to failure (MTTF) – the average time the disk is expected to run continuously without
any failure.
– Typically 3 to 5 years
– Probability of failure of new disks is quite low, corresponding to a “theoretical MTTF” of
500,000 to 1,200,000 hours for a new disk
– E.g., an MTTF of 1,200,000 hours for a new disk means that given 1000 relatively new
disks, on an average one will fail every 1200 hours
– MTTF decreases as disk ages

BITS Pilani,Pilani
Hyderabad Campus
Campus
RAID

RAID: Redundant Arrays of Independent Disks

– high capacity and high speed by using multiple disks in parallel,
– high reliability by storing data redundantly, so that data can be recovered even if a disk fails

BITS Pilani,Pilani
Hyderabad Campus
Campus
Improvement of Reliability via Redundancy

E.g. a system with 100 disks, each with MTTF of 100,000 hours (approx. 11 years), will have a system
MTTF of 1000 hours (approx. 41 days)
– Techniques for using redundancy to avoid data loss are critical with large numbers of disks
Redundancy – store extra information that can be used to rebuild information lost in a disk failure
E.g., Mirroring (or shadowing)
– Duplicate every disk. Logical disk consists of two physical disks.
– Every write is carried out on both disks
• Reads can take place from either disk
– If one disk in a pair fails, data still available in the other
• Data loss would occur only if a disk fails, and its mirror disk also fails before the system is repaired
Mean time to data loss depends on mean time to failure,
and mean time to repair
– E.g. MTTF of 100,000 hours, mean time to repair of 10 hours gives mean time to data loss of 500*10 6
hours (or 57,000 years) for a mirrored pair of disks (ignoring dependent failure modes)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Improvement in Performance via Parallelism

Two main goals of parallelism in a disk system:

1. Load balance multiple small accesses to increase throughput
2. Parallelize large accesses to reduce response time.
Improve transfer rate by striping data across multiple disks.
Block-level striping – with n disks, block i of a file goes to disk (i mod n) + 1
– Requests for different blocks can run in parallel if the blocks reside on different disks
– A request for a long sequence of blocks can utilize all disks in parallel

BITS Pilani,Pilani
Hyderabad Campus
Campus
RAID Levels

Schemes to provide redundancy at lower cost by using disk striping combined with parity bits
 RAID Level 0: Block striping; non-redundant.
 Used in high-performance applications where data loss is not
critical.
 RAID Level 1: Mirrored disks with block striping
 Offers best write performance.
 Popular for applications such as storing log files in a database.

BITS Pilani,Pilani
Hyderabad Campus
Campus
RAID Levels

RAID Level 2: employs parity bits for error correction.

RAID Level 3: employs single parity bit for error
detection and correction.
RAID Level 4: uses block-level striping and keeps a
parity block on a separate disk for corresponding blocks
from other disks for error detection and correction.

BITS Pilani,Pilani
Hyderabad Campus
Campus
RAID Levels (Cont.)

RAID Level 5: Block-Interleaved Distributed Parity; partitions data and parity among all N + 1
disks
E.g., with 5 disks, parity block for nth set of blocks is stored on disk (n mod 5) + 1, with the data
blocks stored on the other 4 disks.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Choice of RAID Level

Level 1 provides much better write performance than level 5

– Level 5 requires more block reads and block writes to write a single block, whereas Level 1
only requires 2 block writes
– Level 1 preferred for high update environments such as log disks
Level 1 had higher storage cost than level 5
– When enough disks have been bought to satisfy required rate of I/O, they often have spare
storage capacity
• so there is often no extra monetary cost for Level 1!
Level 5 is preferred for applications with low update rate,
and large amounts of data
Level 1 is preferred for all other applications
RAID levels 2, 3, 4 of academic interest only

BITS Pilani,Pilani
Hyderabad Campus
Campus
Hardware Issues

Software RAID: RAID implementations done entirely in software, with no special hardware
support
Hardware RAID: RAID implementations with special hardware
– Use non-volatile RAM to record writes that are being executed
– Beware: power failure during write can result in corrupted disk
• Such corrupted data must be detected when power is restored
• Recovery from corruption is similar to recovery from failed disk
• NV-RAM helps to efficiently detected potentially corrupted blocks
• Otherwise all blocks of disk must be read and compared with mirror/parity block

BITS Pilani,Pilani
Hyderabad Campus
Campus
File Organization, Record Organization and
Storage Access

BITS Pilani,Pilani
Hyderabad Campus
Campus
File Organization

The database is stored as a collection of files. Each file is a sequence of records mapped onto
disk blocks. A record is a sequence of fields.
One approach:
–assume record size is fixed
–each file has records of one particular type only
–different files are used for different relations
This case is easiest to implement; will consider variable length records later.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Fixed-Length Records

Simple approach:
– Store record i starting from byte n * (i – 1), where n is the size of each record.
– Record access is simple but records may cross blocks
• Modification: do not allow records to cross block boundaries

Deletion of record i:
alternatives:
– move records i + 1, . . ., n
to i, . . . , n – 1
– move record n to i
– do not move records, but
link all free records on a
free list

BITS Pilani,Pilani
Hyderabad Campus
Campus
Deleting record 3 and compacting

BITS Pilani,Pilani
Hyderabad Campus
Campus
Deleting record 3 and moving last
record

BITS Pilani,Pilani
Hyderabad Campus
Campus
Free Lists

Store the address of the first deleted record in the file header.
Use this first record to store the address of the second deleted record, and so on
Can think of these stored addresses as pointers since they “point” to the location of a record.
Reuse space for normal attributes of free records to store pointers. (No pointers stored in in-use
records.)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Variable-Length Records

Variable-length records arise in database systems in several ways:

– Storage of multiple record types in a file.
– Record types that allow variable lengths for one or more fields such as strings (varchar)
– Record types that allow repeating fields (used in some older data models).

BITS Pilani,Pilani
Hyderabad Campus
Campus
Variable-Length Records: Slotted Page Structure

Slotted page header contains:

– number of record entries
– end of free space in the block
– location and size of each record
Records can be moved around within a page to keep them contiguous with no empty space
between them; entry in the header must be updated.
Pointers should not point directly to record — instead they should point to the entry for the
record in header.
BITS Pilani,Pilani
Hyderabad Campus
Campus
Organization of Records in Files

Heap – a record can be placed anywhere in the file where there is space
Sequential – store records in sequential order, based on the value of the search key of each
record
Hashing – a hash function computed on some attribute of each record; the result specifies in
which block of the file the record should be placed

BITS Pilani,Pilani
Hyderabad Campus
Campus
Sequential File Organization

Suitable for applications that require sequential processing of the entire file
The records in the file are ordered by a search-key

BITS Pilani,Pilani
Hyderabad Campus
Campus
Sequential File Organization

Deletion – use pointer chains

Insertion –locate the position where the record is to be inserted
– if there is free space insert there
– if no free space, insert the record in an overflow block
– In either case, pointer chain must be updated
Need to reorganize the file
from time to time to restore
sequential order

BITS Pilani,Pilani
Hyderabad Campus
Campus
Multitable Clustering File Organization

Store several relations in one file using a

multitable clustering file organization
department

instructor

multitable clustering
of department and
instructor

BITS Pilani,Pilani
Hyderabad Campus
Campus
Multitable Clustering File Organization (cont.)

good for queries involving department instructor, and for queries involving one single
department and its instructors
bad for queries involving only department
results in variable size records
Can add pointer chains to link records of a particular relation

BITS Pilani,Pilani
Hyderabad Campus
Campus
Data Dictionary Storage

The data dictionary (also called system catalog) stores metadata; that is, data about data,
such as

Information about relations

– names of relations
– names, types and lengths of attributes of each relation
– names and definitions of views
– integrity constraints
User and accounting information, including passwords
Statistical and descriptive data
– number of tuples in each relation
Physical file organization information
– How relation is stored (sequential/hash/…)
– Physical location of relation
Information about indices
BITS Pilani,Pilani
Hyderabad Campus
Campus
Relational Representation of System Metadata

Relational representation on disk

Specialized data structures
designed for efficient access,
in memory

BITS Pilani,Pilani
Hyderabad Campus
Campus
Storage Access

A database file is partitioned into fixed-length storage units called blocks. Blocks are units of
both storage allocation and data transfer.
Database system seeks to minimize the number of block transfers between the disk and
memory. We can reduce the number of disk accesses by keeping as many blocks as possible
in main memory.
Buffer – portion of main memory available to store copies of disk blocks.
Buffer manager – subsystem responsible for allocating buffer space in main memory .

BITS Pilani,Pilani
Hyderabad Campus
Campus
Buffer Manager

Programs call on the buffer manager when they need a block from disk.
1. If the block is already in the buffer, buffer manager returns the address of the block in
main memory
2. If the block is not in the buffer, the buffer manager
1. Allocates space in the buffer for the block
1. Replacing (throwing out) some other block, if required, to make space for the new block.
2. Replaced block written back to disk only if it was modified since the most recent time that
it was written to/fetched from the disk.
2. Reads the block from the disk to the buffer, and returns the address of the block in main
memory to requester.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Buffer-Replacement Policies

Most operating systems replace the block least recently used (LRU strategy)
– Idea behind LRU – use past pattern of block references as a predictor of future references
Queries have well-defined access patterns (such as sequential scans), and a database system
can use the information in a user’s query to predict future references
– LRU can be a bad strategy for certain access patterns involving repeated scans of data
• For example: when computing the join of 2 relations r and s by a nested loops
for each tuple tr of r do
for each tuple ts of s do
if the tuples tr and ts match …
– Mixed strategy with hints on replacement strategy provided
by the query optimizer is preferable

BITS Pilani,Pilani
Hyderabad Campus
Campus
Buffer-Replacement Policies (Cont.)

Pinned block – memory block that is not allowed to be written back to disk.
Toss-immediate strategy – frees the space occupied by a block as soon as the final tuple of
that block has been processed
Most recently used (MRU) strategy – system must pin the block currently being processed.
After the final tuple of that block has been processed, the block is unpinned, and it becomes
the most recently used block.
Buffer manager can use statistical information regarding the probability that a request will
reference a particular relation
– E.g., the data dictionary is frequently accessed. Heuristic: keep data-dictionary blocks in
main memory buffer
Buffer managers also support forced output of blocks for the purpose of recovery.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Indexing and Hashing

BITS Pilani,Pilani
Hyderabad Campus
Campus
Indexing and Hashing

Basic Concepts
Ordered Indices
B+-Tree Index Files
Static Hashing
Dynamic Hashing
Multiple-Key Access
Bit-Map Indices

BITS Pilani,Pilani
Hyderabad Campus
Campus
Basic Concepts

Indexing mechanisms used to speed up access to desired data.

– E.g., author catalog in library
Search Key - attribute to set of attributes used to look up records in a file.
An index file consists of records (called index entries) of the form

search-key pointer
Index files are typically much smaller than the original file
Two basic kinds of indices:
– Ordered indices: search keys are stored in sorted order
– Hash indices: search keys are distributed uniformly across “buckets” using a “hash
function”.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Index Evaluation Metrics

Access types supported efficiently. E.g.,

– records with a specified value in the attribute
– or records with an attribute value falling in a specified range of values.
Access time
Insertion time
Deletion time
Space overhead

Multiple indices are possible with multiple search keys.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Ordered Indices

In an ordered index, index entries are stored sorted on the search key value. E.g., author
catalog in library.
Primary index: in a sequentially ordered file, the index whose search key specifies the
sequential order of the file.
– Also called clustering index
– The search key of a primary index is usually the primary key.
Secondary index: an index whose search key specifies an order different from the sequential
order of the file. Also called non-clustering index.
Index-sequential file: ordered sequential file with a primary index.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Dense Index Files

Dense index — Index record appears for every search-key value in the file.
E.g. index on ID attribute of instructor relation

BITS Pilani,Pilani
Hyderabad Campus
Campus
Dense Index Files (Cont.)
Dense index on dept_name, with instructor file sorted on dept_name

BITS Pilani,Pilani
Hyderabad Campus
Campus
Sparse Index Files

Sparse Index: contains index records for only some search-key values.
– Applicable when records are sequentially ordered on search-key
To locate a record with search-key value K we:
– Find index record with largest search-key value < K
– Search file sequentially starting at the record to which the index record points

BITS Pilani,Pilani
Hyderabad Campus
Campus
Sparse Index Files (Cont.)

Compared to dense indices:

– Less space and less maintenance overhead for insertions and deletions.
– Generally slower than dense index for locating records.
Good tradeoff: sparse index with an index entry for every block in file, corresponding to least
search-key value in the block.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Multilevel Index
If primary index does not fit in memory, access becomes expensive.
Solution: treat primary index kept on disk as a sequential file and construct a sparse index on it.
– outer index – a sparse index of primary index
– inner index – the primary index file
If even outer index is too large to fit in main memory, yet another level of index can be created,
and so on.
Indices at all levels must be updated on insertion or deletion from the file.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Multilevel Index (Cont.)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Index Update

Case: Insertion of a record in a file:

BITS Pilani,Pilani
Hyderabad Campus
Campus
(Contd.)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Case: Deletion of a record from a file

BITS Pilani,Pilani
Hyderabad Campus
Campus
(Contd.)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Secondary Indices Example

Secondary index on salary field of instructor

Index record points to a bucket that contains pointers to all the actual records with that particular
search-key value.
Secondary indices have to be dense

BITS Pilani,Pilani
Hyderabad Campus
Campus
B+-Tree Index Files

B+-tree indices are an alternative to indexed-sequential files.

Disadvantage of indexed-sequential files

– performance degrades as file grows, since many overflow blocks get created.
– Periodic reorganization of entire file is required.
Advantage of B+-tree index files:
– automatically reorganizes itself with small, local, changes, in the face of insertions and
deletions.
– Reorganization of entire file is not required to maintain performance.
(Minor) disadvantage of B+-trees:
– extra insertion and deletion overhead, space overhead.
Advantages of B+-trees outweigh disadvantages
– B+-trees are used extensively

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example of B+-Tree

BITS Pilani,Pilani
Hyderabad Campus
Campus
B+-Tree Index Files (Cont.)

A B+-tree is a rooted tree satisfying the following properties:

All paths from root to leaf are of the same length

Each node that is not a root or a leaf has between én/2ù and n pointers (children).
A leaf node has between é(n–1)/2ù and n–1 values
Special cases:
– If the root is not a leaf, it has at least 2 children.
– If the root is a leaf (that is, there are no other nodes in the tree), it can have between 0 and
(n–1) values.

BITS Pilani,Pilani
Hyderabad Campus
Campus
B+-Tree Node Structure

Typical node

– Ki are the search-key values

– Pi are pointers to children (for non-leaf nodes) or pointers to records or buckets of records
(for leaf nodes).
The search-keys in a node are ordered
K1 < K2 < K3 < . . . < Kn–1
(Initially assume no duplicate keys, address duplicates later)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Leaf Nodes in B+-Trees

Properties of a leaf node:

For i = 1, 2, . . ., n–1, pointer Pi points to a file record with search-key value Ki,
If Li, Lj are leaf nodes and i < j, Li’s search-key values are less than or equal to Lj’s search-key
values
Pn points to next leaf node in search-key order

BITS Pilani,Pilani
Hyderabad Campus
Campus
Non-Leaf Nodes in B+-Trees

Non leaf nodes form a multi-level sparse index on the leaf nodes. For a non-leaf node with m
pointers:
– All the search-keys in the subtree to which P1 points are less than K1
– For 2 £ i £ n – 1, all the search-keys in the subtree to which Pi points have values greater
than or equal to Ki–1 and less than Ki
– All the search-keys in the subtree to which Pn points have values greater than or equal to
Kn–1

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example of B+-tree

Leaf nodes must have between 3 and 5 values

(é(n–1)/2ù and n –1, with n = 6).
Non-leaf nodes other than root must have between 3 and 6 children (é(n/2ù and n with n =6).
Root must have at least 2 children.

B+-tree for instructor file (n = 6)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Observations about B+-trees

Since the inter-node connections are done by pointers, “logically” close blocks need not be
“physically” close.
The non-leaf levels of the B+-tree form a hierarchy of sparse indices.
The B+-tree contains a relatively small number of levels
• Level below root has at least 2* én/2ù values
• Next level has at least 2* én/2ù * én/2ù values
• .. etc.
– If there are K search-key values in the file, the tree height is no more than é logén/2ù(K)ù
– thus searches can be conducted efficiently.
Insertions and deletions to the main file can be handled efficiently, as the index can be
restructured in logarithmic time.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Queries on B+-Trees

Find record with search-key value V.

1. C=root
2. While C is not a leaf node {
1. Let i be least value s.t. V < Ki.
2. If no such exists, set C = last non-null pointer in C
3. Else set C = Pi
}
3. Let i be least value s.t. Ki = V
4. If there is such a value i, follow pointer Pi to the desired record.
5. Else no record with search-key value k exists.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Queries on B+-Trees (Cont.)

If there are K search-key values in the file, the height of the tree is no more than élogén/2ù(K)ù.
A node is generally the same size as a disk block, typically 4 kilobytes
– and n is typically around 100 (40 bytes per index entry).

With 1 million search key values and n = 100

– at most log50(1,000,000) = 4 nodes are accessed in a lookup.

Contrast this with a balanced binary tree with 1 million search key values — around 20 nodes are accessed
in a lookup
– above difference is significant since every node access may need a disk I/O, costing around 20 milliseconds

BITS Pilani,Pilani
Hyderabad Campus
Campus
Updates on B+-Trees: Insertion

1. Find the leaf node in which the search-key value would appear
2. If the search-key value is already present in the leaf node
1. Add record to the file
2. If necessary add a pointer to the bucket.
3. If the search-key value is not present, then
1. add the record to the main file (and create a bucket if necessary)
2. If there is room in the leaf node, insert (key-value, pointer) pair in the leaf node
3. Otherwise, split the node (along with the new (key-value, pointer) entry) as discussed in
the next slide.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Updates on B+-Trees: Insertion (Cont.)

Splitting a leaf node:

– take the n (search-key value, pointer) pairs (including the one being inserted) in sorted order. Place the first én/2ù in the original node, and the rest in a
new node.
– let the new node be p, and let k be the least key value in p. Insert (k,p) in the parent of the node being split.
– If the parent is full, split it and propagate the split further up.

Splitting of nodes proceeds upwards till a node that is not full is found.
– In the worst case the root node may be split increasing the height of the tree by 1.

Result of splitting node on insertion of Clearview

Next step: insert entry with (Clearview,pointer-to-new
-node) into parent

BITS Pilani,Pilani
Hyderabad Campus
Campus
B+-Tree Insertion

BITS Pilani,Pilani
Hyderabad Campus
Campus
Insertion in B+-Trees (Cont.)

Splitting a non-leaf node: when inserting (k,p) into an already full internal node N
– Copy N to an in-memory area M with space for n+1 pointers and n keys
– Insert (k,p) into M
– Copy P1,K1, …, K én/2ù-1,P én/2ù from M back into node N
– Copy Pén/2ù+1,K én/2ù+1,…,Kn,Pn+1 from M into newly allocated node N’
– Insert (K én/2ù,N’) into parent N
Read pseudocode in book!

BITS Pilani,Pilani
Hyderabad Campus
Campus
Updates on B+-Trees: Deletion

Find the record to be deleted, and remove it from the main file and from the bucket (if present)
Remove (search-key value, pointer) from the leaf node if there is no bucket or if the bucket has
become empty
If the node has too few entries due to the removal, and the entries in the node and a sibling fit
into a single node, then merge siblings:
– Insert all the search-key values in the two nodes into a single node (the one on the left),
and delete the other node.
– Delete the pair (Ki–1, Pi), where Pi is the pointer to the deleted node, from its parent,
recursively using the above procedure.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Updates on B+-Trees: Deletion

Otherwise, if the node has too few entries due to the removal, but the entries in the node and a
sibling do not fit into a single node, then redistribute pointers:
– Redistribute the pointers between the node and a sibling such that both have more than the
minimum number of entries.
– Update the corresponding search-key value in the parent of the node.
The node deletions may cascade upwards till a node which has én/2ù or more pointers is found.
If the root node has only one pointer after deletion, it is deleted and the sole child becomes the
root.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Examples of B+-Tree Deletion

BITS Pilani,Pilani
Hyderabad Campus
Campus
Examples of B+-Tree Deletion (Cont.)

 Node with Redwood became underfull, and was merged with its sibling
 Root node then has only one child, and is deleted

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example of B+-tree Deletion (Cont.)

 Node with Redwood became underfull

but the sibling has already maximum
no. of pointers and hence cannot
accommodate an additional pointer so
the solution is to put the entries in the
node and a sibling into a single node,
then redistribute pointers:
 Redistribute the pointers between
the node and a sibling such that
both have more than the minimum
number of entries. This is done by
borrowing a singly entry from an
adjacent node (sibling).
 Update the corresponding search-
key value in the parent of the
node.

BITS Pilani,Pilani
Hyderabad Campus
Campus
B+-Tree File Organization

Index file degradation problem is solved by using B+-Tree indices.

Data file degradation problem is solved by using B+-Tree File Organization.
The leaf nodes in a B+-tree file organization store records, instead of pointers.
Leaf nodes are still required to be half full
– Since records are larger than pointers, the maximum number of records that can be stored
in a leaf node is less than the number of pointers in a nonleaf node.
Insertion and deletion are handled in the same way as insertion and deletion of entries in a B +-
tree index.

BITS Pilani,Pilani
Hyderabad Campus
Campus
 B+-Tree File Organization (Cont.)

Good space utilization important since records use more space than pointers.
To improve space utilization, involve more sibling nodes in redistribution during splits and merges
– Involving 2 siblings in redistribution (to avoid split / merge where possible) results in each node having at least entries

Example of B+-tree File Organization

BITS Pilani,Pilani
Hyderabad Campus
Campus
 Multiple-Key Access

Use multiple indices for certain types of queries.

Possible strategies for processing query using indices on single attributes:

1. Use index on branch_name to find branches with branch name Perryridge; scan each record for
balance = 1000
2. Use index on balance to find accounts with a balance of 1000; test branch_name = “Perryridge”.
3. Use branch_name index to find pointers to all records pertaining to the “Perryridge” branch.
Similarly use index on balance. Take intersection of both sets of pointers obtained. Disadvantage: not
efficient in case of many pointers in individual indices but very few pointers in the intersection of both
indices.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Indices on Multiple Attributes

Composite search keys are search keys containing more than one attribute
– E.g. (branch_name, balance)
Lexicographic ordering: (a1, a2) < (b1, b2) if either
– a1 < b1, or
– a1=b1 and a2 < b2

BITS Pilani,Pilani
Hyderabad Campus
Campus
Indices on Multiple Attributes
Suppose we have an index on combined search-key
(branch_name, balance).
With the where clause
where branch_name = “Perryridge” and balance = 1000
the index on (branch_name, balance) can be used to fetch only records that satisfy both
conditions.
– Using separate indices in less efficient — we may fetch many records (or pointers) that
satisfy only one of the conditions.
Can also efficiently handle
where branch_name = “Perryridge” and balance < 1000
But cannot efficiently handle
where branch_name < “Perryridge” and balance = 10000
– May fetch many records that satisfy the first but not the second condition

BITS Pilani,Pilani
Hyderabad Campus
Campus
Other Kinds of Indices

Hash indices
– Used widely for in-memory indices, but not much for on-disk indices
Bitmap indices
– Used for attributes with very few distinct values
– Primary use: answering some aggregate queries very efficiently

BITS Pilani,Pilani
Hyderabad Campus
Campus
Hash-Based Indexes

❖ Static Hash indexes

❖ Dynamic Hash indexes

BITS Pilani,Pilani
Hyderabad Campus
Campus
Static Hashing

A bucket is a unit of storage containing one or more records (a bucket is

typically a disk block).
In a hash file organization we obtain the bucket of a record directly from its
search-key value using a hash function.
Hash function h is a function from the set of all search-key values K to the set of
all bucket addresses B.
Hash function is used to locate records for access, insertion as well as deletion.
Records with different search-key values may be mapped to the same bucket;
thus entire bucket has to be searched sequentially to locate a record.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Hash Functions

Worst hash function maps all search-key values to the same bucket; this makes
access time proportional to the number of search-key values in the file.
An ideal hash function is uniform, i.e., each bucket is assigned the same
number of search-key values from the set of all possible values.
Ideal hash function is random, so each bucket will have the same number of
records assigned to it irrespective of the actual distribution of search-key
values in the file.
Typical hash functions perform computation on the internal binary representation
of the search-key.
– For example, for a string search-key, the binary representations of all the
characters in the string could be added and the sum modulo the number of buckets
could be returned.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example of Hash File Organization

There are 10 buckets,

The binary representation of the ith character is assumed to be the integer i.
The hash function returns the sum of the binary representations of the characters modulo 10
– E.g. h(Perryridge) = 5 h(Mianus) = 7
h(Brighton) = 3 h(Redwood) = 4

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example of Hash File Organization

Hash file organization of account file, using branch_name as key

BITS Pilani,Pilani
Hyderabad Campus
Campus
Handling of Bucket Overflows

Bucket overflow can occur because of

– Insufficient buckets
– Skew in distribution of records. This can occur due to two reasons:
• multiple records have same search-key value
• chosen hash function produces non-uniform distribution of key values
Although the probability of bucket overflow can be reduced, it cannot be eliminated; it is handled by using
overflow buckets.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Handling of Bucket Overflows (Cont.)

Overflow chaining – the overflow buckets of a given bucket are chained together in a
linked list.
Above scheme is called closed hashing.
– An alternative, called open hashing, which does not use overflow buckets, is not
suitable for database applications.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Hash Indices

Hashing can be used not only for file organization, but also for index-structure
creation.
A hash index organizes the search keys, with their associated record pointers,
into a hash file structure.
Strictly speaking, hash indices are always secondary indices
– if the file itself is organized using hashing, a separate primary hash index on it using the same search-key is unnecessary.
– However, we use the term hash index to refer to both secondary index structures and hash organized files.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example of Hash Index

Hash function: Sum of the

digits of search key
account_number modulo 7

BITS Pilani,Pilani
Hyderabad Campus
Campus
Deficiencies of Static Hashing

In static hashing, function h maps search-key values to a fixed set of B of bucket

addresses. Databases grow or shrink with time.
– If initial number of buckets is too small, and file grows, performance will degrade
due to too much overflows.
– If space is allocated for anticipated growth, a significant amount of space will be
wasted initially (and buckets will be underfull).
– If database shrinks, again space will be wasted.
One solution: periodic re-organization of the file with a new hash function
– Expensive, disrupts normal operations

Better solution: allow the number of buckets to be modified dynamically.

BITS Pilani,Pilani
Hyderabad Campus
Campus
Dynamic Hashing

Good for database that grows and shrinks in size

Allows the hash function to be modified dynamically
Extendable hashing – one form of dynamic hashing
– Hash function generates values over a large range — typically b-bit integers, with b
= 32.
– At any time use only a prefix of the hash function to index into a table of bucket
addresses.
– Let the length of the prefix be i bits, 0 £ i £ 32.
• Bucket address table size = 2i. Initially i = 0
• Value of i grows and shrinks as the size of the database grows and shrinks.
– Multiple entries in the bucket address table may point to a bucket .
– Thus, actual number of buckets = 2 i-i j

• Where 2ij are the no. of buckets.

• The number of buckets also changes dynamically due to coalescing and splitting of buckets.

BITS Pilani,Pilani
Hyderabad Campus
Campus
General Extendable Hash Structure

In this structure, i2 = i3 = i, whereas i1 = i – 1

BITS Pilani,Pilani
Hyderabad Campus
Campus
Use of Extendable Hash Structure

Each bucket j stores records with search key values.

All the entries that point to the same bucket have the same values on the first ij bits.
To locate the bucket containing search-key Kl:
1. Compute h(Kl) = X
2. Use the first i high order bits of X as a displacement into bucket address table,
and follow the pointer to appropriate bucket
To insert a record with search-key value Kl
– follow same procedure as look-up and locate the bucket, say j.
– If there is room in the bucket j insert record in the bucket.
– Else the bucket must be split and insertion re-attempted. ( Overflow buckets used instead in some cases.)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Insertion in Extendable Hash Structure (Cont.)

To split a bucket j when inserting record with search-key value Kl:

If i > ij (more than one pointer to bucket j)

– allocate a new bucket z, and set ij = iz = (ij + 1)
– Update the second half of the bucket address table entries originally pointing to j, to point to z
– remove each record in bucket j and reinsert (in j or z)
– recompute new bucket for Kl and insert record in the bucket (further splitting is required if the bucket is
still full)
If i = ij (only one pointer to bucket j)
– If i reaches some limit b, or too many splits have happened in this insertion, create an overflow bucket
(If all records in bucket j have same search key values, splitting will not help)
– Else
• increment i and double the size of the bucket address table.
• replace each entry in the table by two entries that point to the same bucket. Use second pointer to point to new bucket for bucket j and set ij = iz = i
• Rehash each record to bucket j or newly created bucket z. Recompute new bucket address table entry for Kl

BITS Pilani,Pilani
Hyderabad Campus
Campus
Deletion in Extendable Hash Structure

To delete a key value,

– locate it in its bucket and remove it.
– The bucket itself can be removed if it becomes empty (with appropriate updates to the bucket address table).

– Coalescing of buckets can be done (can coalesce only with a “buddy” bucket having same value of i and same i –1 prefix, if it is present)
j
j
– Decreasing bucket address table size is also possible
• Note: decreasing bucket address table size is an expensive operation and should be done only if number of buckets becomes
much smaller than the size of the table

BITS Pilani,Pilani
Hyderabad Campus
Campus
Use of Extendable Hash Structure: Example

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example (Cont.)
 Initial Hash structure; bucket size = 2

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example (Cont.)

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example (Cont.)

 Hash structure after four insertions

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example (Cont.)

 Hash structure after insertion of 7 records

BITS Pilani,Pilani
Hyderabad Campus
Campus
Example (Cont.)

 Hash structure after insertion of all records

BITS Pilani,Pilani
Hyderabad Campus
Campus
Extendable Hashing vs. Other
Schemes
Benefits of extendable hashing:
– Hash performance does not degrade with growth of file
– Buckets can be allocated dynamically
Disadvantages of extendable hashing
– Extra level of indirection to find desired record
– Changing size of bucket address table is an expensive operation

BITS Pilani,Pilani
Hyderabad Campus
Campus
Bitmap Indices

Bitmap indices are a special type of index designed for efficient querying on multiple
keys
Records in a relation are assumed to be numbered sequentially from, say, 0
– Given a number n it must be easy to retrieve record n
• Particularly easy if records are of fixed size
Applicable on attributes that take on a relatively small number of distinct values
– E.g. gender, country, state, …
– E.g. income-level (income broken up into a small number of levels such as 0-9999,
10000-19999, 20000-50000, 50000- infinity)
A bitmap is simply an array of bits

BITS Pilani,Pilani
Hyderabad Campus
Campus
Bitmap Indices (Cont.)

In its simplest form a bitmap index on an attribute has a bitmap for each value of the attribute
– Bitmap has as many bits as records
– In a bitmap for value v, the bit for a record is 1 if the record has the value v for the attribute,
and is 0 otherwise

BITS Pilani,Pilani
Hyderabad Campus
Campus
Bitmap Indices (Cont.)

Bitmap indices are useful for queries on multiple attributes

– not particularly useful for single attribute queries
Queries are answered using bitmap operations
– Intersection (and)
– Union (or)
– Complementation (not)
Each operation takes two bitmaps of the same size and applies the operation on corresponding
bits to get the result bitmap
– E.g. 100110 AND 110011 = 100010
100110 OR 110011 = 110111
NOT 100110 = 011001
– Males with income level L1: 10010 AND 10100 = 10000
• Can then retrieve required tuples.
• Counting number of matching tuples is even faster

BITS Pilani,Pilani
Hyderabad Campus
Campus