DATABASE

TECHNOLOGIES
Parallel and Distributed Databases
Department of Computer Science and Engineering

Acknowledgment: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES

Parallel and Distributed Databases

Parallel Databases
1) Shared-Memory Machines
2) Shared-Disk Machines , NAS/DAS/SAN
3) Shared-Nothing Machines
Intraquery Parallelism
Interquery Parallelism
DATABASE TECHNOLOGIES
Parallel Databases

• A parallel database system seeks to improve performance through parallelization of

various operations such as loading data, building indexes and evaluating queries

• Parallel databases improve processing and input/output speeds by using

multiple CPUs and disks in parallel.
DATABASE TECHNOLOGIES
Parallel Databases

Parallel Algorithms for Query Processing

In addition to two-tier and three-tier architectures there is another type of architecture,
called parallel database architecture, that is prevalent for data-intensive applications.
Three main approaches have been proposed for parallel databases.
They correspond to three different hardware configurations of processors and secondary storage
devices (disks) to support parallelism.
DATABASE TECHNOLOGIES
Parallel Databases

Parallel architectures are classified into three broad groups.

1) Shared-Memory Machines
The most tightly coupled architectures share their main memory.
1) Shared-Disk Machines
● A less tightly coupled architecture shares disk but not memory.
2) Shared-Nothing Machines
● Architectures that are often used for databases do not even share disk; these are
called “shared nothing” architectures,
● Although the processors are in fact interconnected and share data through message
passing.
DATABASE TECHNOLOGIES
Models of Parallelism
DATABASE TECHNOLOGIES
Parallel Databases - 1) Shared-Memory Machines

The most tightly coupled architectures share their main memory.

a) In this architecture, illustrated in Fig., each processor has access to all the memory of
all the processors.
b) That is, there is a single physical address space for the entire machine, rather than
one address space for each processor.
c) Each processor has some local main memory, which it typically uses whenever it can.
However, it has direct access to the memory of other processors when it needs to.
d) Large machines of this class are of the NUMA (non-uniform memory access) type,
meaning that it takes somewhat more time for a processor to access data in a
memory that “belongs” to some other processor than it does to access its “own”
memory, or the memory of processors in its local cluster. However, the difference in
memory-access times are not great in current architectures.
e) All memory accesses, no matter where the data is, take much more time than a
cache access, so the critical issue is whether or not the data a processor needs is in
its own cache.
DATABASE TECHNOLOGIES
Parallel Databases - 2) Shared-Disk Machines

A less tightly coupled architecture shares disk but not memory.

a) In this architecture, suggested by Fig., every processor has its own memory, which is not
accessible directly from other processors.
b) However, the disks are accessible from any of the processors through the communication
network.
c) Disk controllers manage the potentially competing requests from different processors.
d) The number of disks and processors need not be identical, as it might appear from the Figure.
e) This architecture today appears in two forms, depending on the units of transfer between the
disks and processors.
f) Disk farms called network attached storage (NAS) store and transfer files.
g) The alternative, storage area networks (SAN) transfer disk blocks to and from the processors.
DATABASE TECHNOLOGIES
Parallel Databases - 2) Shared-Disk Machines (Cont’d)

We discussed two forms of enterprise-level secondary storage systems earlier.

Both storage area networks (SANs) and network attached storage (NAS) fall into
the shared-disk architecture and lend themselves to parallel processing.
They have different units of data transfer:
● SANs transfer data in units of blocks or pages to and from disks to processors;
● NAS behaves like a file server that transfers files using some file transfer
protocol.
In these systems, as more processors are added, there is more contention for the
limited network bandwidth.
DATABASE TECHNOLOGIES
Secondary Storage Architecture

Directly Attached Storage (DAS)

Network Attached Storage (NAS)
Storage Area Network (SAN)

Fibre Channel and Gigabit Ethernet (GbE) – Image ©Huawei

DATABASE TECHNOLOGIES
Parallel Databases - 3) Shared-Nothing Machines

1) Here, all processors have their own memory and their own disk or disks, as in Figure.
2) All communication is via the network, from processor to processor.
3) For example, if one processor P wants to read tuples from the disk of another processor
Q, then processor P sends a message to Q asking for the data.
4) Processor Q obtains the tuples from its disk and ships them over the network in another
message, which is received by P.
5) As we mentioned, the shared-nothing architecture is the most commonly used
architecture for database systems.
6) Shared-nothing machines are relatively inexpensive to build; one buys racks of commodity
machines and connects them with the network connection that is typically built into the
rack. Multiple racks can be connected by an external network.
DATABASE TECHNOLOGIES
Parallel Databases - 3) Shared-Nothing Machines (Cont’d)
8) But when we design algorithms for these machines we must be aware that it is costly to
send data from one processor to another.
9) Normally, data must be sent between processors in a message, which has considerable
overhead associated with it. Both processors must execute a program that supports the
message transfer, and there may be contention or delays associated with the
communication network as well.
10) Typically, the cost of a message can be broken into a large fixed overhead plus a small
amount of time per byte transmitted.
11) Thus, there is a significant advantage to designing a parallel algorithm so that
communications between processors involve large amounts of data sent at once.
12) For instance, we might buffer several blocks of data at processor P, all bound for processor
Q. If Q does not need the data immediately, it may be much more efficient to wait until we
have a long message at P and then send it to Q.
13) Fortunately, the best known parallel algorithms for database operations can use long
messages effectively.
DATABASE TECHNOLOGIES
Parallel Databases - 3) Shared-Nothing Machines

14) Parallel databases using shared-nothing architecture are relatively inexpensive to build.
1) Today, commodity processors are being connected in this fashion on a rack, and several
racks can be connected by an external network.
15) Each processor has its own memory and disk storage.
16) The shared-nothing architecture affords the possibility of achieving parallelism in query
processing at three levels, which we will discuss below:
1) individual operator parallelism,
2) intraquery parallelism, and
3) interquery parallelism.
DATABASE TECHNOLOGIES
Parallel Databases - 3) Shared-Nothing Machines
Studies have shown that by allocating more
processors and disks, linear speed-up—a linear
reduction in the time taken for operations—is
possible.

Linear scale-up refers to being able to give a

constant sustained performance by increasing the
number of processors and disks proportional to the
size of data.

Both of these are implicit goals of parallel

processing.
DATABASE TECHNOLOGIES
Parallel Databases - 3) Shared-Nothing Machines
DATABASE TECHNOLOGIES
Parallel Databases - Query Parallelism

There are two types of query parallelism: interquery parallelism and intraquery parallelism.
1) Interquery Parallelism
Interquery parallelism refers to the ability of the database to accept queries from multiple
applications at the same time. Each query runs independently of the others, but the database
manager runs all of them at the same time. Db2 database products have always supported this
type of parallelism.
• Intra Query Parallelism
Intraquery parallelism refers to the simultaneous processing of parts of a single query, using
either intrapartition parallelism, interpartition parallelism, or both.
DATABASE TECHNOLOGIES
Parallel Databases -
Query Execution

• We have discussed how each individual operation may be executed by distributing the data among
multiple processors and performing the operation in parallel on those processors.

• A query execution plan can be modeled as a graph of operations.

• To achieve a parallel execution of a query, one approach is to use a parallel algorithm for each
operation involved in the query, with appropriate partitioning of the data input to that operation.

• Another opportunity to parallelize comes from the evaluation of an operator tree where some of the
operations may be executed in parallel because they do not depend on one another.

• These operations may be executed on separate processors. If the output of one of the operations
can be generated tuple-by-tuple and fed into another operator, the result is pipelined parallelism.

• An operator that does not produce any output until it has consumed all its inputs is said to block the
pipelining.
 DATABASE TECHNOLOGIES
Parallel Databases -

Interquery Parallelism

1) Interquery parallelism refers to the execution of multiple queries in parallel.

2) Activities of locking, logging, and so on among processors must be coordinated, and
simultaneous conflicting updates of the same data by multiple processors must be avoided.
3) The main goal behind interquery parallelism is to scale up (i.e., to increase the overall rate
at which queries or transactions can be processed by increasing the number of processors).
4) Because single-processor multiuser systems themselves are designed to support
concurrency control among transactions with the goal of increasing transaction
throughput, database systems using shared memory parallel architecture can achieve this
type of parallelism more easily without significant changes.
5)Cont’d...
DATABASE TECHNOLOGIES
Parallel Databases -
Interquery Parallelism

5) We can speed up the query execution by performing various operations, such as sorting,
selection, projection, join, and aggregate operations, individually using their parallel
execution.
6) We may achieve further speed-up by executing parts of the query tree that are
independent in parallel on different processors.
7) However, it is difficult to achieve interquery parallelism in shared-nothing parallel
architectures or shared-disk architectures.
8) One area where the shared disk architecture has an edge is that it has a more general
applicability, since it, unlike the shared-nothing architecture, does not require data to be
stored in a partitioned manner.
DATABASE TECHNOLOGIES
Parallel Databases -

THANK YOU
Department of Computer Science
and Engineering

20
 DATABASE
TECHNOLOGIES
Map Reduce
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES

Parallel and Distributed Databases

Parallel Databases -
The Map-Reduce Parallelism Framework
DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework

• Map-reduce is a high-level programming system that allows many important database

processes to be written
• User writes code for two functions, map and reduce.
• A master controller divides the input data into chunks and assigns different processors to
execute the map function on each chunk.
• Other processors or perhaps the same ones are then assigned to perform the reduce
function on pieces of the output from the map function.
DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework

The Storage Model

• Assume a massively parallel machine most likely shared-nothing.
• Typically, the processors are commodity computers, mounted in racks with a simple
communication network among the processors on a rank.
• If there is more than one rack, the racks are also connected by a simple network
• Data is stored in files. Typically, the files are very large compared with the files found in
conventional systems.
• Files are divided into chunks, which might be complete cylinders of a disk and are typically
many megabytes
• For resiliency, each chunk is replicated several times so it will not be lost if the disk holding it
crashes
DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework

MapReduce Processing

Mapping – indexing and sorting data into the

desired structure based on the specified key-
value pairs

Shuffling – redistribute mapper’s output

across nodes depending on the key

Reducing – produces summary of previously

mapped and shuffled data
DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework
DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework
DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework

Hadoop Distributed File System

DATABASE TECHNOLOGIES
Parallel Databases - The Map-Reduce Parallelism Framework

Advantages
1. The model is easy to use since it hides the details of parallelization, fault-tolerance, locality
optimization and load balancing

2. A large variety of problems are easily expressible as MapReduce computations.

3. Scales to large clusters of machines comprising thousands of machines. The implementation

makes efficient use of these machine resources and therefore is suitable for use on many of
the large computational problems
THANK YOU
Department of Computer Science
and Engineering

30
 DATABASE
TECHNOLOGIES
Parallel Algorithms on Relations
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES

Parallel and Distributed Databases

Tuple-at-a-Time Operations in Parallel
DATABASE TECHNOLOGIES
Parallel Databases

Tuple-at-a-Time Operations in Parallel

• Consider a shared-nothing machine. If there are p processors, divide relation R’s tuples
evenly among the p processor’s disks.
• To compute σC (R), use each processor to examine the tuples of R on its own disk.
• Store those tuples t in σC (R) at the same processor that has t on its disk. Thus, the result
relation σC (R) is divided among the processors just like R is.
• In case of a projection, the number of tuples in πL(R) at each processor would be the same as
the number of tuples of R at that processor.
• Thus, if R is distributed evenly, so would be its projection. However, a selection could
radically change the distribution of tuples in the result set compared to the distribution of R.
DATABASE TECHNOLOGIES
Parallel Databases

Parallel Algorithms for Full-Relation Operations

• Consider the operation δ(R). If we use a hash function to distribute the tuples of R, then we shall place
duplicate tuples of R at the same processor.
• We can produce δ(R) in parallel by applying a standard algorithm to the portion of R at each processor.
• If we use the same hash function to distribute the tuples of both R and S, then we can take the union,
intersection or difference of R and S by working in parallel on the portions of R and S at each processor.
• Suppose that R and S are not distributed using the same hash function, re-hash the tuples and
distribute them.
• To join R(X,Y ) S(Y,Z), hash the tuples of R and S considering Y attributes to a number of buckets
equal to the number of processors.
• To compute aggregations, hash the tuples of R and S considering the grouping attributes to a number
of buckets equal to the number of processors.
DATABASE TECHNOLOGIES
Parallel Databases

Performance of Parallel Algorithms

• The total work — disk I/O’s and processor cycles cannot be smaller for a parallel machine
than for a uniprocessor.
• Because there are p processors working with p disks, we can expect the elapsed time to be
much smaller for the multiprocessor than for the uniprocessor.
• A unary operation such as σC (R) can be completed in 1/pth of the time it would take to
perform the operation at a single processor, provided relation R is distributed evenly.
• The number of disk I/O’s is essentially the same as for a uniprocessor selection.
DATABASE TECHNOLOGIES
Parallel Databases

Performance of Parallel Algorithms – Join

• Use a hash function on the join attributes that sends each tuple to one of p buckets, where
p is the number of processors.
• To distribute the tuples belonging to one processor, we must read each tuple from disk to
memory, compute the hash function and ship all tuples except the one out of p tuples that
happens to belong to the bucket at its own processor.
• We need to do B(R) + B(S) disk I/O’s to read all the tuples of R and S and determine their
buckets.
• We then must ship ((p − 1)/p) * (B(R) + B(S)) blocks of data across the machine’s internal
interconnection network to their proper processors; only the (1/p)th of the tuples already at
the right processor need not be shipped.
DATABASE TECHNOLOGIES
Parallel Databases

Performance of Parallel Algorithms – Join

• Number of disk I/O’s for the first read of each tuple and the storing away of each tuple by the
processor receiving the tuple during the hash and distribution of tuples = 2(B(R) + B(S)) / p .
• Number of disk I/O’s done by a naive parallel algorithm at each processor = 3(B(R) + B(S)) /p .
• Number of disk I/O’s performed at each processor
= 5(B(R) + B(S)) /p compared to 3(B(R) + B(S)) in a uniprocessor machine.
DATABASE TECHNOLOGIES
Parallel Databases

Performance of Parallel Algorithms – Join example

• Consider R(X,Y ) S(Y,Z), where B(R) = 1000 and B(S) = 500
• Let M = 101 and p = 10. Assume that tuples of R and S are distributed uniformly among the 10 processors
• Hash each tuple of R and S to one of 10 “buckets,” using a hash function h that depends only on the join
attributes Y. Tuples are shipped to the processor corresponding to their “bucket.”
• The total number of disk I/O’s needed to read the tuples of R and S is 1000 + 500 = 1500 or 150 per
processor
• Each processor will have about 15 blocks worth of data for each other processor, so it ships 135 blocks to
the other nine processors. The total communication is thus 1350 blocks
• Arrange such that the processors ship the tuples of S before the tuples of R. Since each processor
receives about 50 blocks of tuples from S, it can store those tuples in a main-memory data structure,
using 50 of its 101 buffers
DATABASE TECHNOLOGIES
Parallel Databases

Performance of Parallel Algorithms – Join example

• When processors start sending R-tuples, each one is compared with the local S-tuples and
resulting joined tuples are output
• The cost of the join is 1500 disk I/O’s
• The elapsed time is primarily the 150 disk I/O’s performed at each processor, plus the time to
ship tuples between processors and perform the main-memory computations
• Elapsed time for 150 disk I/O’s is less than 1/10th of the time to perform the same algorithm
on a uniprocessor machine
THANK YOU
Department of Computer Science
and Engineering

40
 DATABASE
TECHNOLOGIES
Distributed Databases
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

 DATABASE TECHNOLOGIES

Parallel and Distributed Databases

Distributed Database Concepts
1)Parallel Vs Distributed Database
2)Distributed Database Concepts
3)Data Distribution Among Distributed Databases.
4)Homogeneous and Heterogeneous Distributed Databases
5)Advantages of Distributed Databases
6)Additional Functions of DDB
7)Data Fragmentation
a)Horizontal, Vertical and Hybrid Types
8)Data Replication
DATABASE TECHNOLOGIES
Parallel Vs Distributed Database

Parallel Database System:

Data is stored at a single site, can be used to improve query performance

through parallel implementation

Distributed Database System:

Data is stored across several sites, each site managed by DBMS capable of
running independently
 DATABASE TECHNOLOGIES
Distributed Database Concepts

• A distributed database (DDB) is a collection of multiple logically related

database distributed over a computer network

• A Distributed Database Management System (DDBMS) refers to a software

system that manages a Distributed Database while making the distribution
transparent to the user.
DATABASE TECHNOLOGIES
Data Distribution Among Distributed Databases
 DATABASE TECHNOLOGIES
Homogeneous Distributed Databases

In a homogeneous distributed database

▪ All sites have identical software
▪ All sites are aware of each other and agree to cooperate in processing user requests.
▪ Each site surrenders part of its autonomy in terms of right to change schemas or
software
▪ Appears to user as a single system
DATABASE TECHNOLOGIES
Heterogeneous Distributed Databases

In a heterogeneous distributed database

▪ Different sites may use different schemas and software

o Difference in schema is a major problem for query processing

o Difference in software is a major problem for transaction processing
▪ Sites may not be aware of each other and may provide only limited facilities for

cooperation in transaction processing

DATABASE TECHNOLOGIES
Advantages of Distributed Databases

1. Management of distributed data with different levels of transparency: hiding the details

of where each file (table, relation) is physically stored within the system.
2. The possible types of transparency are:

a) Distribution or network transparency – freedom for the user from the operational

details of the network. It can be divided into location and naming transparency.
b) Replication transparency – makes the user unaware of the existence of copies.
c) Fragmentation transparency – makes the user unaware of the existence of

fragments.
 DATABASE TECHNOLOGIES
Advantages of Distributed Databases

2. Increased reliability and availability: two of the most common potential advantages cited for
DDB:
a. Reliability – the probability that a system is running (not down) at a certain point of time
b. Availability – the probability that the system is continuously available during a time interval
3. Improved performance: a DDBMS fragments the database by keeping the data closer to where
it is needed most (data localization reduces CPU and I/O).
4. Easier expansion: adding more data, increasing database sizes, or adding more processors is
much easier in a DDB.
 DATABASE TECHNOLOGIES
Additional Functions of DDB

1) Keeping track of data: the ability to keep track of the data distribution, fragmentation, and
replication,
2) Distributed query processing: the ability to access remote sites via a communication network,
3) Distributed transaction management: the ability to execute transactions that access data from
many sites,
4) Replicated data management: the ability to decide which copy of a replicated data to access
and maintain the consistency of copies of replicated data items.
5) Distributed database recovery: the ability to recover from individual site crashes (or
communication failures),
6) Security: proper management of the security of the data and the authorization/access
privileges of users,
7) Distributed directory (catalog) management: design and policy issues for the placement of
directory.
 DATABASE TECHNOLOGIES
Data Fragmentation

● Techniques that are used to break up a database into logical units, called fragmentation.
• Information concerning data fragmentation, allocation, and replication is stored in a global

directory that is accessed by the DDB applications as needed.

• In a DDB, decisions must be made regarding which site should contain which portion of the

DB.
• A horizontal fragment (or shard) of a relation is a subset of the tuples in the relation (e.g.,
DNO=5).
● Horizontal fragmentation is also known as sharding or horizontal partitioning.
• A vertical fragment of a relation keeps only a subset of the attributes of a relation (e.g., SSN,
AGE).
• A mixed (hybrid) fragment of a relation combines the horizontal and vertical

fragmentations.
 DATABASE TECHNOLOGIES
Data Replication

• Data Replication is the process of storing fragments of data in more than

one site or node. It is useful in improving the availability of data
• If a site fails, there may be other sites that can provide the same data
that was at the failed site
• Speed of query processing can be improved by making a copy of needed
data available at the sites where queries are initiated.
• Replication uses a publishing industry metaphor to represent the
components in a replication topology.
• A replication topology defines the relationship between servers and
copies of data and clarifies the logic that determines how data flows
between servers.
• Publisher and Subscriber.
 DATABASE TECHNOLOGIES
Data Replication

Types of Data Replication:

1. Transactional Replication – In Transactional replication sites receive full initial copies of the database
and then receive updates as data changes. Transactional replication is typically used in server-to-
server environments. It consistently and accurately replicates each change.
2. Snapshot Replication – Snapshot replication distributes data exactly as it appears at a specific
moment in time does not monitor for updates to the data. Snapshot replication is generally used
when data changes are infrequent.
3. Merge Replication – Data from two or more databases is combined into a single database. Merge
replication is the most complex type of replication because it allows both publisher and subscriber to
independently make changes to the database. Merge replication is typically used in server-to-client
environments. It allows changes to be sent from one publisher to multiple subscribers.
THANK YOU
Department of Computer Science
and Engineering

54
54
 DATABASE
TECHNOLOGIES
Distributed Transactions
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES

Distributed Transactions
Distributed Transactions
Distributed Commit
Two-Phase Commit Protocol
Concurrency Control
DATABASE TECHNOLOGIES
Distributed Transactions

Transaction may access data at several sites.

• Each site has a local transaction manager responsible for:
• Maintaining a log for recovery purposes - log manager
• Participating in coordinating the concurrent execution of the transactions executing at
that site.
• Each site has a transaction coordinator, which is responsible for:
• Starting the execution of transactions that originate at the site.
• Distributing sub transactions at appropriate sites for execution.
• Coordinating the termination of each transaction that originates at the site, which
may result in the transaction being committed at all sites or aborted at all sites.
DATABASE TECHNOLOGIES
Distributed Transactions

Transaction may access data at several sites.

• Each site has a local transaction manager:
• Each site has a transaction coordinator:
DATABASE TECHNOLOGIES
Distributed Transaction System Architecture
DATABASE TECHNOLOGIES
Commit Protocols

• Commit protocols are used to ensure atomicity across sites

• A transaction which executes at multiple sites must either be committed at all the
sites, or aborted at all the sites.
• Not acceptable to have a transaction committed at one site and aborted at another.
• The two-phase commit (2 PC) protocol is widely used.
• The three-phase commit (3 PC) protocol is more complicated and more expensive, but
avoids some drawbacks of two-phase commit protocol.
 DATABASE TECHNOLOGIES
Two Phase Commit Protocol - Phase 1

2PC - PHASE 1:
1. The coordinator places a log record <Prepare T> on the log at its site.
2. The coordinator sends to each component’s site the message Prepare T.
3. Each site receiving the message Prepare T decides whether to commit or abort its component
of T.
4. If a site wants to commit its component, it must enter a state called pre-committed. Once in
the pre-committed state, the site cannot abort its component of T without a directive to do
so from the coordinator:
a. Perform whatever steps necessary to be sure the local component of T will not have to abort.
b. Place the record <Ready T> on the local log and flush the log to disk.
c. Send Ready T message to the coordinator.
5. If the site wants to abort its component of T, then it logs the record <Don’t Commit T> and
sends the message Don’t Commit T to the coordinator.
DATABASE TECHNOLOGIES
Two Phase Commit Protocol - Phase 2

2PC - Phase 2
1. If the coordinator has received Ready T from all components of T, only then it
decides to commit T. The coordinator logs <Commit T> at its site and then
sends message Commit T to all sites involved in T.
2. If the coordinator has received Don’t Commit T from one or more sites, it logs
<Abort T> at its site and then sends Abort T message to all sites involved in T.
3. If a site receives a Commit T message, it commits the component of T at that
site, logging <Commit T> as it does.
4. If a site receives the message Abort T, it aborts T and writes the log record
<Abort T>
DATABASE TECHNOLOGIES
Three Phase Commit Protocol

• Phase 1: Obtaining Preliminary Decision: Identical to 2PC Phase 1.

• Every site is ready to commit if instructed to do so.

• Phase 2 of 2PC is split into 2 phases, Phase 2 and Phase 3 of 3PC.

• In phase 2 of 3PC, coordinator makes a decision as in 2PC (called the pre-commit
decision) and records it in multiple (at least K) sites.

• In Phase 3, coordinator sends commit/abort message to all participating sites.

DATABASE TECHNOLOGIES
Concurrency Control

• All concurrency mechanisms must preserve data consistency and complete each atomic
action in finite time.
• Important capabilities are

a) Be resilient to site and communication link failures.

b) Allow parallelism to enhance performance requirements.

c) Incur optimal cost and optimize communication delays.

d) Place constraints on atomic action.

THANK YOU
Department of Computer Science
and Engineering

65
 DATABASE
TECHNOLOGIES
Distributed Query Processing
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES

Parallel and Distributed Databases

Distributed Query Processing

Distributed Query Processing Stages

The Distributed Join Problem
Semi Join
Joins of Many Relations
DATABASE TECHNOLOGIES
Distributed Query Processing

Distributed Query Processing Stages

1. Query Mapping - The input query on distributed data is specified formally using a query
language. It is then translated into an algebraic query on global relations. This translation is
done by referring to the global conceptual schema and does not take into account the
actual distribution and replication of data.
2. Localization - This stage maps the distributed query on the global schema to separate
queries on individual fragments/sites using data distribution and replication information.
3. Global Query Optimization - Optimization consists of selecting a strategy from a list of
candidates that is closest to optimal. Since DDBs are connected by a network, often the
communication costs over the network are the most significant.
4. Local Query Optimization - This stage is common to all sites in the DDB.
DATABASE TECHNOLOGIES
Distributed Query Processing

The Distributed Join Problem

• Consider the join operation R(A,B) S(B,C) where R and S reside at different nodes of a
network

• There are two obvious ways to compute the join.

1. Send a copy of R to the site of S and compute the join there.
2. Send a copy of S to the site of R and compute the join there.

• In a distributed system, the following must be taken into account while evaluating the cost:
1. The cost of a data transmission over the network.
2. The potential gain in performance from having several sites process parts of the query
in parallel.
DATABASE TECHNOLOGIES
Distributed Query Processing

The Distributed Join Example

• Size of the EMPLOYEE relation is 100 * 10,000 = 106 bytes
• Size of the DEPARTMENT relation is 35 * 100 = 3,500 bytes
• πFname,Lname,Dname(EMPLOYEE Dno=Dnumber DEPARTMENT)
• The result set of this query will include 10,000 records.
• The query is submitted at a distinct site 3, which is called
the result site because the query result is needed there.
Neither the EMPLOYEE nor the DEPARTMENT relations are
at site 3.
DATABASE TECHNOLOGIES
Distributed Query Processing

The Distributed Join Example

• Options:
1. Transfer both the EMPLOYEE and the DEPARTMENT relations to Site 3 and perform the
join. A total of 1,000,000 + 3,500 = 1,003,500 bytes must be transferred.
2. Transfer the EMPLOYEE relation to site 2, execute the join at site 2 and send the result
to site 3. The size of the query result is 40 * 10,000 = 400,000 bytes.
■ So 400,000 + 1,000,000 = 1,400,000 bytes must be transferred
3. Transfer the DEPARTMENT relation to site 1, execute the join at site 1 and send the
result to site 3. In this case, 400,000 + 3,500 = 403,500 bytes must be transferred.
DATABASE TECHNOLOGIES
Distributed Query Processing

Semi Join
• Only the relevant part of each relation is shipped to the site of the other relation(s)
• Project the join attributes of S and send the result set to the site where R is
• Compute R πY(S) there
• Send the result set of R πY(S) to the site where S is to compute R S
DATABASE TECHNOLOGIES
Distributed Query Processing

Semi Join Example: πFname,Lname,Dname(EMPLOYEE Dno=Dnumber DEPARTMENT) in site 2

1. Project the join attributes of DEPARTMENT at site 2 and
transfer them to site 1.
Transfer F = πDnumber(DEPARTMENT) whose size is
4 * 100 = 400 bytes
2. Join the transferred file with the EMPLOYEE relation at
site 1 and transfer the required attributes from the
resulting file to site 2. We transfer R = πDno,Fname,Lname (F Dnumber=Dno EMPLOYEE) whose size
is 34 * 10,000 = 340,000 bytes
3. Execute the query by joining the transferred file R with DEPARTMENT and present the result
to the user at site 2. Total bytes transferred = 400 + 340,000 = 340,400
DATABASE TECHNOLOGIES
Distributed Query Processing

Joins of Many Relations

When we need to join three or more relations at different sites,
• We may need several semi-joins to eliminate all the dangling tuples
from the relations before shipping them to other sites for joining.
• There are sets of relation schemas such that no finite sequence of
semi-joins eliminates all dangling tuples.
• It is possible to identify those sets of relation schemas such that
there is a finite way to eliminate dangling tuples by semi-joins.
DATABASE TECHNOLOGIES
Distributed Query Processing

Joins of Many Relations

• Consider natural join of relations R(A, B), S(B,C) and T(C,A)
• R and S join to produce the n tuples {(1, 1, 1), (2, 2, 2),...,(n, n, n)}
• R S T is empty
THANK YOU
Department of Computer Science
and Engineering

76
 DATABASE
TECHNOLOGIES
Distributed Locking
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES

Parallel and Distributed Databases

Distributed Locking

Centralized Lock Systems

Primary-Copy Locking
Global Locks From Local Locks
Distributed Catalog Management
Management Schemes For Distributed Catalogs
DATABASE TECHNOLOGIES
Distributed Locking

• Locking tables is managed by individual sites.

• Component of a transaction at a site can request locks on the data elements only at that site.
Centralized Lock Systems
• Designate one site as the lock site, to maintain a lock table for logical elements, whether or
not they have copies at that site.
• When a transaction wants a lock on logical element X, it sends a request to the lock site,
which grants or denies the lock, as appropriate.
• The usual cost is three messages per lock (request, grant and release) unless the transaction
happens to be running at the lock site.
• The use of a single lock site could become a bottleneck.
• If the lock site crashes, no transaction at any site can obtain locks.
DATABASE TECHNOLOGIES
Distributed Locking

Primary-Copy Locking
• An improvement on the centralized locking approach, one which also allows replicated data, is to
distribute the function of the lock site, but still maintain the principle that each logical element has a
single site responsible for its global lock.
• This distributed-lock method called primary copy, avoids the possibility that the central lock site will
become a bottleneck, while still maintaining the simplicity of the centralized method.
• Each logical element X has one of its copies designated as the “primary copy.” In order to get a lock
on logical element X, a transaction sends a request to the site of the primary copy of X. The site of
the primary copy maintains an entry for X in its lock table and grants or denies the request as
appropriate.
• Most lock requests require three messages, except for those where the transaction and the primary
copy are at the same site.
DATABASE TECHNOLOGIES
Distributed Locking

Global Locks From Local Locks

• Another approach is to synthesize global locks from collections of local locks.
• In these schemes, no copy of a database element X is “primary”; rather they are symmetric,
and local shared or exclusive locks can be requested on any of these copies
• The key to a successful global locking scheme is to require transactions to obtain a certain
number of local locks on copies of X before the transaction can assume it has a global lock on X.
Majority Locking
• A transaction sends one general request (broadcast) for local locks on an element X, which will
be received by all sites. Similarly, the release of locks may be achieved by a single message.
 DATABASE TECHNOLOGIES
Distributed Catalog Management

Management Schemes For Distributed Catalogs:

1. Centralized Catalogs - The entire catalog is stored in one single site. Due to its central nature, it is easy to
implement. However, the advantages of reliability, availability, autonomy, and distribution of processing
load are adversely impacted.
2. Fully Replicated Catalogs - Identical copies of the complete catalog are present at each site. This scheme
facilitates faster reads by allowing them to be answered locally. However, all updates must be broadcast
to all sites. Updates are treated as transactions and a centralized two-phase commit scheme is employed
to ensure catalog consistency
3. Partially Replicated Catalogs - Each site maintains complete catalog information on data stored locally at
that site. Each site is also permitted to cache entries retrieved from remote sites. However, there are no
guarantees that these cached copies will be the most recent and updated. The system tracks catalog
entries for sites where the object was created and for sites that contain copies of this object. Any
changes to copies are propagated immediately to the original (birth) site.
DATABASE TECHNOLOGIES
Distributed Catalog Management

Schema Architecture of Distributed Databases

THANK YOU
Department of Computer Science
and Engineering

84
 DATABASE
TECHNOLOGIES
Peer to Peer Distributed Search
Department of Computer Science and Engineering

Acknowledgement: Authors of the prescribed textbooks and materials sourced online.

DATABASE TECHNOLOGIES
Parallel and Distributed Databases
Peer-to-Peer Distributed Search
Peer-to-Peer Networks
Distributed Hashing Problem
Centralized Solutions for Distributed Hashing Problem
Chord Circles
Links in Chord Circles
Search using Finger Tables
Adding New Nodes
When a Peer Leaves the Network
When a Peer Fails
DATABASE TECHNOLOGIES
Peer-to-Peer Networks

• A peer-to-peer network is a collection of nodes (participating machines) that:

1. Are autonomous: participants do not respect any central control and can join or
leave the network at will.
2. Are loosely coupled: they communicate over a general-purpose network such as
the Internet, rather than being hard-wired together like the processors in a
parallel machine.
3. Are equal in functionality: there is no leader or controlling node.
4. Share resources with one another.
• Each node in the peer-to-peer network has a subset of the data elements, but there is
no centralized index that says where something is located
DATABASE TECHNOLOGIES
Distributed Hashing Problem

Consider the problem of searching records in a (very large) set of key-value pairs.
1. Associated with each key K is a value V. For example, K might be the identifier of a
document. V could be the document itself or it could be the set of nodes at which the
document can be found.
2. If the size of the key-value data is small, we could use a central node that holds the
entire key-value table
3. All nodes would query the central node when they wanted the value V associated with a
given key K
4. A pair of query-response messages would answer any lookup question for any node
5. Alternatively, we could replicate the entire table at each node, so there would be no
messages needed at all
DATABASE TECHNOLOGIES
Distributed Hashing Problem

• The problem becomes more challenging when the key-value table is too large to be
handled by a single node
• Consider this problem, using the following constraints:
1. At any time, only one node among the peers knows the value associated with any
given key K.
2. The key-value pairs are distributed roughly equally among the peers.
3. Any node can ask the peers for the value V associated with a chosen key K. The value
of V should be obtained in a way such that the number of messages sent among the
peers grows much more slowly than the number of peers.
4. The amount of routing information needed at each node to help locate keys must
also grow much more slowly than the number of nodes.
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

If the set of participants in the network is fixed or the set of participants changes slowly, then use
a hash function h that hashes keys into node numbers. We place the key-value pair (K,V ) at the
node h(K).
Chord Circles
1. Arrange the peers in a “chord circle.”
2. Each node knows its predecessor and successor around the circle
3. Nodes also have links to nodes located at an exponentially growing set of distances around
the circle
4. To place a node in the circle, hash its ID i and place it at position h(i). Refer to this node as
Nh(i).
5. The successor of each node is the next higher one, clockwise around the circle.
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

1. The nodes are located around the circle using a hash function h that is capable of
mapping both keys and node ID’s to m-bit numbers, for some m.
2. Suppose m = 6, then there are 64 different locations for nodes around the circle
3. Key-value pairs are also distributed around the circle using the hash function h.
4. If (K,V ) is a key-value pair, then we compute h(K) and place (K,V ) at the lowest
numbered node Nj such that h(K) ≤ j.
1. For example, any (K,V ) pair such that 42 < h(K) ≤ 48 would be stored at N48. If h(K) is any
of 57, 58,..., 63, 0, 1, then (K,V ) would be placed at N1.
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

Links in Chord Circles:

• Each node around the circle stores links to its predecessor and successor
• N1 has successor N8 and predecessor N56. These links are sufficient to send
messages around the circle to look up the value associated with any key
• if N8 wants to find the value associated with a key K such that h(K) = 54, it can
send the request forward around the circle until a node Nj is found such that
j ≥ 54; it would be node N56
• Each node has a finger table that gives the first nodes found at distances around the circle
that are a power of two
• Finger table for N8
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

Search using Finger Tables:

• Suppose we are at node Ni and we want to find the key-value pair (K,V ) where h(K) = j
• If (K,V ) exists, it will be at the lowest-numbered node that is at least j.
• Use the finger table and knowledge of successors to find (K,V ), if it exists, using
at most m+1 messages, where m is the number of bits in the hash values
produced by hash function h
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

Search using Finger Tables Example:

1. Suppose N8 wants to find the value V for key K, where h(K) = 54
2. Since the successor of N8 is N14, and 54 is not in the range 9, 10,..., 14,
N8 knows (K,V ) is not at N14.
1. N8 examines its finger table and finds that all the entries are below 54
2. N8 takes the largest, N42 and sends a message to N42 asking it to look for key K and have
the result sent to N8
3. N42 finds that 54 is not in the range 43, 44,..., 48 between N42 and its successor N48.
4. Thus, N42 examines its own finger table, which is:
5. The last node (in the circular sense) that is less than 54 is N51, so N42 sends a message to
N51, asking it to search for (K,V ) on behalf of N8
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

Search using Finger Tables Example:

• N51 finds that 54 is less than its successor, N56. Thus, if (K,V ) exists, it is at N56.
• N51 sends a request to N56, asking it to reply to N8.
• In general, this recursive algorithm sends no more than m request messages
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

Adding New Nodes:

• Suppose a new node Ni (a node whose ID hashes to i) wants to join the network of peers
• If Ni does not know how to communicate with any peer, it is not possible for Ni to join
• If Ni knows even one peer, Ni can ask that peer what node would be Ni’s successor around
the circle
• The node at which this hypothetical key would reside is the successor of Ni. Suppose that
the successor of Ni is Nj
1. Change predecessor and successor links, so Ni is properly linked into the circle.
2. Rearrange data so Ni gets all the data at Nj that belongs to Ni, that is, key-value pairs
whose key hashes to something i or less.
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

Adding New Nodes Example:

• Suppose we need to add a node N26 i.e. a node whose ID hashes to 26
• Whatever peer N26 contacted will be told that N26’s successor is N32.
• N26 sets its successor to N32 and its predecessor to nil
• The predecessor of N32 remains N21
• N21 runs the stabilization
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

When a Peer Leaves the Network:

• A node can leave the circle at any time. The simple case is when a node leaves “gracefully”
• To leave gracefully, a node:
1. Notifies its predecessor and successor that it is leaving, so they can become each
other’s predecessor and successor.
2. Transfers its data to its successor.
DATABASE TECHNOLOGIES
Centralized Solutions for Distributed Hashing Problem

When a Peer Fails:

1. To avoid total unavailability of data, we can replicate it at several nodes. For example,
place each (K,V ) pair at three nodes: the correct node, its predecessor in the circle, and
its successor.
2. An alternative approach is to cluster nodes into groups of (say) three or more. Nodes in
a cluster replicate their data and can substitute for one another, if one leaves or fails
3. When clusters get too large, they can be split into two clusters that are adjacent on the
circle
4. Clusters that get too small can be combined with a neighbor
THANK YOU
Department of Computer Science
and Engineering

100