Postgresql For Data Architects Discover How To Design Develop and Maintain Your Database Application Effectively With Postgresql Maymala
Postgresql For Data Architects Discover How To Design Develop and Maintain Your Database Application Effectively With Postgresql Maymala
com
https://textbookfull.com/product/learning-postgresql-10-a-beginner-s-
guide-to-building-high-performance-postgresql-database-solutions-juba/
textbookfull.com
https://textbookfull.com/product/new-technologies-for-glutamate-
interaction-neurons-and-glia-1st-edition-maria-kukley/
textbookfull.com
A business framework for international commercialization
of innovative construction products First Edition Ali
Albassami
https://textbookfull.com/product/a-business-framework-for-
international-commercialization-of-innovative-construction-products-
first-edition-ali-albassami/
textbookfull.com
https://textbookfull.com/product/mapping-social-memory-a-
psychotherapeutic-psychosocial-approach-1st-edition-nigel-williams/
textbookfull.com
https://textbookfull.com/product/beautiful-beadwork-from-nature-
melissa-shippee/
textbookfull.com
https://textbookfull.com/product/clinical-neuroanatomy-stephen-g-
waxman/
textbookfull.com
https://textbookfull.com/product/the-handbook-of-formal-methods-in-
human-computer-interaction-benjamin-weyers/
textbookfull.com
Souvenir Nation Relics Keepsakes and Curios from the
Smithsonian s National Museum of American History 1st
Edition William L. Bird Jr.
https://textbookfull.com/product/souvenir-nation-relics-keepsakes-and-
curios-from-the-smithsonian-s-national-museum-of-american-history-1st-
edition-william-l-bird-jr/
textbookfull.com
PostgreSQL for Data Architects
Jayadevan Maymala
BIRMINGHAM - MUMBAI
PostgreSQL for Data Architects
All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the author, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
ISBN 978-1-78328-860-1
www.packtpub.com
Reviewers Proofreaders
Pascal Charest Maria Gould
Bahman Movaqar Clyde Jenkins
Ângelo Marcos Rigo Chris Smith
Hans-Jürgen Schönig Jonathan Todd
Stéphane Wirtel
Indexer
Commissioning Editor Hemangini Bari
Anthony Albuquerque
Graphics
Acquisition Editor Sheetal Aute
Sonali Vernekar Abhinash Sahu
Copy Editor
Relin Hedly
About the Author
When he is not working on open source technologies, he spends time reading and
updating himself on economic and political issues.
I'd like to thank my lovely wife, Nahid, who has taught me how to
be strong.
Ângelo Marcos Rigo has a strong background in web development since
1998, focusing on content management systems. For the past 7 years, he has been
managing, customizing, and developing extensions for Moodle LMS. He can be
reached at his website http://www.u4w.com.br/novosite/index.php for CMS or
Moodle LMS consulting. He has reviewed Moodle Security, Packt Publishing.
Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.PacktPub.
com and as a print book customer, you are entitled to a discount on the eBook copy.
Get in touch with us at [email protected] for more details.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital
book library. Here, you can search, access, and read Packt's entire library of books.
Why subscribe?
• Fully searchable across every book published by Packt
• Copy and paste, print, and bookmark content
• On demand and accessible via a web browser
[i]
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
Table of Contents
[ ii ]
Table of Contents
[ iii ]
Table of Contents
[ iv ]
Table of Contents
[v]
Preface
PostgreSQL is an incredibly flexible and dependable open source relational database.
Harnessing its power will make your applications more reliable and extensible
without increasing costs. Using PostgreSQL's advanced features will save you work
and increase performance, once you've discovered how to set it up.
PostgreSQL for Data Architects will teach you everything you need to learn in order to
get a scalable and optimized PostgreSQL server up and running.
The book starts with basic concepts (such as installing PostgreSQL from source) and
covers theoretical aspects (such as concurrency and transaction management). After
this, you'll learn how to set up replication, use load balancing to scale horizontally,
and troubleshoot errors.
As you continue through this book, you will see the significant impact of
configuration parameters on performance, scalability, and transaction management.
Finally, you will get acquainted with useful tools available in the PostgreSQL
ecosystem used to analyze PostgreSQL logs, set up load balancing, and recovery.
Chapter 2, Server Architecture, covers the important processes started when we start a
PostgreSQL cluster and how they work along with the memory structures to provide
the functionality expected from a database management system.
[ vii ]
Preface
Chapter 3, PostgreSQL – Object Hierarchy and Roles, explains various object types and
objects provided by PostgreSQL. Important concepts such as databases, clusters,
tablespaces, and schemas are covered in this chapter.
Chapter 5, Data Modeling with SQL Power Architect, talks about how we can model
tables and relationships with SQL Power Architect. Some of the aspects that should
be considered when we choose a design tool are also covered in this chapter.
Chapter 6, Client Tools, covers two clients tools (pgAdmin: a UI tool and psql: a
command-line tool). Browsing database objects, generating queries, and generating
the execution plan for queries using pgAdmin are covered. Setting up the
environment variables for connecting from psql, viewing history of SQL commands
executed, and meta-commands are also covered in this chapter.
Chapter 7, SQL Tuning, explains query optimization techniques. To set the context,
some patterns about database use and theory on how the PostgreSQL optimizer
works are covered.
Chapter 8, Server Tuning, covers PostgreSQL server settings that have significant
impact on query performance. These include memory settings, cost settings,
and so on. Two object types: partitions and materialized views are also
covered in this chapter.
Chapter 9, Tools to Move Data in and out of PostgreSQL, covers common tools/utilities,
such as pg_dump, pg_bulkload, and copy used to move data in and out
of PostgreSQL.
Chapter 10, Scaling, Replication, and Backup and Recovery, covers methods that
are usually used for achievability. A step-by-step method to achieve horizontal
scalability using PostgreSQL's streaming replication and pgpool-II is also presented.
Point-in-time recovery for PostgreSQL is also covered in this chapter.
Chapter 12, PostgreSQL – Extras, covers quite a few topics. Some interesting data types
that every data architect should be aware of, a couple of really useful extensions, and
a tool to analyze PostgreSQL log files are covered. It also covers a few interesting
features available in PostgreSQL 9.4.
[ viii ]
Preface
Conventions
In this book, you will find a number of text styles that distinguish among different
kinds of information. Here are some examples of these styles and an explanation of
their meaning.
Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows:
"We will use the following wget command to download the source."
A block of code/SQL at psql prompt as well as the output from the server at psql is
set as follows:
CREATE TABLE emp(id serial, first_name varchar(50));
[ ix ]
Preface
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about
this book—what you liked or disliked. Reader feedback is important for us as it helps
us develop titles that you will really get the most out of.
If there is a topic that you have expertise in and you are interested in either writing
or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to
help you to get the most from your purchase.
[x]
Preface
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you find a mistake in one of our books—maybe a mistake in the text or
the code—we would be grateful if you could report this to us. By doing so, you can
save other readers from frustration and help us improve subsequent versions of this
book. If you find any errata, please report them by visiting http://www.packtpub.
com/submit-errata, selecting your book, clicking on the Errata Submission Form
link, and entering the details of your errata. Once your errata are verified, your
submission will be accepted and the errata will be uploaded to our website or added
to any list of existing errata under the Errata section of that title.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all
media. At Packt, we take the protection of our copyright and licenses very seriously.
If you come across any illegal copies of our works in any form on the Internet, please
provide us with the location address or website name immediately so that we can
pursue a remedy.
We appreciate your help in protecting our authors and our ability to bring you
valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at
[email protected], and we will do our best to address the problem.
[ xi ]
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
Installing PostgreSQL
This chapter gives you an overview of the process to install PostgreSQL from the
source. The system used for installation and providing examples in the following
sections is a 64-bit CentOS (6.4) machine. Other Unix/Linux systems typically have
similar commands. For those using Windows systems, there is a set of utilities
available at http://sourceforge.net/projects/unxutils/, which makes it
possible to execute most of the Unix commands (find, grep, cut, and so on) in the
Windows environment. The steps to be followed to install PostgreSQL on Windows
are very different compared to those for Unix/Linux systems and are not covered in
this chapter.
Installation options
There are many possible ways to install PostgreSQL on a system. For Windows,
downloading the Graphical Installer and using this is the easy way. For Linux systems
such as Red Hat Enterprise Linux or CentOS, we could either use Yellow dog Updater
Modified (yum) or Red Hat Package Manager or RPM Package Manager (rpm)
commands to install PostgreSQL. For Ubuntu, PostgreSQL can be installed using the
apt-get command, which in turn works with Ubuntu's Advanced Packaging Tool
(APT). While these options work, we do not get to see what is happening when we
execute these commands, except, of course, that the database gets installed.
Then there are situations where we might want to build from the source.
Assume that all we have is one production server and one development or staging
server. We are on version 9.3. Version 9.4 is about to be released and there are
quite a few interesting features in 9.4 that we want to try out. If we want to install 9.4
in the test server and use it alongside 9.3, without the installations stepping on each
other's toes, compiling from the source with the --prefix= option and specifying
different installation directories is the right approach. We could also set different
default ports. It's also possible that the new version (source) is ready, but the
package for our Linux distribution is not ready yet.
[1]
Installing PostgreSQL
We might use a flavor of Linux for which an installation package is not available at
all. Installation from source is the way forward in these situations. One advantage
with installing from the source is that we don't have to worry too much about
which package to download, the version of operating system (CentOS 6.3 or 6.4?),
architecture (32 bit or 64 bit), and so on. These are more or less irrelevant. Of course,
we should be using an operating system/architecture that is supported by the
database, but that's about it! We also need to download and install all the tools and
utilities necessary to compile and make the software, in this case, PostgreSQL.
We can see a number of versions all the way down to version 1 when it was called
Postgres95 and up to the latest production and beta versions. If you belong to the
group who believe that one shouldn't try software that is not at least a few months
old, so that its teething issues are resolved, you should opt for the last-but-one
version. It's a good idea to opt for the latest stable version. The latest versions have
added quite a few very useful features, such as materialized views and an improved
set of JSON functions and operators.
Executing this command will give us a window that looks like this:
[2]
Chapter 1
As we can see, the tarred and gzipped source code comes to about 21 MB. As an
aside, the installation files of Oracle—the big RDBMS out here—weighs over 2.2 GB.
The tar command is used to create or extract TapeARchive files. In the preceding
command, the x option is used to extract, v for verbose is used so that we can see
the list of files and folders getting extracted, and the f option is for, well, passing
the name of the file, which will undergo the extraction process. We might need to
provide the z option, so the command will be tar -xzvf if the preceding code in the
tar command does not work. Some versions of tar are intelligent enough to figure
out whether it is a gzipped file or not and will unzip it automatically. The untarred
unzipped files come to around 115 MB.
The find command searches for files meeting specific criteria. Here, we instructed
find to limit itself to scanning just one level of subdirectories using maxdepth 1. We
used the type option along with d to tell find that we need files of type directory, as
shown in the following screenshot:
• src: This directory has most of the core code, namely, code for the backend
processes, optimizer, storage, client utilities (such as psql) and code to
take care of replication, and so on. It also contains the makefiles for various
distributions. For example, we have the files Makefile.hpux, Makefile.
linux, Makefile.openbsd, and Makefile.sco under src/makefile.
[3]
Installing PostgreSQL
• doc: This directory has the source for documentation written in DocBook,
DocBook being an application of Standard Generalized Markup Language
(SGML). It's possible to generate documentation in an HTML format, PDF
format, and a few other formats.
• contrib: This directory is where many extensions are available. These
are add-on modules that do not form part of the core installation, but can
be installed as needed. For example, those who have to connect to other
PostgreSQL databases can install the Foreign Data Wrapper extension:
postgres_fdw. For those who want to access the contents of a file on the
server from a table, there is the file_fdw extension.
• config: This directory contains a few macros that help you configure and
compile the package.
Now let's move on to the dependencies, configuration options, and the actual
installation itself.
A compiler is also necessary. GNU Compiler Collection (GCC) is one such toolset
that is included in almost all the Unix systems. The gcc -v command will provide
you with the version of gcc as well as options with which it was configured on the
system, as shown in the following screenshot:
[4]
Random documents with unrelated
content Scribd suggests to you:
master in his rounds from morning till night—informed me that there
was a revolution in Petersburg and that cannon were firing in the
capital.
On the evening of the next day, Count Komarovsky, a high officer
of the police, was at our house, and told us of the band of
revolutionaries in the Cathedral Square, the cavalry charge, and the
death of Milorádovitch.[26]
26. When Nicholas became Emperor in place of his brother Constantine, the
revolt of the Decembrists took place in Petersburg on December 14, 1825.
Five of the conspirators were afterwards hanged, and over a hundred
banished to Siberia.
31. Elizabeth, daughter of Peter the Great, reigned from 1741 to 1762. Probably
il refers to her father.
It is easy to imagine the harmonious trio that made up this
household: a card-playing father, passionately devoted to horses and
racing and noisy carouses in disreputable company; a daughter
brought up in complete independence and accustomed to do as she
pleased in the house; and a middle-aged blue-stocking suddenly
converted into a bride. As a matter of course, no love was lost
between the stepmother and stepdaughter. In general, real
friendship between a woman of thirty-five and a girl of seventeen is
impossible, unless the former is sufficiently unselfish to renounce all
claim to sex.
The common hostility between stepmothers and step-daughters
does not surprise me in the least: it is natural and even moral. A
new member of the household, who usurps their mother’s place,
provokes repulsion on the part of the children. To them the second
marriage is a second funeral. The child’s love is revealed in this
feeling, and whispers to the orphan, “Your father’s wife is not your
mother.” At one time the Church understood that a second marriage
is inconsistent with the Christian conception of marriage and the
Christian dogma of immortality; but she made constant concessions
to the world, and went too far, till she came up against the logic of
facts—the simple heart of the child who revolts against the absurdity
and refuses the name of mother to his father’s second choice.
The woman too is in an awkward situation when she comes away
from the altar to find a family of children ready-made: she has
nothing to do with them, and has to force feelings which she cannot
possess; she is bound to convince herself and the world, that other
people’s children are just as attractive to her as her own.
Consequently, I don’t blame either the convent-lady or my cousin
for their mutual dislike; but I understand how a young girl
unaccustomed to control was eager to go wherever she could be
free. Her father was now getting old and more submissive to his
learned wife; her brother, the officer, was behaving worse and
worse; in fact, the atmosphere at home was oppressive, and she
finally induced her stepmother to let her go on a visit to us, for some
months or possibly for a year.
§7
The day after her arrival, my cousin turned my usual routine, with
the exception of my lessons, upside down. With a high hand she
fixed hours for us to read together, advised me to stop reading
novels, and recommended Ségur’s General History and The Travels
of Anacharsis.[32] From the ascetic point of view she opposed my
strong inclination to smoke on the sly—cigarettes were then
unknown, and I rolled the tobacco in paper myself: in general, she
liked to preach to me, and I listened meekly to her sermons, if I did
not profit by them. Fortunately, she was not consistent: quite
forgetting her own arrangements, she read with me for amusement
rather than instruction, and often sent out a secret messenger in the
shape of a pantry-boy to buy buckwheat cakes in winter or
gooseberries in summer.
32. Voyage du jeune Anacharsis, by the Abbé Barthélemy, published in 1779.
Ségur was a French historian (1753-1830).
S
OME time in the year 1824 I was walking one day with my
father along the Moscow River, on the far side of the Sparrow
Hills; and there we met a French tutor whom we knew. He had
nothing on but his shirt, was obviously in great alarm, and was
calling out, “Help! Help!” Before our friend had time to pull off his
shirt or pull on his trousers, a Cossack ran down from the Sparrow
Hills, hurled himself into the water, and disappeared. In another
moment he reappeared, grasping a miserable little object, whose
head and hands shook like clothes hung out to dry; he placed this
burden on the bank and said, “A shaking will soon bring him round.”
The bystanders collected fifty roubles for the rescuer. The Cossack
made no pretences but said very honestly, “It’s a sin to take money
for a thing like that; for he gave me no trouble, no more than a cat,
to pull him out. But,” he added, “though I don’t ask for money, if I’m
offered it, I may as well take it. I’m a poor man. So thank you
kindly.” Then he tied up the money in his handkerchief and went
back to his horses grazing on the hill.
My father asked the man’s name and wrote next day to tell his
commanding officer of his gallantry; and the Cossack was promoted
to be a corporal. A few months later the Cossack appeared at our
house and brought a companion, a German with a fair curling wig,
pock-marked, and scented. This was the drowning man, who had
come to return thanks on behalf of the Cossack; and he visited us
afterwards from time to time.
Karl Sonnenberg had taught boys German in several families, and
was now employed by a distant relation of my father’s, who had
confided to him the bodily health and German pronunciation of his
son. This boy, Nikolai Ogaryóv, whom Sonnenberg always called
Niko, attracted me. There was something kind, gentle, and
thoughtful about him; he was quite unlike the other boys whom I
was in the way of seeing. Yet our intimacy ripened slowly: he was
silent and thoughtful, I was lively and feared to trouble him by my
liveliness.
Niko had lost his mother in infancy, and his grandmother died
about the time when my cousin Tatyana left us and went home.
Their household was in confusion, and Sonnenberg, who had really
nothing to do, made out that he was terribly busy; so he brought the
boy to our house in the morning and asked if we would keep him for
the whole day. Niko was frightened and sad; I suppose he loved his
grandmother.
After sitting together for some time, I proposed that we should
read Schiller. I was soon astonished by the similarity of our tastes:
he knew by heart much more than I did, and my favourite passages
were those he knew best; we soon shut the book, and each began
to explore the other’s mind for common interests.
He too was familiar with the unprinted poems of Púshkin and
Ryléev;[33] the difference from the empty-headed boys whom I
sometimes met was surprising. His heart beat to the same tune as
mine; he too had cut the painter that bound him to the sullen old
shore of conservatism; our business was to push off with a will; and
we decided, perhaps on that very first day, to act in support of the
Crown Prince Constantine!
33. One of the five Decembrists who were hanged when the revolt was
suppressed.
Panting and flushed, we stood there and wiped our brows. The
sun was setting, the cupolas of Moscow glittered in his rays, the city
at the foot of the hill spread beyond our vision, a fresh breeze
fanned our cheeks. We stood there leaning against each other; then