Algorithms for Convex Optimization 1st Edition

Vishnoi pdf download

https://ebookmeta.com/product/algorithms-for-convex-
optimization-1st-edition-vishnoi/

Download more ebook from https://ebookmeta.com

We believe these products will be a great fit for you. Click
the link to download now, or visit ebookmeta.com
to discover even more!

Lectures on Modern Convex Optimization 2020-2023:

Analysis, Algorithms, Engineering Applications Aharon
Ben-Tal

https://ebookmeta.com/product/lectures-on-modern-convex-
optimization-2020-2023-analysis-algorithms-engineering-
applications-aharon-ben-tal/

Introduction to Online Convex Optimization 2nd Edition

Elad Hazan

https://ebookmeta.com/product/introduction-to-online-convex-
optimization-2nd-edition-elad-hazan/

The Projected Subgradient Algorithm in Convex

Optimization Alexander J. Zaslavski

https://ebookmeta.com/product/the-projected-subgradient-
algorithm-in-convex-optimization-alexander-j-zaslavski/

Alexander Dallas Bache Scientist and Educator 186 1867

Merle M. Odgers

https://ebookmeta.com/product/alexander-dallas-bache-scientist-
and-educator-186-1867-merle-m-odgers/
From Start-Up to Grown-Up - Grow Your Leadership to
Grow Your Business 1st Edition Alisa Cohn

https://ebookmeta.com/product/from-start-up-to-grown-up-grow-
your-leadership-to-grow-your-business-1st-edition-alisa-cohn/

Engineering Natural Product Biosynthesis Elizabeth

Skellam

https://ebookmeta.com/product/engineering-natural-product-
biosynthesis-elizabeth-skellam/

The Gender of Things How Epistemic and Technological

Objects Become Gendered 1st Edition Maria Rentetzi

https://ebookmeta.com/product/the-gender-of-things-how-epistemic-
and-technological-objects-become-gendered-1st-edition-maria-
rentetzi/

Why Kids Love and Hate School Reflections on Difference

1st Edition Steven P. Jones

https://ebookmeta.com/product/why-kids-love-and-hate-school-
reflections-on-difference-1st-edition-steven-p-jones/

Empire of Knowledge Alexander Vucinich

https://ebookmeta.com/product/empire-of-knowledge-alexander-
vucinich/
Routledge Handbook of Sport History 1st Edition Murray
G. Phillips (Editor)

https://ebookmeta.com/product/routledge-handbook-of-sport-
history-1st-edition-murray-g-phillips-editor/
 Algorithms for Convex Optimization

In the last few years, algorithms for convex optimization have

revolutionized algorithm design, both for discrete and continuous
optimization problems. For problems such as maximum flow,
maximum matching, and submodular function minimization, the
fastest algorithms involve essential methods such as gradient
descent, mirror descent, interior point methods, and ellipsoid
methods. The goal of this self-contained book is to enable
researchers and professionals in computer science, operations
research, data science, and machine learning to gain an in-depth
understanding of these algorithms. The text emphasizes how to
derive key algorithms for convex optimization from first principles
and how to establish precise running time bounds. This modern text
explains the success of these algorithms in problems of discrete
optimization, as well as how these methods have significantly
pushed the state of the art of convex optimization itself.
nisheeth k. vishnoi is A. Bartlett Giamatti Professor of Computer
Science at Yale University. His research areas include theoretical
computer science, optimization, and machine learning. He is a
recipient of the Best Paper Award at IEEE FOCS in 2005, the IBM
Research Pat Goldberg Memorial Award in 2006, the Indian National
Science Academy Young Scientist Award in 2011, and the Best Paper
Award at ACM FAccT in 2019. He was elected an ACM Fellow in
2019. He obtained a bachelor’s degree in computer science and
engineering from IIT Bombay and a PhD in algorithms,
combinatorics, and optimization from Georgia Institute of
Technology.
Algorithms for Convex Optimization
NISHEETH K. VISHNOI
Yale University
University Printing House, Cambridge CB2 8BS, United Kingdom
One Liberty Plaza, 20th Floor, New York, NY 10006, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
314–321, 3rd Floor, Plot 3, Splendor Forum, Jasola District Centre, New Delhi –
110025, India
103 Penang Road, #05–06/07, Visioncrest Commercial, Singapore 238467
Cambridge University Press is part of the University of Cambridge.
It furthers the University’s mission by disseminating knowledge in the pursuit of
education, learning, and research at the highest international levels of excellence.
www.cambridge.org
Information on this title: www.cambridge.org/9781108482028
DOI: 10.1017/9781108699211
© Nisheeth K. Vishnoi 2021
This publication is in copyright. Subject to statutory exception and to the
provisions of relevant collective licensing agreements, no reproduction of any part
may take place without the written permission of Cambridge University Press.
First published 2021
A catalogue record for this publication is available from the British Library.
Library of Congress Cataloging-in-Publication Data
Names: Vishnoi, Nisheeth K., 1976- author.
Title: Algorithms for convex optimization / Nisheeth K. Vishnoi. Description: New
York : Cambridge University Press, [2021] | Includes bibliographical references and
index.
Identifiers: LCCN 2020052071 (print) | LCCN 2020052072 (ebook) | ISBN
9781108482028 (hardback) | ISBN 9781108741774 (paperback) | ISBN
9781108699211 (epub)
Subjects: LCSH: Mathematical optimization. | Convex functions. | Convex
programming.
Classification: LCC QA402.5 .V57 2021 (print) | LCC QA402.5 (ebook) | DDC
515/.882-dc23
LC record available at https://lccn.loc.gov/2020052071
LC record available at https://lccn.loc.gov/2020052072
ISBN 978-1-108-48202-8 Hardback
ISBN 978-1-108-74177-4 Paperback
Cambridge University Press has no responsibility for the persistence or accuracy of
URLs for external or third-party internet websites referred to in this publication
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.
Dedicated to Maya and Vayu
 Contents

Preface
Acknowledgments
Notation

1 Bridging Continuous and Discrete

Optimization
1.1 An Example: The Maximum Flow Problem
1.2 Linear Programming
1.3 Fast and Exact Algorithms via Interior Point Methods
1.4 Ellipsoid Method beyond Succinct Linear Programs

2 Preliminaries
2.1 Derivatives, Gradients, and Hessians
2.2 Fundamental Theorem of Calculus
2.3 Taylor Approximation
2.4 Linear Algebra, Matrices, and Eigenvalues
2.5 The Cauchy-Schwarz Inequality
2.6 Norms
2.7 Euclidean Topology
2.8 Dynamical Systems
2.9 Graphs
2.10 Exercises

3 Convexity
 3.1 Convex Sets
3.2 Convex Functions
3.3 The Usefulness of Convexity
3.4 Exercises

4 Convex Optimization and Efficiency

4.1 Convex Programs
4.2 Computational Models
4.3 Membership Problem for Convex Sets
4.4 Solution Concepts for Optimization Problems
4.5 The Notion of Polynomial Time for Convex
Optimization
4.6 Exercises

5 Duality and Optimality

5.1 Lagrangian Duality
5.2 The Conjugate Function
5.3 KKT Optimality Conditions
5.4 Proof of Strong Duality under Slater’s Condition
5.5 Exercises

6 Gradient Descent
6.1 The Setup
6.2 Gradient Descent
6.3 Analysis When the Gradient Is Lipschitz Continuous
6.4 Application: The Maximum Flow Problem
6.5 Exercises

7 Mirror Descent and the Multiplicative

Weights Update
7.1 Beyond the Lipschitz Gradient Condition
 7.2 A Local Optimization Principle and Regularizers
7.3 Exponential Gradient Descent
7.4 Mirror Descent
7.5 Multiplicative Weights Update
7.6 Application: Perfect Matching in Bipartite Graphs
7.7 Exercises

8 Accelerated Gradient Descent

8.1 The Setup
8.2 Main Result on Accelerated Gradient Descent
8.3 Proof Strategy: Estimate Sequences
8.4 Construction of an Estimate Sequence
8.5 The Algorithm and Its Analysis
8.6 An Algorithm for Strongly Convex and Smooth
Functions
8.7 Application: Linear System of Equations
8.8 Exercises

9 Newton’s Method
9.1 Finding a Root of a Univariate Function
9.2 Newton’s Method for Multivariate Functions
9.3 Newton’s Method for Unconstrained Optimization
9.4 First Take on the Analysis
9.5 Newton’s Method as Steepest Descent
9.6 Analysis Based on a Local Norm
9.7 Analysis Based on the Euclidean Norm
9.8 Exercises

10 An Interior Point Method for Linear

Programming
10.1 Linear Programming
 10.2 Constrained Optimization via Barrier Functions
10.3 The Logarithmic Barrier Function
10.4 The Central Path
10.5 A Path-Following Algorithm for Linear Programming
10.6 Analysis of the Path-Following Algorithm
10.7 Exercises

11 Variants of Interior Point Method and

Self-Concordance
11.1 The Minimum Cost Flow Problem
11.2 An IPM for Linear Programming in Standard Form
11.3 Application: The Minimum Cost Flow Problem
11.4 Self-Concordant Barriers
11.5 Linear Programming Using Self-Concordant Barriers
11.6 Semidefinite Programming Using Self-Concordant
Barriers
11.7 Convex Optimization Using Self-Concordant Barriers
11.8 Exercises

12 Ellipsoid Method for Linear

Programming
12.1 0-1-Polytopes with Exponentially Many Constraints
12.2 Cutting Plane Methods
12.3 Ellipsoid Method
12.4 Analysis of Volume Drop and Efficiency for Ellipsoids
12.5 Application: Linear Optimization over 0-1-Polytopes
12.6 Exercises

13 Ellipsoid Method for Convex

Optimization
13.1 Convex Optimization Using the Ellipsoid Method?
13.2 Application: Submodular Function Minimization
13.3 Application: The Maximum Entropy Problem
13.4 Convex Optimization Using the Ellipsoid Method
13.5 Variants of Cutting Plane Method
13.6 Exercises

Bibliography
Index
 Preface

Convex optimization studies the problem of minimizing a convex

function over a convex set. Convexity, along with its numerous
implications, has been used to come up with efficient algorithms for
many classes of convex programs. Consequently, convex
optimization has broadly impacted several disciplines of science and
engineering.
In the last few years, algorithms for convex optimization have
revolutionized algorithm design, both for discrete and continuous
optimization problems. The fastest-known algorithms for problems
such as maximum flow in graphs, maximum matching in bipartite
graphs, and submodular function minimization involve an essential
and nontrivial use of algorithms for convex optimization such as
gradient descent, mirror descent, interior point methods, and cutting
plane methods. Surprisingly, algorithms for convex optimization have
also been used to design counting problems over discrete objects
such as matroids. Simultaneously, algorithms for convex optimization
have become central to many modern machine learning applications.
The demand for algorithms for convex optimization, driven by larger
and increasingly complex input instances, has also significantly
pushed the state of the art of convex optimization itself.
The goal of this book is to enable a reader to gain an in-depth
understanding of algorithms for convex optimization. The emphasis
is to derive key algorithms for convex optimization from first
principles and to establish precise running time bounds in terms of
the input length. Given the broad applicability of these methods, it is
not possible for a single book to show the applications of these
methods to all of them. This book shows applications to fast
algorithms for various discrete optimization and counting problems.
The applications selected in this book serve the purpose of
illustrating a rather surprising bridge between continuous and
discrete optimization.
The structure of the book. The book has roughly four parts.
Chapters 3, 4, and 5 provide an introduction to convexity, models of
computation and notions of efficiency in convex optimization, and
duality. Chapters 6, 7, and 8 introduce first-order methods such as
gradient descent, mirror descent and the multiplicative weights
update method, and accelerated gradient descent, respectively.
Chapters 9, 10, and 11 present Newton’s method and various
interior point methods for linear programming. Chapters 12 and 13
present cutting plane methods such as the ellipsoid method for
linear and general convex programs. Chapter 1 summarizes the book
via a brief history of the interplay between continuous and discrete
optimization: how the search for fast algorithms for discrete
problems is leading to improvements in algorithms for convex
optimization.
Many chapters contain applications ranging from finding maximum
flows, minimum cuts, and perfect matchings in graphs, to linear
optimization over 0-1-polytopes, to submodular function
minimization, to computing maximum entropy distributions over
combinatorial polytopes.
The book is self-contained and starts with a review of calculus,
linear algebra, geometry, dynamical systems, and graph theory in
Chapter 2. Exercises posed in this book not only play an important
role in checking one’s understanding; sometimes important methods
and concepts are introduced and developed entirely through them.
Examples include the Frank-Wolfe method, coordinate descent,
stochastic gradient descent, online convex optimization, the min-max
theorem for zero-sum games, the Winnow algorithm for
classification, bandit optimization, the conjugate gradient method,
primal-dual interior point method, and matrix scaling.
How to use this book. This book can be used either as a textbook
for a stand-alone advanced undergraduate or beginning graduate-
level course, or as a supplement to an introductory course on convex
optimization or algorithm design. The intended audience includes
advanced undergraduate students, graduate students, and
researchers from theoretical computer science, discrete optimization,
operations research, statistics, and machine learning. To make this
book accessible to a broad audience with different backgrounds, the
writing style deliberately emphasizes the intuition, sometimes at the
expense of rigor.
A course for a theoretical computer science or discrete
optimization audience could cover the entire book. A course on
convex optimization can omit the applications to discrete
optimization and can, instead, include applications as per the choice
of the instructor. Finally, an introductory course on convex
optimization for machine learning could include material from
Chapters 2 to 7.
Beyond convex optimization? This book should also prepare the
reader for working in areas beyond convex optimization, e.g.,
nonconvex optimization and geodesic convex optimization, which are
currently in their formative years.
Nonconvex optimization. One property of convex functions is that
a “local” minimum is also a “global” minimum. Thus, algorithms for
convex optimization, essentially, find a local minimum. Interestingly,
this viewpoint has led to convex optimization methods being very
successful for nonconvex optimization problems, especially those
that arise in machine learning. Unlike convex programs, some of
which can be NP-hard to optimize, most interesting classes of
nonconvex optimization problems are NP-hard. Hence, in many of
these applications, we define a suitable notion of local minimum and
look for methods that can take us to one. Thus, algorithms for
convex optimization are important for nonconvex optimization as
well; see the survey by Jain and Kar (2017).
Geodesic convex optimization. Sometimes, a function that is
nonconvex in a Euclidean space turns out to be convex if we
introduce a suitable Riemannian metric on the underlying space and
redefine convexity with respect to the “straight lines” - geodesics -
induced by the metric. Such functions are called geodesically convex
and arise in optimization problems over Riemannian manifolds such
as matrix Lie groups; see the survey by Vishnoi (2018). The theory
of efficient algorithms for geodesic convex optimization is under
construction, and the paper by Bürgisser et al. (2019) presents some
recent progress.
Other documents randomly have
different content
in concert with the Earl of Mortimer, her paramour, murdered by
thrusting a red-hot iron into his bowels. Pedro the Cruel, King of
Castille and Leon in 1350, merited his surname owing to his cruel
treatment of his two brothers, whom he murdered, and his queen,
whom he poisoned. Ivan II., Czar of Russia (reigned 1533 to 1584),
was styled The Terrible on account of the cruelties he inflicted
upon all who offended his autocracy. Frederick I., of Germany
(reigned 1152 to 1190), was surnamed Barbarossa from his red
beard, barba being Latin for beard; while for his bombardment of
Messina in 1848 Ferdinand, King of Naples, was nicknamed Bomba.
Philippe, Duc d’Orleans, the father of Louis Philippe, King of France,
assumed the name of Egalité when he joined the Republican party
in 1789. Of a truth, if “Equality” was what this not unworthy Prince
aspired to, he enjoyed it to the full, for he lost his head under the
guillotine in common with more than twenty thousand of his fellow-
citizens.
 NATIONAL NICKNAMES.
Brother Jonathan, the popular nickname of the United States,
arose out of the person of Jonathan Trumbull, the Governor of
Connecticut, whom General Washington never failed to consult in
cases of emergency. “We must refer the matter to Brother
Jonathan!” he was wont to exclaim when no other officer could offer
any practical suggestion to aid him out of a difficulty; and true
enough, “Brother Jonathan” proved himself in every instance equal
to the confidence reposed in him. Another stock nickname for the
United States is Uncle Sam. This originated from a vulgar
misconception of the initial letters “U. S.” (United States) for those of
the well-known sobriquet of an official whose business it was to
mark them on all Government property. The numerous
acquaintances of this person understood that the goods so marked
had passed through the hands of “Uncle Sam,” and the joke
becoming public it spread far and wide, until in the end it was
considered far too good to be allowed to drop. The term Yankee
finds its origin in the native attempt to pronounce the word
“English,” but approaching no nearer to the sound than Yengees, the
name bestowed upon the English colonists by the Indians of
Massachusetts, and afterwards given to the New Englanders by the
British soldiers during the American War.
The nickname of the typical Englishman, John Bull, was derived
from Dr. Arbuthnot’s satire of this title published in 1721. There was
also a real person of the name of John Bull, well known as the
composer of “God Save the King”; but he died just a hundred years
before Dr. Arbuthnot’s performance was heard of. Of a still later date
is the national English nickname, Mrs. Grundy, which arose out of
the passage, “What will Mrs. Grundy say?” from Thomas Morton’s
drama, “Speed the Plough,” produced in 1798. The proverbial
prudishness of the English people in matters affecting art, could
scarcely be better expressed than under the style of The British
Matron. The British soldier is popularly referred to under the
general designation of Tommy Atkins, because “Thomas Atkins”
was a fictitious name that figured in the soldiers’ monthly statement
of accounts.
The Irish as a nation are invariably alluded to as Pat or Paddy,
being short for Patrick, their most common Christian name, selected
in honour of St. Padhrig, or Patrick (born 373, died 466); the Scots
as Sandie or Sawney, a contraction of Alexander, their most
popular Christian name; and the Welsh as Taffy, a corruption of
Davy, and short for David, the name of their Archbishop and Saint
(born 490, died 554).
The national nickname of the Chinese is John Chinaman, in
imitation of our own characteristic “John Bull.” Even now a Chinaman
addresses every Englishman he meets as “John,” which is his idea of
our most popular name. Hence, British sailors in the Chinese waters
from the first returned the compliment, so to speak, by alluding to
each Celestial with whom they came in contact as “John Chinaman.”
The Chinese are also called Pigtails, on account of their Tartar
tonsure and braided queue. By the Indians of North America
Europeans are styled Pale Faces; while the Europeans designate
the Indians Red Skins, both terms having reference to the
complexion. The word Nigger is a corruption of Negro, derived
from niger, the Latin for black. The reason why a negro generally
bears the name of Sambo is because Zambo is the native term used
to designate the offspring of a black person and a mulatto. The word
Mulatto is Spanish, derived from the Latin mulus, a mule, and
signifying a mixed breed. A Mulatto may be either the offspring of a
negress by a white man, or of a white woman by a negro.
 BIRDS.
The following owe their names to their characteristic note:—the
Cuck-oo, the Pee-wit, the Cur-lew, the Chick-a-dee, and the
Whip-poor-will. The Trumpeter of South America is so called on
account of its loud, clear, and trumpet-like cry. The word
Nightingale is a modern form of the Anglo-Saxon nihtegale,
indicative of a bird that sings by night, agreeably to its component
parts, niht, night, and gale, a songster. The Night-Jar bears its
name because the sound it emits resembles the whirring of a
spinning-wheel. The Mocking-bird possesses the power of
imitating the notes of other birds; while the Humming-bird is
remarkable for the humming sound that proceeds from its wings as
it speeds through the air.
Several birds are named after the colour or some other
characteristic of their plumage. Among these we have the
Greenfinch and the Goldfinch, the term Finch from the Anglo-
Saxon finc, denoting a small singing bird; the Greenlet expressing a
tiny green bird peculiar to South America; the Jay, a corruption of
gai, its French name, alluding to its gay or showy appearance; the
Blue-bird, common in the United States, the upper half of which is
blue; the Blackbird, so called from its sable aspect; the Starling,
owing to the specks at the extremities of its feathers; the
Flamingo, of South America and Africa, from its flaming colour; the
Oriole, an Australian bird of golden-yellow plumage, agreeably to
the Latin aureolus, golden; and the Lyre-bird, also a native of
Australia, so denominated on account of the sixteen feathers of the
tail which when folded form in appearance a perfect lyre. The British
song-bird known as the Red-poll receives its name from the tuft of
red feathers upon its head; whereas the South African Secretary-
bird is so called because a tuft of feathers on each side of its head
are supposed to resemble quill pens stuck behind the ear. The South
American Birds of Paradise are indeed a beautiful species, all the
colours of the rainbow being represented in their plumage; and the
same may be said of the Love-birds, so designated from the
extraordinary affection which they exhibit towards one another. The
Kingfisher is regarded as the king of fisher-birds, or those that dive
for fish as their prey, by reason of his gay plumage.
The Lapwing derives its name from the loud flapping noise made
by its wings during flight; the Wagtail, from the incessant wagging
of its tail; and the Scissors-tail—found only in South America—
from the peculiar nature of its tail, which, like a pair of scissors,
opens and shuts in the course of its rapid passage through the air
and so entraps the flies upon which it preys. The Hangbird is so
called from its habit of suspending its nest from the limb of a tree;
the Weaver-bird, from the wonderful intertwining of twigs and
grass displayed in the construction of its nest; and the Tailor-bird,
from the skill it displays in constructing its nest by stitching together
the leaves of plants.
Among corruptions of the names of birds it will be sufficient to
mention the Widow-bird, properly the Whydaw-bird, after the
territory in Africa of which it is a native; the Martin, from the Latin
murustenco, or wall-swallow, shortened into murten, and
mispronounced marten; and the Muscovy Duck, which, so far from
claiming a Muscovite origin, is merely a musk duck, a species
somewhat larger than our common duck.
The Swift derives its name from its rapid flight; the Passenger-
pigeon, from its migratory habits; the Skylark, from mounting to
the sky and singing as it flies; and the Chaffinch, from its
preference for chaff above every other kind of food. The Diver is
remarkable for its habit of diving; the Sandpiper inhabits the sea-
beach; and the Chimney-swallow builds his nest in an ordinary
house chimney. The Horn-bill, the Boat-bill, the Spoon-bill, and
the Duck-bill are respectively so named on account of the
resemblance of their bills to the articles, and in the last-mentioned
case to the bird, indicated; while the Cross-bill has its mandibles
crossed in opposite directions. The Pouter-pigeon is so called from
the pouting, or bulging out, of its breast; the Ring-dove, from the
white ring around its neck; and the Wryneck, from the curious
manner in which it turns its neck over its shoulder when surprised.
The Woodcock is found in the underwood of a forest, while the
Woodpecker pecks holes in the bark of trees in search for insects.
Chief among the birds which derive their names from the
countries to which they originally belonged are the Guinea-fowl,
brought from Guinea, West Africa; the Brahma-fowl, from the
neighbourhood of the Brahmapootra River in India; the Bantam,
from Bantam in Java; the Barb, from Barbary, and the Turkey,
which, although an American bird, was long believed to have been
imported from European Turkey. Another native of North America
received its name of the Baltimore-bird from the fact that its
colours corresponded with those which occurred in the arms of Lord
Baltimore, the Governor of Maryland, in which State it principally
abounds. The Canary was first brought from the Canary Islands in
1500. The Petrel, a sea-bird usually associated with storms,
expresses the Anglicized form of the Italian petrillo, a diminutive of
Peter, in allusion to St. Peter walking on the sea, and the frequent
appearance of this bird standing as it were on the surface of the
water.
 RELIGIOUS ORDERS.
Strictly speaking, the members of the various Religious Orders, in
this country at least, are not Monks, but Friars. Only those who live
completely isolated from the rest of mankind, as did St. Anthony, are
entitled to the former designation, which, in common with the term
Monastery, comes from the Greek monos, alone. Consequently, a
Religious House is incorrectly described as a Monastery unless each
individual within its walls occupies a separate cell, both by night and
by day, and never suffers himself to have the least communion with
his neighbour. Failing compliance with such a rule, the term
Convent, derived from the Latin con, together, and venire, to come,
is more fittingly applicable. This designation, however, is now borne
by an institution reserved for a community of Nuns, so called from
the Italian nonna, a grandmother, because they originally comprised
only very aged women; albeit it was formerly the custom to speak of
Monasteries and Convents without discrimination. An Abbey always
indicated a Religious House in connection with a Church, as, for
example, Westminster Abbey, the abode of the community attached
to the West Minster, presided over by an Abbot, so styled in
accordance with the Syriac and Latin abba, a father, or, in the case
of a female community, by an Abbess; whereas a Priory denoted a
lesser or branch establishment placed at some distance from the
Abbey, and controlled by a Prior (or Prioress), signifying one who
had a prior claim over the rest to the office of Abbot (or Abbess) in
the original community.
A Friar, on the other hand, is—conformably to the Latin fratre and
the French frère, a brother—what the term implies, viz., one of a
brotherhood. In olden times there existed four distinct and powerful
Orders of Friars. These were the Dominicans, founded by St.
Dominic to preach away the Albigensian heresies, also known as the
Black Friars, on account of their black habits, and in France as the
Jacobins, because their first convent was situated in the Rue St.
Jacques, Paris; the Franciscans, or Grey Friars, named after St.
Francis d’Assissi; the Carmelites, or White Friars of Mount
Carmel; and the Augustines, or Austin Friars, whose origin is
ascribed to St. Augustin or Austin, the first Archbishop of Canterbury,
who died in 605. Eventually a fifth Order, styled the Trinitarians, or
Friars of the Holy Trinity, otherwise the Crutched Friars, so called
from the cross (Latin cruciati, crossed) embroidered on their habit,
came into existence.
Referring to the Franciscans, those who conformed to the austere
rules laid down by their founder were denominated Observant
Friars, while those who, as time wore on, began to live in convents
and coveted lands, chapels, and books, received the name of
Conventional Friars. Out of the Franciscans there have sprung
two lesser Orders, so to speak, chiefly distinguished by a slight
change in the details appertaining to the habit worn by them. These
are the Capuchins, so called from the capuce, or pointed cowl, that
they wear, and the Cordeliers, from the knotted cord which
encircles their waist in place of a girdle. In effect, however, these
two offshoots of the Franciscans are the same, and subject to the
like rules, as the parent institution.
Having disposed of the Friars, let us now turn to the Monks
properly so called. Originally the sole existing order of monks was
that of the Benedictines as established by St. Benedict, who
introduced the monastic system into Western Europe in the year
529. No less than twelve large Monasteries were raised by him
before he died; but notwithstanding the austere rules which
obtained among the Benedictines, these were yet considered too lax
by some individual members of the Order, with the result that first
one and then another “Reformed Order” sprang into existence, the
latest being in each case distinguished for a still more rigorous rule
than that of its immediate predecessor. Thus, we now have the
Carthusians, our English designation for the monks of La
Chartreuse near Grenoble, by whom the celebrated liqueur known as
Chartreuse is prepared; the Cistercians, or monks of Citeau; and
the Cluniacs, or monks of Cluny, respectively named after the
vicinity of their original monastery in France; while the Bernardines
received their title from St. Bernard, who founded the famous
Hospice of Mont St. Bernard in the year 962. From the Carthusians,
also, there have sprung the Basilians founded by St. Basil, and
from the Cistercians, the Trappists, or monks of La Trappe,
originally established in the French district so denominated.
Foremost among the Religious Orders not comprised in any of the
brotherhoods cited above are the Jesuists, properly styled “The
Society of Jesus,” an organization founded upon a military basis by
St. Ignatius Loyola in 1534, which extends its influence all over the
globe. Next in point of importance come the Servites, otherwise
“The Religious Servants of the Holy Virgin,” established by seven
Florentine merchants in 1283; the Passionists, a community of
priests solemnly agreed to preach “Jesus Christ and Him crucified,”
founded by Paul Francis, better known as St. Paul of the Cross, in
1737; and the Redemptorists, or preachers of the Redemption,
also styled the Liguorians, after St. Francis Liguori, who originated
this Order in 1732. Each of these, except, of course, the lay
members of the Jesuists, are professedly Monks; and yet these are
not really Monks, but Friars, because they live in community, and at
times mingle freely with the people. In short, they are Missionary
Friars.
 PAPER AND PRINTING.
The word Paper comes from the Latin papyrus, and Greek
papyros, the designation of an Egyptian plant from whose reeds the
earliest kind of writing material was obtained. Parchment is an
Anglicized form of the French parchemin, from the Greek
pergamenos, named after the ancient city of Pergamos, in Asia
Minor, where the skins of goats were first prepared for writing upon
at a time when Ptolemy prohibited the exportation of the papyrus
from Egypt.
Hand-paper was originally so called from its watermark, which
was that of a hand; Pot-Paper, of a pot; Post-paper, of a post-
horn; Crown-paper, of a crown; and Foolscap, of a fool’s head
with the cap and bells. India-paper formerly came from the Far
East, whereas Nepaul-paper is made in the district of Nepaul,
Northern India. Cap-paper is so designated because, prior to being
used by grocers for wrapping up sugar and other commodities sold
by weight, it is folded into a cap-like form. Among papers of a stiffer
kind, that are chiefly intended for drawing upon, we have Elephant,
so called from its large size (28 inches by 23), Cartridge-paper,
originally manufactured for soldiers’ cartridges, and Bristol-board,
formerly made only at Bristol.
By the term Folio, derived from the Latin folium, a leaf, is meant
a sheet of paper folded but once, thus making two leaves or four
pages; a Quarto (written 4to), is a sheet folded into quarters or
four leaves, making eight pages; an Octavo (8vo), so styled in
accordance with the Latin octa, eight, one folded into eight leaves or
sixteen pages; a Duodecimo (12mo), the Latin for “two and ten,”
one making twelve leaves or twenty-four pages, and so forth.
When Caxton set up the first printing press in this country, in the
year 1476, there were many among the vulgar who regarded it as
an invention of the devil; and the clergy, no doubt, fostered this
idea, foreseeing that in the event of the Bible being distributed to
the masses by this means, the way would be thrown open to the
production of spurious editions of Sacred Writ, and the perversion of
religious doctrine, which up to that period had been free to all who
chose to attend daily instruction in the monasteries. Hence, printing
was popularly described as “The Black Art,” while the boys who took
the sheets off the press, from the fact of generally smearing their
faces with ink, came to be known as Imps or Devils. This will explain
why a printer’s errand-boy still bears the nickname of a Printer’s
Devil.
Our Parliamentary Records, Debates, Reports of Meetings, and
Accounts, have received the name of Hansards because they are
printed by the large printing firm established by Luke Hansard in
1752; whereas a Blue Book is so called on account of its stiff cover
of blue paper. The French Government reports are styled Yellow
Books for a similar reason. The term Book comes from the Danish
bog, a beech-tree, which abounds in Denmark and whose wood is
much used for engraving-blocks. The Leaf of a book is in allusion to
the ancient custom of writing on the bark of trees; while Volume is
derived from the Latin volvo, I roll, relative to the Egyptian papyri,
each one of which when rolled up formed a document or volume
complete in itself. A storehouse for books is styled a Library, in
accordance with the Latin librarium, a book-case, derived in the first
instance from liber, a book.
A Pamphlet owes its description to Pamphila, a Greek lady who
left behind her a kind of commonplace book containing notes,
epitomes, and anecdotes. The French equivalent for a pamphlet is a
Brochure, so called from the verb brocher, to stitch, because such
a book consists only of a few pages stitched together. The word
Chart comes from the Latin charta, a leaf of paper; a chart,
therefore, is not printed on canvas like a map, but on a single sheet
of paper. Map traces its origin from mappa, a Punic word which
signified a signal-cloth, also a napkin, because in ancient times
military and other landmarks were sketched upon a cloth in the
absence of parchment and paper. Nowadays, a book of maps
designed for school use is called an Atlas, after the fabled King of
Mauritania of this name, who was believed by the ancients to
support the world on his shoulders. The figure of Atlas with the
globe first appeared as a frontispiece to “Mercator’s Projections,”
published about the year 1560.
A Cartoon, as we understand the term, is a representation of
political significance, usually coloured and printed on stiff paper. To
some extent this kind of publication owes its origin to the celebrated
‘Cartoons’ of Raffaelle, now in the South Kensington Museum, so
called because they were drawn upon cartone, the Italian for
pasteboard. A Broadside consists of a large sheet of paper having
the matter printed straight across, instead of in columns, so as to
admit of being read at one broad view. The reverse side of the sheet
is left blank. A Poster bears its name from the fact that formerly the
sidewalks of London streets, instead of being paved as now, were
distinguished from the centre, or sedan-chair and riding way, by a
series of posts; and upon these, theatrical and other announcements
were posted. In France, the theatre bills are exhibited upon the
lamp-posts on the Boulevards in a similar manner. In conclusion, the
distinction between Booksellers and Stationers was originally this: the
former were itinerant sellers of books, like hawkers, and pedlars,
whereas the latter had stalls at the corners of streets or in open
market; and as the stationarii, or stationary booksellers, were
enabled to display a more varied stock than the itinerants who
carried books only, such articles as writing-paper, pens, ink, and
other materials in course of time received the name of Stationery.
 POLITICAL NICKNAMES.
The utmost difficulty exists in reconciling the various opinions
expressed by different authors concerning the origin of the terms
Whig and Tory. And yet, if we but consider the reasons why these
nicknames were first bestowed upon the two great political factions
of this country during the reign of Charles II., we may possibly attain
a much-desired end. In the year 1648 (temp. Charles I.) there
occurred a rising, or sally, of the peasantry inhabiting the south-
western districts of Scotland against the Royalists. This was known
as the Whigamore Raid, the term whigamore being applied to the
teamsters and ploughmen of those parts because they used the
twin-syllabic cry of “Whi-gam!” to drive their horses. When,
therefore, in the early days of the Restoration, the ultra-Protestant
party opposed certain measures of the Government, the Catholics
reproached them with favouring the fanatical opinions of the Scottish
Covenanters and Whigamores, and styled them Whigs. In return
the Protestants bestowed upon their opponents the nickname of
Tories, the familiar designation of a band of Irish outlaws who
sought refuge in the bog districts of Ireland. The word Tory, or
rather Toree, is Irish, signifying a robber. From that time down to the
present Toryism has been considered to denote a steadfast
adherence to constitutional principles and the maintenance of royalty
and the peerage, as opposed to the progressive and more liberal
views appertaining to Whiggism, which advocates constitutional
reform and a moderate extension of democratical powers. The word
Liberal was first employed by Lord Byron and his friends as the title
of a periodical intended to set forth the political aims of the
advanced Whig party in 1828. The term Conservative (derived
from the Latin con, together, and servare, to keep, to preserve) first
appeared in an article in The Quarterly Review, January, 1830, and
was permanently adopted by the Tory party on the passing of the
Reform Bill two years afterwards. The still more advanced section of
the Whig party which came into prominence in 1816 were styled
Radicals, or Radical Reformers, from their desire to institute a
thorough reformation in the national policy. In our own day the
Radicals and the Democrats may be set down as one and the same
party; while the Socialists eminently carry out the principles of the
primitive Radicals of the time of Charles I., who styled themselves
Levellers because they strove to reduce society to a common level.
The word Democrat is derived from the Greek demos, people, and
kratein, to govern; therefore denoting one who upholds the principle
of government by the people themselves, and diametrically opposed
to an Aristocracy (Greek, aristos, best, and kratein, to govern), or
government by the bravest and best. These terms were first brought
into notice by the French Revolutionists of 1790.
Adverting to the protracted struggle between the Royalists under
Charles I. and the Parliamentarians under the Cromwellian
Parliament, no two nicknames could have been more suggestive of
their origin than those respectively of the Cavaliers and the
Roundheads. The latter arose out of the Puritan fashion of
cropping the hair close round the head, the former from the cavalier
manner in which a number of gentlemen offered themselves as a
permanent escort to the King after he had been subjected to insult
in December, 1641. The word Cavalier is synonymous with the
French chevalier, a mounted knight, from cheval, a horse, derived
from the Latin caballus, and the Greek kaballes, an inferior horse.
The Protestants in Ireland received the name of Orangemen
owing to their adherence to William III., Prince of Orange, while the
Roman Catholics were styled Jacobites from their adhesion to
James II., Jacobus being the Latin form of the King’s name. The
Peep o’ Day Boys were so called because they broke into the
houses of the people at dawn of day in quest of arms; and the
White Boys, from the white smocks they wore over their clothing.
The depredations of both these insurgents were finally put an end to
by the Insurrection Acts, passed in 1786-7. The secret brotherhood
of the Fenians, organized for the overthrow of the English rule in
Ireland, derived its name from Fiona Mac Cumhal, better known as
Fingal, after whom Fingal’s Cave is designated. The correct
interpretation of the Gaelic word Fenian is “a hunter.” Another secret
society of quite recent origin is that of the Irish Invincibles,
established, as was publicly stated by Carey the Informer, for the
“making of history by killing tyrants.” Their title is due to the boast
that they defy extermination. The Ribbonmen take their name from
the distinctive badge which they wear. Emergency Men are the
more active members of the Irish Defence Association. The
Separists and the Parnellites are one and the same, sworn to
support the measures of Mr. Parnell and the Irish National Party in
promoting Home Rule for Ireland. The now familiar word
Boycotting, in connection with Irish affairs, arose out of the
troubles experienced by Captain Boycott, of Lough Mask Farm, near
Ballinrobe, County Mayo, the land agent of Lord Erne. His house was
besieged, his labourers were threatened, his crops remained
ungathered, and tradesmen refused to supply him with goods. This
occurred on the 11th and 12th of November, 1880, after which the
military was despatched to his aid, and a “Boycott Fund” subscribed
for his benefit. The expression “to boycott” a man practically means
to place him beyond the pale of civilization.
The lowest order of the French Revolutionists were denominated
Sansculottes, literally, “without breeches,” because they rejected
those very serviceable articles of attire as being emblematical of the
aristocracy. The same term was also applied to the Republican
leaders as a reproach for the negligence of their dress; but after a
time they themselves adopted the title with pride. The Red
Republicans were so called for a two-fold reason. In the first place,
they did not hesitate to steep their hands in human blood to
accomplish their political aims; and, secondly, they wore the red cap,
symbolical of Liberty from the days of the Romans downwards. The
two antagonistic parties of the Revolution were styled The
Mountain and The Plain for the reason that the former sat upon
the most elevated benches in the Hall of Assembly, while the latter
occupied the ground floor. The Plain was for the most part composed
of the Girondists, or deputies from the Department of the Gironde.
The Hats and the Caps were the two great political factions in
Sweden, so called on account of the French chapeaux worn by the
partisans of the French interest on the one side, and the Russian
caps worn by the partisans of the Russian interest on the other.
Apropos of Russia, the word Nihilist (derived from the Latin nihil,
nothing), originally denoted a social rather than a political party
opposed to the tyranny of custom. Its significance is well expressed
by Turgeneff, who first introduced it in his novel “Fathers and Sons,”
published in 1862:—“A Nihilist is a man who bows before no
authority, who accepts no principle without examination, no matter
what credit the principle has.” At the present day a Nihilist is a
revolutionary Socialist of the most pronounced degree.
The Italian Carbonari, being the plural of carbonaro, a coal-man,
a charcoal-burner, who first came into notice in 1820, assumed their
designation from the fact of their meetings being originally held in
the huts of the charcoal-burners, and because they held charcoal to
be the symbol of purification. The Black Cloaks were the upper
classes of Naples, distinguished by the colour and quality of their
cloaks from the Lazzari, or beggars. Regarded as a political party,
the Neapolitan Black Cloaks no longer exist; but the Lazzaroni, so
called from the Hospital of St. Lazarus, which serves for their refuge,
are still to be met with in all quarters of the city. Then, again, we
must not omit mention of the Guelphs and the Ghibellines, names
of two powerful families whose rival partisanship of the Papal and
the Imperial supremacy in Italy threatened the peace of Europe
during the long period embraced between the years 1250 and 1500.
The word Federal comes from the Latin fœdus, a league or
compact. A federal form of government is one under which a
number of States, while retaining their individual institutions and
autonomy, unite together for purposes of defence and for a larger
national existence, delegating to a representative national
government certain specified powers. The most noteworthy
examples in history of this form of government are the Achaean
League, the Swiss Republic, and the United States of America. In the
early history of the United States the term “Federal” was applied to
that one of the two great political parties which was supposed to be
more particularly in sympathy with English standards and to favour
an English alliance, and which desired a strong central government.
Their opponents, who preferred a French alliance, and who opposed
a strong central government, were then termed “Republicans.”
About 1830 the “Federals” became Whigs, and in 1856 they
assumed the name of Republicans (from res publica, the State),
the States-rights party having in the meantime taken the name of
“Democrats” (from demos, the people). During the civil war of 1861-
1865 the Northerners were all termed “Federals” (or by their
opponents “Yankees” or “Yanks”), while the Southerners had taken
the name of Confederates, because their Constitution instituted a
weaker central government and favoured the independent action of
the several States.
The Southerners were also given the nickname of “Corn-feds,” in
allusion to the chief article of their diet. The term Yankee above
alluded to dates back to the seventeenth century, and is a
modification of the name “Yengees,” an attempt by the
Massachusetts Indians to pronounce the name “English.”
By the residents of the Northern States, the term is limited to the
inhabitants of the six States of New England. During the civil war of
1861-1865, the members of a political faction in the North received
the name of Copperheads, because they were regarded as secret
foes to the national cause. The allusion was to the poisonous
copperhead serpent, which gives no warning of its approach. The
Know-nothings were a secret political party in the United States
(1848-1860), whose chief aim was the checking of foreign
immigration and the political influence of foreigners by the repeal of
the Naturalization Laws, and the reserving of public appointments
for native-born Americans. The answer given by its members to all
questions about the party organization was, “I don’t know.”
 The Tammany Ring was the name used to designate an
organization among certain officials and their backers in the city of
New York in 1870-1871, who succeeded for a time in enriching
themselves at the expense of the city. The ring was overthrown in
1871, and its leaders imprisoned or banished. The name of the ring
arose from Tammany Hall, the headquarters of a society originally
founded (in 1805) for benevolent purposes, but which had become a
political power, and which is again (1892) controlling the government
of New York.
The term Mugwump first came into political use in 1884. It was
then applied to voters who had been “Republicans,” but who, on
various grounds, preferred the Democratic candidate Cleveland to
the Republican Blaine, and who succeeded in electing the former. It
has since been given generally to citizens, who, while actively
interested in politics, refuse to be bound closely by party ties,
considering causes such as free trade, civil service reform, honest
money, &c., as more important than party labels or party success.
The name “Mugwump” is said to be derived from an Indian word
signifying “wise chief.”
The Chartists constituted an enormous body of the people of this
country who, soon after the passing of the Reform Bill in 1832,
loudly clamoured for “The People’s Charter,” of which the six
principal points were these:—Universal Suffrage, Vote by Ballot,
Annual Parliaments, Payment of the Members, Abolition of the
Property Qualification, and the Equalization of Electorial Rights.
William Lovett, the author of this document, died in August, 1877.
The English war party, who sided with the Turks in the Russo-Turkish
struggle of 1878 received the nickname of Jingoes, or The Jingo
Party, from the chorus of Macdermott’s famous music-hall song,
commencing—
 “We don’t want to fight, but, by Jingo, if we do!”

“Jingo” is a corruption of Jainko, the Basque term for God. Hence

the expression, “By Jingo!” is properly a direct appeal to the Deity. A
Protectionist is one who advocates the protection of home-
produce and manufactures against foreign competition by the
imposition of import duties.
 FLOWERS.
The name of Forget-me-not originated in the following legend:—
A German knight and his lady were walking on the bank of the
Danube, when the fair one saw a beautiful tuft of Myosotis palustris
growing in the water, and expressed a wish to have it. With
chivalrous alacrity the knight at once plunged into the river and
gathered his prize; but before he could regain the steep and slippery
bank, encumbered as he was by his heavy armour, he was drawn by
the treacherous eddy into a deep pool. Finding he could not save
himself, he threw the flowers ashore to his mistress as he sank, and
uttered with his last breath the words “Vergess mein nicht!”
(“Forget-me-not!”) Hence this flower has come to be universally
regarded as the emblem of fidelity.
Mignonette, the diminutive of Mignon, the French for “darling,” is
so called on account of its delicate fragrance. The Carnation owes
its name to the Latin caro, flesh, in relation to its colour. Geranium
comes from the Greek and Latin geranos, a crane; this genus of
plants having a beak-like torus, or receptacle. It is also known as
Crane’s-bill for the same reason. Pansy is an Anglicized form of
the French pensée, “thoughts,” this being the sentiment expressed
by the flower.
The Camellia was named after G. J. Camelli, the German botanist
and missionary (died 1690), by whom it was introduced into Europe
from the East; the Dahlia, after Andrew Dahl, the Swedish botanist,
who discovered it in Mexico in 1784; and the Fuchsia, after Leonard
Fuchs, who brought it from Mexico about the year 1542. The
Victoria Regia was so called because it was introduced into this
country from British Guiana soon after the accession of Queen
Victoria. The Adonis is said to have sprung out of the blood of
Adonis, the beautiful youth who was gored to death by a boar; and
the Hyacinth is supposed to have originated in a similar manner
after Hyacinth had fallen a victim to the jealousy of Zephyr. The
Aspasia bears the name of Aspasia of Miletus, the mistress of
Pericles. The term Orchid comes from the Greek orchis, a testicle,
all plants of this family being distinguished by double testicles. The
Sweetbriar is literally a “fragrant thorn”; the Lilac betrays its
Eastern origin in the Turkish leilak, and Persian lilaj; while the term
Lavender is derived from the Latin lavere, to wash, because the
essential oil obtained from this shrub enters into the composition of
a favourite scent.
The Dog-rose was so called by the Greeks from the belief that
the root of this particular rose-tree was efficacious in curing the bite
of a mad dog. The Damask-rose was brought to England from
Damascus by Dr. Linacre in 1540. The Cabbage-rose is thick and
compact like a cabbage. The Christmas-rose makes its appearance
about Christmas-time. The word Primrose, agreeably to the Latin
prima rosa, signifies the first rose, or flower, of spring.
The Mayflower, otherwise the Hawthorn, the Anglo-Saxon for
“hedge-thorn,” appears in flower in the month of May, while Gilly-
flower is merely a corruption of “July flower.” The Tiger-flower is
streaked like a tiger. Daffodil is a corruption of “d’Asphodele,” the
French name of this flower. Hollyhock is not “Holy Oak,” but the
Anglo-Saxon holi-hoc, or marsh mallow. The Noon-tide, or Noon-
flower, closes its petals at noon; the Convolvulus, so called from
the Latin con, together, and volvere, to roll, does the like at sunset,
in common with the ordinary field Daisy, which owes its name, a
corruption of the Anglo-Saxon doeges-eaye, literally “the day’s eye,”
to this circumstance. The Buttercup was originally so designated in
accordance with an old-established idea that the yellow hue of
butter was attributable to the fact of these flowers being eaten by
cattle. However, as the buttercups are invariably avoided by the
cattle, the proverbial wisdom of our forefathers must for once in a
way be discredited. Cowslip is a corruption of “cows’ leek.”
 The very common supposition that the Sunflower inclines
towards the sun is entirely erroneous, as has been proved by
observation. This flower merely takes its name from its form and
colour. On the other hand, if its etymology be correct, the
Heliotrope does actually turn towards the sun, the word helios
being Greek for sun, and tropos, to turn. The Goldylocks is so
called on account of its tufts of yellow flowers; whereas the
Marigold, which bears yellow flowers, was named in honour of
Queen Mary. Both these, with the Sunflower, belong to the
Chrysanthemum (Greek chrusos, gold, and anthemos, flower)
family. The word Rhododendron, we may add, comes from the
Greek rhodon, rose, and dendron, tree.
The Passion-flower symbolizes in its tints and several parts the
various attributes of Christ’s Passion, as follows:—The white tint,
purity; the blue tint, heaven; the leaf, the spear; the five anthers,
the five wounds; the tendrils, the whips and cords; the column of
the ovary, the pillar of the cross; the stamens, the hammers; the
three styles, the nails; the fleshy thorns within the flowers, the
crown of thorns; the calyx, the nimbus, or glory. In addition to the
foregoing the passion-flower remains open for three days, and this is
supposed to correspond with the three years’ ministry of the
Redeemer.
Lastly, the flower known as the Stock received its name from the
fact that it was principally sold in the old Stocks Market displaced by
the building of the present Mansion House in the year 1737; the
market itself having derived its title from a pair of stocks that stood
there.