Convex Optimization Algorithms 1st Edition Dimitri

P. Bertsekas download pdf

https://ebookultra.com/download/convex-optimization-algorithms-1st-
edition-dimitri-p-bertsekas/

Visit ebookultra.com today to download the complete set of

ebook or textbook!
 We believe these products will be a great fit for you. Click
the link to download now, or visit ebookultra.com
to discover even more!

Nonlinear Programming 3rd Edition Dimitri Bertsekas

https://ebookultra.com/download/nonlinear-programming-3rd-edition-
dimitri-bertsekas/

Optimization Algorithms on Matrix Manifolds P.-A. Absil

https://ebookultra.com/download/optimization-algorithms-on-matrix-
manifolds-p-a-absil/

Convex Optimization Introductory Course 1st Edition

Mikhail Moklyachuk

https://ebookultra.com/download/convex-optimization-introductory-
course-1st-edition-mikhail-moklyachuk/

Linear and Convex Optimization 1st Edition Michael H.

Veatch

https://ebookultra.com/download/linear-and-convex-optimization-1st-
edition-michael-h-veatch/
Nonsmooth Mechanics and Convex Optimization 1st Edition
Yoshihiro Kanno

https://ebookultra.com/download/nonsmooth-mechanics-and-convex-
optimization-1st-edition-yoshihiro-kanno/

Convex optimization and Euclidean distance geometry no

bibliogr Jon Dattorro

https://ebookultra.com/download/convex-optimization-and-euclidean-
distance-geometry-no-bibliogr-jon-dattorro/

Optimization Using Evolutionary Algorithms and

Metaheuristics Applications in Engineering 1st Edition
Kaushik Kumar
https://ebookultra.com/download/optimization-using-evolutionary-
algorithms-and-metaheuristics-applications-in-engineering-1st-edition-
kaushik-kumar/

The Design of Approximation Algorithms 1st Edition David

P. Williamson

https://ebookultra.com/download/the-design-of-approximation-
algorithms-1st-edition-david-p-williamson/

An Introduction to Optimization 2nd Edition Edwin K. P.

Chong

https://ebookultra.com/download/an-introduction-to-optimization-2nd-
edition-edwin-k-p-chong/
Convex Optimization Algorithms 1st Edition Dimitri P.
Bertsekas Digital Instant Download
Author(s): Dimitri P. Bertsekas
ISBN(s): 9781886529281, 1886529280
Edition: 1
File Details: PDF, 18.40 MB
Year: 2015
Language: english
Convex Optimization Algorithms

Dimitri P. Bertsekas
Massachusetts Institute of Technology

WWW site for book information and orders

http://www.athenasc.com

®
Athena Scientific, Belmont, Massachusetts
Athena Scientific
Post Office Box 805
Nashua, NH 03061-0805
U.S.A.

Email: [email protected]
WWW: http://www.athenasc.com

© 2015 Dimitri P. Bertsekas

All rights reserved. No part of this book may be reproduced in any form
by any electronic or mechanical means (including photocopying, recording,
or information storage and retrieval) without permission in writing from
the publisher.

Publisher's Cataloging-in-Publication Data

Bertsekas, Dimitri P.
Convex Optimization Algorithms
Includes bibliographical references and index
1. Nonlinear Programming 2. Mathematical Optimization. I. Title.
T57.8.B475 2015 519.703
Library of Congress Control Number: 2002092168

ISBN-10: 1-886529-28-0, ISBN-13: 978-1-886529-28-1

Contents
1. Convex Optimization Models: An Overview p. 1
1. 1. Lagrange Duality . . . . . . . . . . . p. 2
1.1.1. Separable Problems - Decomposition p. 7
1.1.2. Partitioning . . . . . . . . . . . p. 9
1.2. Fenchel Duality and Conic Programming . p. 10
1.2.1. Linear Conic Problems . . . . p. 15
1.2.2. Second Order Cone Programming p. 17
1.2.3. Semidefinite Programming p. 22
1.3. Additive Cost Problems . . p. 25
1.4. Large Number of Constraints . p. 34
1.5. Exact Penalty Functions p. 39
1.6. Notes, Sources, and Exercises p. 47

2. Optimization Algorithms: An Overview . p. 53

2.1. Iterative Descent Algorithms . . . . . . . . . . . . p. 55
2.1.1. Differentiable Cost Function Descent - Unconstrained
Problems p. 58
2.1.2. Constrained Problems - Feasible Direction Methods p. 71
2.1.3. Nondifferentiable Problems - Subgradient Methods p. 78
2.1.4. Alternative Descent Methods . . . . . . . . p. 80
2.1.5. Incremental Algorithms . . . . . . . . . . p. 83
2.1.6. Distributed Asynchronous Iterative Algorithms p. 104
2.2. Approximation Methods . . . . . . . . . . p. 106
2.2.1. Polyhedral Approximation . . . . . . . . p. 107
2.2.2. Penalty, Augmented Lagrangian, and Interior
Point Methods . . . . . . . . . . . . p. 108
2.2.3. Proximal Algorithm, Bundle Methods, and .
Tikhonov Regularization . . . . . . . . . p. 110
2.2.4. Alternating Direction Method of Multipliers p. 111
2.2.5. Smoothing of Nondifferentiable Problems p. 113
2.3. Notes, Sources, and Exercises . . . . . . . p. 119

3. Subgradient Methods p. 135

3.1. Subgradients of Convex Real-Valued Functions p. 136

iii
 iv Contents

3.1.l. Characterization of the Subdifferential . . p. 146

3.2. Convergence Analysis of Subgradient Methods p. 148
3.3. E-Subgradient Methods . . . . . . . . . . p. 162
3.3.l. Connection with Incremental Subgradient Methods p. 166
3.4. Notes, Sources, and Exercises . . . . . . . . . . . . p. 167

4. Polyhedral Approximation Methods . . . . p. 181

4.1. Outer Linearization - Cutting Plane Methods p. 182
4.2. Inner Linearization - Simplicial Decomposition p. 188
4.3. Duality of Outer and Inner Linearization . p. 194
4.4. Generalized Polyhedral Approximation p. 196
4.5. Generalized Simplicial Decomposition . . p. 209
4.5.l. Differentiable Cost Case . . . . . . p. 213
4.5.2. Nondifferentiable Cost and Side Constraints p. 213
4.6. Polyhedral Approximation for Conic Programming p. 217
4. 7. Notes, Sources, and Exercises . . . . . . . . . . p. 228

5. Proximal Algorithms p. 233

5.1. Basic Theory of Proximal Algorithms p. 234
5.1.l. Convergence . . . . . p. 235
5.1.2. Rate of Convergence . . . . . . p. 239
5.1.3. Gradient Interpretation . . . . p. 246
5.1.4. Fixed Point Interpretation, Overrelaxation,
and Generalization . . . . . . p. 248
5.2. Dual Proximal Algorithms . . . . . . p. 256
5.2.l. Augmented Lagrangian Methods p. 259
5.3. Proximal Algorithms with Linearization p. 268
5.3.l. Proximal Cutting Plane Methods . p. 270
5.3.2. Bundle Methods . . . . . . . . p. 272
5.3.3. Proximal Inner Linearization Methods p. 276
5.4. Alternating Direction Methods of Multipliers p. 280
5.4.l. Applications in Machine Learning . . . p. 286
5.4.2. ADMM Applied to Separable Problems p. 289
5.5. Notes, Sources, and Exercises . p. 293

6. Additional Algorithmic Topics p. 301

6.1. Gradient Projection Methods . . . . . . p. 302
6.2. Gradient Projection with Extrapolation . p. 322
6.2.l. An Algorithm with Optimal Iteration Complexity p. 323
6.2.2. Nondifferentiable Cost - Smoothing . . p. 326
6.3. Proximal Gradient Methods . . . . . . . p. 330
6.4. Incremental Subgradient Proximal Methods p. 340
6.4. l. Convergence for Methods with Cyclic Order p. 344
 Contents V

6.4.2. Convergence for Methods with Randomized Order p. 353

6.4.3. Application in Specially Structured Problems p. 361
6.4.4. Incremental Constraint Projection Methods p. 365
6.5. Coordinate Descent Methods . . . . . . . . . p. 369
6.5.1. Variants of Coordinate Descent . . . . . . p. 373
6.5.2. Distributed Asynchronous Coordinate Descent p. 376
6.6. Generalized Proximal Methods . . . . . . . . . p. 382
6.7. E-Descent and Extended Monotropic Programming p. 396
6. 7.1. E-Subgradients . . . . . . . . . . . . . p. 397
6.7.2. E-Descent Method . . . . . . . . . . . . p. 400
6.7.3. Extended Monotropic Programming Duality p. 406
6.7.4. Special Cases of Strong Duality . . . . . . p. 408
6.8. Interior Point Methods . . . . . . . . . . . p. 412
6.8.1. Primal-Dual Methods for Linear Programming p. 416
6.8.2. Interior Point Methods for Conic Programming p. 423
6.8.3. Central Cutting Plane Methods p. 425
6.9. Notes, Sources, and Exercises . . . . p. 426

Appendix A: Mathematical Background p. 443

A.l. Linear Algebra . . . . p. 445
A.2. Topological Properties p. 450
A.3. Derivatives . . . . . p. 456
A.4. Convergence Theorems p. 458

Appendix B: Convex Optimization Theory: A Summary p. 467

B. l. Basic Concepts of Convex Analysis . . p. 467
B.2. Basic Concepts of Polyhedral Convexity p. 489
B.3. Basic Concepts of Convex Optimization p. 494
B.4. Geometric Duality Framework p. 498
B.5. Duality and Optimization . . . . . . p. 505

References p. 519

Index . . . p. 557
 ATHENA SCIENTIFIC
OPTIMIZATION AND COMPUTATION SERIES

l. Convex Optimization Algorithms, by Dimitri P. Bertsekas, 2015,

ISBN 978-1-886529-28-1, 576 pages
2. Abstract Dynamic Programming, by Dimitri P. Bertsekas, 2013,
ISBN 978-1-886529-42-7, 256 pages
3. Dynamic Programming and Optimal Control, Two-Volume Set,
by Dimitri P. Bertsekas, 2012, ISBN 1-886529-08-6, 1020 pages
4. Convex Optimization Theory, by Dimitri P. Bertsekas, 2009,
ISBN 978-1-886529-31-1, 256 pages
5. Introduction to Probability, 2nd Edition, by Dimitri P. Bertsekas
and John N. Tsitsiklis, 2008, ISBN 978-1-886529-23-6, 544 pages
6. Convex Analysis and Optimization, by Dimitri P. Bertsekas, An-
gelia Nedic, and Asuman E. Ozdaglar, 2003, ISBN 1-886529-45-0,
560 pages
7. Nonlinear Programming, 2nd Edition, by Dimitri P. Bertsekas,
1999, ISBN 1-886529-00-0, 791 pages
8. Network Optimization: Continuous and Discrete Models, by Dim-
itri P. Bertsekas, 1998, ISBN 1-886529-02-7, 608 pages
9. Network Flows and Monotropic Optimization, by R. Tyrrell Rock-
afellar, 1998, ISBN 1-886529-06-X, 634 pages
10. Introduction to Linear Optimization, by Dimitris Bertsimas and
John N. Tsitsiklis, 1997, ISBN 1-886529-19-1, 608 pages
11. Parallel and Distributed Computation: Numerical Methods, by
Dimitri P. Bertsekas and John N. Tsitsiklis, 1997, ISBN 1-886529-
01-9, 718 pages
12. Neuro-Dynamic Programming, by Dimitri P. Bertsekas and John
N. Tsitsiklis, 1996, ISBN 1-886529-10-8, 512 pages
13. Constrained Optimization and Lagrange Multiplier Methods, by
Dimitri P. Bertsekas, 1996, ISBN 1-886529-04-3, 410 pages
14. Stochastic Optimal Control: The Discrete-Time Case, by Dimitri
P. Bertsekas and Steven E. Shreve, 1996, ISBN 1-886529-03-5,
330 pages

vi
 ABOUT THE AUTHOR

Dimitri Bertsekas studied Mechanical and Electrical Engineering at the

National Technical University of Athens, Greece, and obtained his Ph.D.
in system science from the Massachusetts Institute of Technology. He has
held faculty positions with the Engineering-Economic Systems Department,
Stanford University, and the Electrical Engineering Department of the Uni-
versity of Illinois, Urbana. Since 1979 he has been teaching at the Electrical
Engineering and Computer Science Department of the Massachusetts In-
stitute of Technology (M.1.T.), where he is currently the McAfee Professor
of Engineering.
His teaching and research spans several fields, including deterministic
optimization, dynamic programming and stochastic control, large-scale and
distributed computation, and data communication networks. He has au-
thored or coauthored numerous research papers and sixteen books, several
of which are currently used as textbooks in MIT classes, including "Nonlin-
ear Programming," "Dynamic Programming and Optimal Control," "Data
Networks," "Introduction to Probability," "Convex Optimization Theory,"
as well as the present book. He often consults with private industry and
has held editorial positions in several journals.
Professor Bertsekas was awarded the INFORMS 1997 Prize for Re-
search Excellence in the Interface Between Operations Research and Com-
puter Science for his book "Neuro-Dynamic Programming" (co-authored
with John Tsitsiklis), the 2001 AACC John R. Ragazzini Education Award,
the 2009 INFORMS Expository Writing Award, the 2014 AACC Richard
Bellman Heritage Award for "contributions to the foundations of determin-
istic and stochastic optimization-based methods in systems and control,"
and the 2014 Khachiyan Prize for "life-time accomplishments in optimiza-
tion." In 2001, he was elected to the United States National Academy of
Engineering for "pioneering contributions to fundamental research, practice
and education of optimization/control theory, and especially its application
to data communication networks."

vii
Preface

There is no royal way to geometry

(Euclid to king Ptolemy of Alexandria)

Interest in convex optimization has become intense due to widespread ap-

plications in fields such as large-scale resource allocation, signal processing,
and machine learning. This book aims at an up-to-date and accessible de-
velopment of algorithms for solving convex optimization problems.
The book complements the author's 2009 "Convex Optimization The-
ory" book, but can be read independently. The latter book focuses on
convexity theory and optimization duality, while the present book focuses
on algorithmic issues. The two books share mathematical prerequisites,
notation, and style, and together cover the entire finite-dimensional convex
optimization field. Both books rely on rigorous mathematical analysis, but
also aim at an intuitive exposition that makes use of visualization where
possible. This is facilitated by the extensive use of analytical and algorith-
mic concepts of duality, which by nature lend themselves to geometrical
interpretation.
To enhance readability, the statements of definitions and results of
the "theory book" are reproduced without proofs in Appendix B. Moreover,
some of the theory needed for the present book, has been replicated and/or
adapted to its algorithmic nature. For example the theory of subgradients
for real-valued convex functions is fully developed in Chapter 3. Thus the
reader who is already familiar with the analytical foundations of convex
optimization need not consult the "theory book" except for the purpose of
studying the proofs of some specific results.
The book covers almost all the major classes of convex optimization
algorithms. Principal among these are gradient, subgradient, polyhedral
approximation, proximal, and interior point methods. Most of these meth-
ods rely on convexity (but not necessarily differentiability) in the cost and
constraint functions, and are often connected in various ways to duality. I
have provided numerous examples describing in detail applications to spe-
cially structured problems. The reader may also find a wealth of analysis
and discussion of applications in books on large-scale convex optimization,
network optimization, parallel and distributed computation, signal process-
ing, and machine learning.
The chapter-by-chapter description of the book follows:
Chapter 1: Here we provide a broad overview of some important classes of
convex optimization problems, and their principal characteristics. Several

ix
X Preface

problem structures are discussed, often arising from Lagrange duality the-
ory and Fenchel duality theory, together with its special case, conic duality.
Some additional structures involving a large number of additive terms in
the cost, or a large number of constraints are also discussed, together with
their applications in machine learning and large-scale resource allocation.
Chapter 2: Here we provide an overview of algorithmic approaches, focus-
ing primarily on algorithms for differentiable optimization, and we discuss
their differences from their nondifferentiable convex optimization counter-
parts. We also highlight the main ideas of the two principal algorithmic
approaches of this book, iterative descent and approximation, and we illus-
trate their application with specific algorithms, reserving detailed analysis
for subsequent chapters.
Chapter 3: Here we discuss subgradient methods for minimizing a con-
vex cost function over a convex constraint set. The cost function may be
nondifferentiable, as is often the case in the context of duality and machine
learning applications. These methods are based on the idea of reduction
of distance to the optimal set, and include variations aimed at algorithmic
efficiency, such as E-subgradient and incremental subgradient methods.
Chapter 4: Here we discuss polyhedral approximation methods for min-
imizing a convex function over a convex constraint set. The two main
approaches here are outer linearization (also called the cutting plane ap-
proach) and inner linearization (also called the simplicial decomposition
approach). We show how these two approaches are intimately connected
by conjugacy and duality, and we generalize our framework for polyhedral
approximation to the case where the cost function is a sum of two or more
convex component functions.
Chapter 5: Here we focus on proximal algorithms for minimizing a convex
function over a convex constraint set. At each iteration of the basic proxi-
mal method, we solve an approximation to the original problem. However,
unlike the preceding chapter, the approximation is not polyhedral, but
rather it is based on quadratic regularization, i.e., adding a quadratic term
to the cost function, which is appropriately adjusted at each iteration. We
discuss several variations of the basic algorithm. Some of these include
combinations with the polyhedral approximation methods of the preced-
ing chapter, yielding the class of bundle methods. Others are obtained
via duality from the basic proximal algorithm, including the augmented
Lagrangian method ( also called method of multipliers) for constrained op-
timization. Finally, we discuss extensions of the proximal algorithm for
finding a zero of a maximal monotone operator, and a major special case:
the alternating direction method of multipliers, which is well suited for
taking advantage of the structure of several types of large-scale problems.
Chapter 6: Here we discuss a variety of algorithmic topics that sup-
plement our discussion of the descent and approximation methods of the
Preface xi

preceding chapters. We first discuss gradient projection methods and vari-

ations with extrapolation that have good complexity properties, including
Nesterov's optimal complexity algorithm. These were developed for differ-
entiable problems, and can be extended to the nondifferentiable case by
means of a smoothing scheme. Then we discuss a number of combinations
of gradient, subgradient, and proximal methods that are well suited for
specially structured problems. We pay special attention to incremental
versions for the case where the cost function consists of the sum of a large
number of component terms. We also describe additional methods, such
as the classical block coordinate descent approach, the proximal algorithm
with a nonquadratic regularization term, and the E-descent method. We
close the chapter with a discussion of interior point methods.
Our lines of analysis are largely based on differential calculus-type
ideas, which are central in nonlinear programming, and on concepts of hy-
perplane separation, conjugacy, and duality, which are central in convex
analysis. A traditional use of duality is to establish the equivalence and
the connections between a pair of primal and dual problems, which may in
turn enhance insight and enlarge the set of options for analysis and compu-
tation. The book makes heavy use of this type of problem duality, but also
emphasizes a qualitatively different, algorithm-oriented type of duality that
is largely based on conjugacy. In particular, some fundamental algorithmic
operations turn out to be dual to each other, and whenever they arise in
various algorithms they admit dual implementations, often with significant
gains in insight and computational convenience. Some important examples
are the duality between the subdifferentials of a convex function and its
conjugate, the duality of a proximal operation using a convex function and
an augmented Lagrangian minimization using its conjugate, and the dual-
ity between outer linearization of a convex function and inner linearization
of its conjugate. Several interesting algorithms in Chapters 4-6 admit dual
implementations based on these pairs of operations.
The book contains a fair number of exercises, many of them sup-
plementing the algorithmic development and analysis. In addition a large
number of theoretical exercises (with carefully written solutions) for the "the-
ory book," together with other related material, can be obtained from the
book's web page http://www.athenasc.com/convexalgorithms.html, and
the author's web page http://web.mit.edu/dimitrib/www /home.html. The
MIT OpenCourseWare site http://ocw.mit.edu/index.htm, also provides
lecture slides and other relevant material.
The mathematical prerequisites for the book are a first course in
linear algebra and a first course in real analysis. A summary of the relevant
material is provided in Appendix A. Prior exposure to linear and nonlinear
optimization algorithms is not assumed, although it will undoubtedly be
helpful in providing context and perspective. Other than this background,
the development is self-contained, with proofs provided throughout.
xii Preface

The present book, in conjunction with its "theory" counterpart may

be used as a text for a one-semester or two-quarter convex optimization
course; I have taught several variants of such a course at MIT and else-
where over the last fifteen years. Still the book may not provide all of the
convex optimization material an instructor may wish for, and it may need
to be supplemented by works that aim primarily at specific types of con-
vex optimization models, or address more comprehensively computational
complexity issues. I have added representative citations for such works,
which, however, are far from complete in view of the explosive growth of
the literature on the subject.
The book may also be used as a supplementary source for nonlinear
programming classes that are primarily focused on classical differentiable
nonconvex optimization material (Kuhn-Tucker theory, Newton-like and
conjugate direction methods, interior point, penalty, and augmented La-
grangian methods). For such courses, it may provide a nondifferentiable
convex optimization component.
I was fortunate to have several outstanding collaborators in my re-
search on various aspects of convex optimization: Vivek Borkar, Jon Eck-
stein, Eli Gafni, Xavier Luque, Angelia Nedic, Asuman Ozdaglar, John
Tsitsiklis, Mengdi Wang, and Huizhen (Janey) Yu. Substantial portions of
our joint research have found their way into the book. In addition, I am
grateful for interactions and suggestions I received from several colleagues,
including Leon Bottou, Steve Boyd, Tom Luo, Steve Wright, and particu-
larly Mark Schmidt and Lin Xiao who read with care major portions of the
book. I am also very thankful for the valuable proofreading of parts of the
book by Mengdi Wang and Huizhen (Janey) Yu, and particularly by Ivan
Pejcic who went through most of the book with a keen eye. I developed
the book through convex optimization classes at MIT over a fifteen-year
period, and I want to express appreciation for my students who provided
continuing motivation and inspiration.
Finally, I would like to mention Paul Tseng, a major contributor
to numerous topics in this book, who was my close friend and research
collaborator on optimization algorithms for many years, and whom we
unfortunately lost while he was still at his prime. I am dedicating the book
to his memory.

Dimitri P. Bertsekas
[email protected]
January 2015
1
Convex Optimization Models:
An Overview

Contents

1.1. Lagrange Duality . . . . . . . . p. 2

1.1.1. Separable Problems - Decomposition p. 7
1.1.2. Partitioning . . . . . . . . . . . p.- 9
1.2. Fenchel Duality and Conic Programming . p. 10
1.2.1. Linear Conic Problems . . . . p. 15
1.2.2. Second Order Cone Programming p. 17
1.2.3. Semidefinite Programming p. 22
1.3. Additive Cost Problems . . p. 25
1.4. Large Number of Constraints . p. 34
1.5. Exact Penalty Functions p. 39
1.6. Notes, Sources, and Exercises p. 47

1
 2 Convex Optimization Models: An Overview Chap. 1

In this chapter we provide an overview of some broad classes of convex

optimization models. Our primary focus will be on large challenging prob-
lems, often connected in some way to duality. We will consider two types
of duality. The first is Lagrange duality for constrained optimization, which
is obtained by assigning dual variables to the constraints. The second is
Fenchel duality together with its special case, conic duality, which involves
a cost function that is the sum of two convex function components. Both
of these duality structures arise often in applications, and in Sections 1.1
and 1.2 we provide an overview, and discuss some examples.t
In Sections 1.3 and 1.4, we discuss additional model structures in-
volving a large number of additive terms in the cost, or a large number
of constraints. These types of problems also arise often in the context of
duality, as well as in other contexts such as machine learning and signal
processing with large amounts of data. In Section 1.5, we discuss the exact
penalty function technique, whereby we can transform a convex constrained
optimization problem to an equivalent unconstrained problem.

1.1 LAGRANGE DUALITY

We start our overview of Lagrange duality with the basic case of nonlin-
ear inequality constraints, and then consider extensions involving linear
inequality and equality constraints. Consider the problemt
mm1m1ze f (x)
(1.1)
subject to x EX, g(x) ::::; 0,
where X is a nonempty set,
g(x) = (g1(x), ... ,gr(x))',
and f: X H ~ and gj : X H ~, j = 1, ... , r, are given functions. We refer
to this as the primal problem, and we denote its optimal value by f*. A
vector x satisfying the constraints of the problem is referred to as feasible.
The dual of problem (1.1) is given by
maximize q(µ)
(1.2)
subject to µ E ~r,

t Consistent with its overview character, this chapter contains few proofs,
and refers frequently to the literature, and to Appendix B, which contains a full
list of definitions and propositions (without proofs) relating to nonalgorithmic
aspects of convex optimization. This list reflects and summarizes the content
of the author's "Convex Optimization Theory" book [Ber09]. The proposition
numbers of [Ber09] have been preserved, so all omitted proofs of propositions in
Appendix B can be readily accessed from [Ber09].
t Appendix A contains an overview of the mathematical notation, terminol-
ogy, and results from linear algebra and real analysis that we will be using.
Sec. 1.1 Lagrange Duality 3

where the dual function q is

q(µ) ={ ~:EX L(x, µ) ifµ 2'. 0,

otherwise,

and L is the Lagrangian function defined by

L(x, µ) = .f(x) + µ'g(x), XE X, µ E ~r;

(cf. Section 5.3 of Appendix B).

Note that the dual function is extended real-valued, and that the
effective constraint set of the dual problem is

The optimal value of the dual problem is denoted by q*.

The weak duality rela tion, q* ::::; .f *, always holds. It is easily shown
by writing for allµ 2'. 0, and x E X with g(x ) ::::; 0,
T

q(µ) = inf L(z,µ)::::; L(x,µ) = .f(x) + L_µ1g1 (x )::::; .f(x),

zEX
j= l

so that
q* = sup q(µ) = supq(µ)::::; inf .f(x) = .f*.
µ E1Rr µ 2: 0 xEX,g(x)<'.'.O

We state this formally as follows (cf. Prop. 4.1.2 in Appendix B) .

Proposition 1.1.1: (Weak Duality Theorem) Consider problem

(1.1). For any feasible solution x and anyµ E ~r, we have q(µ) ::::; f(x).
Moreover, q* ::::; f *.

When q* = .f*, we say that strong duality holds. The following propo-
sition gives necessary and sufficient conditions for strong duality, and pri-
mal and dual optimality (see Prop. 5.3.2 in Appendix B).

Proposition 1.1.2: (Optimality Conditions) Consider problem

(1.1). There holds q* = f*, and (x*, µ*)area primal and dual optimal
solution pair if and only if x* is feasible, µ* 2'. 0, and

x* E argminL(x,µ*), µ1g1 (x* ) = 0, j = 1, ... ,r.

xEX
4 Convex Optimization Models: An Overview Chap. 1

Both of the preceding propositions do not require any convexity as-

sumptions on f, g, and X. However, generally the analytical and algo-
rithmic solution process is simplified when strong duality (q* = f*) holds.
This typically requires convexity assumptions, and in some cases conditions
on ri( X), the relative interior of X, as exemplified by the following result,
given in Prop. 5.3.l in Appendix B. The result delineates the two principal
cases where there is no duality gap in an inequality-constrained problem.

Proposition 1.1.3: (Strong Duality - Existence of Dual Opti-

mal Solutions) Consider problem (1.1) under the assumption that
the set X is convex, and the functions f, and g1, ... , 9r are convex.
Assume further that f * is finite, and that one of the following two
conditions holds:
(1) There exists x EX such that gj(x) < 0 for all j = 1, ... , r.
(2) The functions 9i, j = 1, ... , r, are affine, and there exists x E
ri(X) such that g(x) $ 0.
Then q* = f* and there exists at least one dual optimal solution.
Under condition (1) the set of dual optimal solutions ~s also compact.

Convex Programming with Inequality and Equality Constraints

Let us consider an extension of problem (1.1), with additional linear equal-

ity constraints. It is our principal constrained optimization model under
convexity assumptions, and it will be referred to as the convex programming
problem. It is given by
minimize f (x)
(1.3)
subject to x EX, g(x) $ 0, Ax= b,

where X is a convex set, g(x) = (g1(x), ... ,gr(x))', f : X H ~ and

= 1, ... , r, are given convex functions, A is an m x n matrix,
gj : X H ~, j
and b E ~m.
The preceding duality framework may be applied to this problem by
converting the constraint Ax = b to the equivalent set of linear inequality
constraints
Ax$ b, -Ax$ -b,
with corresponding dual variables >.+ ~ 0 and >.- ~ 0. The Lagrangian
function is
f(x) + µ'g(x) + (>.+ - >.-) 1 (Ax - b),
and by introducing a dual variable
Sec. 1.1 Lagrange Duality 5

with no sign restriction, it can be written as

L(x, µ, >..) = f(x) + µ'g(x) + >..'(Ax - b).

The dual problem is

maximize inf L(x, µ, >..)

xEX
subject to µ ~ 0, >.. E ~m.

In this manner, Prop. 1.1.3 under condition (2), together with Prop. 1.1.2,
yield the following for the case where all constraint functions are linear.

Proposition 1.lA: (Convex Programming - Linear Equality

and Inequality Constraints) Consider problem (1.3).
(a) Assume that f* is finite, that the functions gj are affine, and
that there exists x E ri(X) such that Ax= band g(x) :s; 0. Then
q* = f* and there exists at least one dual optimal solution.
(b) There holds f* = q*, and (x*,µ*,>..*) are a primal and dual
optimal solution pair if and only if x* is feasible, µ* ~ 0, and

x* E argminL(x,µ*,>..*), µjgj(X*) = 0, j = 1, ... , r.

xEX

In the special case where there are no inequality constraints:

mm1m1ze f (x)
(1.4)
subject to x E X, Ax = b,

the Lagrangian function is

L(x, >..) = f(x) + >.. (Ax - b),

1

and the dual problem is

maximize inf L( x, >..)

xEX
subject to >.. E ~m.

The corresponding result, a simpler special case of Prop. 1.1.4, is given in

the following proposition.
6 Convex OpUrnizaUon Models: An Overview Chap. 1

Proposition 1.1.5: (Convex Programming - Linear Equality

Constraints) Consider problem (1.4).
(a) Assume that f* is finite and that there exists x E ri(X) such
that Ax = b. Then f* = q* and there exists at least one dual
optimal solution.
(b) There holds f* = q*, and (x*,>.*) are a primal and dual optimal
solution pair if and only if x* is feasible and

x* E argminL(x,>.*).
xEX

The following is an extension of Prop. 1.l.4(a) to the case where the

inequality constraints may be nonlinear. It is the most general convex
programming result relating to duality in this section (see Prop. 5.3.5 in
Appendix B).

Proposition 1.1.6: (Convex Programming - Linear Equality

and Nonlinear Inequality Constraints) Consider problem (1.3).
Assume that f* is finite, that there exists x E X such that Ax = b
and g(x) < 0, and that there exists x E ri(X) such that Ax= b. Then
q* = f* and there exists at least one dual optimal solution.

Aside from the preceding results, there are alternative optimality con-
ditions for convex and nonconvex optimization problems, which are based
on extended versions of the Fritz John theorem; see [Be002] and [BOT06],
and the textbooks [Ber99] and [BN003]. These conditions are derived us-
ing a somewhat different line of analysis and supplement the ones given
here, but we will not have occasion to use them in this book.

Discrete Optimization and Lower Bounds

The preceding propositions deal mostly with situations where strong du-
ality holds (q* = f*). However, duality can be useful even when there is
duality gap, as often occurs in problems that have a finite constraint set
X. An example is integer programming, where the components of x must
be integers from a bounded range (usually O or 1). An important special
case is the linear 0-1 integer programming problem

minimize c' x
subject to Ax .:; b, Xi = 0 or 1, i = 1, ... , n,
Sec. 1.1 Lagrange Duality 7

where x = (x1, ... , Xn),

A principal approach for solving discrete optimization problems with
a finite constraint set is the branch-and-bound method, which is described
in many sources; see e.g., one of the original works [LaD60], the survey
[BaT85], and the book [NeW88]. The general idea of the method is that
bounds on the cost function can be used to exclude from consideration
portions of the feasible set. To illustrate, consider minimizing F(x) over
x E X, and let Y1, Y2 be two subsets of X. Suppose that we have bounds

Then, if F2 ::::; F 1 , the solutions in Y1 may be disregarded since their cost

cannot be smaller than the cost of the best solution in Y2 • The lower bound
F 1 can often be conveniently obtained by minimizing f over a suitably
enlarged version of Y1, while for the upper bound F2, a value f(x), where
x E Y2, may be used.
Branch-and-bound is often based on weak duality (cf. Prop. 1.1.1) to
obtain lower bounds to the optimal cost of restricted problems of the form
minimize f (x)
(1.5)
subject to x EX, g(x)::::; 0,
where X is a subset of X; for example in the 0-1 integer case where X
specifies that all Xi should be O or 1, X may be the set of all 0-1 vectors
x such that one or more components Xi are fixed at either O or 1 (i.e., are
restricted to satisfy x; = 0 for all x E X or Xi = 1 for all x E X). These
lower bounds can often be obtained by finding a dual-feasible (possibly
dual-optimal) solution µ 2: 0 of this problem and the corresponding dual
value
q(µ) = in( {f(x) + µ'g(x) }, (1.6)
xEX

which by weak duality, is a lower bound to the optimal value of the re-
stricted problem (1.5). In a strengthened version of this approach, the
given inequality constraints g(x) ::::; 0 may be augmented by additional in-
equalities that are known to be satisfied by optimal solutions of the original
problem.
An important point here is that when X is finite, the dual function
q of Eq. (1.6) is concave and polyhedral. Thus solving the dual problem
amounts to minimizing the polyhedral function -q over the nonnegative
orthant. This is a major context within which polyhedral functions arise
in convex optimization.

1.1.1 Separable Problems - Decomposition

Let us now discuss an important problem structure that involves Lagrange

duality and arises frequently in applications. Here x has m components,
Random documents with unrelated
content Scribd suggests to you:
Eugénie, do not avoid her. The time has come when she ought to
know you as you are. Yes, we have at last arrived at the decisive
hour which Victor spoke of the night before he died. Mlle. Eugénie
must now be enabled to appreciate you as you deserve. She must
pity you.... She must love you! If this is not the case, however sad it
will be to give up an illusion without which it seems impossible to be
happy, renounce it, and acknowledge without shrinking: 'She does
not love me; she never will love me; she is not the wife God destines
me.' But do not act hastily. Believe me, if she is intended for you,
whatever has been done, nothing is lost. But it is my opinion she is
intended for you."
These words did Louis good. "I hope you are not deceived," said he,
"and this very hope revives me. I will try to believe you are right. We
will do nothing hastily, therefore. But do you not think I could now
venture to disclose my sentiments to Mlle. Eugénie, if I have a
favorable opportunity, and see it will give no offence? One
consideration alone restrains me—I fear being suspected of seeking
her hand from interested motives."
"The time for such suspicions is past. If Eugénie still cherishes them,
it will lower her in my estimation. She is twenty-two years of age.
She has a good deal of heart and an elevated mind, and is capable
of deciding her own destiny. I therefore approve of your plan. If she
loves you, she will have the courage to avow it to her parents. If she
does not love you, she has sufficient courage to make it evident to
you."
"How I wish the question already decided!"
"No youthful impulsiveness! You need more than ever to be
extremely cautious while feeling your way. Your situation is one of
great delicacy. Act, but with deliberation."
Such was pretty nearly the advice I gave Louis, often stopping to
give vent to my grief, which was as profound as ever. He left me
quite comforted. Though he did not say so, for fear of being
deceived, he thought Eugénie loved him, and believed, with her on
his side, he should triumph over every obstacle. When a person is in
love, he clings to hope in spite of himself, even when all is evidently
lost.

CHAPTER XXV.
ALL IS LOST!-THE PROSPECT BRIGHTENS.

Louis spent several evenings in succession with me. He briefly

related how the day had passed, and afterwards took up the
different events, and enlarged upon them. He often found enough to
talk about for hours upon the sometimes ungrateful theme. I can
still see him sitting opposite my mother and myself in the arbor in
the little garden behind our house. Everything was calm and
delightful around us in those beautiful autumn evenings. Louis alone
was troubled. In vain we tried to restore peace to his soul: it was
gone!
I never comprehended so thoroughly all the power of love as then.
The profound sadness in which I was at that time overwhelmed
rendered me inaccessible to such passionate outbreaks—such fits of
elevation and depression as Louis was then subject to. I gazed at
him with a cool, dispassionate eye, but with the affectionate
compassion with which we regard a friend who is trying to make
himself unhappy. I was astonished; sometimes I was even—yes, I
acknowledge it—irritated to see how utterly he gave himself up to
the passion he had allowed to develop so rapidly in his heart.
Doubtless my poor friend remained resigned to the will of God, but
not so completely as he thought. It is true, even when his mind was
apparently the most agitated, we felt that piety was the overruling
principle; but then, what a struggle there was between the divine
Spirit, which always seeks to infuse calmness, and the gusts of
passion that so easily result in a tempest!
Ah! I loved my husband too sincerely, and I recall other loves too
pure, to dare assert that love is wrong. But believe me, my young
friend, I do not exaggerate in adding that, if love is not always
censurable, it is in danger of being so. We are told on every hand
that love ennobles the heart and tends to elevate the mind; that it is
the mainspring of great enterprises, and destructive of egotism. Yes,
sometimes; ... but for love to effect such things, what watchfulness
must not a person exercise over himself! How much he must distrust
his weakness! What incessant recourse he must have to God!
Without this, the love that might ennoble is only debasing, and to
such a degree as to lead unawares, so to speak, to the commission
of acts unworthy, not only of a Christian, but a man.
Allow me, my friend, continued Madame Agnes, to make use of a
comparison, common enough, but which expresses my idea better
than any other. Love is like generous wine. It must be used with
sobriety and caution. Taken to excess, it goes to the head, and
makes a fool of the wisest. You are young. You have never loved.
Beware of the intoxication to which I allude! If you ever do love,
watch over yourself; pray with fervor that God will give you the
grace of self-control. The moment love becomes a passion—an
overruling passion—ah! how its victim is to be pitied! When reason
and conscience require it, you can—I mean with the divine
assistance—banish love from the heart where it reigns; but believe
me, it will leave you as an enemy leaves the country it has invaded—
with fearful destruction behind. And first of all, it destroys one's
peace of mind. The soul in which passion has reigned continues to
bear marks of its ravages a long time after its extinction!...
Louis had arrived at this deplorable state; he had not full control
over his heart; his happiness depended on the success of his love.
Eugénie's image beset him everywhere. The word is hard, I confess,
but it is true. He attached undue importance to whatever had the
least bearing on this predominant thought. One day, he announced
he had seen Albert walking with a melancholy air. He was sad, then.
But why should he be sad unless his cousin had treated him coldly?
And Louis hastily added by way of conclusion: "Mlle. Eugénie knows
all I have to annoy me; she follows me in thought, she participates
in my sorrows, she repays me for them...." Another day he had
really seen her. She passed by his window, lovelier than ever, but
more thoughtful. She was doubtless as anxious as he to be freed
from the suspense in which they both were.
At last he came with important news. He had had the unhoped-for
happiness of meeting Eugénie. She was advancing towards him,
blushing with embarrassment, and was the first to greet him, with
an expression so friendly as to leave no doubt of her sentiments. He
returned her salutation, but was so overpowered with emotion that
he could scarcely speak. After some words of no importance, he
said: "I am going to leave you, mademoiselle."
Eugénie replied that she should regret to see him go. Then, as if to
intimate he had enemies in the house, she added: "More than one—
I wish I could say all—will be as afflicted as I at your departure. I
refer to those you have benefited, and to whom you might continue
to do good."
"Yes," said Louis, "it is hard to have to leave my work incomplete.
However limited it is, my soul is in it. But I must not make myself
out a better Christian than I am. It is not my work I shall leave with
the most regret...." He dared not complete the expression of his
thought.
Eugénie, generally so self-restrained, was visibly affected and
intimidated. She was about to reply, when Mme. Smithson suddenly
made her appearance. It looked as if she kept watch over her
daughter. When she saw her talking with Louis, she could not
conceal her annoyance. Saluting him in a freezing, insolent manner,
she said: "Eugénie, what are you doing here? Your cousin is hunting
everywhere for you to go to town with him!"
"There is no hurry," replied Eugénie, resuming her habitual coolness
and dignity. She went away, taking leave of Louis with a visible air of
decided sympathy.
This brief interview was sufficient to render Louis' hopes legitimate. I
agreed with him that Eugénie would have behaved very differently if
she regarded him with antipathy, or even with indifference.
"There is no doubt she knows all that has taken place," said I to my
friend. "If there is any plot against you, she cannot fail to be aware
of it, or, at least, suspect it. Under such circumstances, the very fact
of her showing you unmistakable sympathy is a sufficient proof that
she loves you."
At this time, an occurrence took place that had an unfortunate effect
on me, and created new difficulties in Louis' path. It was then in the
latter part of the month of September. The summer had been rainy
and unpleasant. The rains increased in September, and soon caused
an alarming rise in all the rivers. I was then at the end of my stay in
the little village of St. M——, where I lived unknown to the
Smithsons. Faithful to my request, Louis had told no one of my
temporary residence in the vicinity.
Excuse me for giving you here some topographical details, perhaps
somewhat difficult to comprehend, but necessary for you to know in
order to understand what follows.
St. M—— is situated in a charming valley. In ordinary weather, the
current of the Loire is below the level of the valley through which it
winds with a majestic sweep. When a rise occurs, the plain would at
once be inundated were it not protected by a dike which the water
cannot cross. This dike did not extend to Mr. Smithson's
manufactory, though but a short distance from St. M——. When,
therefore, the river got very high, the mill ran the risk of being
inundated. The dwelling-house alone was out of danger, being on an
eminence beyond the reach of the waters of the Loire, even when it
joined, swelled by the junction, the small stream that drove Mr.
Smithson's machinery.
Having given you some idea of that region, I will now resume my
story. One evening, then, towards the end of my stay at St. M——,
Louis told me the Loire was rising fast. He assured me, however,
before leaving, that there was no danger. "No matter how strong or
high the current," he said, "the dike secures you from all danger. It
is as firm as a rock."
My friend was mistaken. The bank had certain weak places which
the water had undermined without any one's being aware of it.
Towards eleven o'clock, there was a tremendous noise in every
direction. People were screaming and rushing around the house: the
dike had given way! The water had reached the ground floor. My
mother, my sister, and myself were lodged on the first story. The
proprietor, beside himself, and frightened enough to alarm every one
else, came up to tell us we must make haste to escape; his house
was not solid; we were in danger of being carried away.
"The water is only rising slowly," he said. "By wading two or three
hundred yards, we can reach the causeway. There we shall be safe;
for the ground is firm, and the causeway extends to St. Denis. The
inundation cannot reach that place, for it is built on a height."
I did not lose my presence of mind in the midst of the alarm. Victor's
death had destroyed all attachment to life. If my mother and sister
had not been in danger as well as myself, I should have remained
where I was, trusting in God, not believing I was under any moral
obligation to escape from a house which might withstand more than
was supposed; as it did, in fact. But my mother and sister lost all
reason, so to speak. Wild with terror, they fled, and I followed them.
When we got down to the ground floor, we found the water had
risen to the height of about six inches. There was a mournful sound
in every direction which made us tremble. We sprang towards the
causeway. I was at that time in delicate health. I had been suddenly
roused from sleep. The distance I had to wade through the cold
water had a fearful effect on me. When we reached the causeway,
they had to carry me to St. Denis: I was incapable of walking.
While we were thus flying from danger, Louis committed a series of
generous but imprudent acts which became a source of fresh
difficulties to him. He was sitting alone in his chamber, when, about
half-past ten, he heard a dull crash like a discharge of artillery at a
distance. He hastily ran down into the court, entered the porter's
lodge, and inquired where the noise came from that had alarmed
him.
"I do not know, monsieur," replied the man, "but I have an idea that
the levée has given way. At a great inundation twenty years ago, the
Loire made a large hole in the dike, which caused a similar noise. I
know something about it, for I was then living near...."
This was enough to alarm Louis, and just then a man passed with a
torch in his hand, crying breathlessly: "The dike has given way at St.
M——! Help! Quick! The village will be inundated!"
These words redoubled Louis' terror. St. M—— would be inundated;
perhaps it was already.... I was there ill, and knew no one!
"Is there any danger of the water's reaching us?" asked Louis of the
porter.
"The mill? Yes, ... but not Mr. Smithson's: that is impossible. The
house stands twenty feet above the river."
Eugénie and her parents, then, had nothing to fear. I alone was in
danger—in so great a danger that there was not a moment to be
lost.
"Go and tell Mr. Smithson all that has happened," said Louis. "I am
going away. I am obliged to. I shall be back in half an hour, or as
soon as I can."
Of all the sacrifices Louis ever made, this was the most heroic. In
fact, had he remained at his post, he might have saved the
machinery, that was quite a loss to Mr. Smithson. Instead of that, he
hurried off without any thought of the construction his enemies
might put on his departure. To complete the unfortunate
complication, Mr. Smithson had an attack of the gout that very day.
When I afterwards alluded to his imprudence in thus risking his
dearest interests, as well as life itself, Louis replied: "I knew Eugénie
had nothing to fear; whereas, you were in danger. I had promised
Victor on his death-bed to watch over you as he would himself. It
was my duty to do as I did. If it were to do over again, I should do
the same. Did Victor hesitate when he sprang into the water to save
me? And he did not know who I was."
The house I had just left was about half a league from the mill. The
water was beginning to reach the highway, though slowly. Louis kept
on, regardless of all danger, and arrived at our house in feverish
anxiety. I had been gone about fifteen minutes, and the water was
much higher than when we left. Louis learned from a man who
remained in a neighboring house that I was safe: we had all escaped
by the causeway before there was any danger. He added that I must
be at St. Denis by that time. Louis, reassured as to my fate,
succeeded in reaching another road, more elevated, but not so
direct to the mill. This road passed just above the Vinceneau house.
When Louis arrived opposite the house, he saw the water had
reached it. He heard screams mingled with oaths that came from the
father, angry with his wife and daughter. Having returned home a
few moments before, the drunken man was resisting the efforts of
both women to induce him to escape. Louis appeared as if sent by
Providence. He at once comprehended the state of affairs. His look
overawed the drunken man, who left the house. They all four
proceeded toward the mill. There was no nearer place of refuge. The
first people they saw at their arrival were Durand, Albert, and some
workmen. An insolent smile passed over Albert's face. He evidently
suspected Louis of having abandoned everything for the purpose of
saving Madeleine Vinceneau. But he did not dare say anything. Louis
intimidated him much more than he could have wished. He resolved,
however, to make a good use of what he had seen. Louis at once felt
how unfortunate this combination of circumstances was, but the
imminent danger they were in forced him to exertion. It was feared
the walls of the manufactory might give way under the action of the
water, if it got much higher, and it was gradually rising.
Louis set to work without any delay. The workmen, who had
hastened from every part of the neighborhood to take refuge at Mr.
Smithson's, began under his direction to remove the machinery that
was still accessible. They afterwards propped up the walls, and,
when these various arrangements were completed, Louis, who had
taken charge of everything, occupied himself in providing temporary
lodgings for the people driven out by the inundation.
Mme. Smithson and her daughter had come down to render
assistance. The refugees were lodged in various buildings on a level
with the house. Louis would have given everything he possessed for
the opportunity of exchanging a few words with Eugénie at once, in
order to forestall the odious suspicions Albert would be sure to
excite in her mind. But he was obliged to relinquish the hope. Mme.
Smithson and Albert followed her like a shadow. Louis could not
approach her without finding one or the other at her side. Overcome
by so fatiguing a night, he went towards morning to take a little
repose. He felt sure fresh mortifications awaited him in consequence
of what had just taken place, and he was right.
When he awoke after a few hours' sleep, his first care was to go and
see Mr. Smithson. He related what he had done, without concealing
the fact of his abandoning the mill to go to my assistance. Mr.
Smithson was suffering severely from the gout. He was impatient at
such a time to be on his feet, and was chafing with vexation.
"I cannot blame you, monsieur," he said. "The life of a friend is of
more consequence than anything else. Whatever be the material
loss I may have to endure at this time in consequence of your
absence, I forbear complaining. But it was unfortunate things should
happen so. If I had only been able to move!... But no.... You will
acknowledge, monsieur, that I am the victim of misfortune.... Did
you succeed, after all, in saving the person whose fate interested
you more than anything else?..."
"She had made her escape before my arrival. I hurried back, but, on
the way, a new incident occurred. An unfortunate family was on the
point of perishing. I brought them with me, as there was no nearer
asylum."
"Are these people employed at the mill?"
"The woman works here; her husband elsewhere."
"What is their name?"
"Vinceneau."
"I think I have heard of them. The father is a drunkard; the mother
is an indolent woman."
"You may have learned these facts from Mlle. Eugénie, who takes an
interest in the family, I believe. I recommended them to her."
"Was that proper?... I have every reason to think otherwise.... But it
is done. We will say no more about it. And since I am so
inopportunely confined to my bed, I must beg you to continue to
take charge in my place, watch over the safety of the inundated
buildings, provide for the wants of the people who have taken refuge
here, and, above all, have everything done in order."
Louis was uneasy and far from being satisfied. There was a certain
stiffness and ill-humor in Mr. Smithson's manner that made him think
Albert had reported his return to the mill with the Vinceneau family.
He attempted an explanation on this delicate subject.
"Mon Dieu! you seem very anxious about such a trifling affair," said
Mr. Smithson. "It appears to me there is something of much more
importance to be thought of now.... It is high time to try to remedy
the harm done last night...."
Louis felt that, willing or not, he must await a more propitious time.
He went away more depressed than ever.
The whole country around was inundated. I was obliged to send a
boat for news concerning my young friend, and give him information
about myself. The unfortunate people who had taken refuge at Mr.
Smithson's were at once housed and made as comfortable as
possible. It happened that Durand and some others were put in the
same building with the Vinceneau family. Nothing occurred the first
day worth relating. Louis watched in vain for an opportunity of
seeing and speaking to Eugénie. He only saw her at a distance. The
next morning—O unhoped-for happiness!—he met her on her way to
one of the houses occupied by the refugees. She looked at him so
coldly that he turned pale and his limbs almost gave way beneath
him. But Eugénie was not timid. She had sought this interview, and
was determined to attain her object.
"Whom have you put in that house?" she asked, pointing to the one
assigned to the Vinceneaus, which was not two steps from the small
building occupied by Louis himself.
"The Vinceneau family and some others," replied Louis.
At that name, Eugénie's lips contracted. An expression of displeasure
and contempt passed across her face. Then, looking at Louis with a
dignity that only rendered her the more beautiful, she said: "Then
you still have charge of them? I thought you gave them up to me."
"I have had nothing to do with them till within two days,
mademoiselle. It was enough to know you took an interest in their
condition." He then briefly related all that had taken place the night
of the inundation, and ended by speaking of the letter I had written
to relieve his anxiety. He finished by presenting the letter to Eugénie,
under the pretext of showing her the reproaches I addressed him. I
wrote him that, before troubling himself about me, he ought to have
been sure he was not needed at Mr. Smithson's.
Eugénie at first declined reading the letter. Then she took it with a
pleasure she endeavored to conceal. Before reading it, she said:
"Why did you not tell me your friend was at St. M——?"
"I have been greatly preoccupied for some time, and I seldom see
you, mademoiselle. It was in a manner impossible to tell you that my
poor friend had come here to be quiet and gain new strength in
solitude."
"I should have been pleased to see her." So saying, Eugénie, without
appearing to attach any importance to it, read my letter from
beginning to end.
Thus all Albert and Mme. Smithson's calculations were defeated.
There is no need of my telling you the inference Louis' enemies had
drawn from the interest he had manifested in the Vinceneau family.
"He left everything to save them, or rather, to save that girl," said
Mme. Smithson. "He would have let us all perish rather than not
save her."
My being at St. M——, and my letter, threw a very different light on
everything. Thenceforth, Louis, dismissed by her father, and
calumniated by her mother and Albert, was, in Eugénie's eyes, a
victim. And he had risked his own life to save that of his friend. It is
said that noble hearts, especially those of women, regard the rôle of
victim as an attractive one.
When Eugénie left Louis, there was in the expression of her eyes,
and in the tone of her voice, something so friendly and
compassionate that he felt happier than he had for a long time.... To
obtain this interview, Eugénie had been obliged to evade not only
her mother's active vigilance, but that of her cousin and Fanny. This
vigilance, suspended for a moment, became more active than ever
during the following days. It was impossible to speak to Louis; but
she saw him sometimes, and their eyes spoke intelligibly....
The water receded in the course of a week. Louis profited thereby to
come and see me, and make me a sharer in his joy. I was then
somewhat better. I passed the night of the inundation in fearful
suffering, but felt relieved the following day. My dreadful attack of
paralysis did not occur till some weeks afterwards. I little thought
then I had symptoms of the seizure that has rendered my life so
painful.
The refugees were still living at the manufactory, the Vinceneau
family among them. Louis had scarcely returned to his room that
night, when he heard a low knock at his door, and Madeleine
Vinceneau presented herself before him.
TO BE CONTINUED.
 THE NAPOLEONIC IDEA AND ITS
CONSEQUENCES.
FROM THE CIVILTA CATTOLICA.

I.

For several weeks past, we have heard much of Louis Napoleon

Bonaparte.[17] Nothing less than his mournful physical death, on the
9th of January, 1873, was needed to draw him from the oblivion to
which Italian liberals consigned him after his political death of
September 2, 1870. It would seem that from the imperial grave
opened at Chiselhurst went forth a bitter reproach against the
unexampled ingratitude of those who saw the tombstone of Sedan
close over his empire with mute impassibility and secret joy. Now to
the cowardly silence of two years succeeds an uproar of elegies and
praises. Remorse for having left the conqueror of Solferino in the
mire of the Meuse is lulled to sleep by the wailing of hired mourners;
as if the shame of basely forsaking him could be masked behind a
block of unblushing marble.
No man was ever more fatal to himself than Napoleon III. All which
was his by usurpation or right turned against him in the end. His
worst humiliations were the work of his own hands. He destroyed
himself, and the words of the Christian Demosthenes were truer of
him than of others: Nemo nisi a se ipso læditur.
Now, by a final mockery of fortune, he is punished after death by
having bier and tomb dishonored with the apotheosis of the Italian
party who laud to the skies the weapon that worked his ruin—the
ruling idea of his reign.
This idea, which necessarily failed because it was impracticable, and
in its failure reduced him to nothing, is his sole title to compassion or
glory in the opinion of this faction. But as the cruel irony contains a
historical lesson, useful for the present and the future, we will study
it by the light of facts, incontestable except to the blind.

II.

Such were the contradictions, perplexities, and duplicity of Louis

Napoleon Bonaparte upon the throne, that he was often believed to
be a prince reigning at hap-hazard. Indeed, it is said, now that he
has left the earth, that the history of his incomprehensible reign will
be the most difficult work ever undertaken. This seems to us a
mistake, if a distinction be made between the man and the prince,
his life and his reign. The man and his life will always seem
inextricable, for he used all means that suited his convenience, and
in their choice gave preference to no moral rule or principle of
honesty; following openly or hiddenly the mutable interest of each
day. But the prince and his reign, in spite of apparent contradictions,
are easily understood by the simple study of the political end which
he invariably proposed to himself.
This end is not hidden. His youthful writings, and the series of his
imperial documents, read by the light of the actions of his
administration, make it plain. He aimed at reestablishing and
consolidating in his dynasty the power of the First Empire, and at the
elevation of France to the headship of Europe, reorganized in its
territorial divisions according to the law of nationality, and in its
institutions in accordance with the forms of Cæsarean democracy.
An author who has read his books, and confronted them with the
achievements of his reign, thus sums up the new Napoleonic idea
constantly pursued by Louis in his youth, middle life, and old age, in
exile, in prison, and on the throne:
"Peoples distributed according to their needs and instincts, belonging
each to a self-elected country, provided each with a constitution
fixed yet democratic; devoted at their choice to works of civil
industry destined to transform the world; Europe, free in her various
nations, consolidated almost into a federated republic, with France
as its centre; France aggrandized and forming the clasp in the strong
chain of free intercourse; universal exhibitions to encourage nations
in the exchange of reciprocal visits; European congresses, where
governments, laying aside arms, could compose their differences;
Paris, the imperial city par excellence, wonderfully embellished,
raised to the honors of capital of the world, metropolis of wealth and
wisdom, under the wing of the Napoleonic eagle, offering to the two
hemispheres the rarest discoveries in science, masterpieces of art,
exquisite refinements of luxury and civilization."[18]
Divisum imperium cum Jove Cæsar habet!
Such was the intoxicating dream of the life and reign of Napoleon
III., the idea which he believed himself created to carry out—a
combination of the designs of Henry IV. and the aspirations of
Augustus, mounted on the frail pedestal of the principles of 1789.
In fact, proceeds our author, "Within and without the confines of the
Empire, this idea was reduced to two words: reconstruction and
reconciliation, based upon the principles of the French Revolution.
Here was to be the general synthesis of all external and internal
politics in France and Europe: Reconstruction of nations founded on
national will within and without; effected by a single instrument—
universal suffrage—applied to the determination of the nationality as
well as of the sovereign and the government; reconciliation of
nations among themselves, and of the divers classes composing
them, thanks to an equal satisfaction of the rights and interests of
all."[19]
That nothing might be wanting to the enchantment of his fair
dream, the young prisoner of Ham contemplated a double mission of
giving peace and glory to France. "War was to consolidate peace,
imperial battles were to give repose to the world. Thus the famous
device, The Empire and Peace, came to bear a sublime significance."
[20]

In short, the Napoleonic idea had for its ultimate aim the
aggrandizement and European omnipotence of France under the
dynasty of the Bonapartes, through the universal means of popular
suffrage with plébiscites, forming a basis of a new national and
international right, opposed to the old historical right of peoples. The
other three principles of territorial compensation, non-intervention
and accomplished facts, were special means and passing aids to be
used according to opportunity for carrying out intentions.

III.

Louis Napoleon received his political education from his uncle exiled
in the Island of St. Helena, and from the Carbonari, among whom
Ciro Menotti enrolled him in Tuscany, in the year 1831.[21] In these
two schools he acquired the fundamental idea of reconstructing
European countries according to nationality. But he did not see that,
in the hands of Napoleon I. and of the Carbonari, this idea was a
strong weapon of destruction, not a practical or powerful argument
for reconstruction. Bonaparte, gaoler of European potentates, and
the Carbonari, persecuted by them, wished to use it to destroy the
order of things established by the Holy Alliance in the treaty of
Vienna of 1815, upon the right, more or less defined, of legitimacy.
On the pretext of restoring political nationality to peoples, the first
Napoleon bequeathed to his heirs the command to excite Italy and
Hungary against Austria; Poland against Russia and Prussia; Greece
and the Christian principalities against Turkey; Ireland, Malta, and
the Ionian Isles against England; hoping that the changes originating
in this movement, and the gratitude of these nations, would make
easy to his heirs the extension of French boundaries and the
recovery of the imperial crown.
The Carbonari worked with the same pretext to overthrow princes
and substitute themselves, with a view of introducing into states
their anti-Christian and anti-social systems.
The so-called principle of nationality resolved itself, then, with
Napoleon I. and the Carbonari, into a pure engine of war—into a
battery which, after destroying the bulwarks of the opposite principle
of legitimacy, should give into their hands nations and kingdoms.
That Louis Napoleon, in prison, a fugitive, a conspirator, should
support himself with this flattering principle, and dexterously dazzle
with it the eyes of those who could help him to recover the sceptre
of France, can be easily understood; but that, after obtaining this
sceptre by a network of circumstances wholly foreign to the principle
of nationality, he should adopt that principle as the final aim of his
empire and the corner-stone of his own greatness and of French
power—this, in truth, is hard to understand.
But that it was the case is only too clear. He spent the twenty years
of his dominion over France in coloring the design which he had
puzzled out twenty years before, dreaming over the memories of St.
Helena, and plotting in the collieries of the Carbonari.

IV.

To a sagacious mind which had well weighed the true worth of the
Napoleonic idea, even before the new emperor attempted its
fulfilment, terrible dangers and obstacles must have presented
themselves.
After a succession of wars and successful conspiracies had led
nations to an independent reconstruction within natural frontiers,
what increase of territory could have accrued to France?
Suppose Italy, Poland, Hungary, and Iberia adjusted on this
principle, would their power have remained so equalized as to leave
France secure of preponderance?
If Germany had been so reconstructed, to the certain advantage of
Prussia, was there not a risk of exposing France to a shock which
might have proved fatal?
According to the theory of natural limits, the aggrandizement which
France could have demanded in compensation for protection and
successful warfare would have been reduced to some additions
towards the Alps, the Pyrenees, and in Flanders; to a few thousand
square kilometres, and perhaps three or four millions of inhabitants.
Towards the Rhine, we cannot see what the Empire could have
claimed without contradicting the theory itself. Germany has
maintained that Alsace and half of Lorraine, incorporated with
French soil, are German, and has forced them to a legal annexation
to her territory. Now, were these slender acquisitions, so
disproportioned to the acquisitions of neighboring countries, worth
the cost of turning Europe upside down, and subjecting France to a
chance of political and military ruin?
Louis Napoleon rejoiced in the thought of one day resuscitating the
fair name of Italy, extinguished for many years, and restoring it to
provinces so long deprived of it. This sounds well; but was this
resurrection to end in a united kingdom, or in the simple
emancipation from foreign rule? And granted that unity could not be
prevented, and that it should prove equal to the imaginary union of
Spain and Portugal, was it really advantageous to create alongside of
France, from a platonic love of nationality, two new states of twenty-
five millions of souls each, capable of supplanting her later in the
Mediterranean.[22] And if Prussia, taking advantage of the loss of
Italy and Hungary to her rival Austria, had united in a single political
and military body the scattered members of Germany, would it have
been useful and hopeful for France to feel herself pressed on the
other side by a kingdom or empire of fifty millions of inhabitants, a
military race of the first order?
Moreover, what would have become of the Roman Pontiff in this
renovation of countries, governments, and juridical laws. The Pope is
a great moral power, the greatest in the world. If his independence
were to give way before the principle of nationality, what would
become of his religious liberty, so necessary to the public quiet of
consciences. Could a pope, subject to an Italy constructed in any
way soever, increase the light, peace, and tranquillity of France and
the rest of Europe? Would the palace of the Vatican, changed into a
prison, have accorded with the imagined splendors of the Tuileries?
Finally, a new international and national right, which should have
sanctioned, in accordance with popular suffrage, the obligation of
non-intervention and accomplished facts, far from reconciling nations
and various classes of citizens among themselves by superseding the
inalienable right of nature, would have become a firebrand of civil
discord, an incentive to foreign wars, and a germ of revolutions
which would have plunged Europe into the horrors of socialism.
An eagle eye was not needed to see and foresee these weighty
dangers. However affairs might have turned, even if they had
succeeded according to every wish, it is indubitable that the ship of
Napoleonic politics, following in its navigation the star of this idea,
must eventually have struck on three rocks, each one hard enough
to send ship and pilot to the bottom: the Papacy, Germany, and
Revolution. The Papacy, oppressed by the Italy of the Carbonari,
would have taken from France her greatest moral force. Germany, in
one way or another, strongly united in her armies, would have tried,
as in 1813, to overwhelm the Empire. Revolution, kindled and fed
from without, would have gathered strength in France to the ruin of
the Empire.
These rocks were not only visible, but palpable to touch. Napoleon
III. saw them, felt them, and used all the licit and illicit arts of his
administration to avoid them. In vain; it was impossible. He should
not have followed the guidance of his enchantress, his idea;
following it, perdition was inevitable.

V.

Perhaps history offers no other example of a man who has grasped

the sceptre under conditions so propitious for good and so opposed
to evil as those under which Louis Napoleon Bonaparte began his
reign; or of one who has so pertinaciously abused his advantages to
his own ruin and that of others.
The vote of the better and larger portion of the French nation had
raised him to the throne, that he might save them from the hydra of
socialism, and stop the course of political changes in France. Europe,
just recovering from terrible agitations, welcomed his elevation as a
pledge of order and peace. Catholics of every country rejoiced over
it almost as the reward of the uncontested restoration in Rome of
the principality of S. Peter. Interest and conscience seemed to unite
in inducing him to take the triumphal road of justice which must lead
to certain glory.
But cum in honore esset non intellexit.[23] He seemed to wish to
take this path. But, in fact, he showed that he was preparing to
follow another by the ephemeral light of that idea which he
worshipped on the imperial throne with the same devotion which he
had professed in prison and in exile.
The Crimean war, to a participation in which he invited little
Piedmont, predestined by him to enjoy the benefits of Italian
resurrection, helped him to cut the knot of the Holy Alliance, to
humble Russia and set her at enmity with Austria, to create by a
plébiscite the first of his national unities—that of the Roumanian
Principalities—and to introduce at the Congress of Paris that
subalpine diplomacy which, endorsed by him, sowed the seeds of
the contemplated Italian war.
Meanwhile, the daggers and bombs of the Pianori, Tibaldi, and Orsini
came to remind him that, before being Emperor of the French, he
had been an Italian Carbonaro, and that he was expected to keep
his oaths. It is said that, after the explosion of Orsini's bombshell, a
friend of the assassin, to whom Napoleon complained confidentially
of this party persecution, replied: "You have forgotten that you are
an Italian."
"What shall I do?" asked his majesty.
"Serve your country."
"Very good. But I am Emperor of the French, a nation hard to
govern. Can I sacrifice the interests of my people to accommodate
those of Italy?"
"No one will prevent you from studying the interests of France when
you have promulgated the independence and secured the unity of
your country. Italy first of all."[24]
But he had less need of spurring than was supposed.
After the secret negotiations of Plombières, he attacked Austria in
the plains of Lombardy, and, having subdued her, he inaugurated the
resurrection of Italy according to his idea, which, presiding over the
work, showed itself unveiled, with all the magnificence of territorial
compensation, universal suffrage, non-intervention, and
accomplished facts, as we all know.

VI.

But the Napoleonic ship got lost irreparably among the three rocks
above named. Between the Mincio and the Adige it met Germany in
threatening guise; in Rome, the betrayed pontiff rose up; and in
Paris revolution lifted her savage head. For eleven years Bonaparte
struggled to save the ship from the straits into which his Italian
enterprise had driven it; but the more earnest his efforts, the worse
became the entanglement, until the tempest of 1870 split the vessel
in the midst with awful shipwreck.
His crimes towards the Pope, the ignoble artifice of insults couched
in reverential terms, of perfidy, lies, and hypocrisy, alienated from
him not only Catholics, but all those who honored human loyalty and
natural probity. The so-called Roman question, a compendium of the
whole Italian question, ruined the credit of Napoleon III., unmasked
him, and made him appear as inexorable history will show him to
posterity—a monster of immorality, to use the apt expression of one
of his former sycophants.[25]
Prussia, after checking him at the Mincio in 1859, cut short in his
hands the thread of the web woven in 1863 to regenerate Poland on
the plan of Italy. God did not permit a good and noble cause like
that of Poland to be contaminated by the influence of the Napoleonic
idea; and this seems to us an indication that he reserves to her a
restoration worthy of herself and of her faith. Prussia also held him
at bay during the Danish war, into which he threw himself with
closed eyes, in the mad hope of conquering Mexico, and making it
an empire after his own idea. This whim cost France a lake of blood,
many millions of francs, and an indelible stain; it cost the
unfortunate Maximilian of Austria his life, and his gifted wife her
reason. Prussia solemnly mocked at him in the other war of 1866,
when, leagued with Italy by his consent, she attacked the Austrian
Empire.
It was the beginning of that political and military unity of Germany
which was destined to make him pay dear for the work of unity
accomplished beyond the Alps by so many crimes.[26]
Lastly, Prussia, choosing the occasion of the vacancy of the Spanish
throne, and seconded by him in the promotion of an Iberian unity
like that of Italy, and prepared by a subalpine marriage, drew him
into the toils where he left his crown and his honor.
Step by step with the barriers opposed by Prussia to the foolish
policy of Napoleon III. in Europe went the anxieties caused in the
empire by revolution. Losing gradually the support of the honest
Catholic plurality of the French, he thought to reinforce himself by
flattering his enemy, demagogism, and by unchaining gradually
passions irreligious, anarchical, destructive to civilization. Taking all
restraint from the press, he removed every bar to theatrical license,
gave unchecked liberty to villany, free course to nefarious impiety
and a Babylonish libertinism, and finished by opening the doors to
public schools of socialism. But as outside France his duplicity and
cowardly frauds had drawn upon him the hatred and contempt of
accomplices and beneficiaries, so at home they excited discontent
and distrust among all parties.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com