Visit https://ebookultra.

com to download the full version and

explore more ebooks or textbooks

Optimization Algorithms on Matrix Manifolds P.-A.

Absil

_____ Click the link below to download _____

https://ebookultra.com/download/optimization-algorithms-on-
matrix-manifolds-p-a-absil/

Explore and download more ebooks or textbooks at ebookultra.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Convex Optimization Algorithms 1st Edition Dimitri P.

Bertsekas

https://ebookultra.com/download/convex-optimization-algorithms-1st-
edition-dimitri-p-bertsekas/

A Survey of Matrix Theory and Matrix Inequalities Marvin

Marcus

https://ebookultra.com/download/a-survey-of-matrix-theory-and-matrix-
inequalities-marvin-marcus/

Optimization Using Evolutionary Algorithms and

Metaheuristics Applications in Engineering 1st Edition
Kaushik Kumar
https://ebookultra.com/download/optimization-using-evolutionary-
algorithms-and-metaheuristics-applications-in-engineering-1st-edition-
kaushik-kumar/

Hyperfunctions on Hypo Analytic Manifolds AM 136 1st

Edition Paulo Cordaro

https://ebookultra.com/download/hyperfunctions-on-hypo-analytic-
manifolds-am-136-1st-edition-paulo-cordaro/
Wave Equations on Lorentzian Manifolds and Quantization
1st Edition Christian Bar

https://ebookultra.com/download/wave-equations-on-lorentzian-
manifolds-and-quantization-1st-edition-christian-bar/

The Design of Approximation Algorithms 1st Edition David

P. Williamson

https://ebookultra.com/download/the-design-of-approximation-
algorithms-1st-edition-david-p-williamson/

An Introduction to Optimization 2nd Edition Edwin K. P.

Chong

https://ebookultra.com/download/an-introduction-to-optimization-2nd-
edition-edwin-k-p-chong/

Existence and Regularity of Minimal Surfaces on Riemannian

Manifolds MN 27 Jon T. Pitts

https://ebookultra.com/download/existence-and-regularity-of-minimal-
surfaces-on-riemannian-manifolds-mn-27-jon-t-pitts/

Differentiable Manifolds 2nd Edition Lawrence Conlon

https://ebookultra.com/download/differentiable-manifolds-2nd-edition-
lawrence-conlon/
Optimization Algorithms on Matrix Manifolds P.-A. Absil
Digital Instant Download
Author(s): P.-A. Absil; R. Mahony; Rodolphe Sepulchre
ISBN(s): 9781400830244, 1400830249
Edition: Course Book
File Details: PDF, 1.18 MB
Year: 2009
Language: english
Optimization Algorithms on
Matrix Manifolds
This page intentionally left blank
Optimization Algorithms on
Matrix Manifolds

P.-A. Absil
R. Mahony
R. Sepulchre

PRINCETON UNIVERSITY PRESS

PRINCETON AND OXFORD

Copyright 
c 2008 by Princeton University Press

Published by Princeton University Press

41 William Street, Princeton, New Jersey 08540

In the United Kingdom: Princeton University Press

3 Market Place, Woodstock, Oxfordshire OX20 1SY

All Rights Reserved

Library of Congress Control Number: 2007927538

ISBN: 978-0-691-13298-3

British Library Cataloging-in-Publication Data is

available

This book has been composed in Computer Modern in

LATEX

The publisher would like to acknowledge the authors

of this volume for providing the camera-ready copy
from which this book was printed.

Printed on acid-free paper. ∞

press.princeton.edu

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1
To our parents
This page intentionally left blank
 Contents

List of Algorithms xi

Foreword, by Paul Van Dooren xiii

Notation Conventions xv

1. Introduction 1

2. Motivation and Applications 5

2.1 A case study: the eigenvalue problem 5
2.1.1 The eigenvalue problem as an optimization problem 7
2.1.2 Some benefits of an optimization framework 9
2.2 Research problems 10
2.2.1 Singular value problem 10
2.2.2 Matrix approximations 12
2.2.3 Independent component analysis 13
2.2.4 Pose estimation and motion recovery 14
2.3 Notes and references 16

3. Matrix Manifolds: First-Order Geometry 17

3.1 Manifolds 18
3.1.1 Definitions: charts, atlases, manifolds 18
3.1.2 The topology of a manifold* 20
3.1.3 How to recognize a manifold 21
3.1.4 Vector spaces as manifolds 22
3.1.5 The manifolds Rn×p and Rn×p ∗ 22
3.1.6 Product manifolds 23
3.2 Differentiable functions 24
3.2.1 Immersions and submersions 24
3.3 Embedded submanifolds 25
3.3.1 General theory 25
3.3.2 The Stiefel manifold 26
3.4 Quotient manifolds 27
3.4.1 Theory of quotient manifolds 27
3.4.2 Functions on quotient manifolds 29
3.4.3 The real projective space RPn−1 30
3.4.4 The Grassmann manifold Grass(p, n) 30
3.5 Tangent vectors and differential maps 32
viii CONTENTS

3.5.1 Tangent vectors 33

3.5.2 Tangent vectors to a vector space 35
3.5.3 Tangent bundle 36
3.5.4 Vector fields 36
3.5.5 Tangent vectors as derivations∗ 37
3.5.6 Differential of a mapping 38
3.5.7 Tangent vectors to embedded submanifolds 39
3.5.8 Tangent vectors to quotient manifolds 42
3.6 Riemannian metric, distance, and gradients 45
3.6.1 Riemannian submanifolds 47
3.6.2 Riemannian quotient manifolds 48
3.7 Notes and references 51

4. Line-Search Algorithms on Manifolds 54

4.1 Retractions 54
4.1.1 Retractions on embedded submanifolds 56
4.1.2 Retractions on quotient manifolds 59
4.1.3 Retractions and local coordinates* 61
4.2 Line-search methods 62
4.3 Convergence analysis 63
4.3.1 Convergence on manifolds 63
4.3.2 A topological curiosity* 64
4.3.3 Convergence of line-search methods 65
4.4 Stability of fixed points 66
4.5 Speed of convergence 68
4.5.1 Order of convergence 68
4.5.2 Rate of convergence of line-search methods* 70
4.6 Rayleigh quotient minimization on the
sphere 73
4.6.1 Cost function and gradient calculation 74
4.6.2 Critical points of the Rayleigh quotient 74
4.6.3 Armijo line search 76
4.6.4 Exact line search 78
4.6.5 Accelerated line search: locally optimal conjugate gradient 78
4.6.6 Links with the power method and inverse iteration 78
4.7 Refining eigenvector estimates 80
4.8 Brockett cost function on the Stiefel
manifold 80
4.8.1 Cost function and search direction 80
4.8.2 Critical points 81
4.9 Rayleigh quotient minimization on the
Grassmann manifold 83
4.9.1 Cost function and gradient calculation 83
4.9.2 Line-search algorithm 85
4.10 Notes and references 86

5. Matrix Manifolds: Second-Order Geometry 91

5.1 Newton’s method in Rn 91
5.2 Affine connections 93
CONTENTS ix

5.3 Riemannian connection 96

5.3.1 Symmetric connections 96
5.3.2 Definition of the Riemannian connection 97
5.3.3 Riemannian connection on Riemannian submanifolds 98
5.3.4 Riemannian connection on quotient manifolds 100
5.4 Geodesics, exponential mapping, and
parallel translation 101
5.5 Riemannian Hessian operator 104
5.6 Second covariant derivative* 108
5.7 Notes and references 110

6. Newton’s Method 111

6.1 Newton’s method on manifolds 111
6.2 Riemannian Newton method for real-valued functions 113
6.3 Local convergence 114
6.3.1 Calculus approach to local convergence analysis 117
6.4 Rayleigh quotient algorithms 118
6.4.1 Rayleigh quotient on the sphere 118
6.4.2 Rayleigh quotient on the Grassmann manifold 120
6.4.3 Generalized eigenvalue problem 121
6.4.4 The nonsymmetric eigenvalue problem 125
6.4.5 Newton with subspace acceleration: Jacobi-Davidson 126
6.5 Analysis of Rayleigh quotient algorithms 128
6.5.1 Convergence analysis 128
6.5.2 Numerical implementation 129
6.6 Notes and references 131

7. Trust-Region Methods 136

7.1 Models 137
7.1.1 Models in Rn 137
7.1.2 Models in general Euclidean spaces 137
7.1.3 Models on Riemannian manifolds 138
7.2 Trust-region methods 140
7.2.1 Trust-region methods in Rn 140
7.2.2 Trust-region methods on Riemannian manifolds 140
7.3 Computing a trust-region step 141
7.3.1 Computing a nearly exact solution 142
7.3.2 Improving on the Cauchy point 143
7.4 Convergence analysis 145
7.4.1 Global convergence 145
7.4.2 Local convergence 152
7.4.3 Discussion 158
7.5 Applications 159
7.5.1 Checklist 159
7.5.2 Symmetric eigenvalue decomposition 160
7.5.3 Computing an extreme eigenspace 161
7.6 Notes and references 165

8. A Constellation of Superlinear Algorithms 168

x CONTENTS

8.1 Vector transport 168

8.1.1 Vector transport and affine connections 170
8.1.2 Vector transport by differentiated retraction 172
8.1.3 Vector transport on Riemannian submanifolds 174
8.1.4 Vector transport on quotient manifolds 174
8.2 Approximate Newton methods 175
8.2.1 Finite difference approximations 176
8.2.2 Secant methods 178
8.3 Conjugate gradients 180
8.3.1 Application: Rayleigh quotient minimization 183
8.4 Least-square methods 184
8.4.1 Gauss-Newton methods 186
8.4.2 Levenberg-Marquardt methods 187
8.5 Notes and references 188

A. Elements of Linear Algebra, Topology, and Calculus 189

A.1 Linear algebra 189
A.2 Topology 191
A.3 Functions 193
A.4 Asymptotic notation 194
A.5 Derivatives 195
A.6 Taylor’s formula 198

Bibliography 201

Index 221
List of Algorithms

1 Accelerated Line Search (ALS) 63

2 Armijo line search for the Rayleigh quotient on S n−1 76
3 Armijo line search for the Rayleigh quotient on Grass(p, n) 86
4 Geometric Newton method for vector fields 112
5 Riemannian Newton method for real-valued functions 113
6 Riemannian Newton method for the Rayleigh quotient on
S n−1 119
7 Riemannian Newton method for the Rayleigh quotient on
Grass(p, n) 121
8 Riemannian Newton method for the Rayleigh quotient on
Grass(p, n) 124
9 Jacobi-Davidson 127
10 Riemannian trust-region (RTR) meta-algorithm 142
11 Truncated CG (tCG) method for the trust-region subprob-
lem 144
12 Truncated CG method for the generalized eigenvalue prob-
lem 164
13 Geometric CG method 182
14 Riemannian Gauss-Newton method 186
This page intentionally left blank
 Foreword

Constrained optimization is quite well established as an area of research, and

there exist several powerful techniques that address general problems in that
area. In this book a special class of constraints is considered, called geomet-
ric constraints, which express that the solution of the optimization problem
lies on a manifold. This is a recent area of research that provides powerful
alternatives to the more general constrained optimization methods. Clas-
sical constrained optimization techniques work in an embedded space that
can be of a much larger dimension than that of the manifold. Optimization
algorithms that work on the manifold have therefore a lower complexity and
quite often also have better numerical properties (see, e.g., the numerical
integration schemes that preserve invariants such as energy). The authors
refer to this as unconstrained optimization in a constrained search space.
The idea that one can describe difference or differential equations whose
solution lies on a manifold originated in the work of Brockett, Flaschka,
and Rutishauser. They described, for example, isospectral flows that yield
time-varying matrices which are all similar to each other and eventually
converge to diagonal matrices of ordered eigenvalues. These ideas did not
get as much attention in the numerical linear algebra community as in the
area of dynamical systems because the resulting difference and differential
equations did not lead immediately to efficient algorithmic implementations.
An important book synthesizing several of these ideas is Optimization and
Dynamical Systems (Springer, 1994), by Helmke and Moore, which focuses
on dynamical systems related to gradient flows that converge exponentially
to a stationary point that is the solution of some optimization problem.
The corresponding discrete-time version of this algorithm would then have
linear convergence, which seldom compares favorably with state-of-the-art
eigenvalue solvers.
The formulation of higher-order optimization methods on manifolds grew
out of these ideas. Some of the people that applied these techniques to ba-
sic linear algebra problems include Absil, Arias, Chu, Dehaene, Edelman,
Eldén, Gallivan, Helmke, Hüper, Lippert, Mahony, Manton, Moore, Sepul-
chre, Smith, and Van Dooren. It is interesting to see, on the other hand, that
several basic ideas in this area were also proposed by Luenberger and Gabay
in the optimization literature in the early 1980s, and this without any use
of dynamical systems.
In the present book the authors focus on higher-order methods and in-
clude Newton-type algorithms for optimization on manifolds. This requires
xiv FOREWORD

a lot more machinery, which cannot currently be found in textbooks. The

main focus of this book is on optimization problems related to invariant
subspaces of matrices, but this is sufficiently general to encompass well the
two main aspects of optimization on manifolds: the conceptual algorithm
and its convergence analysis based on ideas of differential geometry, and the
efficient numerical implementation using state-of-the-art numerical linear al-
gebra techniques.
The book is quite deep in the presentation of the machinery of differen-
tial geometry needed to develop higher-order optimization techniques, but it
nevertheless succeeds in explaining complicated concepts with simple ideas.
These ideas are then used to develop Newton-type methods as well as other
superlinear methods such as trust-region methods and inexact and quasi-
Newton methods, which precisely put more emphasis on the efficient numer-
ical implementation of the conceptual algorithms.
This is a research monograph in a field that is quickly gaining momentum.
The techniques are also being applied to areas of engineering and robotics, as
indicated in the book, and it sheds new light on methods such as the Jacobi-
Davidson method, which originally came from computational chemistry. The
book makes a lot of interesting connections and can be expected to generate
several new results in the future.

Paul Van Dooren January 2007

 Notation Conventions

M, N manifolds
x, y points on a manifold
ξ, η, ζ, χ tangent vectors or vector fields
ξx , ηx , ζx , χx tangent vectors at x
ϕ, ψ coordinate charts
A, B square matrices
W , X, Y , Z matrices
W, X , Y, Z linear subspaces

Conventions related to the definition of functions are stated in Section A.3.

This page intentionally left blank
 Chapter One

Introduction

This book is about the design of numerical algorithms for computational

problems posed on smooth search spaces. The work is motivated by matrix
optimization problems characterized by symmetry or invariance properties
in the cost function or constraints. Such problems abound in algorithmic
questions pertaining to linear algebra, signal processing, data mining, and
statistical analysis. The approach taken here is to exploit the special struc-
ture of these problems to develop efficient numerical procedures.
An illustrative example is the eigenvalue problem. Because of their scale in-
variance, eigenvectors are not isolated in vector spaces. Instead, each eigendi-
rection defines a linear subspace of eigenvectors. For numerical computation,
however, it is desirable that the solution set consist only of isolated points in
the search space. An obvious remedy is to impose a norm equality constraint
on iterates of the algorithm. The resulting spherical search space is an em-
bedded submanifold of the original vector space. An alternative approach is
to “factor” the vector space by the scale-invariant symmetry operation such
that any subspace becomes a single point. The resulting search space is a
quotient manifold of the original vector space. These two approaches provide
prototype structures for the problems considered in this book.
Scale invariance is just one of several symmetry properties regularly en-
countered in computational problems. In many cases, the underlying symme-
try property can be exploited to reformulate the problem as a nondegenerate
optimization problem on an embedded or quotient manifold associated with
the original matrix representation of the search space. These constraint sets
carry the structure of nonlinear matrix manifolds. This book provides the
tools to exploit such structure in order to develop efficient matrix algorithms
in the underlying total vector space.
Working with a search space that carries the structure of a nonlinear man-
ifold introduces certain challenges in the algorithm implementation. In their
classical formulation, iterative optimization algorithms rely heavily on the
Euclidean vector space structure of the search space; a new iterate is gen-
erated by adding an update increment to the previous iterate in order to
reduce the cost function. The update direction and step size are generally
computed using a local model of the cost function, typically based on (ap-
proximate) first and second derivatives of the cost function, at each step. In
order to define algorithms on manifolds, these operations must be translated
into the language of differential geometry. This process is a significant re-
search program that builds upon solid mathematical foundations. Advances
2 CHAPTER 1

in that direction have been dramatic over the last two decades and have led
to a solid conceptual framework. However, generalizing a given optimization
algorithm on an abstract manifold is only the first step towards the objective
of this book. Turning the algorithm into an efficient numerical procedure is a
second step that ultimately justifies or invalidates the first part of the effort.
At the time of publishing this book, the second step is more an art than a
theory.
Good algorithms result from the combination of insight from differential
geometry, optimization, and numerical analysis. A distinctive feature of this
book is that as much attention is paid to the practical implementation of
the algorithm as to its geometric formulation. In particular, the concrete
aspects of algorithm design are formalized with the help of the concepts of
retraction and vector transport, which are relaxations of the classical geomet-
ric concepts of motion along geodesics and parallel transport. The proposed
approach provides a framework to optimize the efficiency of the numerical
algorithms while retaining the convergence properties of their abstract geo-
metric counterparts.
The geometric material in the book is mostly confined to Chapters 3 and 5.
Chapter 3 presents an introduction to Riemannian manifolds and tangent
spaces that provides the necessary tools to tackle simple gradient-descent op-
timization algorithms on matrix manifolds. Chapter 5 covers the advanced
material needed to define higher-order derivatives on manifolds and to build
the analog of first- and second-order local models required in most optimiza-
tion algorithms. The development provided in these chapters ranges from
the foundations of differential geometry to advanced material relevant to
our applications. The selected material focuses on those geometric concepts
that are particular to the development of numerical algorithms on embed-
ded and quotient manifolds. Not all aspects of classical differential geometry
are covered, and some emphasis is placed on material that is nonstandard
or difficult to find in the established literature. A newcomer to the field of
differential geometry may wish to supplement this material with a classical
text. Suggestions for excellent texts are provided in the references.
A fundamental, but deliberate, omission in the book is a treatment of the
geometric structure of Lie groups and homogeneous spaces. Lie theory is
derived from the concepts of symmetry and seems to be a natural part of
a treatise such as this. However, with the purpose of reaching a community
without an extensive background in geometry, we have omitted this material
in the present book. Occasionally the Lie-theoretic approach provides an
elegant shortcut or interpretation for the problems considered. An effort
is made throughout the book to refer the reader to the relevant literature
whenever appropriate.
The algorithmic material of the book is interlaced with the geometric ma-
terial. Chapter 4 considers gradient-descent line-search algorithms. These
simple optimization algorithms provide an excellent framework within which
to study the important issues associated with the implementation of practi-
cal algorithms. The concept of retraction is introduced in Chapter 4 as a key
INTRODUCTION 3

step in developing efficient numerical algorithms on matrix manifolds. The

later chapters on algorithms provide the core results of the book: the devel-
opment of Newton-based methods in Chapter 6 and of trust-region methods
in Chapter 7, and a survey of other superlinear methods such as conjugate
gradients in Chapter 8. We attempt to provide a generic development of
each of these methods, building upon the material of the geometric chap-
ters. The methodology is then developed into concrete numerical algorithms
on specific examples. In the analysis of superlinear and second-order meth-
ods, the concept of vector transport (introduced in Chapter 8) is used to
provide an efficient implementation of methods such as conjugate gradient
and other quasi-Newton methods. The algorithms obtained in these sections
of the book are competitive with state-of-the-art numerical linear algebra
algorithms for certain problems.
The running example used throughout the book is the calculation of in-
variant subspaces of a matrix (and the many variants of this problem). This
example is by far, for variants of algorithms developed within the proposed
framework, the problem with the broadest scope of applications and the
highest degree of achievement to date. Numerical algorithms, based on a ge-
ometric formulation, have been developed that compete with the best avail-
able algorithms for certain classes of invariant subspace problems. These
algorithms are explicitly described in the later chapters of the book and,
in part, motivate the whole project. Because of the important role of this
class of problems within the book, the first part of Chapter 2 provides a
detailed description of the invariant subspace problem, explaining why and
how this problem leads naturally to an optimization problem on a matrix
manifold. The second part of Chapter 2 presents other applications that can
be recast as problems of the same nature. These problems are the subject
of ongoing research, and the brief exposition given is primarily an invitation
for interested researchers to join with us in investigating these problems and
expanding the range of applications considered.
The book should primarily be considered a research monograph, as it
reports on recently published results in an active research area that is ex-
pected to develop significantly beyond the material presented here. At the
same time, every possible effort has been made to make the book accessible
to the broadest audience, including applied mathematicians, engineers, and
computer scientists with little or no background in differential geometry. It
could equally well qualify as a graduate textbook for a one-semester course in
advanced optimization. More advanced sections that can be readily skipped
at a first reading are indicated with a star. Moreover, readers are encouraged
to visit the book home page1 where supplementary material is available.
The book is an extension of the first author’s Ph.D. thesis [Abs03], itself a
project that drew heavily on the material of the second author’s Ph.D. the-
sis [Mah94]. It would not have been possible without the many contributions
of a quickly expanding research community that has been working in the area

1 http://press.princeton.edu/titles/8586.html
4 CHAPTER 1

over the last decade. The Notes and References section at the end of each
chapter is an attempt to give proper credit to the many contributors, even
though this task becomes increasingly difficult for recent contributions. The
authors apologize for any omission or error in these notes. In addition, we
wish to conclude this introductory chapter with special acknowledgements
to people without whom this project would have been impossible. The 1994
monograph [HM94] by Uwe Helmke and John Moore is a milestone in the
formulation of computational problems as optimization algorithms on man-
ifolds and has had a profound influence on the authors. On the numerical
side, the constant encouragement of Paul Van Dooren and Kyle Gallivan
has provided tremendous support to our efforts to reconcile the perspectives
of differential geometry and numerical linear algebra. We are also grateful
to all our colleagues and friends over the last ten years who have crossed
paths as coauthors, reviewers, and critics of our work. Special thanks to Ben
Andrews, Chris Baker, Alan Edelman, Michiel Hochstenbach, Knut Hüper,
Jonathan Manton, Robert Orsi, and Jochen Trumpf. Finally, we acknowl-
edge the useful feedback of many students on preliminary versions of the
book, in particular, Mariya Ishteva, Michel Journée, and Alain Sarlette.
Other documents randomly have
different content
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright law
in the United States and you are located in the United States, we do
not claim a right to prevent you from copying, distributing,
performing, displaying or creating derivative works based on the
work as long as all references to Project Gutenberg are removed. Of
course, we hope that you will support the Project Gutenberg™
mission of promoting free access to electronic works by freely
sharing Project Gutenberg™ works in compliance with the terms of
this agreement for keeping the Project Gutenberg™ name associated
with the work. You can easily comply with the terms of this
agreement by keeping this work in the same format with its attached
full Project Gutenberg™ License when you share it without charge
with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the
terms of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.

1.E. Unless you have removed all references to Project Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project Gutenberg™
work (any work on which the phrase “Project Gutenberg” appears,
or with which the phrase “Project Gutenberg” is associated) is
accessed, displayed, performed, viewed, copied or distributed:
 This eBook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this eBook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is derived

from texts not protected by U.S. copyright law (does not contain a
notice indicating that it is posted with permission of the copyright
holder), the work can be copied and distributed to anyone in the
United States without paying any fees or charges. If you are
redistributing or providing access to a work with the phrase “Project
Gutenberg” associated with or appearing on the work, you must
comply either with the requirements of paragraphs 1.E.1 through
1.E.7 or obtain permission for the use of the work and the Project
Gutenberg™ trademark as set forth in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is posted

with the permission of the copyright holder, your use and distribution
must comply with both paragraphs 1.E.1 through 1.E.7 and any
additional terms imposed by the copyright holder. Additional terms
will be linked to the Project Gutenberg™ License for all works posted
with the permission of the copyright holder found at the beginning
of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files containing a
part of this work or any other work associated with Project
Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute this

electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the Project
Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if you
provide access to or distribute copies of a Project Gutenberg™ work
in a format other than “Plain Vanilla ASCII” or other format used in
the official version posted on the official Project Gutenberg™ website
(www.gutenberg.org), you must, at no additional cost, fee or
expense to the user, provide a copy, a means of exporting a copy, or
a means of obtaining a copy upon request, of the work in its original
“Plain Vanilla ASCII” or other form. Any alternate format must
include the full Project Gutenberg™ License as specified in
paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™ works
unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or providing

access to or distributing Project Gutenberg™ electronic works
provided that:

• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
 about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt
that s/he does not agree to the terms of the full Project
Gutenberg™ License. You must require such a user to return or
destroy all copies of the works possessed in a physical medium
and discontinue all use of and all access to other copies of
Project Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project Gutenberg™

electronic work or group of works on different terms than are set
forth in this agreement, you must obtain permission in writing from
the Project Gutenberg Literary Archive Foundation, the manager of
the Project Gutenberg™ trademark. Contact the Foundation as set
forth in Section 3 below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on, transcribe
and proofread works not protected by U.S. copyright law in creating
the Project Gutenberg™ collection. Despite these efforts, Project
Gutenberg™ electronic works, and the medium on which they may
be stored, may contain “Defects,” such as, but not limited to,
incomplete, inaccurate or corrupt data, transcription errors, a
copyright or other intellectual property infringement, a defective or
damaged disk or other medium, a computer virus, or computer
codes that damage or cannot be read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except for

the “Right of Replacement or Refund” described in paragraph 1.F.3,
the Project Gutenberg Literary Archive Foundation, the owner of the
Project Gutenberg™ trademark, and any other party distributing a
Project Gutenberg™ electronic work under this agreement, disclaim
all liability to you for damages, costs and expenses, including legal
fees. YOU AGREE THAT YOU HAVE NO REMEDIES FOR
NEGLIGENCE, STRICT LIABILITY, BREACH OF WARRANTY OR
BREACH OF CONTRACT EXCEPT THOSE PROVIDED IN PARAGRAPH
1.F.3. YOU AGREE THAT THE FOUNDATION, THE TRADEMARK
OWNER, AND ANY DISTRIBUTOR UNDER THIS AGREEMENT WILL
NOT BE LIABLE TO YOU FOR ACTUAL, DIRECT, INDIRECT,
CONSEQUENTIAL, PUNITIVE OR INCIDENTAL DAMAGES EVEN IF
YOU GIVE NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you

discover a defect in this electronic work within 90 days of receiving
it, you can receive a refund of the money (if any) you paid for it by
sending a written explanation to the person you received the work
from. If you received the work on a physical medium, you must
return the medium with your written explanation. The person or
entity that provided you with the defective work may elect to provide
a replacement copy in lieu of a refund. If you received the work
electronically, the person or entity providing it to you may choose to
give you a second opportunity to receive the work electronically in
lieu of a refund. If the second copy is also defective, you may
demand a refund in writing without further opportunities to fix the
problem.

1.F.4. Except for the limited right of replacement or refund set forth
in paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of damages.
If any disclaimer or limitation set forth in this agreement violates the
law of the state applicable to this agreement, the agreement shall be
interpreted to make the maximum disclaimer or limitation permitted
by the applicable state law. The invalidity or unenforceability of any
provision of this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the Foundation,

the trademark owner, any agent or employee of the Foundation,
anyone providing copies of Project Gutenberg™ electronic works in
accordance with this agreement, and any volunteers associated with
the production, promotion and distribution of Project Gutenberg™
electronic works, harmless from all liability, costs and expenses,
including legal fees, that arise directly or indirectly from any of the
following which you do or cause to occur: (a) distribution of this or
any Project Gutenberg™ work, (b) alteration, modification, or
additions or deletions to any Project Gutenberg™ work, and (c) any
Defect you cause.

Section 2. Information about the Mission

of Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new computers.
It exists because of the efforts of hundreds of volunteers and
donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project Gutenberg™’s
goals and ensuring that the Project Gutenberg™ collection will
remain freely available for generations to come. In 2001, the Project
Gutenberg Literary Archive Foundation was created to provide a
secure and permanent future for Project Gutenberg™ and future
generations. To learn more about the Project Gutenberg Literary
Archive Foundation and how your efforts and donations can help,
see Sections 3 and 4 and the Foundation information page at
www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-profit
501(c)(3) educational corporation organized under the laws of the
state of Mississippi and granted tax exempt status by the Internal
Revenue Service. The Foundation’s EIN or federal tax identification
number is 64-6221541. Contributions to the Project Gutenberg
Literary Archive Foundation are tax deductible to the full extent
permitted by U.S. federal laws and your state’s laws.

The Foundation’s business office is located at 809 North 1500 West,

Salt Lake City, UT 84116, (801) 596-1887. Email contact links and up
to date contact information can be found at the Foundation’s website
and official page at www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission of
increasing the number of public domain and licensed works that can
be freely distributed in machine-readable form accessible by the
widest array of equipment including outdated equipment. Many
small donations ($1 to $5,000) are particularly important to
maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws regulating

charities and charitable donations in all 50 states of the United
States. Compliance requirements are not uniform and it takes a
considerable effort, much paperwork and many fees to meet and
keep up with these requirements. We do not solicit donations in
locations where we have not received written confirmation of
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where

we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make

any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About

Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.
Project Gutenberg™ eBooks are often created from several printed
editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com