Get ebook downloads in full at ebookmeta.

com

Probabilistic Numerics Computation as Machine

Learning 1st Edition Philipp Hennig

https://ebookmeta.com/product/probabilistic-numerics-
computation-as-machine-learning-1st-edition-philipp-hennig/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Probabilistic Machine Learning: Advanced Topics Kevin P.

Murphy

https://ebookmeta.com/product/probabilistic-machine-learning-advanced-
topics-kevin-p-murphy/

ebookmeta.com

Probabilistic Machine Learning Advanced Topics Draft 1st

Edition Kevin P. Murphy

https://ebookmeta.com/product/probabilistic-machine-learning-advanced-
topics-draft-1st-edition-kevin-p-murphy/

ebookmeta.com

Probabilistic Machine Learning for Finance and Investing -

EARLY RELEASE 1st Edition Deepak Kanungo

https://ebookmeta.com/product/probabilistic-machine-learning-for-
finance-and-investing-early-release-1st-edition-deepak-kanungo/

ebookmeta.com

King of Queens King University 4 1st Edition Nikki

Pennington

https://ebookmeta.com/product/king-of-queens-king-university-4-1st-
edition-nikki-pennington/

ebookmeta.com
Mastering ARKit: Apple’s Augmented Reality App Development
Platform 1st Edition Jayven Nhan

https://ebookmeta.com/product/mastering-arkit-apples-augmented-
reality-app-development-platform-1st-edition-jayven-nhan/

ebookmeta.com

Computational Quantum Chemistry, Second Edition Ram Yatan

Prasad

https://ebookmeta.com/product/computational-quantum-chemistry-second-
edition-ram-yatan-prasad/

ebookmeta.com

Tabata Training: The Science and History of HIIT 1st

Edition Izumi Tabata

https://ebookmeta.com/product/tabata-training-the-science-and-history-
of-hiit-1st-edition-izumi-tabata/

ebookmeta.com

Southern African Wildlife 3rd Edition Mike Unwin

https://ebookmeta.com/product/southern-african-wildlife-3rd-edition-
mike-unwin/

ebookmeta.com

Best-Selling House Plans, 4th Edition: Over 360 Dream-Home

Plans in Full Color Creative Homeowner

https://ebookmeta.com/product/best-selling-house-plans-4th-edition-
over-360-dream-home-plans-in-full-color-creative-homeowner/

ebookmeta.com
Advanced Nanomaterials and Their Applications in Renewable
Energy 2nd Edition Tian-Hao Yan

https://ebookmeta.com/product/advanced-nanomaterials-and-their-
applications-in-renewable-energy-2nd-edition-tian-hao-yan/

ebookmeta.com
 Probabilistic Numerics
Probabilistic numerical computation formalises the connection between machine learning and
applied mathematics. Numerical algorithms approximate intractable quantities from computable
ones. They estimate integrals from evaluations of the integrand, or the path of a dynamical system
described by differential equations from evaluations of the vector field. In other words, they
infer a latent quantity from data. This book shows that it is thus formally possible to think of
computational routines as learning machines, and to use the notion of Bayesian inference to build
more flexible, efficient, or customised algorithms for computation.
The text caters for Masters’ and PhD students, as well as postgraduate researchers in artificial
intelligence, computer science, statistics, and applied mathematics. Extensive background material
is provided along with a wealth of figures, worked examples, and exercises (with solutions) to
develop intuition.

Philipp Hennig holds the Chair for the Methods of Machine Learning at the University of
Tübingen, and an adjunct position at the Max Planck Institute for Intelligent Systems. He has
dedicated most of his career to the development of Probabilistic Numerical Methods. Hennig’s
research has been supported by Emmy Noether, Max Planck and ERC fellowships. He is a co-
Director of the Research Program for the Theory, Algorithms and Computations of Learning
Machines at the European Laboratory for Learning and Intelligent Systems (ELLIS).
Michael A. Osborne is Professor of Machine Learning at the University of Oxford, and a co-
Founder of Mind Foundry Ltd. His research has attracted £10.6M of research funding and has
been cited over 15,000 times. He is very, very Bayesian.
Hans P. Kersting is a postdoctoral researcher at INRIA and École Normale Supérieure in Paris,
working in machine learning with expertise in Bayesian inference, dynamical systems, and
optimisation.

Published online by Cambridge University Press

 ‘This impressive text rethinks numerical problems through the lens of probabilistic inference and
decision making. This fresh perspective opens up a new chapter in this field, and suggests new and
highly efficient methods. A landmark achievement!’
– Zoubin Ghahramani, University of Cambridge

‘In this stunning and comprehensive new book, early developments from Kac and Larkin have been
comprehensively built upon, formalised, and extended by including modern-day machine learn-
ing, numerical analysis, and the formal Bayesian statistical methodology. Probabilistic numerical
methodology is of enormous importance for this age of data-centric science and Hennig, Osborne,
and Kersting are to be congratulated in providing us with this definitive volume.’
– Mark Girolami, University of Cambridge and The Alan Turing Institute

‘This book presents an in-depth overview of both the past and present of the newly emerging
area of probabilistic numerics, where recent advances in probabilistic machine learning are used to
develop principled improvements which are both faster and more accurate than classical numerical
analysis algorithms. A must-read for every algorithm developer and practitioner in optimization!’
– Ralf Herbrich, Hasso Plattner Institute

‘Probabilistic numerics spans from the intellectual fireworks of the dawn of a new field to its
practical algorithmic consequences. It is precise but accessible and rich in wide-ranging, principled
examples. This convergence of ideas from diverse fields in lucid style is the very fabric of good
science.’
– Carl Edward Rasmussen, University of Cambridge

‘An important read for anyone who has thought about uncertainty in numerical methods; an
essential read for anyone who hasn’t’
– John Cunningham, Columbia University

‘This is a rare example of a textbook that essentially founds a new field, re-casting numerics on
stronger, more general foundations. A tour de force.’
– David Duvenaud, University of Toronto

‘The authors succeed in demonstrating the potential of probabilistic numerics to transform the way
we think about computation itself.’
– Thore Graepel, Senior Vice President, Altos Labs

Published online by Cambridge University Press

 PHILIPP HENNIG
Eberhard–Karls–Universität Tübingen, Germany
MICHAEL A. OSBORNE
University of Oxford
H A N S P. K E R S T I N G
École Normale Supérieure, Paris

PROBABILISTIC NUMERICS
C O M P U TAT I O N A S M A C H I N E L E A R N I N G

Published online by Cambridge University Press

 University Printing House, Cambridge CB2 8BS, United Kingdom
One Liberty Plaza, 20th Floor, New York, NY 10006, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
314–321, 3rd Floor, Plot 3, Splendor Forum, Jasola District Centre, New Delhi – 110025, India
103 Penang Road, #05-06/07, Visioncrest Commercial, Singapore 238467

Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of
education, learning, and research at the highest international levels of excellence.

www.cambridge.org
Information on this title:www.cambridge.org/9781107163447
DOI: 10.1017/9781316681411
© Philipp Hennig, Michael A. Osborne and Hans P. Kersting 2022
This publication is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.
First published 2022
Printed in the United Kingdom by TJ Books Limited, Padstow Cornwall
A catalogue record for this publication is available from the British Library.
ISBN 978-1-107-16344-7 Hardback
Cambridge University Press has no responsibility for the persistence or accuracy of
URLs for external or third-party internet websites referred to in this publication
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.

Published online by Cambridge University Press

 To our families.

Published online by Cambridge University Press

 Measurement owes its existence to Earth
Estimation of quantity to Measurement
Calculation to Estimation of quantity
Balancing of chances to Calculation
and Victory to Balancing of chances.

Sun Tzu – The Art of War

§4.18: Tactical Dispositions
Translation by Lionel Giles, 1910

Published online by Cambridge University Press

 Contents
Acknowledgements page ix
Symbols and Notation xi
Introduction 1

I Mathematical Background 17
1 Key Points 19
2 Probabilistic Inference 21
3 Gaussian Algebra 23
4 Regression 27
5 Gauss–Markov Processes: Filtering and SDEs 41
6 Hierarchical Inference in Gaussian Models 55
7 Summary of Part I 61

II Integration 63
8 Key Points 65
9 Introduction 69
10 Bayesian Quadrature 75
11 Links to Classical Quadrature 87
12 Probabilistic Numerical Lessons from Integration 107
13 Summary of Part II and Further Reading 119

III Linear Algebra 123

14 Key Points 125
15 Required Background 127
16 Introduction 131
17 Evaluation Strategies 137
18 A Review of Some Classic Solvers 143
19 Probabilistic Linear Solvers: Algorithmic Scaffold 149
20 Computational Constraints 169
21 Uncertainty Calibration 175

Published online by Cambridge University Press

 viii Contents

22 Proofs 183
23 Summary of Part III 193

IV Local Optimisation 195

24 Key Points 197
25 Problem Setting 199
26 Step-Size Selection – a Case Study 203
27 Controlling Optimisation by Probabilistic Estimates 221
28 First- and Second-Order Methods 229

V Global Optimisation 243

29 Key Points 245
30 Introduction 247
31 Bayesian Optimisation 251
32 Value Loss 259
33 Other Acquisition Functions 267
34 Further Topics 275

VI Solving Ordinary Differential Equations 279

35 Key Points 281
36 Introduction 285
37 Classical ODE Solvers as Regression Methods 289
38 ODE Filters and Smoothers 295
39 Theory of ODE Filters and Smoothers 317
40 Perturbative Solvers 331
41 Further Topics 339

VII The Frontier 349

42 So What? 351

VIII Solutions to Exercises 357

References 369
Index 395

Published online by Cambridge University Press

 Acknowledgements

Many people helped in the preparation of this book. We, the

authors, extend our gratitude to the following people, without
whom this book would have been impossible.
We are particularly grateful to Mark Girolami for his involve-
ment during the early stages of this book as a project. Though
he could not join as an author in the end, he provided a lot of
support and motivation to make this book a reality.
We would like to deeply thank the many people who offered
detailed and thoughtful comments on drafts of the book: On-
drej Bajgar, Nathanael Bosch, Jon Cockayne, Michael Cohen,
Paul Duckworth, Nina Effenberger, Carl Henrik Ek, Giacomo
Garegnani, Roman Garnett, Alexandra Gessner, Saad Hamid,
Marius Hobbhahn, Toni Karvonen, Nicholas Krämer, Emilia
Magnani, Chris Oates, Jonathan Schmidt, Sebastian Schulze,
Thomas Schön, Arno Solin, Tim J. Sullivan, Simo Särkkä, Filip
Tronarp, Ed Wagstaff, Xingchen Wan, and Richard Wilkinson.

Philipp Hennig

I would like to thank my research group, not just for thorough

proof-reading, but for an intense research effort that contributed
substantially to the results presented in this book. And, above
all, for wagering some of their prime years, and their career, on
me, and on the idea of probabilistic numerics:
Edgar Klenske, Maren Mahsereci, Michael Schober, Simon
Bartels, Lukas Balles, Alexandra Gessner, Filip de Roos, Frank
Schneider, Emilia Magnani, Niklas Wahl and Hans-Peter Wieser,
Felix Dangel, Frederik Künstner, Jonathan Wenger, Agustinus
Kristiadi, Nicholas Krämer, Nathanael Bosch, Lukas Tatzel,
Thomas Gläßle, Julia Grosse, Katharina Ott, Marius Hobbhahn,
Motonobu Kanagawa, Filip Tronarp, Robin Schmidt, Jonathan
Schmidt, Marvin Pförtner, Nina Effenberger, and Franziska
Weiler.
I am particularly grateful to those among this group who

https://doi.org/10.1017/9781316681411.001 Published online by Cambridge University Press

 x Acknowledgements

have contributed significantly to the development of the probnum

library, which would not exist at all, not even in a minimal state,
without their commitment.
Last but not least, I am grateful to my wife Maike Kaufman.
And to my daughters Friederike und Iris. They both arrived
while we worked on this book and drastically slowed down
progress on it in the most wonderful way possible.

Michael A. Osborne

I would like to thank Isis Hjorth, for being the most valuable
source of support I have in life, and our amazing children
Osmund and Halfdan – I wonder what you will think of this
book in a few years?

Hans P. Kersting

I would like to thank my postdoc adviser, Francis Bach, for

giving me the freedom to allocate sufficient time to this book.
I am grateful to Dana Babin, my family, and my friends for
their continuous love and support.

https://doi.org/10.1017/9781316681411.001 Published online by Cambridge University Press

 Symbols and Notation xi

Symbols and Notation

Bold symbols (x) are used for vectors, but only where the fact that a variable is a vector is relevant.
Square brackets indicate elements of a matrix or vector: if x = [ x1 , . . . , x N ] is a row vector, then
[ x]i = xi denotes its entries; if A ∈ R n×m is a matrix, then [ A]ij = Aij denotes its entries. Round
brackets (·) are used in most other cases (as in the notations listed below).

Notation Meaning
a∝c a is proportional to c: there is a constant k such that a = k · c.
A ∧ B, A ∨ B The logical conjunctions “and” and “or”; i.e. A ∧ B is true iff
both A and B are true, A ∨ B is true iff ¬ A ∧ ¬ B is false.
AB The Kronecker product of matrices A, B. See Eq. (15.2).
A
B The symmetric Kronecker product. See Eq. (19.16).
AB The element-wise product (aka Hadamard product) of two
matrices A and B of the same shape, i.e. [ A  B]ij = [ A]ij · [ B]ij .
#– #– #–
A,  A A is the vector arising from stacking the elements of a matrix A
#–
row after row, and its inverse (A =  A). See Eq. (15.1).
cov p ( x, y) The covariance of x and y under p. That is,
cov p ( x, y) := E p ( x · y) − E p ( x )E p (y).
C q (V, R d ) The set of q-times continuously differentiable functions from
V to R d , for some q, d ∈ N.
δ( x − y) The Dirac delta, heuristically characterised by the property

f ( x )δ( x − y) dx = f (y) for functions f : R  R.
δij The Kronecker symbol: δij = 1 if i = j, otherwise δij = 0.
det( A) The determinant of a square matrix A.
diag( x) The diagonal matrix with entries [diag( x)]ij = δij [ x ]i .
dωt The notation for an an Itô integral in a stochastic differential
equation. See Definition 5.4.
x
erf ( x ) The error function erf( x ) := √2π 0 exp(−t2 ) dt.

Ep( f ) The expectation of f under p. That is, E p ( f ) := f ( x ) dp( x ).
E |Y ( f ) The expectation of f under p( f | Y ).
∞
Γ(z) The Gamma function Γ(z) := 0 x z−1 exp(− x ) dx. See
Eq. (6.1).
G(·; a, b) The Gamma distribution with shape a > 0 and rate b > 0, with
a a −1
probability density function G(z; a, b) := bΓz( a) e−bz .
GP ( f ; μ, k) The Gaussian process measure on f with mean function μ and
covariance function (kernel) k. See §4.2
H p (x) The (differential) entropy of the distribution p( x ).

That is, H p ( x ) := − p( x ) log p( x ) dx. See Eq. (3.2).
H ( x | y) The (differential) entropy of the cond. distribution p( x | y).
That is, H ( x | y) := H p(·|y) ( x ).
I ( x ; y) The mutual information between random variables X and Y.
That is, I ( x ; y) := H ( x ) − H ( x | y) = H (y) − H (y | x ).

https://doi.org/10.1017/9781316681411.002 Published online by Cambridge University Press

 xii Symbols and Notation

Notation Meaning
I, IN The identity matrix (of dimensionality N): [ I ]ij = δij .
I (· ∈ A) The indicator function of a set A.
Kν The modified Bessel function for some parameter ν ∈ C.
∞
That is, Kν ( x ) := 0 exp(− x · cosh(t))cosh(νt) dt.
L The loss function of an optimization problem (§26.1), or the
log-likelihood of an inverse problem (§41.2).
M The model M capturing the probabilistic relationship between
the latent object and computable quantities. See §9.3.
N, C, R, R + The natural numbers (excluding zero), the complex numbers,
the real numbers, and the positive real numbers, respectively.
N ( x; μ, Σ) = p( x ) The vector x has the Gaussian probability density function
with mean vector μ and covariance matrix Σ. See Eq. (3.1).
N (μ, Σ) ∼ X The random variable X is distributed according to a Gaussian
distribution with mean μ and covariance Σ.
O(·) Landau big-Oh: for functions f , g defined on N, the notation
f (n) = O( g(n)) means that f (n)/g(n) is bounded for n  ∞.
p(y | x ) The conditional the probability density function for variable Y
having value y conditioned on variable X having value x.
rk( A) The rank of a matrix A.
span{ x1 , . . . , xn } The linear span of { x1 , . . . , xn }.
St(·; μ, λ1 , λ1 ) The Student’s-t probability density function with parameters
μ ∈ R and λ1 , λ2 > 0, see Eq. (6.9).
tr( A) The trace of matrix A, That is, tr( A) = ∑i [ A]ii .
A The transpose of matrix A: [ A ]ij = [ A] ji .
U a,b The uniform distribution with probability density function
p(u) := I (u ∈ ( a, b)), for a < b.
V p (x) The variance of x under p. That is, V p ( x ) := cov p ( x, x ).
V |Y ( f ) The variance of f under p( f | Y ). That is, H ( x | y) :=

− log p( x | y) dp( x | y).
W (V, ν) The Wishart distribution with probability density function
−1
W ( x; V, ν) ∝ | x |(ν− N −1)/2 e−1/2 tr(V x) . See Eq. (19.1).
x⊥y x is orthogonal to y, i.e. x, y = 0.
x := a The object x is defined to be equal to a.
Δ
x=a The object x is equal to a by virtue of its definition.
xa The object x is assigned the value of a (used in pseudo-code).
X∼p The random variable X is distributed according to p.
1, 1d A column vector of d ones, 1d := [1, . . . , 1] ∈ R d .
∇ x f ( x, t) The gradient of f w.r.t. x. (We omit subscript x if redundant.)

https://doi.org/10.1017/9781316681411.002 Published online by Cambridge University Press

 Introduction

The Uncertain Nature of Computation

Computation is the resource that has most transformed our

age. Its application has established and extracted value from
globe-spanning networks, mobile communication, and big data.
As its importance to humankind continues to rise, computation
has come at a substantial cost. Consider machine learning, the
academic discipline that has underpinned many recent tech-
nological advances. The monetary costs of modern massively
parallel graphics processing units (GPUs), that have proved so
valuable to much of machine learning, are prohibitive even to
many researchers and prevent the use of advanced machine
learning in embedded and other resource-limited systems. Even
where there aren’t hard resource constraints, there will always
be an economic incentive to reduce computation use. Trou-
blingly, there is evidence1 that current machine learning models 1
R. Schwartz et al. “Green AI” (2019);
D. Patterson. “The Carbon Footprint of
are also the cause of avoidable carbon emissions. Computations
Machine Learning Training Will Plateau,
have to become not just faster, but more efficient. Then Shrink” (2022).
Much of the computation consumption, particularly within
machine learning, is due to problems like solving linear equa-
tions, evaluating integrals, or finding the minimum of nonlinear
functions, all of which will be addressed in different chapters
in this text. These so-called numerical problems are studied by
the mathematical subfield of numerics (short for: numerical analy-
sis). What unites these problems is that their solutions, which
are numbers, have no analytic form. There is no known way to
assign an exact numerical value to them purely by structured,
rule-based thought. Methods to compute such numbers, by a
computer or on paper, are of an approximate nature. Their re-
sults are not exactly right, and we do not know precisely how
far off they are from the true solution; otherwise we could just
add that known difference (or error) and be done.

That we are thus uncertain about the answer to a numerical

https://doi.org/10.1017/9781316681411.003 Published online by Cambridge University Press

 2 Introduction

problem is one of the two central insights of probabilistic numerics

(pn). Accordingly, any numerical solver will be uncertain about
the accuracy of its output – and, in some cases, also about
its intermediate steps. These uncertainties are not ignored by
classical numerics, but typically reduced to scalar bounds. As
a remedy, pn brings statistical tools for the quantification of
uncertainty to the domain of numerics.
What, precisely, are the benefits of quantifying this numerical
uncertainty with probability measures?
For a start, a full probability distribution is a richer output
than a sole approximation (point estimate). This is particularly
useful in sequences of computations where a numerical solu-
tion is an input to the next computational task – as is the case
in many applications, ranging from the elementary (matrix in-
version for least squares) to the complex (solving differential
equations for real-word engineering systems). In these settings,
a probabilistic output distribution provides the means to propa-
gate uncertainty to subsequent steps.
But pn’s uncertainty-aware approach does not stop with
quantifying the uncertainty over the final output. Rather, it offers
an uncertainty-aware alternative to the design of numerical
methods in two ways:
Oftentimes, numerical uncertainty already appears within nu-
merical algorithms because its intermediate values are subject to
approximations themselves. The resulting intra-algorithmic accu-
mulation of such uncertainties calls for an appropriate amount
of caution. It appears, e.g., whenever expensive function evalua-
tions are replaced by cheaper simulations (such as when cheaper
surrogate functions are used in global optimisation); or when
imprecise steps are iterated (such as when ODE solvers concate-
nate their extrapolation steps). Probabilistic numerical methods
account for these uncertainties using probability measures. This
enables such methods to make smarter uncertainty-aware deci-
sions, which becomes most salient through their formulation as
probabilistic agents (as detailed below).
Moreover, probability distributions allow to more precisely
encode the expected structure of the numerical problem into
the solver: Numerical tasks can be solved by any number of
algorithms, and it is difficult to choose among them. Not all
algorithms for, say, integration, work on all integration prob-
lems. Some require the integrand to be highly regular, others
only that it be continuous, or even just integrable at all. If their
requirements are met, they produce a sequence of numbers
that converge to the intractable solution. But they do not all

https://doi.org/10.1017/9781316681411.003 Published online by Cambridge University Press

Other documents randomly have
different content
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the free

distribution of electronic works, by using or distributing this work (or
any other work associated in any way with the phrase “Project
Gutenberg”), you agree to comply with all the terms of the Full
Project Gutenberg™ License available with this file or online at
www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand, agree
to and accept all the terms of this license and intellectual property
(trademark/copyright) agreement. If you do not agree to abide by all
the terms of this agreement, you must cease using and return or
destroy all copies of Project Gutenberg™ electronic works in your
possession. If you paid a fee for obtaining a copy of or access to a
Project Gutenberg™ electronic work and you do not agree to be
bound by the terms of this agreement, you may obtain a refund from
the person or entity to whom you paid the fee as set forth in
paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only be

used on or associated in any way with an electronic work by people
who agree to be bound by the terms of this agreement. There are a
few things that you can do with most Project Gutenberg™ electronic
works even without complying with the full terms of this agreement.
See paragraph 1.C below. There are a lot of things you can do with
Project Gutenberg™ electronic works if you follow the terms of this
agreement and help preserve free future access to Project
Gutenberg™ electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright law in
the United States and you are located in the United States, we do
not claim a right to prevent you from copying, distributing,
performing, displaying or creating derivative works based on the
work as long as all references to Project Gutenberg are removed. Of
course, we hope that you will support the Project Gutenberg™
mission of promoting free access to electronic works by freely
sharing Project Gutenberg™ works in compliance with the terms of
this agreement for keeping the Project Gutenberg™ name
associated with the work. You can easily comply with the terms of
this agreement by keeping this work in the same format with its
attached full Project Gutenberg™ License when you share it without
charge with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the terms
of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.

1.E. Unless you have removed all references to Project Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project Gutenberg™
work (any work on which the phrase “Project Gutenberg” appears, or
with which the phrase “Project Gutenberg” is associated) is
accessed, displayed, performed, viewed, copied or distributed:
 This eBook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this eBook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is derived

from texts not protected by U.S. copyright law (does not contain a
notice indicating that it is posted with permission of the copyright
holder), the work can be copied and distributed to anyone in the
United States without paying any fees or charges. If you are
redistributing or providing access to a work with the phrase “Project
Gutenberg” associated with or appearing on the work, you must
comply either with the requirements of paragraphs 1.E.1 through
1.E.7 or obtain permission for the use of the work and the Project
Gutenberg™ trademark as set forth in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is posted

with the permission of the copyright holder, your use and distribution
must comply with both paragraphs 1.E.1 through 1.E.7 and any
additional terms imposed by the copyright holder. Additional terms
will be linked to the Project Gutenberg™ License for all works posted
with the permission of the copyright holder found at the beginning of
this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files containing a
part of this work or any other work associated with Project
Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute this

electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1 with
active links or immediate access to the full terms of the Project
Gutenberg™ License.
1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if you
provide access to or distribute copies of a Project Gutenberg™ work
in a format other than “Plain Vanilla ASCII” or other format used in
the official version posted on the official Project Gutenberg™ website
(www.gutenberg.org), you must, at no additional cost, fee or expense
to the user, provide a copy, a means of exporting a copy, or a means
of obtaining a copy upon request, of the work in its original “Plain
Vanilla ASCII” or other form. Any alternate format must include the
full Project Gutenberg™ License as specified in paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™ works
unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or providing

access to or distributing Project Gutenberg™ electronic works
provided that:

• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using
the method you already use to calculate your applicable
taxes. The fee is owed to the owner of the Project
Gutenberg™ trademark, but he has agreed to donate
royalties under this paragraph to the Project Gutenberg
Literary Archive Foundation. Royalty payments must be
paid within 60 days following each date on which you
prepare (or are legally required to prepare) your periodic tax
returns. Royalty payments should be clearly marked as
such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4,
“Information about donations to the Project Gutenberg
Literary Archive Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt
that s/he does not agree to the terms of the full Project
 Gutenberg™ License. You must require such a user to
return or destroy all copies of the works possessed in a
physical medium and discontinue all use of and all access
to other copies of Project Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full

refund of any money paid for a work or a replacement copy,
if a defect in the electronic work is discovered and reported
to you within 90 days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project Gutenberg™

electronic work or group of works on different terms than are set
forth in this agreement, you must obtain permission in writing from
the Project Gutenberg Literary Archive Foundation, the manager of
the Project Gutenberg™ trademark. Contact the Foundation as set
forth in Section 3 below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on, transcribe
and proofread works not protected by U.S. copyright law in creating
the Project Gutenberg™ collection. Despite these efforts, Project
Gutenberg™ electronic works, and the medium on which they may
be stored, may contain “Defects,” such as, but not limited to,
incomplete, inaccurate or corrupt data, transcription errors, a
copyright or other intellectual property infringement, a defective or
damaged disk or other medium, a computer virus, or computer
codes that damage or cannot be read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except

for the “Right of Replacement or Refund” described in paragraph
1.F.3, the Project Gutenberg Literary Archive Foundation, the owner
of the Project Gutenberg™ trademark, and any other party
distributing a Project Gutenberg™ electronic work under this
agreement, disclaim all liability to you for damages, costs and
expenses, including legal fees. YOU AGREE THAT YOU HAVE NO
REMEDIES FOR NEGLIGENCE, STRICT LIABILITY, BREACH OF
WARRANTY OR BREACH OF CONTRACT EXCEPT THOSE
PROVIDED IN PARAGRAPH 1.F.3. YOU AGREE THAT THE
FOUNDATION, THE TRADEMARK OWNER, AND ANY
DISTRIBUTOR UNDER THIS AGREEMENT WILL NOT BE LIABLE
TO YOU FOR ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL,
PUNITIVE OR INCIDENTAL DAMAGES EVEN IF YOU GIVE
NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you

discover a defect in this electronic work within 90 days of receiving it,
you can receive a refund of the money (if any) you paid for it by
sending a written explanation to the person you received the work
from. If you received the work on a physical medium, you must
return the medium with your written explanation. The person or entity
that provided you with the defective work may elect to provide a
replacement copy in lieu of a refund. If you received the work
electronically, the person or entity providing it to you may choose to
give you a second opportunity to receive the work electronically in
lieu of a refund. If the second copy is also defective, you may
demand a refund in writing without further opportunities to fix the
problem.

1.F.4. Except for the limited right of replacement or refund set forth in
paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of damages.
If any disclaimer or limitation set forth in this agreement violates the
law of the state applicable to this agreement, the agreement shall be
interpreted to make the maximum disclaimer or limitation permitted
by the applicable state law. The invalidity or unenforceability of any
provision of this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the

Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and distribution
of Project Gutenberg™ electronic works, harmless from all liability,
costs and expenses, including legal fees, that arise directly or
indirectly from any of the following which you do or cause to occur:
(a) distribution of this or any Project Gutenberg™ work, (b)
alteration, modification, or additions or deletions to any Project
Gutenberg™ work, and (c) any Defect you cause.

Section 2. Information about the Mission of

Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new computers.
It exists because of the efforts of hundreds of volunteers and
donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project Gutenberg™’s
goals and ensuring that the Project Gutenberg™ collection will
remain freely available for generations to come. In 2001, the Project
Gutenberg Literary Archive Foundation was created to provide a
secure and permanent future for Project Gutenberg™ and future
generations. To learn more about the Project Gutenberg Literary
Archive Foundation and how your efforts and donations can help,
see Sections 3 and 4 and the Foundation information page at
www.gutenberg.org.
 Section 3. Information about the Project
Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-profit
501(c)(3) educational corporation organized under the laws of the
state of Mississippi and granted tax exempt status by the Internal
Revenue Service. The Foundation’s EIN or federal tax identification
number is 64-6221541. Contributions to the Project Gutenberg
Literary Archive Foundation are tax deductible to the full extent
permitted by U.S. federal laws and your state’s laws.

The Foundation’s business office is located at 809 North 1500 West,

Salt Lake City, UT 84116, (801) 596-1887. Email contact links and up
to date contact information can be found at the Foundation’s website
and official page at www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission of
increasing the number of public domain and licensed works that can
be freely distributed in machine-readable form accessible by the
widest array of equipment including outdated equipment. Many small
donations ($1 to $5,000) are particularly important to maintaining tax
exempt status with the IRS.

The Foundation is committed to complying with the laws regulating

charities and charitable donations in all 50 states of the United
States. Compliance requirements are not uniform and it takes a
considerable effort, much paperwork and many fees to meet and
keep up with these requirements. We do not solicit donations in
locations where we have not received written confirmation of
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where

we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make

any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.

Project Gutenberg™ eBooks are often created from several printed

editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.
back