Download Full Version ebookmass - Visit ebookmass.

com

Optimizations and Programming: Linear, Non-linear,

Dynamic, Stochastic and Applications with Matlab
Abdelkhalak El Hami

https://ebookmass.com/product/optimizations-and-programming-
linear-non-linear-dynamic-stochastic-and-applications-with-
matlab-abdelkhalak-el-hami/

OR CLICK HERE

DOWLOAD NOW

Discover More Ebook - Explore Now at ebookmass.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Reliability of High-Power Mechatronic Systems 2: Aerospace

and Automotive Applications Issues,Testing and Analysis
Abdelkhalak El Hami (Editor)
https://ebookmass.com/product/reliability-of-high-power-mechatronic-
systems-2-aerospace-and-automotive-applications-issuestesting-and-
analysis-abdelkhalak-el-hami-editor/
ebookmass.com

Linear Algebra and Its Applications, Global Edition Lay

https://ebookmass.com/product/linear-algebra-and-its-applications-
global-edition-lay/

ebookmass.com

Reliability Of High Power Mechatronics Systems 1

Abdelkhalak El Hami (Editor)

https://ebookmass.com/product/reliability-of-high-power-mechatronics-
systems-1-abdelkhalak-el-hami-editor/

ebookmass.com

Pediatric Primary Care 6th Edition Catherine E. Burns

(Editor)

https://ebookmass.com/product/pediatric-primary-care-6th-edition-
catherine-e-burns-editor/

ebookmass.com
The Child Psychotherapy Progress Notes Planner
(PracticePlanners) 5th Edition – Ebook PDF Version

https://ebookmass.com/product/the-child-psychotherapy-progress-notes-
planner-practiceplanners-5th-edition-ebook-pdf-version/

ebookmass.com

The War Nurse Tracey Enerson Wood

https://ebookmass.com/product/the-war-nurse-tracey-enerson-wood/

ebookmass.com

(eTextbook PDF) for Starting Out with Java: From Control

Structures through Objects, 7th Edition

https://ebookmass.com/product/etextbook-pdf-for-starting-out-with-
java-from-control-structures-through-objects-7th-edition/

ebookmass.com

A Line in the Sand Teri Wilson

https://ebookmass.com/product/a-line-in-the-sand-teri-wilson-2/

ebookmass.com

Titanium for Consumer Applications: Real-World Use of

Titanium Francis Froes

https://ebookmass.com/product/titanium-for-consumer-applications-real-
world-use-of-titanium-francis-froes/

ebookmass.com
Developmental Mathematics: Prealgebra, Beginning Algebra,
& Intermediate Algebra 2nd Edition Julie Miller

https://ebookmass.com/product/developmental-mathematics-prealgebra-
beginning-algebra-intermediate-algebra-2nd-edition-julie-miller-2/

ebookmass.com
Optimizations and Programming
 Digital Sciences Set
coordinated by
Abdelkhalak El Hami

Volume 1

Optimizations and
Programming

Linear, Nonlinear, Dynamic, Stochastic

and Applications with Matlab

Abdelkhalak El Hami
Bouchaib Radi
First published 2021 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers,
or in the case of reprographic reproduction in accordance with the terms and licenses issued by the
CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the
undermentioned address:

ISTE Ltd John Wiley & Sons, Inc.

27-37 St George’s Road 111 River Street
London SW19 4EU Hoboken, NJ 07030
UK USA

www.iste.co.uk www.wiley.com

© ISTE Ltd 2021

The rights of Abdelkhalak El Hami and Bouchaib Radi to be identified as the authors of this work have
been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Control Number: 2020948478

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library
ISBN 978-1-84821-953-3
 Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Part 1. Programmation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3. Geometry of the linear program . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1. Polyhedra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.2. Extreme points and vertices . . . . . . . . . . . . . . . . . . . . . . 6
1.4. Graphical solving of a linear program . . . . . . . . . . . . . . . . . . . 6
1.5. Simplex algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5.1. Basic solutions and basic feasible solutions . . . . . . . . . . . . . . 9
1.5.2. Simplex tableau . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.3. Change of feasible basis . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5.4. Existence and uniqueness of an optimal solution . . . . . . . . . . . 14
1.6. Initialization of the simplex algorithm . . . . . . . . . . . . . . . . . . . 15
1.6.1. Big M method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.6.2. Auxiliary program or Phase I . . . . . . . . . . . . . . . . . . . . . . 17
1.6.3. Degeneracy and cycling . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.6.4. Geometric structure of realizable solutions . . . . . . . . . . . . . . 21
1.7. Interior-point algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.8. Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.8.1. Duality theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.9. Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.9.1. Lagrangian relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.10. Postoptimal analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.10.1. Effect of modifying b . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.10.2. Effect of modifying c . . . . . . . . . . . . . . . . . . . . . . . . . 32
vi Optimizations and Programming

1.11. Application to an inventory problem . . . . . . . . . . . . . . . . . . . 34

1.11.1. Optimal solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.11.2. Sensitivity to variation in stock . . . . . . . . . . . . . . . . . . . . 35
1.11.3. Dual problem of the competitor . . . . . . . . . . . . . . . . . . . 36
1.12. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Chapter 2. Integer Programming . . . . . . . . . . . . . . . . . . . . . . . 41

2.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.2. Solving methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.2.1. Branch-and-bound method . . . . . . . . . . . . . . . . . . . . . . . 42
2.2.2. The branch-and-cut method . . . . . . . . . . . . . . . . . . . . . . 44
2.3. Binary programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.1. Knapsack problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.2. Investment problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.4. Decomposition principle . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.4.1. Benders decomposition . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.5. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Chapter 3. Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . 65

3.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2. Solving strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.3. Discrete DP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.1. Bellman’s equation and the principle of optimality . . . . . . . . . 68
3.3.2. Approach of the method . . . . . . . . . . . . . . . . . . . . . . . . 70
3.3.3. A few examples of DP . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.3.4. Solving an LP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3.5. Shortest path problem . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.6. Knapsack problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.3.7. Stock management problem . . . . . . . . . . . . . . . . . . . . . . 81
3.4. Continuous DP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.4.1. Hamilton–Jacobi equation . . . . . . . . . . . . . . . . . . . . . . . 84
3.4.2. Application to a consumption-savings model . . . . . . . . . . . . . 84
3.5. Stochastic DP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.5.1. Decision-chance process . . . . . . . . . . . . . . . . . . . . . . . . 85
3.5.2. Solving method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.5.3. Application to a contract problem . . . . . . . . . . . . . . . . . . . 86
3.5.4. Optimal binary search tree . . . . . . . . . . . . . . . . . . . . . . . 87
3.6. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Chapter 4. Stochastic Programming . . . . . . . . . . . . . . . . . . . . . 93

4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.2. Presentation of the problem . . . . . . . . . . . . . . . . . . . . . . . . . 94
 Contents vii

4.3. Optimal feedback in an open loop . . . . . . . . . . . . . . . . . . . . . 94

4.4. Stochastic linear programming . . . . . . . . . . . . . . . . . . . . . . . 95
4.4.1. Models with probability thresholds on the constraints . . . . . . . . 96
4.5. Stochastic linear programs with recourse . . . . . . . . . . . . . . . . . 96
4.5.1. L-shaped method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.5.2. Multicut L-shaped method . . . . . . . . . . . . . . . . . . . . . . . 99
4.5.3. Interior linearization method . . . . . . . . . . . . . . . . . . . . . . 100
4.6. Nonlinear stochastic programming . . . . . . . . . . . . . . . . . . . . . 100
4.6.1. Approaches to two-step problems with recourse . . . . . . . . . . . 100
4.6.2. Regularized decomposition method . . . . . . . . . . . . . . . . . . 101
4.6.3. Methods based on the Lagrangian . . . . . . . . . . . . . . . . . . . 101
4.6.4. Frank–Wolfe method for problems with simple recourse . . . . . . 103
4.6.5. Approximation by sampling average: Monte Carlo method . . . . . 105
4.6.6. Stochastic gradient method . . . . . . . . . . . . . . . . . . . . . . . 106
4.7. Stochastic dynamic programming . . . . . . . . . . . . . . . . . . . . . 107
4.7.1. Markov decision process . . . . . . . . . . . . . . . . . . . . . . . . 108
4.7.2. Scenario tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.8. Application to the reliability of mechanical systems . . . . . . . . . . . 111
4.8.1. Position and modeling of the reliability problem . . . . . . . . . . . 113
4.9. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Part 2. Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Chapter 5. Combinatorial Optimization . . . . . . . . . . . . . . . . . . . 129

5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.2. Symmetric TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.2.1. Historical overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.2.2. Solving methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.3. Asymmetric traveling salesman problem . . . . . . . . . . . . . . . . . 140
5.3.1. Variants of the ATSP . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.3.2. Mathematical formulations . . . . . . . . . . . . . . . . . . . . . . . 142
5.3.3. Methods for solving the ATSP . . . . . . . . . . . . . . . . . . . . . 144
5.4. Vehicle routing problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.4.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.4.2. Fields of application . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.4.3. Parameters of the VRP . . . . . . . . . . . . . . . . . . . . . . . . . 150
5.4.4. Variants of the VRP . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.4.5. Mathematical formulation of the VRP . . . . . . . . . . . . . . . . . 153
5.4.6. Algorithmic complexity . . . . . . . . . . . . . . . . . . . . . . . . . 155
5.5. Selective routing problem . . . . . . . . . . . . . . . . . . . . . . . . . . 156
5.5.1. Problems similar to the VRP . . . . . . . . . . . . . . . . . . . . . . 157
5.5.2. Mathematical formulation . . . . . . . . . . . . . . . . . . . . . . . 157
5.6. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
viii Optimizations and Programming

Chapter 6. Unconstrained Nonlinear Programming . . . . . . . . . . . 161

6.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
6.2. Mathematical formulation . . . . . . . . . . . . . . . . . . . . . . . . . 161
6.2.1. Existence and uniqueness results . . . . . . . . . . . . . . . . . . . . 162
6.3. Optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
6.4. Quadratic problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.4.1. Gradient method with optimal step size . . . . . . . . . . . . . . . . 163
6.4.2. Conjugate gradient method . . . . . . . . . . . . . . . . . . . . . . . 164
6.5. Newton’s algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.6. Methods of descent and linear search . . . . . . . . . . . . . . . . . . . 165
6.6.1. Presentation of methods of descent . . . . . . . . . . . . . . . . . . 165
6.6.2. Method of greatest slope . . . . . . . . . . . . . . . . . . . . . . . . 167
6.6.3. Acceptable step size . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
6.6.4. Linear search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.6.5. Newton’s method with linear search . . . . . . . . . . . . . . . . . . 170
6.7. Quasi-Newton methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.7.1. DFP and BFGS methods . . . . . . . . . . . . . . . . . . . . . . . . 172
6.8. Relaxation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.9. Gradient method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.10. Least squares problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.10.1. Gauss–Newton method . . . . . . . . . . . . . . . . . . . . . . . . 176
6.10.2. Levenberg–Marquardt algorithm . . . . . . . . . . . . . . . . . . . 178
6.10.3. Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
6.11. Direct search methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.11.1. Nelder–Mead algorithm . . . . . . . . . . . . . . . . . . . . . . . . 181
6.11.2. Torczon method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.12. Application to an identification problem . . . . . . . . . . . . . . . . . 183
6.13. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.13.1. The fminsearch function . . . . . . . . . . . . . . . . . . . . . . . 187
6.13.2. The fminunc function . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.13.3. Relaxation method . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Chapter 7. Constrained Nonlinear Optimization . . . . . . . . . . . . . 193

7.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.2. Mathematical formulation . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.3. Lagrange multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
7.4. Optimization with inequality constraints . . . . . . . . . . . . . . . . . 195
7.4.1. First-order conditions of optimality . . . . . . . . . . . . . . . . . . 195
7.4.2. Presentation of saddle points . . . . . . . . . . . . . . . . . . . . . . 197
7.4.3. Saddle point and optimization . . . . . . . . . . . . . . . . . . . . . 198
7.4.4. Convex case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.5. Constrained minimization algorithms . . . . . . . . . . . . . . . . . . . 201
7.5.1. Relaxation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
 Contents ix

7.5.2. Projection method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

7.5.3. Exterior penalty method . . . . . . . . . . . . . . . . . . . . . . . . 204
7.5.4. Uzawa’s algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.6. Newton algorithms: SQP method . . . . . . . . . . . . . . . . . . . . . 206
7.6.1. Equality constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
7.6.2. Inequality constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 209
7.7. Application to structure optimization . . . . . . . . . . . . . . . . . . . 210
7.8. Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.8.1. The fmincon function . . . . . . . . . . . . . . . . . . . . . . . . . . 219
7.8.2. The fminbnd function . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7.8.3. Penalty method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Appendix 1. Reminders from Linear Algebra . . . . . . . . . . . . . . . 231

Appendix 2. Reminders about functions from Rn into R . . . . . . . . 241

Appendix 3. Optimization Toolbox . . . . . . . . . . . . . . . . . . . . . . 245

Appendix 4. Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
 Preface

Optimization is the domain of mathematics that studies how to minimize (or

maximize) a certain objective, for example, an economic parameter or some type of
energy. This is both an ancient topic, with the earliest optimization problems going
back to Euclid, and a relatively new topic, with the recent development of numerical
methods such as linear programming, which only truly began to flourish in the
second half of the 20th century. As an illustration, imagine the problem of searching
for a path that joins two points on a map in the shortest time. Some parameters of the
problem might be uncertain; for example, we might encounter a traffic jam on the
road. Optimization seeks to find the best compromise between the various possible
risks.

In the presence of uncertainty, the optimizer (or decider) will often take advantage
of information about the system that becomes available dynamically, i.e. gradually
over time. For example, we will gradually discover which roads are most susceptible
to traffic jams over time, as we try them out. The difficulty of an optimization problem
is closely linked to the amount of information needed to make an optimal decision.
A problem is said to be large if this quantity of information is too big for classical
solving techniques to be applied by brute force.

This book is divided into two parts: programming (Part 1) and optimization
(Part 2). In the programming part, we present a collection of tools for operations
research, including linear programming, integer programming, binary programming,
dynamic programming and stochastic programming. Operations research first arose
in the United Kingdom during the Second World War, when scientific methods were
used to study various aspects of military operations. Since then, it has become a key
element of decision-making processes in various commercial, industrial and
governmental contexts, offering a systematic way to apprehend the ever increasing
complexity of the management problems faced by both the private and public sectors.
xii Optimizations and Programming

Following its success in military matters during the Second World War,
operations research was applied to operational problems in the industry and private
sector for many years. In the last decade or so, its applications have been extended to
other domains, such as economics, finance, marketing and corporate planning. More
recently, operations research has been used to manage healthcare and education
systems, solve environmental problems and in other areas of public interest. Its
principal users are manufacturing, distribution and retail companies in the mining,
energy, transportation and construction sectors, as well as service companies such as
banks, and various government bodies. Noteworthy examples of recent applications
of operations research include logistical studies, railway safety, packaging design,
strategic workforce planning, aerial transportation, forestry operations, nuclear fuel
optimization, production planning, and so on.

The optimization part of this book is divided into three chapters: combinatorial
optimization, nonlinear optimization without constraints and nonlinear optimization
with constraints. It is entirely dedicated to numerical optimization algorithms, their
theoretical foundations and convergence properties, their implementation and
application, and other practical aspects. The objective is to familiarize readers with
these numerical algorithms in order to understand how they behave in practice, how
to properly take advantage of Matlab as a tool, how to design and adequately
implement such algorithms and how to correctly diagnose any difficulties that might
arise.

Each chapter starts with a few reminders of key results, but readers should not
hesitate to consult the references listed at the end of the book. This book is organized
according to a strictly linear approach. As a general rule, the concepts are illustrated
with examples. Each chapter ends with an example in Matlab.

Acknowledgments

We would like to thank everyone who contributed, directly or indirectly, to the

writing of this book and especially, the engineering and PhD students of INSA Rouen
whom we have had the pleasure of supervising over the past few years.

Bouchaib R ADI
Abdelkhalak E L H AMI
November 2020
 PART 1

Programmation
 1

Linear Programming

1.1. Introduction

Linear programming can be defined as a mathematical technique for solving

management problems. For example, suppose that a manager faced with various
options wishes to determine the optimal way to use the resources of a company to
achieve a specific objective, such as maximizing the utility or minimizing the cost.
The company’s problems can usually be modeled as a linear program (LP) consisting
of a certain number of resources [HIL 90]. For instance, the labor, raw materials
and capital are all resources available in limited quantities that must be distributed
optimally over various manufacturing processes. The approach to solving this type of
problem is divided into two key steps:
– model the problem with linear equations or inequalities that allow us to properly
identify and structure the constraints satisfied by the variables of the model. The
contribution of each variable to the company’s overarching objective must also be
defined as a function that will be optimized;
– find the mathematical optimum using specific linear programming techniques.
We will study three methods for solving the various types of linear programming
problems. The first is based on graphical solving and is therefore limited to two or
three variables. The second method is more algebraic; it motivates the third method
presented in this chapter, which is known as the simplex method (or algorithm).

1.2. Definitions

D EFINITION 1.1.– An LP is in canonical form if it is expressed as follows:

⎧
⎪ maxn < c, x >
⎨ x∈R
⎪ s.t. Ax ≤ b [1.1]
⎩ x≥0

Optimizations and Programming: Linear, Nonlinear, Dynamic, Stochastic and Applications

with Matlab, First Edition. Abdelkhalak El Hami and Bouchaib Radi.
© ISTE Ltd 2021. Published by ISTE Ltd and John Wiley & Sons, Inc.
4 Optimizations and Programming

A ∈ Mp,n (R), b ∈ Rp , and x ∈ Rn . We write x = (x1 , . . . , xn )T ≥ 0 if

x1 , . . . , xn ≥ 0.

An LP is in standard form if it is expressed as follows:

⎧
⎨ max cT x
s.t. Ax = b
⎩
x≥0

Every linear problem can be expressed in canonical form: min cT x = − max cT x.

T HEOREM 1.1.– Every LP in standard form can be expressed in canonical form and
vice versa.

Terminology

– The function cT x is the objective function, cost function or loss function.

– The components of the vector x are called decision variables.
– If the vector x satisfies all constraints, then x is an admissible solution.
– The set of all admissible solutions is the admissible set or admissible region.
– An admissible solution x∗ that maximizes the loss function (i.e. cT x∗ ≥ cT x for
every admissible x) is called an optimal admissible solution or optimal solution.

E XAMPLE 1.1.– A factory manufactures two products P1 and P2 while consuming

certain resources: equipment, labor and raw materials. The needs are listed in the
following table. Each resource is available in limited quantities.

P1 P2 Availability
Equipment 3 9 81
Labor 4 5 55
Raw materials 2 1 20

The two products P1 and P2 yield effective profits of 6 dhs and 4 dhs per unit.
The goal is to determine which (possibly non-integer) quantities of the products P1
and P2 the factory should produce to maximize the total profit from selling these two
products, subject to the availability of the resources.

Let x1 and x2 be the quantities of the products P1 and P2 , respectively.

The LP may be expressed as follows: maximize z(x1 , x2 ) = 6x1 + 4x2 subject to:
 Linear Programming 5

– availability of resources:

3x1 + 9x2 ≤ 81
4x1 + 5x2 ≤ 55
2x1 + x2 ≤ 20

– positivity of variables: x1 , x2 ≥ 0

1.3. Geometry of the linear program

1.3.1. Polyhedra

D EFINITION 1.2.– A polyhedron is a set that can be described as

P = {x ∈ Rn | Ax ≤ b},

where A is an m × n matrix and b is a vector in Rm . Note that the admissible set of

an LP is a polyhedron.

D EFINITION 1.3.– A polyhedron P is said to be bounded if there exists a constant c

such that x ≤ c for every x ∈ P.

Figure 1.1. Bounded polyhedron (left) and unbounded polyhedron

(right). For a color version of this figure, see
www.iste.co.uk/radi/optimizations.zip

D EFINITION 1.4.– Let a be a non-zero vector of Rn , and let b be a scalar.

– The set {x ∈ Rn |aT x = b} is said to be a hyperplane.
– The set {x ∈ Rn |aT x ≥ b} is said to be a half-space.

Note that a hyperplane is the boundary of the corresponding half-space and the
vector a is perpendicular to the hyperplane that it defines [TEG 12].
6 Optimizations and Programming

1.3.2. Extreme points and vertices

Let P be a polyhedron and x∗ ∈ P. The point x∗ is an extreme point if and only if

x∗ is a vertex and x∗ is a basic admissible solution.

D EFINITION 1.5.– Let P be a polyhedron. A vector x ∈ P is an extreme point of P if

it cannot be expressed as a convex combination of two other points of P.

D EFINITION 1.6.– Let P be a polyhedron. A vector x ∈ P is a vertex of P if there

exists c such that
cT x < c T y
for every y in P not equal to x.

T HEOREM 1.2.– In linear programming, if the admissible domain is neither empty nor
the whole of Rn , it is a convex polytope with finitely many vertices that can either be
bounded or unbounded. If an extreme point exists, it is attained at one of the vertices
of the polytope. A point in the interior of the domain is never an extreme point if
f = 0. If the polytope is bounded, f attains both a minimum and a maximum on it.

This theorem gives us a graphical solving method.

1.4. Graphical solving of a linear program

The graphical method works by plotting lines and searching for a solution as
follows:
– identify the admissible domain;
– identify the contours;
– contours perpendicular to the vector c, and therefore mutually parallel;
– each value of z is associated with a contour;
– the value of z increases in the direction of c.

E XAMPLE 1.2.– Consider, again, the example from earlier. Its mathematical model is
defined by the following linear program:
⎧
⎪
⎪ max 6x1 + 4x2
⎪
⎪ (x1 ,x2 )∈R2
⎪
⎪
⎪
⎨ s.t. 3x1 + 9x2 ≤ 81
⎪ 4x1 + 5x2 ≤ 55
⎪
⎪
⎪
⎪ 2x1 + x2 ≤ 20
⎪
⎪
⎩ x1 , x 2 ≥ 0
 Linear Programming 7

The intersection of the half-planes determined by the lines corresponding to each

constraint represents the set of solutions that satisfy the constraints (shaded area). The
direction of the cost function is given by

z(x1 , x2 ) = 6x1 + 10x2 = constant.

 
This gives us the optimal solution x = 15 2 , 5 by sliding the line with this
direction upwards, while keeping it parallel, until its intersection with the y-axis is
maximal. Graphical solutions are of course only applicable with programs of, at
most, three variables (see Figure 1.2).

Figure 1.2. Graphical solution. For a color version of this

figure, see www.iste.co.uk/radi/optimizations.zip
8 Optimizations and Programming

E XAMPLE 1.3.– Consider the linear program:

min z = −x1 − x2
s.t. x1 + 2x2 ≤ 3
[1.2]
2x1 + x2 ≤ 3
x1 , x2 ≥ 0

The graphical solution is shown in Figure 1.3. We have z ∗ = −2, with x∗1 = 1 and
x∗2 = 1.

Figure 1.3. Graphical solution with isoprofit lines. For a color version of
this figure, see www.iste.co.uk/radi/optimizations.zip
 Linear Programming 9

1.5. Simplex algorithm

If a linear programming problem in standard form has an optimal solution, then a

basic admissible solution that is optimal exists. The simplex algorithm is an
algorithm for solving linear optimization problems. Introduced by George Dantzig in
1947, it was probably the earliest algorithm for minimizing functions on sets defined
by inequalities. The algorithm moves from one basic admissible solution to the next
while reducing the cost.

The idea of the method is as follows:

– Let x0 be a basic admissible solution.
– For k = 0, . . . , find an adjacent basic admissible solution xk+1 such that
cT xk+1 < cT xk .
– Until no adjacent basic admissible solution improves the objective.

This gives a local minimum, which is in fact a global minimum in linear

programming.

1.5.1. Basic solutions and basic feasible solutions

Consider a system of m linear equations Ax = b with n variables (m ≤ n).

D EFINITION 1.7.– A basic solution of the system of equations Ax = b is obtained by

setting n − m variables to zero and solving the system for the remaining p variables.
The solution of the system of p equations in m unknowns is assumed to be unique. The
n − p variables set to zero are said to be non-basic variables, and the p remaining
variables are said to be basic variables. An LP admits, at most, Cnm basic solutions. If
the basic solution also satisfies the non-negativity constraints, it is said to be a basic
feasible solution.

D EFINITION 1.8.– A basic solution is said to be degenerate if at least one basic

variable is zero. This type of solution is obtained when the number of lines passing
through an extreme point is greater than the number of decision variables. A basic
feasible solution whose m basic variables are positive is said to be a non-degenerate
basic feasible solution.

R EMARK.– Each basic feasible solution corresponds to an extreme point. However,

there can be more than one basic feasible solution for the same extreme point. This
occurs when the basic feasible solution is degenerate.

R EMARK.– The number of basic solutions quickly becomes very large, even in
modestly sized models. For example, a model in standard form with 12 constraints
12
and 25 variables can have up to C25 = 5 200 300 basic solutions.
10 Optimizations and Programming

1.5.2. Simplex tableau

Suppose that we wish to solve

⎧
⎪
⎨ max z(x1 , x2 , x3 , x4 ) = 4x1 + 3x2
s.t. 3x1 + 4x2 + x3 = 12
⎪
⎩
7x1 + 2x2 + x4 = 14, xi ≥ 0 (i = 1, . . . , 4)


3410
Let c = (4, 3, 0, 0) , b = (12, 14) , (A, I2 ) =
T T
. It is easy to see that
7201
x = (x1 , x2 , x3 , x4 )T = (0, 0, 12, 14)T satisfies the constraints. Thus, x is a basic
feasible solution with basis J = {3, 4} (the third and fourth components of x), whose
basic variables are x3 = 12 and x4 = 14.

Note that x is not optimal: z(x) = cT x = 0. The optimization procedure

constructs a sequence of tables, called simplex tableaus. The first tableau summarizes
the data c, b, A and the basic variables. The last row is equal to −cT .

x 1 x2 x3 x4
3 4 1 0 xJ1 = x3 = 12
7 2 0 1 xJ2 = x4 = 14
-4 -3 0 0 z(x) = 0

Table 1.1. First simplex tableau

General case

Consider the problem

⎧
⎪
⎨ max z(x) = c x
T

(P ) s.t. Ax = b,
⎪
⎩
x≥0

where x ∈ Rn+ , c ∈ Rn , b ∈ Rm , and A is an m × n matrix. We may assume without

loss of generality that rank A = m < n.

D EFINITION 1.9.– Let A = (a1 , a2 , . . . , an ) (where aj is the jth column of A), and,
for J ⊂ {1, 2, . . . , n}, let AJ = {aj , j ∈ J}.
– J ⊂ {1, 2, . . . , n} is a basis of (P) if |J| = m and if rank AJ = rank (aj , j ∈
J) = m;
 Linear Programming 11

– let x = (x1 , . . . , xn )T ∈ Rn+ . Then xj is a basic variable (respectively,

non-basic variable) if j ∈ J (respectively, j ∈
/ J). We write xJ = (xj , j ∈ J);
– basic feasible solution: x = (x1 , . . . , xn )T ∈ Rn+ such that xj = 0 for j ∈
/ J
and such that Ax = AJ xJ = b.

R EMARK.– The advantage of passing from the canonical form to the standard form
is that we immediately obtain a basic feasible solution that can be used as a starting
point for the simplex algorithm. The basic variables are the slack variables.

1.5.3. Change of feasible basis

Let x be a basic feasible solution of (P). Then

z(x) = cT x = cJ xJ where cJ = (cj , j ∈ J).

Our goal is to find another feasible basis J¯ and a basic feasible solution x̄ such
that z(x̄) > z(x) (meaning that x̄ is better than x). The simplex method proceeds by
replacing one of the basic variables xr with a non-basic variable xk . We say that
– the variable xr enters the basis J : xr → x̄r = 0;
– the variable xk leaves the basis J¯: xk = 0 → x̄k > 0.

Thus, J¯ = (J − {r}) ∪ {k}. We need rules to choose r and k. These rules are as
follows:
– Choose r such that

xJr xJj
= min , aik > 0, i ∈ J . [1.3]
ark aik

– Choose k such that

zk − ck = min {zj − cj , zj < cj ∃i ∈ J | aij > 0}. [1.4]

j=1,...,n

E XAMPLE 1.4.– Returning to the previous example: J = {3, 4}, c3 = c4 = 0, and so

zj = 0 for j = 1, 2, 3, 4. Thus, zj − cj = −cj , j = 1, 2, 3, 4. Hence, k = 1.

To choose r,

xJr xJi 12 14
= min ; ai1 > 0, i ∈ J = min , = 2.
ar1 ai1 3 7
12 Optimizations and Programming

Thus, r = 2, x4 leaves the basis, and x1 enters the basis. The new basis is J¯ =
{3, 1}. We have

¯ a11 J 3 ¯ xJ
x̄J1 = x̄3 = xJ1 − x2 = 12 − 14 = 6x̄J2 = x̄1 = 2 = 2.
a21 7 a21
¯ ¯
Hence, x̄ = (2, 0, 6, 0)T and z(x̄) = cJ1 x̄J1 + cJ2 x̄J2 = c3 x̄3 + c1 x̄1 = 8. Passing
from J = {3, 4} to J¯ = {3, 1} increases the value of z to 8, but this value is not yet
maximal.

Calculating the new tableau

We now apply the transformation x → x̄ that increased the value of z. Since the
value z(x̄) (> z(x)) is not necessarily maximal in general, we may need to repeat the
steps for choosing r and k several times until we find a basic feasible solution that is
also a maximum of z.
¯
To do this, we need a second tableau where xJ is replaced by x̄J , interpreted in
the same way as the first. The same linear transformation that allowed us to pass from
x to x̄ is applied to the columns of A.

The matrix A = (aij , i = 1, . . . , m, j = 1, . . . , n) is replaced by Ā = (āij , i =

1, . . . , m, j = 1, . . . , n) as follows:
⎛ ⎞ ⎛ ⎞
a1j ā1j
⎜ . ⎟ ⎜ . ⎟ aik arj arj
aj = ⎜ . ⎟ ⎜ . ⎟
⎝ . ⎠ → ā = ⎝ . ⎠ , where āij = aij − ark ( if i = r) and ārj = ark .
j

amj āmj
[1.5]

This gives


n 
n
aij xj = bi ⇔ āij xj = b̄i , [1.6]
j=1 j=1

¯
where b̄i , i = 1, . . . , m, are the new basic variables x̄Ji .

The last row of the new tableau is computed in the same way:
arj
z̄j − cj = zj − cj − (zk − ck ). [1.7]
ark
 Linear Programming 13

E XAMPLE 1.5.– If we apply the above procedure to our example, we obtain Table 1.2.

x 1 x2 x3 x 4
¯
0 22/7 1 -3/7 xJ1 = x3 = 6
¯
1 2/7 0 1/7 xJ2 = x1 = 2
0 -13/7 0 4/7 z(x̄) = 8

Table 1.2. Second simplex tableau (J¯ = {3, 1})

As we saw above, the value of z increases from 0 to 8, but there are still negative
values in the last row of the tableau, so we need to perform another change of basis by
¯ This gives:
applying the formulas [1.4] and [1.3] after substituting J.
¯
x̄Ji 2 6 21 21
k = 2, min ; i = 1, 2 = min , = min 7, = ,
āi2 2/7 22/7 11 11

r = 1, x3 leaves the basis. J˜ = {2, 1} is the new basis.

The computation with this new basis is presented in Table 1.3.

x1 x2 x 3 x4
˜
0 1 7/22 -3/22 x̃J1 = x2 = 21/11
˜
1 0 -1/11 2/11 x̃J2 = x1 = 16/11
0 0 13/22 7/22 z(x̃) = 127/11

Table 1.3. Third simplex tableau (J˜ = {2, 1})

The value of z now increases from 8 to 12711 . This value is maximal because every
value in the final row is non-negative. The optimal solution is therefore x̃ = (x̃1 , x̃2 ) =
( 16 21
11 , 11 ). This solution is unique because no further change of basis is possible.

E XAMPLE 1.6.– Consider the linear problem:

min z = −800x1 − 300x2

s.t. 2x1 + x2 ≤ 400
x1 ≤ 150 [1.8]
x2 ≤ 200
x 1 , x2 ≥ 0
14 Optimizations and Programming

Let us introduce slack variables x3 , x4 and x5 :

min z = −800x1 − 300x2

s.t. 2x1 + x2 + x3 = 400
x1 + x4 = 150 [1.9]
x2 + x5 = 200
x1 , x2 , x3 , x5 ≥ 0

This gives the following initial tableau with the basis (x3 , x4 , x5 ):

x1 x2 x3 x4 x5 θ
2 1 1 0 0 400 200
T0 = 1 0 0 1 0 150 150 ←
0 1 0 0 1 200
-800 -300 0 0 0 0

The new basis is (x3 , x1 , x5 ) given as:

x 1 x2 x3 x4 x5 θ
0 1 1 -2 0 100 100 ←
T1 = 1 0 0 1 0 150
0 1 0 0 1 200 200
0 -300 0 800 0 120000

The next basis is (x2 , x1 , x5 ) given as:

x1 x2 x3 x4 x5
0 1 1 -2 0 100
T2 = 1 0 0 1 0 150
0 0 -1 2 1 100
0 0 300 200 0 150000

Since the reduced costs are all positive, this tableau is optimal. The optimal
solution is x1 = 150 and x2 = 100, giving an optimal value of z = −1, 500, 000 for
the objective function.

1.5.4. Existence and uniqueness of an optimal solution

After writing out the first simplex tableau and adding zj = −cj to the last row,
there are three possible cases:
 Linear Programming 15

1) zj − cj ≥ 0 for j = 1, . . . , n. The basic feasible solution x is already optimal.

This optimal solution may or may not be unique;
2) there exists j ∈ {1, . . . , n} such that zj −cj < 0 and aij ≤ 0 for every i ∈ J. In
this case, the domain of realizable solutions is unbounded and the problem is ill-posed,
since max z(x) = +∞;
3) the usual case: there exists j ∈ {1, . . . , n} such that zj − cj < 0, and there
exists i ∈ J such that aij > 0. The change in basis described earlier is now possible
and should be performed, possibly several times, until case (1) is reached.

Could the simplex algorithm ever fail to terminate if case (3) leads to a loop? The
answer is yes, and examples have been successfully constructed. However, they are
very rare in practice.

Let us therefore state two important theorems about the simplex method.

T HEOREM 1.3.– Let x ∈ Rn+ be a basic realizable solution of (P ) with respect to


a basis J (|J| = m = rank(A)). Let zj = ci aij . If zj − cj ≥ 0 for every
i∈J
j = 1, . . . , n, then x is an optimal basic realizable solution.

T HEOREM 1.4.– Let x ∈ Rn+ be a basic realizable solution of (P ) with respect to


a basis J (|J| = m = rank(A)). Let zj = ci aij . Suppose that aij ≤ 0 for
i∈J
every i ∈ J and for every j ∈ {1, . . . , n} such that zj − cj < 0. Then the set
{z(x), x is a realizable solution} is unbounded.

R EMARK.– For a comprehensive discussion of the efficiency of the simplex method,

refer to [CHV 83].

1.6. Initialization of the simplex algorithm

This section presents two techniques for finding a basic realizable solution to
initialize the simplex algorithm: the first is the Big M method, and the second is the
Phase I method.

1.6.1. Big M method

Suppose that we wish to solve the LP

max z(x) = cT x
(P ) s.t. Ax = b,
x≥0
16 Optimizations and Programming

where A is an m × n matrix, rank A = m < n.

By adding slack variables (with positive or negative signs), we can always reduce
the LP to the form (P) described above, with b ∈ Rm + . If there is no obvious basic
realizable solution to initialize the simplex algorithm, we proceed by adding artificial
variables yi ≥ 0 to the constraints:

Ax = b is replaced by Ax + y = b, y = (y1 , y2 , . . . , ym )T ∈ Rm
+.

The new constraints are not equivalent to the initial constraints. The yi > 0 are
penalized by replacing the objective function z(x) with


m
z  = cT x − M yi ,
i=1

where M is some very large positive value. We will choose yi = bi , i = 1, . . . , m, as

a basic feasible solution to serve as the initial solution.

Since the yi significantly reduce the value of z  , they will disappear from the basis
over the course of the simplex algorithm. As they are non-basic variables, their value
will be equal to zero, and the LP that is ultimately solved is the same as the program
(P).

E XAMPLE 1.7.– Consider the LP

max z(x) = −x1 + 2x2 − 2x3

s.t. x1 + x2 − x3 = 3
−x1 + 3x2 = −4
x1 , x2 , x3 ≥ 0.

The new program is given as

max z  (x, y) = (−1 + 2M )x1 + (2 − 2M )x2 + (−2 − M )x3

s.t. x1 + x2 − x3 + y1 = 3
−x1 + 3x2 + y2 = −4
x1 , x2 , x3 , y1 , y2 ≥ 0.
 Linear Programming 17

The first tableau is as follows:

c -1+2M 2-2M -2-M 0 0

Basic variables x1 x2 x3 y1 y2
y1 = 3 1 1 -1 1 0
y2 = 4 1 -3 0 0 1
z  (x) = −7M 1-2M -2+2M 2+M 0 0

The second tableau is as follows:

c -1+2M 2-2M -2-M 0 0

Basic variables x1 x2 x3 y1 y2
x1 = 3 1 1 -1 1 0
y2 = 1 0 -4 1 -1 1
z  (x) = −3 − M 0 -3+4M 3-M -1+2M 0

The third tableau is as follows:

c -1+2M 2-2M -2-M 0 0

Basic variables x1 x2 x3 y1 y2
x1 = 4 1 -3 0 0 1
x3 = 1 0 -4 1 -1 1
z  (x) = −6 0 9 0 2+M -3+M

The unique optimal solution is given by x1 = 4, x2 = 0, x3 = 1, and

z(x1 , x2 , x3 ) = −6.

1.6.2. Auxiliary program or Phase I

Suppose that the LP that we wish to solve is in the standard form:

max z(x) = cT x
s.t. Ax = b, x ≥ 0.

The auxiliary program method proceeds by letting M → +∞ in (PM ) and solving

the program obtained in the limit. Let

1  c m
z  (x, y) = z (x, y) = x− yi .
M M i=1
18 Optimizations and Programming

Maximizing z  (x, y) is equivalent to maximizing z  (x, y), or maximizing


m
lim z  (x, y) = − yi .
M →+∞
i=1

We therefore solve the auxiliary program (PA ) given by

⎧ 
m
⎪
⎪ 
⎪
⎨ max z (x, y) = − yi
(PA ) i=1
⎪
⎪ s.t. Ax + y = b,
⎪
⎩
x ∈ Rn+ , y ∈ Rm
+ , b ∈ R+ .
m

The simplex algorithm can now be applied to (PA ) starting from the basic
realizable solution y = b.
– First case: max z  (y) < 0. There exists yi > 0, i ∈ {1, 2, . . . , m}. In this case,
(P ) does not have any realizable solutions.
– Second case: max z  (y) = 0. The optimal solution of (PA ) can be used as
an initial basic realizable solution for applying the simplex algorithm to (P ). The
objective function of (P ) is expressed in terms of non-basic variables, then the simplex
algorithm is applied to (P ).


R EMARK.– For a minimization problem, we add yi instead.

Let us go into more detail for a minimization problem. The initial tableau of the
auxiliary problem is as follows:

A b
−cTB A | O −cTB b

where B = I, cB = (1, 1, 1, . . .)T , and the matrices A and b relate to the auxiliary
program. For the original variables i, we have ci = 0.

If (x∗ , y ∗ ) is a zero-cost optimal solution of the auxiliary problem, then the

artificial variable in the basis is necessarily zero, so the solution is degenerate.

If we assume that the kth basic variable is artificial, consider the kth row of the
tableau and select the element of this row in column j, where j is the index of a
variable in the original problem. If the element is non-zero: k leaves the basis and j
enters the basis, and we pivot the tableau. But what if no such element exists? This
 Linear Programming 19

would mean that the matrix A does not have full rank, and the row in question
corresponds to a redundant constraint.

Once the optimal tableau of the auxiliary problem has been obtained and all
artificial variables have left the basis, we obtain the initial tableau of the original
problem P1 by deleting any columns that relate to the artificial variables and
calculating the reduced initial costs: cT − cTB B −1 A and −cTB B −1 b in the box
reserved for the cost function. The following example shows how this method works.

E XAMPLE 1.8.– Consider the following linear programming problem:

min z = x1 + x2 + x3 − 2x4
s.c. x1 + x2 + x3 = 1
[1.10]
x2 + x4 = 1
x 1 , x2 , x3 , x4 ≥0

We will first perform Phase I of the simplex algorithm. After adding artificial
variables, we obtain the tableau as follows:

x1 x2 x3 x4 x5 x6 θ
1 1 1 0 1 0 1 1←
T0 = 0 1 0 1 0 1 1
-1 -2 -1 -1 0 0 -2
↑

x1 x2 x3 x4 x5 x6 θ
1 1 1 0 1 0 1 1←
T1 = 0 1 0 1 0 1 1 1
0 -1 0 -1 1 0 -1
↑

x1 x2 x3 x4 x5 x6 θ
1 1 1 0 1 0 1
T2 = -1 0 -1 1 -1 1 0 0←
1 0 1 -1 2 0 0
↑

x1 x2 x3 x4 x5 x6
1 1 1 0 1 0 1
T3 =
-1 0 -1 1 -1 1 0
0 0 0 0 1 1 0
20 Optimizations and Programming

There are no longer any artificial variables in the basis. We have found the optimal
tableau of the auxiliary problem, and therefore an admissible basic solution for the
initial problem. Since Phase I is now complete, delete the columns corresponding to
the artificial variables and compute the reduced costs to obtain the following tableau:

x1 x2 x3 x4 θ
1 1 1 0 1 1←
T0 = -1 0 -1 1 0
-2 0 -2 0 -1
↑

x1 x2 x3 x4
1 1 1 0 1
T1 =
0 1 0 1 1
0 2 0 0 1
This tableau is optimal. The optimal solution is x1 = 1, x2 = 0, x3 = 0 and
x4 = 1, giving an optimal value of z = −1.

1.6.3. Degeneracy and cycling

Suppose that the current basic admissible solution is degenerate (i.e. at least one
basic variable is zero), and let us recall a possible unfavorable scenario that can occur
when the simplex algorithm is iterated. Performing a change of basis yields a value
for the cost function that is greater than or equal to the previous value. The method
is only guaranteed to converge if z ∗ < z0 ; in fact, if the previous basic solution is
degenerate, it might be the case that z ∗ = z0 .

If this phenomenon occurs multiple times over consecutive iterations, the

algorithm can return to a basis that was already considered earlier. In other words, it
returns to the same vertex twice. This is called cycling.

Although degeneracy is a frequently encountered phenomenon in linear

programming, cycling is not. Some examples have been given in the literature
[GAS 75]. Despite the rarity of cycling, it would be desirable to specify a method
that avoids it and therefore guarantees that the simplex method will converge. There
are several possible choices for a rule that avoids the inconvenience of cycling.

In the past, the most common technique was the so-called method of small
perturbations. This method applies an infinitesimal geometric modification to the
vector d in order to force it to be expressible as a linear combination of m vectors,
with strictly positive coefficients in some basis. Each vertex is then defined as the
intersection of n hyperplanes [HIL 90].
 Linear Programming 21

More recently, Bland [BLA 77] suggested modifying the rules for choosing the
change of basis. Bland’s rule proceeds as follows:
– for the variable entering the basis, choose the smallest index j such that the
reduced cost is negative;
– for the variable leaving the basis, if there is equality in θ∗ , choose the variable
with the lowest index.

1.6.4. Geometric structure of realizable solutions

Earlier, when we solved linear programs graphically, the optimal solutions were on
the boundary of the convex set of realizable solutions. If there was a unique optimal
solution, it was an extreme point.

Recall the canonical form of a linear program:

⎧
⎪
⎨ max z(x) = c x
T

(P ) s.t. Ax ≤ b,
⎪
⎩
x ≥ 0,

where the matrix A is assumed to be an m × n matrix of rank m (< n). The standard
form is then given by
⎧
⎪
⎪ max z(x) = cT x
⎪
⎨ x
(Ps ) s.t. (A, Im ) = b,
⎪
⎪ e
⎪
⎩
x ∈ Rn+ , e ∈ Rm
+

It is possible to show the following result.


Let Γ (respectively, ) be the set of feasible solutions of (P ) (respectively of
(Ps )):

Γ = {x ∈ Rn ; Ax ≤ b, x ≥ 0},
 
 x x
= , (A, Im ) = 0, x ∈ Rn+ , e ∈ Rm
+ .
e e


Then Γ and are closed and convex sets.
22 Optimizations and Programming

Recall that the simplex method only uses the basic solutions
 of (Ps ). The next
theorem tells us that these are in fact the extreme points of .
 
x0 x0
T HEOREM 1.5.– is a basic feasible solution of (Ps ) if and only if is an
e0 e0

extreme point of .

Examples where the solution is not unique show that the optimal solutions lie on
the boundary:

T HEOREM 1.6.– Every optimalsolution of (P ) (respectively of (Ps )) belongs to the

boundary of Γ (respectively of ).

A question presents itself: can there be optimal solutions but no optimal basic
feasible solutions? If so, the simplex method, which only considers basic solutions,
would not be able to find an optimal solution. The following theorem tells us that this
does not happen.

T HEOREM 1.7.– If (Ps ) has an optimal solution, then (Ps ) has an optimal basic
feasible solution.

1.7. Interior-point algorithm

The simplex algorithm moves along a sequence of vertices of the polyhedron and
converges rapidly on average. However, Klee and Minty [KLE 72] found
pathological examples where the simplex algorithm visits almost every vertex, which
can potentially be an enormous number. The existence of theoretical linear programs
that derail the simplex algorithm motivated research to find an algorithm that has
polynomial complexity in m and n, even in the worst-case scenarios. Khachian
[KHA 79] found the first such algorithm, based on the ellipsoid method from
nonlinear optimization. Although it is faster than the simplex method in Minty’s
problems, it never succeeded in establishing itself because it tended to be much
slower than the simplex algorithm on real problems in practice. Nevertheless, as a
mathematical result, it inspired research into interior-point algorithms.

In 1984, Karmarkar [KAR 84] found a new algorithm. The computation time in
the worst-case scenario is proportional to n3.4 L, where L denotes the number of bits
required to encode the tableaus A, b and c that define the linear program. The algorithm
is too complicated for computations by hand; known as the interior-point algorithm, it
enters into the polyhedron itself to converge on the optimal vertex more quickly.
 Linear Programming 23

Today, some variants of this algorithm are able to compete with the simplex
algorithm. Some software programs already include an interior-point algorithm in
addition to the simplex algorithm. On some types of linear programs, interior-point
algorithms can be 10 times faster than the simplex algorithm, but nevertheless, there
are still many cases where the simplex algorithm is quicker [PRI 11].

E XAMPLE 1.9.– Consider the LP

max z = 2x1 + x2
s.t. x1 + x2 ≤ 4
−x1 + x2 ≤ 2
x1 ≤ 2
0.5x1 − x2 ≤ 0
x1 , x 2 ≥ 0

The optimum is x∗ = (2, 2)T with cost z ∗ = 6. Starting from the interior point
x0 = (1, 1)T and without a safety factor, the algorithm performs two iterations: it
considers the point x1 = (2, 1.591)T , followed by the point x∗ .

1.8. Duality

Duality is an essential notion in linear programming. The duality operation

associates any so-called primal linear programming problem with another such
problem, known as the dual, that is defined solely in terms of the data of the primal
problem.

Duality is interesting for two reasons:

– the dual problem has an important economic interpretation that offers another
perspective of the primal problem;
– there are straightforward and powerful mathematical relations between the
primal and dual problems; this will allow us to develop new algorithms that will be
more efficient in many situations.

Suppose that we have the following primal program:

⎧
⎪
⎨ max z(x) = cx
(P ) s.t. Ax ≤ b,
⎪
⎩
x ∈ Rn+
24 Optimizations and Programming

Its dual program is as follows:

⎧
⎪
⎨ min w(y) = yb
(D) s.t. yA ≥ c,
⎪
⎩
y ∈ Rm +

R EMARK.– The primal and dual programs satisfy the following relations:
– m = number of constraints of (P ) = number of variables of (D),
– n = number of variables of (P ) = number of constraints of (D).

If (P ) has two constraints, (D) has two variables and can be solved graphically,
regardless of the number of variables of (P ).

So, what is the form of (D) when (P ) is not in canonical form?

In such a case, we can determine (D) by reducing (P ) to canonical form, as shown

in the following example.

E XAMPLE 1.10.– Consider the LP:

⎧
⎨ max z(x) = c x
⎪ T
 
(P ) A b
⎪
⎩ s.t. Ax = b, x ≥ 0 ⇔ x≤ , x≥0
−A −b

By the above definition, the dual of (P ) is given by

⎧ 
⎪
⎪ b
⎪
⎨ min w(u, v) = (u, v) −b = (u − v)b
⎪
(D) 
⎪
⎪ A
⎪
⎩ s.t. (u, v) −A = (u − v)A ≥ c, u, v ∈ R+ .
⎪ m

Setting y = u − v gives

(D) min w(y) = yb s.t. yA ≥ c, y with no sign restriction (denoted (y)).

 Linear Programming 25

In general, we have:

Primal Dual
Maximization Minimization
Coefficient of z Right-hand side of the constraints
Right-hand side of the constraints Coefficient of w
⎧ ⎧
⎪
⎨= ⎪
⎨ no sign constraint
Constraint ≤ Variable ≤
⎪
⎩ ⎪
⎩
≥ ≥
⎧ ⎧
⎪
⎨ no sign constraint ⎪
⎨=
Variable ≥ Constraint ≤
⎪
⎩ ⎪
⎩
≤ ≥

T HEOREM 1.8.– The dual of the dual is the primal.

1.8.1. Duality theorem

Now that we have defined the dual of a linear program, let us study the links
between the solutions of programs that are dual to one another.

T HEOREM 1.9.– Let (x, y) be a feasible solution of (P̄ ) and (u, v) a feasible solution
of (D̄). Then:
1) z(x, y) ≤ w(u, v);
2) z(x, y) = w(u, v) ⇒ (x, y) and (u, v) are optimal solutions of (P̄ ) and (D̄).

T HEOREM 1.10.– One of the following three cases is satisfied:

1) (P̄ ) and (D̄) have optimal solutions and max z(x, y) = min w(u, v);
2) one of (P̄ ) and (D̄) has a feasible solution, but not the other;
3) neither (P̄ ) nor (D̄) have feasible solutions.

T HEOREM 1.11.– Let (x, y) (respectively, (u, v)) be feasible solutions of (P̄ )
(respectively, (D̄)). Then (x, y) and (u, v) are optimal solutions of (P̄ ) and (D̄) if
and only if the following statements hold:
26 Optimizations and Programming

– if one constraint is strictly an inequality in (P̄ ) (respectively, (D̄)), then the

corresponding variable of (D̄) (respectively, of (P̄ )) is zero;
– if the value of a restricted variable (i.e. a positive variable or a negative variable)
in one of the programs (P̄ ) or (D̄) is not zero, then the corresponding constraint in the
other program is an equality.

This theorem can alternatively be stated as follows:

T HEOREM 1.12.– Let x and p be admissible solutions of the primal and dual
programs, respectively. Then the vectors x and p are, respectively, optimal solutions
of the two problems if and only if:

pi (aTi x − bi ) = 0 ∀i
(cj − pT Aj )xj = 0 ∀j

Application

Consider the LP

min 13x1 + 10x2 + 6x3

s.t. 5x1 + x2 + 3x3 = 8
3x1 + x2 = 3
xi ≥ 0

The dual of this program is:

max 8p1 + 3p2

s.t. 5p1 + 3p2 ≤ 13
p1 + p2 ≤ 10
3p1 ≤ 6

The solution of the primal is x∗ = (1, 0, 1)T .

The optimal dual solution can be constructed from additional slack conditions:
– the condition pi (aTi x∗ − bi ) = 0 is satisfied because x∗ is admissible;
– the condition (cj − pT Aj )xj = 0 is satisfied for j = 2:
- for j = 1, this condition becomes:

5p1 + 3p2 = 13.

Discovering Diverse Content Through
Random Scribd Documents
discourses, recollecting my experience as the dupe of similar
expectations.
“I was tempted, nevertheless, by a Greek who had a process with
cinnabar, which failed. At the same time I became acquainted with a
strange gentleman, newly arrived, who often, in my presence, sold
the fruit of his operations to the goldsmiths. I was a long time
frequenting his company, but he did not consent to inform me of his
secret. At last I prevailed over him, but it was only a refinement of
metals more ingenious than the rest. I failed not to write to the abbé,
at Toulouse, enclosing a copy of the process of the stranger, and
imagining that I had attained some useful knowledge, he advised me
to remain another year at Paris, since I had made so good a
beginning.
“After all, as to the philosophers’ stone, I succeeded no better than
before. I had been three years at Paris, and my money was nearly
expended, when I had a letter from the abbé, informing me that he
had something to communicate, and that I should join him as soon
as possible.
“On my arrival at Toulouse, I found that he had a letter from King
Henry of Navarre, who was a lover of philosophy, and who requested
that I should proceed to Pau, in Berne, to teach him the secret I had
received from the stranger at Paris. He would recompense me with
three or four thousand crowns. The mention of this sum exhilarated
the abbé, and he never let me rest till I set out to wait on the prince. I
arrived at Pau in May 1542. I found the prince a very curious
personage. By his command I went to work, and succeeded
according to the process I knew. When it was finished I obtained the
expected recompense, but although the king wished to serve me
further, he was dissuaded by the lords of his court, even by those
who had engaged me to come to him. He dismissed me with great
acknowledgments, desiring me to see if there was anything in his
estates which would gratify me, such as confiscations or the like, and
that he would give them to me with pleasure. These promises, which
meant nothing, did not lead me to entertain the hopes of a courtier,
and I returned to the abbé at Toulouse.
 “On my road I heard of a religious man, who was very skilful in
natural philosophy. I went to visit him; he lamented my misfortunes,
and said, with a friendly zeal, that he advised me to amuse myself no
longer with these various particular operations, which were all false
and sophistical, but that I should rather peruse the best books of the
ancient philosophers, as well to know the true matter as the right
order that should be pursued in the practice of this science.
“I felt the truth of this safe counsel, but before I put it in execution, I
went to see my friend at Toulouse, to give him an account of the
eight hundred crowns that we had put in common, and to divide with
him the recompense I had received from the King of Navarre. If he
proved not content with all I told him, he was still less so at the
resolution I had taken to discontinue my operations. Of our eight
hundred crowns, we had but eighty-six left. I departed from him, and
returned home, intending to go to Paris, and there remain until I was
fixed in my theory of reading the works of the adepts. I reached Paris
in 1546, and remained there a year, assiduously studying the Turba
Philosophorum, the good Trévisan, the “Remonstrance of Nature,”
and some other of the best books. But as I had no first principles, I
knew not on what to determine.
“At length I went out of my solitude, not to see my old
acquaintances, the searchers after particular tinctures and minor
works, but to frequent those who proceeded in the great process by
the books of the genuine adepts. I was, nevertheless, disappointed
herein, by the confusion and disagreement of their theories, by the
variety of their works, and of their different operations. Excited by a
sort of inspiration, I gave myself up to the study of Raymond Lully
and Arnold de Villa Nova. My reading and meditation continued
another year. I then formed my plan, and only waited to sell the
remainder of my land to enable me to go home, and put my
resolution into practice. I commenced at Christmas, 1549, and after
some preparations, having procured everything that was necessary, I
began my process, not without inquietude and difficulty. A friend said
to me:—“What are you going to do? have you not lost enough by this
delusion?” Another assured me that if I continued to purchase so
much coal, I should be suspected of counterfeiting coin, of which he
had already heard a rumour. Another said I ought to follow my
business of a lawyer. But I was chiefly tormented by my relations,
who reproached me bitterly with my conduct, and threatened to bring
the officers of justice into my house to break my furnaces in pieces.
“I leave you to judge my trouble and grief at this opposition. I found
no consolation but in my work, which prospered from day to day, and
to which I was very attentive. The interruption of all commerce, which
was occasioned by the plague, gave me the opportunity of great
solitude, in which I could examine with undisturbed satisfaction the
success of the three colours which mark the true work. I thus arrived
at the perfection of the tincture, and made an essay of its virtue on
common quicksilver, on Easter Monday, 1550. In less than an hour it
was converted into pure gold. You may guess how joyful I was, but I
took care not to boast. I thanked God for the favour he shewed me,
and prayed that I should be permitted to use it but for His glory.
“The next day I set out to find the abbé, according to the promise
we gave each other, to communicate our discoveries. On my way, I
called at the house of the religious man who had assisted me by his
good advice. I had the grief to find that both he and the abbé had
been dead about six months. I did not go back to my house, but
sought another place, to await the arrival of one of my relations
whom I had left at my dwelling. I sent him a procuration to sell all that
I possessed, both house and furniture, to pay my debts, and to
distribute the remainder among those of my relations who were in
want. He soon after rejoined me, and we set out for Lausanne, in
Switzerland, resolved to pass our days without ostentation in some
of the celebrated cities of Germany.”
In his unknown retreat[V] the adept recorded his adventures and
experiences when in search of the philosophical stone, ut divertarem
bonos piosque vivos, à sophisticationibus, ad viam rectam
perfectionis in hoc opere divino. His little work is entitled simply
Opusculum Chemicum; it opens with the romantic narrative which I
have cited almost in extenso. It calls Hermes magnus propheta
noster, insists that the art is the gift of God alone on the authority of
all the initiates, and quotes so largely from previous writers that it
can scarcely be considered an original work on the Hermetic
philosophy.
The life of Bernard Trévisan has abundantly testified to the
physical nature of his object, which is amply confirmed by this
treatise. The methods of projection upon metals, the composition of
precious stones, and the application of the tincture as a medicine for
the human body, are successively considered. One grain of the
divinum opus, dissolved in white wine, transmutes that liquor into a
rich citron colour, and has innumerable hygienic uses.

FOOTNOTES:
[V] See Appendix I.
 BERIGARD OF PISA.
The following account of a transmutation performed by himself, is
recorded by the celebrated Italian philosopher, Claude Berigard, and
will be found on the twenty-fifth page of his Circulus Pisanus,
published at Florence in 1641.
“I did not think that it was possible to convert quicksilver into gold,
but an acquaintance thought proper to remove my doubt. He gave
me about a drachm of a powder nearly of the colour of the wild
poppy, and having a smell like calcined sea-salt. To avoid all
imposition, I purchased a crucible, charcoal, and quicksilver, in which
I was certain that there was no gold mixed. Ten drachms of
quicksilver which I heated on the fire were on projection transmuted
into nearly the same weight of good gold, which stood all tests. Had I
not performed this operation in the most careful manner, taking every
precaution against the possibility of doubt, I should not have believed
it, but I am satisfied of the fact.”
 CHARNOCK.
Thomas Charnock was born in the Isle of Thanet, in the year
1524. He calls himself an unlettered scholar, and student in
astronomy and philosophy. He practised surgery, and, though he
knew only the rudiments of Latin, it appears that he was famous in
the neighbourhood of Salisbury, where he had established himself,
for his accomplishments in the liberal sciences. He had two masters
in alchemy, the first being Sir James S——, a priest, dwelling in the
cloisters, near Salisbury, who informed Charnock that he did not
derive his knowledge from any living adept, but that by meditation
upon the words of the philosophers, he had mastered the principal
secrets of alchemy as he lay in his bed, and had accordingly
succeeded in making the silver powder.
The other master who instructed Charnock was a blind man, led
by a boy, whom the neophyte accidentally discovered at an inn
among other travellers, by a few words of the occult chemistry, which
he perceived in his conversation. As soon as the company had
retired, Charnock questioned the speaker, and requested instruction
in natural philosophy. To this the adept objected that he was
unacquainted with his interrogator, saying he would render up his
knowledge to God who gave it, if he did not meet with a certain
Master Charnock, the fame of whose learning and charity had
reached him.
At these words Charnock made himself known, and the old man
discoursed with him for an hour, during which time he found him
expert in many mysteries of the sacred science. He promised
Charnock that if he made a vow not to reveal the secret for gold,
preferment, or through affection for great men, but only at death to
one who was truly devoted to the search into nature, he would make
him the heir of his knowledge. Accordingly, on the following Sunday
they received the Eucharist together, and then, withdrawing into the
middle of a large field, the boy was sent away out of hearing, and, in
a few words, the blind man uttered “the mystery of mineral
prudence.” Their conversations were continued for nine days. The
secrets of alchemy were disclosed, and the adept also related his
own private history, acquainting Charnock that his name was William
Bird, that he had been a prior of Bath, and had defrayed the expense
of repairing the abbey church from treasure which he had acquired
by means of the red and white elixirs. At the suppression of the
abbey, he concealed the inestimable powder in the wall, and
returning in ten days it was gone. He found a few rags in the place
where he had left it. This misfortune almost deprived him of his
senses; he wandered about, and lost his sight. He was therefore
unable to repeat his process, and continued to travel over the
country, led by a boy. He had received his Hermetic knowledge from
a servant of Ripley.
At the time of this communication, Charnock was twenty-eight
years old, and two years after his first master fell sick while attending
his furnace for the completion of the red stone. He sent for
Charnock, made him the heir of his work, and died after giving him
instructions how to proceed. Charnock began his operations on the
materials left by his leader, and was much perplexed by the difficulty
of keeping the fire equal. He often started out of his sleep to examine
the fuel; but after all his care, which continued during the space of
several months, the frame of wood that covered the furnace took fire
during a short period of his absence, and when, smelling the
burning, he ran up to his laboratory, he discovered that his work was
completely destroyed. This occurred on January 1, 1555. To repair
the mischief he was obliged to recommence at the first part of the
process, and he hired a servant to assist in taking care of the fire. In
the course of two months certain signs filled him with hopes of
success, when his dependence on his servant proved the ruin of his
work. He discovered that this unfaithful assistant would let the fire
nearly out, and then, to conceal his neglect, would rekindle it with
grease till it was so hot as to scorch the matter beyond recovery.
In the third attempt, Charnock resolved to proceed without help.
His fire cost him three pounds a week, and he was obliged to sell
some rings and jewels to maintain it. He made good progress in the
course of eight months, and expected to be rewarded in a little time
for all his labours; but at this critical period he was impressed to
serve as a soldier at the siege of Calais. Furious with
disappointment, he took a hatchet, smashed his glasses, furnace,
and apparatus, and threw them out of the house.
He wrote his “Breviary of Philosophy” in 1557, and the “Enigma of
Alchemy” in 1572, with a memorandum, dated 1574, when he was
fifty years old. Therein he declares his attainment of the gold-
producing powder when his hairs were white. The “Breviary” claims
to describe all the vessels and instruments which are required in the
science; a potter, a joiner, and a glassmaker must lend their several
services. The address of one of these artificers, specially
recommended by the author, is said to be Chiddinfold in Sussex; he
could manufacture egg-shaped glasses which opened and shut “as
close as a hair.” The regulation of the philosophical fire is described
in this curious poem, but the rest of its information is of a purely
autobiographical kind.
 GIOVANNI BRACCESCO.
This alchemist of Brescia flourished in the sixteenth century. He
was the author of a commentary on Geber, which is not supposed to
cast much light on the obscurities of the Arabian philosopher. The
most curious of his original treatises is Legno della Vita, vel quale si
dichiara la medecina per la quale i nostri primi padri vivevano nove
cento anni, Rome, 1542, 8vo.—“The Wood of Life, wherein is
revealed the medicine by means of which our Primeval Ancestors
lived for Nine Hundred Years.” This work, together with La
Esposizione di Geber Filosophe, Venice, 1544, 8vo, was translated
into Latin, and may be found in the collections of Gratarole and
Mangetus. They were also published separately under the title De
Alchimia dialogi duo, Lugd., 1548, 4to. The Wood of Life is one of
the innumerable names given by the alchemists to the matured and
perfected stone, the composition whereof is the accomplishment of
the magnum opus. It is more generally denominated the Universal
Balsam or Panacea, which cures all diseases and insures to its most
blessed possessor an unalterable youth. The name Wood of Life is
bestowed by the Jews on the two sticks which confine the scroll of
the Law. They are convinced that a simple contact with these sacred
rods strengthens the eyesight and restores health. They also hold
that there is no better means of facilitating the accouchement of
females than to cause them to behold these vitalising sticks, which,
however, they are in no wise permitted to touch.[W]
The work of Braccesco is written in the form of a dialogue, and is
explanatory of the Hermetic principles of Raymond Lully, one of the
interlocutors, who instructs an enthusiastic disciple in the arcane
principles of the divine art, the disciple in question being in search of
a safeguard against the numerous infirmities and weaknesses of the
“humid radical.” Such a medicine is declared by the master to be
extracted from a single substance, which is the sophic aqua
metallorum. The dialogue is of interest, as it shows the connection in
the mind of the writer between the development of metallic perfection
and the physical regeneration of humanity.

FOOTNOTES:
[W] Dictionnaire des Sciences Occultes, i. p. 232.
 LEONARDI FIORAVANTI.
Doctor, surgeon, and alchemist of the sixteenth century, this Italian
was a voluminous author, who is best known by his “Summary of the
Arcana of Medicine, Surgery, and Alchemy,” published in octavo at
Venice in 1571, and which has been reprinted several times. It
contains an application of Hermetic methods and principles to the
science of medicine, but the author’s account of the petra
philosophorum shows the designation to be of a purely arbitrary kind,
for it is a mixture of mercury, nitre, and other substances, intended to
act on the stomach, and has no connection with the transmuting
lapis of the alchemical sages.
 JOHN DEE.
The life of this pseudo-adept, and of Edward Kelly, his companion
in alchemy, is involved in a cloud of necromancy and magico-
Hermetical marvels, so that the fabulous and historical elements are
not to be easily separated.
The true name of Edward Kelly is supposed to have been Talbot.
He is said to have been born at Worcester in 1555, and to have
followed the profession of a lawyer in London. His talents in
penmanship appear to have been utilised in the falsification of
deeds. He was prosecuted at Lancaster, according to a narrative of
his enemies, for an offence of this nature, and was condemned to
lose his ears. By some he is said to have suffered this punishment,
[X] by others to have evaded it, seeking safety in Wales, where he
lodged at an obscure inn, and concealed his identity by adopting a
new name. During this sojourn an old manuscript was shown him by
the innkeeper, which was indecipherable by himself or his
neighbours. The so-called Edward Kelly, being initiated into the
mysteries of ancient writing, discovered it to be a treatise on
transmutation, and his curiosity was highly excited. He inquired as to
its history, and was told that it had been discovered in the tomb of a
bishop who had been buried in a neighbouring church, and whose
tomb had been sacrilegiously uptorn by some wretched heretical
fanatics at that epoch of furious religiomania and rampant
Elizabethan persecution. The object of this desecration was the
discovery of concealed treasures in the resting-place of the prelate,
to whom immense riches were attributed by popular tradition. The
impiety was, however, rewarded by nothing but the manuscript in
question, and two small ivory bottles, respectively containing a
ponderous red and white powder. These pearls beyond price were
rejected by the pigs of apostasy; one of them was shattered on the
spot, and its ruddy, celestine contents for the most part lost. The
remnant, together with the remaining bottle and the unintelligible
manuscript, were speedily disposed of to the innkeeper in exchange
for a skinful of wine. The unbroken bottle was transferred by the new
owner as a plaything to his children, but the providence which in the
main overwatches the accomplishment of the sublime act preserved
its contents intact. When Edward Kelly, with an assumed antiquarian
indifference about objects which were more curious than valuable,
offered a pound sterling for all the articles, a bargain was promptly
effected. The lawyer was by no means an alchemist, but he believed
himself possessed of a Hermetic treasure; he determined, at all
risks, to return to London, and consult with his friend Dr Dee, who
abode in a cottage at Mortlake, and who, in matters of magical
devilment, and in the tortuosities of the occult, was considered a
man of men.
Whether he had been accused of forgery, whether he had lost his
ears, or not, the discovery of Edward Kelly caused the necromantic
doctor to be blind to his faults or his crimes; he at once set to work in
his company, in the year 1579, and in the month of December a
stupendous success was the crown of their labour in common. The
richness of Kelly’s tincture proved to be one upon two hundred and
seventy-two thousand two hundred and thirty; but they lost much
gold in experiments before they knew the extent of its power. In Dr
Dee’s “Diary in Germany” he mentions the book of St Dunstan,
which is probably the manuscript of Kelly, and also the powder
“found at the digging in England,” which indicates some foundation
for the narrative just given. The place where the treasure was
obtained is reported to have been the ruins of Glastonbury Abbey,
founded by St Dunstan. The last abbot was hanged by Henry VIII. for
his adherence to the Papal cause.
Kelly appears to have taken up his quarters at Islington. In June
1583 an attachment was issued against him for coining, of which his
companion declares him guiltless. In the following September, Dr
Dee, his wife and children, and Edward Kelly, with his wife,
accompanied by a certain Lord Albert Alasco, of Siradia, in Poland,
departed from London for Cracow. As soon as they had arrived in
the north of Germany, Dr Dee received a letter from one of his
friends in England, informing him that his library at Mortlake had
been seized and partially destroyed, on the vulgar report of his
unlawful studies, and that his rents and property were sequestered.
Despite the possession of the Donum Dei, all parties appear to have
been in considerable penury in consequence.
In 1585 we find them at Prague, then the metropolis of alchemy,
and the headquarters of adepts and adeptship. Edward Kelly and his
companions presently abounded in money, and the owner of the
Hermaic Benediction made no secret of his prize or his powers,
indulged in all kinds of extravagance, performed continual
projections for himself and his friends, as well as for many persons
of distinction who sought his acquaintance. Much of the result was
distributed. The transmutations of Kelly at this period are attested by
several writers, including Gassendus. The most authenticated and
remarkable, according to Figuier, is that which took place in the
house of the imperial physician, Thaddeus de Hazek, when, by the
mediation of a single drop of a red oil, Kelly transmuted a pound of
mercury into excellent gold, the superabundant virtue of the agent
leaving in addition at the bottom of the crucible a small ruby. Dr
Nicholas Barnaud, the assistant of Hazek, and an alchemical writer,
whose works are as rare as they are reputable, was a witness of this
wonder, and subsequently himself manufactured the precious metal,
the désir désiré, with the assistance of Edward Kelly.
The report spread, and the adept was invited by the Emperor
Maximilian II. to the Court of Germany, where his transmutations
raised him into highest favour; he was knighted, and created Marshal
of Bohemia. Now perfectly intoxicated, he posed as a veritable
adept, who was able to compose the inestimable projecting powder.
This gave a handle to the enemies whom his exaltation had made
him; they persuaded the Emperor to practically imprison this living
philosophical treasure, and to extract his alchemical secret. His
misfortunes now began. Absolute inability to obey the imperial
mandate and compose a considerable quantity of the stone
philosophical, was interpreted as a contumacious refusal; he was
cast into a dungeon, but on engaging to comply with the demand if
he had the liberty to seek assistance, he was speedily set free,
whereupon he rejoined Dr Dee, and they again set to work in
concert. The Book of St Dunstan indicated the use but not the
preparation of the powder, and their experiments, vigilantly
overwatched to prevent the escape of Kelly, proved entirely futile. In
the desperation which succeeded their failure, the outrageous
disposition of Kelly broke out, and he murdered one of his guards.
He was again imprisoned, his companion, for the most part,
remaining unmolested, and employing his opportunities, it is said, to
interest Queen Elizabeth in the fate of the Emperor’s prisoner. She
claimed the alchemist as her subject, but his recent crime had
rendered him obnoxious to the laws of the empire, and he was still
detained in his dungeon.
In 1589, Dr Dee set out himself for England. He halted at Bremen,
and was there visited by Henry Khunrath, one of the greatest adepts
of the age. The Landgrave of Hesse sent him a complimentary letter,
and was presented in return with twelve Hungarian horses. Dr Dee
arrived in England after an absence of six years; he was received by
the Queen, who subsequently visited him at his house, presented
him with two hundred angels to keep his Christmas, and gave him a
license in alchemy. Sir Thomas Jones offered him his Castle of
Emlin, in Wales, for a dwelling; he was made Chancellor of St Paul’s,
and in 1595, Warden of Manchester College. He repaired thither with
his wife and children, and was installed in February 1596. He does
not appear to have accomplished any transmutation after his return
to England. In 1607 we again find him at Mortlake, living on the
revenue which he derived from Manchester, but subject to much
persecution by the Fellows of that College. He died in 1608, at the
age of eighty years.
The Hermetic abilities of Kelly were always believed in by the
Emperor; he continued to detain him, hoping to extract his secret.
Some friends of the unfortunate alchemist endeavoured, in the year
1597, to effect his escape by means of a rope, but he fell from the
window of his prison, and died of the injuries which he received.
During his confinement he composed a treatise on the
philosophical stone, and the Diary of Dr Dee was published from a
genuine Ashmolean manuscript in 1604. The son of John Dee
became physician to the Czar at Moscow, and in his Fasciculus
Chemicus, he states that, in early youth, he witnessed transmutation
repeatedly for the space of seven years.
The metrical account of Sir Edward Kelly’s work in the Theatrum
Chemicum Britannicum informs all who are broiling in the kitchen of
Geber to burn their books “and come and learn of me,” for they can
no more compound the Elixir Vitæ and the precious stone than they
can manufacture apples. The progenitor of magnesia, wife to the
gold of the philosophers, is not a costly thing. The philosophical gold
is not common but Hermetic sulphur, and magnesia is essential
mercury.
The Testamentum Johannis Dee Philosophi Summi ad Johannem
Gwynn, transmissum 1568, is lucidly worded as follows in its
reference to the magnum opus:—

“Cut that in Three which Nature hath made one,

Then strengthen yt, even by it self alone;
Wherewith then cutte the powdered sonne in twayne,
By length of tyme, and heale the wounde againe.
The self same sonne troys yet more, ye must wounde,
Still with new knives, of the same kinde, and grounde;
Our monas trewe thus use by Nature’s Law,
Both binde and lewse, only with rype and rawe,
And aye thank God who only is our Guyde,
All is ynough, no more then at this tyde.”

FOOTNOTES:
[X] Morhof, Epistola ad Langlelotum de Metallorun
Transmutatione.
 HENRY KHUNRATH.
This German alchemist, who is claimed as a hierophant of the
psychic side of the magnum opus, and who was undoubtedly aware
of the larger issues of Hermetic theorems, must be classed as a
follower of Paracelsus. He was a native of Saxony, born about the
year 1560. He perambulated a large portion of Germany, and at the
age of twenty-eight received the degree of medical doctor at the
University of Basle. He practised medicine at Hamburg and
afterwards at Dresden, where he died in obscurity and poverty, on
the 9th of September 1601, aged about forty-five years. The
Amphitheatrum Sapientiæ Æternæ solius veræ, Christiano
Kabbalisticum divino magicum, &c., published in folio in 1609, is the
most curious and remarkable of his works, some of which still remain
in manuscript.[Y] It was left unfinished by its author, appearing four
years after his decease, with a preface and conclusion by his friend
Erasmus Wohlfahrt.
The prologue directs the aspirant to the supreme temple of
everlasting wisdom to know God and Jesus Christ whom He hath
sent, to know also himself, and the mysteries of the macrocosmos.
The whole treatise is purely mystical and magical. The seven steps
leading to the portals of universal knowledge are described in an
esoteric commentary on some portions of the Wisdom of Solomon.
The lapis philosophorum is declared to be identical with the Ruach
Elohim who brooded over the face of the waters during the first
period of creation. The Ruach Elohim is called vapor virtutis Dei, and
the internal form of all things. The perfect stone is attained through
Christ, and, conversely, the possession of that treasure gives the
knowledge of Christ. The Amphitheatrum Sapientiæ Æternæ seems
to be the voice of the ancient chaos, but its curious folding plates are
exceedingly suggestive.
 FOOTNOTES:
[Y] Chausepié, Dictionnaire.
 MICHAEL MAIER.
This celebrated German alchemist, one of the central figures of
the Rosicrucian controversy in Germany, and the greatest adept of
his age, was born at Ruidsburg, in Holstein, towards the year 1568.
In his youth he applied himself closely to the study of medicine, and
establishing himself at Rostock, he practised that art with so much
success that he became physician to the Emperor Rudolph II., by
whom he was ennobled for his services. Some adepts,
notwithstanding, succeeded in enticing him from the practical path
which he had followed so long into the thorny tortuosities of
alchemical labyrinths. Il se passionna pour le grand œuvre and
scoured all Germany to hold conferences with those whom he
imagined to be in possession of transcendent secrets. The
Biographie Universelle declares that he sacrificed his health, his
fortune, and his time to these “ruinous absurdities.” According to
Buhle,[Z] he travelled extensively; and on one occasion paid a visit to
England, where he made the acquaintance of the Kentish mystic,
Robert Fludd.
He appears as an alchemical writer a little before the publication of
the Rosicrucian manifestoes. In the controversy which followed their
appearance, and which convulsed mystic Germany, he took an early
and enthusiastic share, defending the mysterious society in several
books and pamphlets. He is supposed to have travelled in search of
genuine members of the “College of Teutonic Philosophers R.C.,”
and, failing to meet with them, is said to have established a
brotherhood of his own on the plan of the Fama Fraternitatis. These
statements rest on inadequate authority, and there is better ground
for believing that he was initiated, towards the close of his life, into
the genuine order. A posthumous tract of Michael Maier, entitled
“Ulysses,” was published in 1624 by one of his personal friends, who
added to the same volume the substance of two pamphlets which
had already appeared in German, but which, by reason of their
importance, were now translated into Latin for the benefit of the
literati of Europe. The first was entitled Colloquium
Rhodostauroticum trium personarum, per Famam et Confessionem
quodamodo revelatam de Fraternitate Roseæ Crucis. The second
was an Echo Colloquii, by Benedict Hilarion, writing in the name of
the Rosicrucian Fraternity. It appears from these pamphlets that
Maier was admitted into the mystical order, but when or where is
uncertain. He became the most voluminous alchemical writer of his
period, publishing continually till his death in the year 1622.
Many of his works are Hermetic elaborations of classical
mythology, and are adorned with most curious plates. They are all
hopelessly obscure, if his Rosicrucian apologies be excepted; the
latter are not deficient in ingenuity, but they are exceedingly
laboured, and, of course, completely unsatisfactory. He does not
appear to have been included among the adepts, and he is now
almost forgotten. His chemical knowledge is buried in a multitude of
symbols and insoluble enigmas, and believers in spiritual chemistry
will not derive much comfort or profit from his writings.

FOOTNOTES:
[Z] See De Quincey’s “Rosicrucians and Freemasons.”
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookmass.com