A-Generalized-Reduced-Gradient-Algorithm-for-Solving-Large_1989_IFAC-Proceed
A-Generalized-Reduced-Gradient-Algorithm-for-Solving-Large_1989_IFAC-Proceed
45
46 J. L. D. Fac6
submitted to linear constraints and simple bounds We introduce certain priority principles in order
on the variables: to exhibit an easy working structure in the GRG
basic matrix:
n
Max {f(x)IAx=b, al :£ x :£ a2' X6R }, A(mxn) is a
P1 - introduce in the basis as many state
rank m matrix and f is a differentiable nonlinear
variables as you can.
function.
P2 - if a state variable is at a bound look for a
Abadie and Carpentier (1966) extended the approach substitute among the controls of the same
to the case of general nonlinear equality time period.
constraints:
P3 - if you cannot use P2, keep the state variable
in the "working basis" and introduce in the
Max {f(x) ICi (x)=O, i=l ,2, ... ,m, a:; x:; b, x 6 Rn},
"GRG basis" a control of a preceding period
f and ci are differentiable functions on an open
and build the elementary transformation
set containing P = {a :;; x S b } .
matrix (pivoting).
The GRG method consists in the solution of a P4 - do a LU decomposition of each diagonal block.
sequence of unconstrained nonlinear programmes
in a reduced dimension space, a manifold defined Using these principles we can exhibit basic matrix
by the nonlinear equations, that stops when a factorizations that minimize the fill-in in the
constrained local optimal solution is reached basis representation and present good numerical
(a Kuhn-Tucker point). stability.
procedure FORW (x 8 ,u 8 ), and, by adjusting 8 a QN part and a CG part, and has been considered
if necessary, find a new point so that: the best one; but recent numerical comparisons
made independently by Liu and Nocedal (1988) and
8 8
L(x ,u ) > L(x,u); by Gilbert and Lemarechal (1988) show a better
performance of Nocedal's method. This method is a
(iii) go to step 1; BFGS that uses the m most recent changes in the
gradients and the solution estimates to compute a
QN search direction.
Remarks Concerning the Algorithm
(a) line search: as conjugate gradients are very Consideration of the Simple Bounds on the Variables
sensitive to line searches, in step 4(i) we
compute an optimal stepsize as in the original We shall consider two approaches to deal with these
GRG method [see Abadie (1975)]. constraints trying to preserve the superlinear
convergence property of the unconstrained methods.
(b) in step 4(ii) a basic variable may hit a The first one was introduced by Rosen (1960) - it
bound: we do a change-of-basis choosing the is based on gradient projections, manifold
entering variable by the application of the suboptimizations and active set strategies;
same principles used for the reinversions. Lenard (1979) presents a numerical comparative
study about these procedures.
(c) when the number of factors in the basic matrix
representation attains a predetermined number Stimulated by the deficiencies of these methods in
a reinversion is automatically done. large-scale problems, Murtagh and Saunders (1978)
presented a reduced gradient for linear constraints
(d) different levels of anti-degeneracy techni- extending the revised simplex techniques to a
ques are employed to avoid cycling risks. nonlinear objective function implemented in their
code MINOS that incorporates bound constraints.
(e) time delays have only a passive role in the Some of the n-m independent variables (called
matrix factorization strategy, been considered nonbasic) are fixed at a bound and they use a
by a specific sparse matrix procedure, not conjugate gradient or a BFGS subprogramme to solve
interfering in the principles stated before. the sucessive unconstrained manifold optimizations
and introduce matrices updates to manage the two
(f) other type of constraints can be incorporated situations: (i) a superbasic variable hits a bound
to the formulation (1)-(4), like supplementary and becomes nonbasic, and (ii) a basic variable
inequalities by the introduction of slack hits a bound and is exchanged with a superbasic.
variables and the augmentation of the state
vector dimension. Shanno and Marsten (1982) proposed a similar
reduced gradient formulation with Shanno's
(g) if the initial state vector is free, we just memoryless quasi-Newton CG method trying to choose
consider the control vector of the first the largest set of superbasics at each reinversion,
period with m+n components. and a more efficient treatment of the binding
situations. Comparative numerical experiments
(h) a Fortran code has been designed by Faco also show the advantage in allowing looser conver-
(1977), and different models for applications gence criteria in the initial manifolds.
in electric energy generation, macroeconomy,
and fishery management have been solved by The other approach was introduced by Bertsekas
Faco (1979, 1980, 1983, 1988). (1982a, b) to solve the problem
Min {[(x) Ix ~ 0, x € Rn} proposing projected Newton
methods of the form:
LIMITED-STORAGE QUASI-NEWTON METHODS
xk+1 = [ xk.- <:kDkgk ] + , where.[0]+ d enotes proJect1on
.,
Conjugate gradient algorithms have been widely on the pos1t1ve orthant, uk 1S a stepsize obtained
used to optimize large-scale problems because they by an Armijo-like rule and Dk is a symmetric
use very few memory locations, O(n), but their positive definite matrix partly diagonal.
convergences are quite slow; quasi-Newton methods
are very fast but need a large amount of storage, A key property of this technique is the capacity
0(n2), to approximate the objective function to identify the manifold of binding constraints
hessian matrix. Common properties of these at a solution in a finite number of iterations.
methods were analyzed by many people like Nazareth To avoid feasible directions algorithms zigzag
(1979), Shanno (1978), Buckley (1978), Nocedal phenomenous he suggests certain enlargements of
(1980), etc. the binding sets by the definition of scalars €k
to measure the proximity to the bounds. An
Limited-storage methods combine properties of extension to deal with double bounds is also
Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi- described.
Newton method and conjugate gradients methods
using variable amount of available extra-storage Bonnans (1983) proposes a projected variable
(a controlled mUltiple of n). The performance of metric method for bound constrained optimization
CG methods can be substantialy increased by the following Bertsekas techniques proving superlinear
use of supplementary information. convergence for a different linestep choice:
Wolfe's rule [see Wolfe (1969), Lemarechal (1981)].
We consider the method CONMIN by Shanno and Phua Bonnans (1987) presents some extensions to limited-
(1978), VSCG by Buckley and LeNir (1983) and memory methods.
L-BFGS by Nocedal (1980) proposed to solve large-
scale unconstrained nonlinear programmes. All of Conn, Gould and Toint (1988) extend the global
them present theoretical superlinear convergence. convergence properties of trust region algorithms
from More (1982) to the case with simple bounds,
Buckley-LeNir's method is an extension of the including results for nonconvex functions by the
memoryless quasi-Newton algorithm introduced by Bertsekas's projected operator methodology.
Shanno (1978) that uses 2 pairs of vectors to
approximate the second order information, to a
predetermined variable number of vectors depending
on the available extra storage. The algorithm has
48 J. L D. Faco
Model
Modifications in the GRECO Algorithm Use the
Following Techniques Min C
1- Define the set of active bounds:
k s. t.
< k<
I~={j la j =x.=a.+€ and g. < 0 or
J J J
X. =X. +
l,t 1,t-1 kEM
I (Qkt-+Vkt_ ) -Q. t-V, t'
b. _E:k:£x~:;b. and g. > O} .I: i ' T , T 1, 1,
J J J J
The tunas fisheries along the southeastern coast Abadie, J. (1970). Application of the GRG method
in Brazil are important economical activities. to optimal control problems. In J. Abadie
Optimal Control models have been built by Thomaz, (Ed.), Integer & Nonlinear Programming, North
Gomes and Face (1985), Thomaz (1986) and Face Holland, Amsterdam. pp. 191-211.
(1988) considering interacting mUltispecies fish Abadie, J. (1975). Methode du gradient reduit
populations submitted to increasing fishery generalise. Note EDF HI 1756/00, Clamart,
pression. Regulation and limitation of the France.
quantities captured are most important to achieve Abadie, J.,and M. Bichara ( 1973). Resolution nu-
a maximum sustainable yield. The following model merique de certains problemes de commande
is a modified version of the Lotka-Volterra multi- optimale. Rev. Fran~. d'Autom., Inform. et
species interaction and a bioeconomic objective Rech. Oper., 7, 77-105.
function. Abadie, J., and J.-Carpentier (1966). Generalisa-
tion de la methode du gradient reduit de Wolfe
Three competitive tuna species are considered. au cas de contraintes non lineaires. In D.B.
Hertz & J. Melese (Eds.), 4th IFORS Conf.,
We can represent the dynamic ecosystem by coupled Wiley, New York. pp. 1041-1053.
difference equations: Bertsekas, D. (1982a). Constrained Optimization
and Lagrange Multipli er Methods. Academic
xi(t+1)-xi(t)=[ri-ailxl (t)-ai2x2(t)-ai,x3(t)] Press, New York.
Bertsekas, D. (1982b). Projected Newton methods
xi(t), for optimization problems with simple
constraints. SIAM J. Contr. & Opt., 20,
where, 221-246. --
Bonnans, J.F. (1983). A variant of a projected
xi(t) = biomass of population i at time t,
variab le metric method for bound constrained
- r = net growth rate of population i, optimization problems, RR242, INRIA-
i
Rocquencourt, France.
- a ij = !~~i:!~~~a~i~~c~,Of the population on
Bonnans, J.F. (1987). Methodes a metrique vari-
able et programmation quadratique sucessive.
for i=l,2,3, and t=O,l,2, ... ,T. Note EDF-INRIA, France.
Buckley, A. (1978). A combined conjugate gradient
The inhibitor effects ai· are estimated taking quasi-Newton minimization algorithm. Math.
into account the s clust~rs generat ed by the Progr., 15, 200-210.
optimal spatial distr ibu t ion, Buckley, A., and A. LeNir (1983). QN-like variable
storage conjugate gradients. Math. Progr., ~,
r. 155-173.
1
a .. K. Conn, A.R., N.I.M. Gould, and P.L. Toint (1988).
lJ
C ij
1 Global convergence of a class of trust region
algorithms for optimization with simple
where,
bounds. SIAM J. Num. An., 25, 433-460.
Drud, A. (1985). CONOPT: a GRG-Code for large
s
sparse dynamic nonlinear optimization pro-
L pih Pjh
blems. Math. Progr., 31, 153-191.
h=i
c .. i,j=1 ,2, ... ,il, Face, J.L.D. (1977). Commande optimale des syste-
lJ s l mes dynamiques non-lineaires a retards avec
L P.
in contraintes dlinegalites sur lletat et la
h=i
commande: une specialisation de la methode
GRG. Dr.lng theSiS, Unlversite Pierre et
and:
Marie Curie, Paris, France.
percentage of the population i in the Face, J.L.D. (1979). Application of the GRECO
cluster h, algorithm to the optimal generation
scheduling for electric power systems. X Int.
- K.1 carrying capacity of the environment for
Symp. on Math. Progr., Montreal, Canada-.-----
the population i in absence of other
Face, J.L.D. (1980). Optimization of a dynamical
competing species (saturation level ).
planning model by the GRG method. V. Symp.
Uber OR, KHln, Germany.
Let us cons ider the optimal control model:
Face, J.L.D. (1983). Optimization of a dynamical
ecosystem by Nonlinear Programming. 11th IFIP
T 3
-Ot Conf. Syst. Mod. & Opt., Copenhagen, Danmark.
Max J= L e L diui(t), Face, J.L.D. (1988). Mathematical Programming
t=O i=l
solutions for fishery management. In G. Mitra
(Ed.), Mathematical Models for Decision
so that,
Support, NATO ASI series, v.F48, Springer-
3
Verlag, Heidelberg pp. 197-205.
x. (t+1)-x. (t)=[r.-
1 1 1
L a .. x.J (t) Ix. (t)-u. (t),
j =1 lJ 1 1 Gilbert, J.C., and C. Lemarechal (1988). Some
numerical experiments with variable-storage
0i ~ ui(t) $ Si ' quasi-Newton algorithms. IIASA work.p. WP-88-
-121, Laxenburg, Austria.
Lemarechal, C. (1981). A view of line searches.
i=1,2,3, and t=O,l,2, .. . ,T,
In A. Auslender, W. Oettli & J. Stoer. (Eds.),
Optimization & Optimal Control , Lect. Notes
where, 30, Springer-Verlag, Heidelberg. pp. 59-78.
Lenard, M. (1979). A computational study of
- ui(t) = harvesting rate of population i,
active set strategies in nonlinear program-
- d = unitary profit of harvesting population i, ming with linear const raints. Math. Progr.,
i
16, 81-97.
- T = planning horizon,
Liu,-n.C., and J. Nocedal (1988). On the limited
- ai' Si' a i and b i bounds on xi and u i ' memory BFGS method for l arge scale optimiza-
tion. Tech. Rep. NAM 3, North Western Univ.,
- xi(O) known, and 6 = medium discount rate on
Evanston , USA.
[0, T] .
50 J. L. D. Fac6