Interior-Point Methods in Model Predictive Control

Stephen Wright
University of Wisconsin-Madison

ICIAM, Zurich, July, 2007

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

1 / 30

Introduction
Model Predictive Control has strong ties to optimization. Many control problems can be viewed as highly structured optimization problems. In MPC, need to solve in real time so eciency is important. Interior-point methods are good at exploiting structure. Each subproblem has the same structure dimension and nonzero distribution in contrast to active-set methods. Can leverage any eciencies in the subproblem solution technique across the whole algorithm and across all timepoints. Hence, there is a natural match between interior-point methods and MPC. Of course, optimization has many other contributions to make to control and MPC.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 2 / 30

Outline

Exploiting Structure in Control Problems

Interior-point methods

The MPC Context: Parametrized Optimization

Warm starting interior-point methods Active-set methods and homotopy Enumeration methods Nonlinear MPC and parametrized NLP

Robust MPC: Two Paradigms

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

3 / 30

Linear MPC
min (u, x) =
u,x

1 2

T T T (xk Qxk + uk Ruk + 2xk Muk ), k=0

subject to x0 = xj , xk+1 = Axk + Buk , Duk Gxk d.

An optimization problems with innitely many variables: xj is the current state of the system (or an estimate) xk is the state at k sampling steps into the future uk is the input at state k. Assume R symmetric positive denite, Q MT
Stephen Wright (UW-Madison)

M R

positive semidenite.
ICIAM 2007 4 / 30

Interior-Point Methods in MPC

Reduction to Finite Dimensions

If the auxiliary constraints Duk Gxk d become inactive after a certain stage N, we can derive a control law uk = Kxk , k N, and a total cost-to-go 1 T x QxN 2 N for the tail of the sequence, by solving a Riccati equation: Q = Q + K T RK + (MK + K T M T ) + (A + BK )T Q(A + BK ). Can thus pose the following nite-dimensional linear MPC problem: min (u, x) =
u,x

1 2

N1 k=0

1 T T T T (xk Qxk + uk Ruk + 2xk Muk ) + xN QxN 2

subject to x0 = xj and xk+1 = Axk + Buk , Duk Gxk d,

Stephen Wright (UW-Madison)

k = 0, 1, . . . , N 1 k = 0, 1, . . . , N 1.
ICIAM 2007 5 / 30

Interior-Point Methods in MPC

Many possible enhancements: endpoint constraints FxN = 0 to force unstable modes to vanish at the end of the horizon. soft constraints on xk , enforced with
1

or sum-of-squares penalties.

rate-of-change input constraints, formulated via change of variables. The essentially stagewise structure can be exploited in interior-point methods.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

6 / 30

Interior-Point Methods: Convex QP

Given a convex QP of the form min 1 T x Qx + c T x s.t. Ax b, 2

we can write optimality conditions as follows: Qx + c AT = 0, Ax + s = b,

0 s 0. A general iterate of a primal-dual method (x, s, ) satises s > 0, > 0, and the pairwise products si i are roughly similar: i si = (average value of the i si s) (central path condition).
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 7 / 30

def

Primal-dual interior-point steps are obtained from these linear systems: rc x Q AT 0 A 0 I = rb , rs s 0 S where S = diag(s1 , s2 , . . . ), = diag(1 , 2 , . . . ), and rc = Qx AT + c, rb = Ax b, rxs = Se e. Take step (x, s, ) (x, s, ) + (x, s, ), choosing to retain the property (, s) > 0. (Impose some other heuristics on that have the practical eect of keeping the next iterate near the central path.)
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 8 / 30

By eliminating s we obtain Q AT A 1 S x = rc rb 1 rs .

For the linear MPC problem, we can do a symmetric permutation of this coecient matrix, using a stagewise ordering the primal and dual variables. We obtain a banded coecient matrix, where the bandwidth is proportional to the number of primal-dual variables at each stage. Can thus solve the linear system in O(N) operations (N = number of stages), but cubic in the dimension of the state and input vectors. Can also devise a specialized block elimination scheme that is equivalent to solving a time-dependent Riccati equation.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

9 / 30

Banded Matrix Structure

(u1 ) R B D

(p1 ) B T

(1 ) D T

(x2 ) (u2 )

(p2 )

(2 )

(x3 )

I S1 1 1 I Q MT A G M R B D AT B T GT D T I S2 1 2 I

Can apply a banded LU-factorization.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

10 / 30

Alternative (Rao, W, Rawlings 1997) uses a Riccati substitution scheme. Eliminate the i from this system to obtain: (u1 ) R1 B (p1 ) B T I I Q2 T M2 A M2 R2 B AT B T I
1 where R1 = R + D T 1 S1 D, etc.

(x2 ) (u2 )

(p2 )

(x3 )

I Q3

Seek parametrization of the costate solution components in terms of the states: pk1 = k xk k , k = N, N 1, . . . , 2, where k and k are solved using a discrete-time Riccati recurrence starting with N = QN .
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 11 / 30

Software

The problem structure can be exploited independently of the algorithmic strategy. This suggests an object-oriented approach. The interior-point convex QP code OOQP is an example of such a code. To make it ecient for MPC, need to supply C++ code for data structures for the variables, right-hand side, and problem data, and routines to manipulate these structure; code to solve the linear system eciently. Many users of OOQP appear to have used it in this way. e.g. the MPT (Multi-Parametric Toolbox) at http://control.ee.ethz.ch/mpt/

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

12 / 30

The MPC Context

In MPC, we solve a problem like this at timepoint tj , with given initial state xj , and use the u0 component of the solution as the input. We can view the linear MPC problem as a structured parametrized QP, where xj is the parameter. At the next timepoint tj+1 , formulate and solve a new problem, with new initial state xj+1 . Its usually similar, but shifted forward one timepoint; changes due to disturbances; changes due to model error. Generally xj+1 is not the same as the x1 predicted from the model. Still, we may be able to use the current solution as a warm start. Can interior-point methods exploit this warm start? What about alternatives?
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 13 / 30

Warm Starting?
When the model is good and the disturbances are small, the solution of the MPC problem at one time point can be used to derive a good approximate solution of the problem at the next time point just shift the primal-dual solution by one stage. However this does not give a good starting point for the interior-point solver, as it requires an interior starting point, i.e. one that satises the inequality constraints in the optimality conditions strictly. (Theoretically, interior-point methods need to start from a certain neighborhood of a central path, and certainly from a primal-dual point that satises the inequality constraints strictly.) We can massage the interior-point solution to move it away from the boundary, but the improvement over a cold start is usually modest (e.g. factor of 3). For large disturbances, use a cold start.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 14 / 30

Early approaches for general LP/QP included Yildirim and Wright (2002) and Gondzio and Grothey (2003). For example: Shift small iterates to some positive value. Retrace along the central path (moving back into the interior) and do least-squares steps to recover feasibility for the perturbed problem. Perform repeated corrections: several steps of Newtons method for a xed duality gap . Gondzio and Grothey (2006) describe an unblocking scheme, which nds an adjustment to and s such that a full primal-dual step can be taken from (x, s, ) without violating (, s) 0. More news on warm starting of interior-point methods: Grotheys talk.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

15 / 30

Parametrized QP
General form of parametrized QP: min 1 T x Qx + c T x s.t. Ax b + Et, 2

where t IR is the parameter. Optimality conditions are a parametrized monotone linear complementarity problem: Qx + c AT = 0, Ax + s = b + Et, 0 s 0. In the strictly convex case, the primal-dual solution (x(t), s(t), (t)) is a single valued, continuous, piecewise-linear function of the parameter t. The kinks occur where the inequality constraints become active and inactive. Denote A(t) = {i | si (t) = 0}.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 16 / 30

Active Set/Pivoting

Given a primal-dual solution and active set A(t0 ) for some t0 , change to t1 and determine new solution and new active set A(t1 ). A dual-feasible active-set method is ideal, since the solution (x, s, ) for t = t0 is dual feasible also for t = t1 . (The constraints Qx + c AT = 0 and 0 still hold, but Ax + s = b + Et1 is generally violated.) Goldfarb-Idnani dual active-set algorithm (1983) can be used to recover the solution. Bartlett & Biegler (2006) describe an implementation in the code QPSchur that is ecient for linear MPC. Uses object-oriented design, Schur complement form to retain structure as the active set changes, modications for numerical stability and for use of dual-infeasible initial points.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

17 / 30

Pivoting/Homotopy Along a Line

Reduce the parametrization to one dimension by forming a line between t0 and t1 and tracking the pivots (changes in A(t)) along this line. LCP / optimality conditions form of the 1D parametrization is Qx + c AT = 0, Ax + s = b + E (t0 + (t1 t0 )), 0 s 0. Starting with = 0, nd next breakpoint (value of where one of the positive i s or positive si s becomes zero. Make the appropriate change to the active set and move along the (linear) solution path to the next breakpoint. (Ferreau, Bock, Diehl, 2007). This approach has been explored in more general contexts: QP or LCP with a single parameter.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 18 / 30

Enumeration

Although the number of possible active sets A(t) is astronomical (possibly exponential in the number of inequality constraints), only a relatively small number of possibilities are visited in practice. Can precompute solutions for a number of given active sets A. Given a t, we can check whether A is the active set for t by performing a few matrix-vector multiplies / vector additions. If so, the primal-dual solution can be read o. Many authors have studied variants of this approach, e.g. (Pannocchia, Rawlings, W, 2006) who consider a partial enumeration approach that maintains a table of data for the most recently occurring active sets A.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

19 / 30

Nonlinear MPC
Have nonlinear dynamics: xk+1 = fk (xk , uk ), k = 0, 1, 2, . . . with x0 = xj as before. Stagewise constraints may be present as before (including nonlinear constraints). Objective is also stagewise (often quadratic, still). The resulting problem is a structured nonlinear program with parameter xj . Can solve it via SQP, with subproblem solved with a linear MPC solver; apply a nonlinear interior-point method directly, exploiting structure in the solution of each linear system.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

20 / 30

Parametrized Nonlinear Programming

Consider this form of parametrized nonlinear programming: P(t) : min (x) s.t. c(x, t) 0.
x

(cf. the parametrized QP mentioned earlier). KKT optimality conditions: (x) +

x c(x, t)

= 0,

c(x, t) + s = 0, 0 s 0. Given an approximate solution (x, s, ) for P(t0 ) and we wish to nd a solution for P(t1 ). If the approximate solution to P(t0 ) is strictly feasible, can apply implicit function theorem to the central-path conditions, which are: (x) +
x c(x, t)

= 0,

c(x, t) + s = 0, Se e = 0.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 21 / 30

Get sensitivities by solving: x(t0 ) W (x0 , 0 , t0 ) 0 x c(x0 , t0 ) x c(x0 , t0 )T 0 I (t0 ) = 0 S0 0 s (t0 )

xt c(x0 , t0 )0 t c(x0 , t0 )

(The matrix is the same one used in the latest interior-point iteration.) Then dene the approximation to (x(t1 ), s(t1 ), (t1 )) as (x0 , s0 , 0 ) + (x(t0 ), x(t0 ), (t0 ))(t1 t0 ). This approach works best if the active set does not change from t0 to t1 , and when is small. More details in Bieglers talk (?)

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

22 / 30

Alternative: Linearization of KKT conditions

Linearize the optimality conditions and solve the resulting linear complementarity problem: (x0 ) + W (x0 , 0 , t0 )(x1 x0 ) + c(x0 , t0 ) +
x c(x0 , t0 )(x1 x c(x0 , t1 )1

= 0,

x0 ) +

t c(x0 , t0 )(t1

t0 ) + s1 = 0, 0 s1 1 0.

More complicated to solve than a linear system of equations, but possibly tolerates changes in the active set better; does not require interiority of (x0 , s0 , 0 ); can take further Newton-like steps to improve accuracy.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

23 / 30

Robust MPC
Discrete-time system with disturbances: xk+1 = Ak xk + Bk uk + Rk dk , (x0 = xj ) yk = Ck xk + Dk dk , k = 0, 1, 2, . . . , N, with disturbances dk t . Require constraints: (xk , uk ) Kk , xN+1 K . (1)

One possible robust approach: x N, consider only inputs u0 , u1 , . . . , uN such that (1) are satised for all realizations of d0 , d1 , . . . , dN in the disturbance sets 1 , 2 , . . . , N . In MPC, use u0 as the input. This approach is conservative. It does not allow for the for the fact that later inputs uk can be dened using feedback from earlier outputs yk (which contain information about earlier disturbances dk ).
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 24 / 30

Alternative robust approach: Derive a feedback law for uk based on observations y0 , y1 , . . . , yk : Seek gk and Gk such that
t

uk = gk +
=0

Gk y , k = 0, 1, . . . , N.

Require the gk and Gk to be such that for all possible disturbances dk , the constraints (1) are still satised. Clearly less conservative, as it allows feedback in the choice of uk , and hence response to earlier disturbances. We obtain a feasibility problem in the gk and Gk . But the constraints are extremely nonlinear (even if the specications of Kk and K are linear).

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

25 / 30

Ben-Tal, Boyd, Nemirovskii (2005) proposed purication of the output sequence. Assume that an auxiliary sequence yk is computed alongside the actual trajectory: xk+1 = Ak xk + Bk uk (0 = xj ) x yk = Ck xk k = 0, 1, 2, . . . , N. Dene puried outputs: v = y y , = 0, 1, . . . , k, and dene a linear control law using the v :
t def

uk = hk +
=0

Hk v .

The feasibility problem of nding the hk and Hk such that (1) is satised for all disturbances is more tractable than for the earlier parametrization. If the Kk and K are polyhedral and the uncertain sets k are boxes, the constraints on hk and Hk are a (convex) system of linear equalities and inequalities.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 26 / 30

Solving Robust MPC with Interior-Point Methods

(Goulart, Kerrigan, Ralph, 2006). Model has the form xk+1 = Axk + Buk + Rdk , k = 0, 1, 2, . . . , N (x0 = xj ), where disturbance satsies dk

1 and there are additional constraints

Cxk + Duk b. Seek linear control law using the disturbances Rdk from previous stages:
t

uk = hk +
=0

Hk Rd , k = 0, 1, 2, . . . , N.

Minimize a convex quadratic function of the (uk , xk ) for the disturbance-free trajectory: xk+1 = Ak + Buk , k = 0, 1, 2, . . . , N (0 = xj ). x x Can express as a convex QP in the variables hk and Hk , parametrized by xj . But it is large and dense, with O(N 2 ) unknowns. Each iteration of an interior-point method would require O(N 6 ) operations.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 27 / 30

Reformulation
Obtain a structured formulation by reintroducing states and inputs explicitly. The unknowns include a nominal (disturbance-free) state/input sequence, plus an input/control sequence for each basis function in the disturbance space. (There are O(Nl) of these, where l is the number of columns in the disturbance basis matrix R.) The Hessian and Jacobian of the resulting QP have a block-banded / bordered structure. A solution to the linear system at each interior-point iteration can be found in O(N 3 (m + n)3 ) operations. The control-law variables Hk and hk are no longer needed. Just take the rst nominal input u0 to be the input. More details in Kerrigans talk (?)

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

28 / 30

Robust MPC: Another Paradigm

(Diehl, 2006) Consider a linear model xk+1 = Axk + Buk + c, k = 0, 1, . . . , N, where at each stage k, the data (A, B, c) is drawn from a polytopic set: (A, B, c) conv{(A1 , B1 , c1 ), (A2 , B2 , c2 ), . . . , (AK , BK , cK )}. (Possibly a dierent realization at each stake k.) Aim to choose inputs u0 , u1 , . . . , uN to minimize a max of quadratic objectives over the worst-case set of realizations of the (A, B, c)s. Solve by constructing a scenario tree of realizations, branching in time. Constraints represent transitions between parent-child node pairs, and constraints at individual nodes. also the max of quadratic objectives is represented by a quadratic constraint. Leads to a structured QCQP, solved with an interior-point method. More details in Diehls talk.
Stephen Wright (UW-Madison) Interior-Point Methods in MPC ICIAM 2007 29 / 30

Conclusions

Many interesting algorithmic ideas continue to arise in MPC. Optimization algorithms and tools (interior-point methods) continue to play an important role. Robust, stochastic, and nonlinear approaches, which are the focus of much modern optimization research, are particularly important in many MPC applications.

Stephen Wright (UW-Madison)

Interior-Point Methods in MPC

ICIAM 2007

30 / 30