Automatica 93 (2018) 149–160

Contents lists available at ScienceDirect

Automatica
journal homepage: www.elsevier.com/locate/automatica

Linear predictors for nonlinear dynamical systems: Koopman

operator meets model predictive control✩
Milan Korda *, Igor Mezić
University of California, Santa Barbara, United States

article info a b s t r a c t
Article history: This paper presents a class of linear predictors for nonlinear controlled dynamical systems. The basic
Received 17 March 2017 idea is to lift (or embed) the nonlinear dynamics into a higher dimensional space where its evolution is
Received in revised form 5 September 2017 approximately linear. In an uncontrolled setting, this procedure amounts to numerical approximations
Accepted 26 January 2018
of the Koopman operator associated to the nonlinear dynamics. In this work, we extend the Koopman
operator to controlled dynamical systems and apply the Extended Dynamic Mode Decomposition (EDMD)
to compute a finite-dimensional approximation of the operator in such a way that this approximation
Keywords:
Koopman operator has the form of a linear controlled dynamical system. In numerical examples, the linear predictors
Model predictive control obtained in this way exhibit a performance superior to existing linear predictors such as those based
Data-driven control design on local linearization or the so called Carleman linearization. Importantly, the procedure to construct
Optimal control these linear predictors is completely data-driven and extremely simple – it boils down to a nonlinear
Lifting transformation of the data (the lifting) and a linear least squares problem in the lifted space that can be
readily solved for large data sets. These linear predictors can be readily used to design controllers for
the nonlinear dynamical system using linear controller design methodologies. We focus in particular on
model predictive control (MPC) and show that MPC controllers designed in this way enjoy computational
complexity of the underlying optimization problem comparable to that of MPC for a linear dynamical
system with the same number of control inputs and the same dimension of the state-space. Importantly,
linear inequality constraints on the state and control inputs as well as nonlinear constraints on the state
can be imposed in a linear fashion in the proposed MPC scheme. Similarly, cost functions nonlinear in
the state variable can be handled in a linear fashion. We treat both the full-state measurement case and
the input–output case, as well as systems with disturbances/noise. Numerical examples demonstrate the
approach.1
© 2018 Elsevier Ltd. All rights reserved.

1. Introduction The key step in obtaining accurate predictions of a nonlinear

dynamical system as the output of a linear dynamical system is
This paper presents a class of linear predictors for nonlinear lifting of the state-space to a higher dimensional space, where
controlled dynamical systems. By a predictor, we mean an artificial the evolution of this lifted state is (approximately) linear. For
dynamical system that can predict the future state (or output) of uncontrolled dynamical systems, this idea can be rigorously jus-
a given nonlinear dynamical system based on the measurement
tified using the Koopman operator theory (Mezić & Banaszuk,
of the current state (or output) and given the current and future
2004; Mezić, 2005). The Koopman operator is a linear operator
inputs of the system. We focus on predictors possessing a linear
that governs the evolution of scalar functions (often referred to as
structure that allows established linear control design method-
ologies to be used to design controllers for nonlinear dynamical observables) along trajectories of a given nonlinear dynamical sys-
systems. tem. A finite-dimensional approximation of this operator, acting on
a given finite-dimensional subspace of all functions, can be viewed
✩ The material in this paper was not presented at any conference. This paper as a predictor of the evolution of the values of these functions
was recommended for publication in revised form by Associate Editor Riccardo along the trajectories of the nonlinear dynamical system and hence
Scattolini under the direction of Editor Ian R. Petersen. also as a predictor of the values of the state variables themselves
* Corresponding author.
E-mail addresses: [email protected] (M. Korda),
provided that they lie in the subspace of functions the operator is
[email protected] (I. Mezić).
truncated on. In the uncontrolled context, the idea of representing
1 The source code for the numerical examples is available from https://github. a nonlinear dynamical system by an infinite-dimensional linear
com/MilanKorda/KoopmanMPC/raw/master/KoopmanMPC.zip. operator goes back to the seminal works of Carleman (1932),

https://doi.org/10.1016/j.automatica.2018.03.046
0005-1098/© 2018 Elsevier Ltd. All rights reserved.
150 M. Korda, I. Mezić / Automatica 93 (2018) 149–160

Koopman (1931) and Koopman and von Neuman (1932). The po- 2. Linear predictors — basic idea
tential usefulness of such linear representations for prediction and
control was suggested in Mezić and Banaszuk (2004). We consider a discrete-time nonlinear controlled dynamical
In this work, we extend the definition of the Koopman operator system
to controlled dynamical systems by viewing the controlled dynam-
ical system as an uncontrolled one evolving on an extended state- x+ = f (x, u), (1)
space given by the product of the original state-space and the space n m
where x ∈ R is the state of the system, u ∈ U ⊂ R the control
of all control sequences. Subsequently, we use a modified version input, x+ is the successor state and f the transition mapping. The
of the Extended Dynamic Mode Decomposition (EDMD) (Williams, input–output case is treated in Section 7.1.
Kevrekidis, & Rowley, 2015) to compute a finite-dimensional ap- The focus of this paper is the prediction of the trajectory of (1)
proximation of this controlled Koopman operator. In particular, we given an initial condition x0 and the control inputs {u0 , u1 , . . .}. In
impose a specific structure on the set of observables appearing in particular we are looking for simple predictors possessing a linear
the EDMD such that the resulting approximation of the operator structure which are suitable for linear control design methodolo-
has the form of a linear controlled dynamical system. Importantly, gies such as model predictive control (MPC) (Mayne et al., 2000).
the procedure to construct these linear predictors is completely The predictors investigated are assumed to be of the form of a
data-driven (i.e., does not require the knowledge of the underlying controlled linear dynamical system
dynamics) and extremely simple — it boils down to a nonlinear z + = Az + Bu, (2)
transformation of the data (the lifting) and a linear least squares x̂ = Cz ,
problem in the lifted space that can readily solved for large data
sets using linear algebra. On the numerical examples tested, the where z ∈ RN with (typically) N ≫ n and x̂ is the prediction of x,
linear predictors obtained in this way exhibit a predictive per- B ∈ RN ×m and C ∈ Rn×N . The initial condition of the predictor (2)
formance superior compared to both Carleman linearization as is given by
well as local linearization methods. For a related work on extend- ψ1 (x0 )
⎡ ⎤
ing Koopman operator methods to controlled dynamical systems, ⎢ . ⎥
z0 = ψ (x0 ) := ⎣ .. ⎦ , (3)
see Brunton, Brunton, Proctor, and Kutz (2016), Proctor, Brunton,
and Kutz (2016, in press), Williams, Hemati, Dawson, Kevrekidis, ψN (x0 )
and Rowley (2016). See also Surana (2016), Surana and Banaszuk where x0 is the initial condition of (1) and ψi : Rn → R, i =
(2016) and Mauroy and Goncalves (2016) for the use of Koopman 1, . . . , N, are user-specified (typically nonlinear) lifting functions.
operator methods for state estimation and nonlinear system iden- The state z is referred to as the lifted state since it evolves on a
tification, respectively. higher-dimensional, lifted, space.2 Importantly, the control input
Finally, we demonstrate in detail the use of these predictors for u ∈ U of (2) remains unlifted and hence linear constraints on
model predictive control (MPC) design; see the survey (Mayne, the control inputs can be imposed in a linear fashion. Notice also
Rawlings, Rao, & Scokaert, 2000) or the book (Grüne & Pannek, that the predicted state x̂ is a linear function of the lifted state
2011) for an overview of MPC. In particular, we show that these z and hence also linear constraints on the state can be readily
predictors can be used to design MPC controllers for nonlinear imposed. Fig. 1 depicts this idea.
dynamical systems with computational complexity comparable to Predictors of this form lend themselves immediately to linear
MPC controllers for linear dynamical systems with the same num- feedback control design methodologies. Importantly, however, the
ber of control inputs and states. Indeed, the resulting MPC scheme resulting feedback controller will be nonlinear in the original state
is extremely simple: In each time step of closed-loop operation x even though it may be (but is not required to be) linear in the
it involves one evaluation of a family of nonlinear functions (the lifted state z. Indeed, from any feedback controller κlift : RN → Rm
for (2), we obtain a feedback controller κ : Rn → Rm for the
lifting) to obtain the initial condition of the linear predictor and the
original system (1) defined by
solution of a convex quadratic program affinely parametrized by
this lifted initial condition. Importantly, nonlinear cost functions κ (x) := κlift (ψ(x)). (4)
and constraints can be handled in a linear fashion by including
all nonlinear terms appearing in these functions among the lifting The idea is that if the true trajectory of x generated by (1) and
functions. Therefore, the proposed scheme can be readily used the predicted trajectory of x̂ generated by (2) are close for each
for predictive control of nonlinear dynamical systems, using the admissible input sequence, then the optimal controller for (2)
tailored and extremely efficient solvers for linear MPC (in our case should be close to the optimal controller for (1). In Section 3 we
will see how the linear predictors (2) can be derived within the
qpOASES Ferreau, Kirches, Potschka, Bock, & Diehl, 2014), thereby
Koopman operator framework extended to controlled dynamical
avoiding the troublesome and computationally expensive solution
systems.
of nonconvex optimization problems encountered in classical non-
Note, however, that in general one cannot hope that a trajec-
linear MPC schemes (Grüne & Pannek, 2011).
tory of a linear system (2) will be an accurate prediction of the
The paper is organized as follows: In Section 2 we describe trajectory of a nonlinear system for all future times. Nevertheless,
the problem setup and the basic idea behind the use of linear if the predictions are accurate on a long enough time interval,
predictors for nonlinear dynamical systems. In Section 3 we derive these predictors can facilitate the use of linear control systems
these linear predictors as finite-dimensional approximations to
the Koopman operator extended to nonlinear dynamical systems. 2 In general, the term ‘‘lifted state’’ may be misleading as, in principle, the same
In Section 4 we describe a numerical algorithm for obtaining approach can be applied to dynamical systems with states evolving on a (possibly
these linear predictors. In Section 5 we describe the use of these infinite-dimensional) space M, with finitely many observations (or outputs) h(x) =
predictors for model predictive control. In Section 7 we discuss [h1 (x), . . . , hp (x)]⊤ available at each time instance; the lifting is then applied to
these output measurements (and possibly their time-delayed versions), i.e., ψi (x)
extensions of the approach to input–output systems (Section 7.1)
in (19) becomes ψi (h(x)) for some functions ψi : Rp → R. In other words, rather
and to systems with disturbances/noise (Section 7.2). In Section 8 than the state itself we lift the output of the dynamical system. For a detailed
we present numerical examples. treatment of the input–output case, see Section 7.1.
 M. Korda, I. Mezić / Automatica 93 (2018) 149–160 151

Fig. 1. Linear predictor for a nonlinear controlled dynamical system — z is the lifted state evolving on a higher-dimensional state space, x̂ is the prediction of the true state
x and ψ is a nonlinear lifting mapping. This predictor can then be used for control design using linear methods, in our case linear MPC.

design methodologies. Especially suited for this purpose is model where S is the left shift operator, i.e. (S u)(i) = u(i + 1), and u(i)
predictive control (MPC) that uses only finite-time predictions to denotes the ith element of the sequence u.
generate the control input. The Koopman operator K : H → H associated to (8) is
We will briefly mention in Section 3.2.2 how more complex, bi- defined by
linear, predictors of the form
(Kφ )(χ ) = φ (F (χ )) (9)
z + = Az + (Bz)u, (5)
x̂ = Cz for each φ : R × ℓ(U ) → R belonging to some space of
n

observables H.
can be obtained within the same framework and argue that pre- The Koopman operator (9) is a linear operator fully describing
dictors of this form can be asymptotically tight (in a well defined the non-linear dynamical system (1) provided that H contains
sense). Nevertheless, predictors of the from (5) are not immedi- the components of the non-extended state3 xi , i = 1, . . . , n.
ately suited for linear control design and hence in this paper we For example, spectral properties of the operator K should provide
focus on the linear predictors (2). information on spectral properties of the nonlinear dynamical
system (1).
3. Koopman operator — rationale behind the approach

We start by recalling the Koopman operator approach for the 3.2. EDMD for controlled systems
analysis of an uncontrolled dynamical system
In this paper, however, we are not interested in spectral proper-
x+ = f (x). (6) ties but rather in time-domain prediction of trajectories of (1). To
this end, we construct a finite-dimensional approximation to the
The Koopman operator K : F → F is defined by operator K which will yield a predictor of the form (2). In order
to do so, we adapt the extended dynamic mode decomposition
(Kψ )(x) = ψ (f (x)) (7)
algorithm (EDMD) of Williams et al. (2015) to the controlled
for every ψ : R → R belonging to F , which is a space of functions
n
setting. The EDMD is a data-driven algorithm to construct finite-
(often referred to as observables) invariant under the action of the dimensional approximations to the Koopman operator. The algo-
Koopman operator. Importantly, the Koopman operator is linear rithm assumes that a collection of data (χj , χj+ ), j = 1, . . . , K
(but typically infinite-dimensional) even if the underlying dynam- satisfying χj+ = F (χj ) is available and seeks a matrix A (the trans-
ical system is nonlinear. Crucially, this operator fully captures all pose of the finite-dimensional approximation of K) minimizing
properties of the underlying dynamical system provided that the
K
space of observables F contains the components of the state xi , ∑ 2
i.e, the mappings x ↦ → xi belong to F for all i ∈ {1, . . . , n}. ∥φ(χj+ ) − Aφ(χj )∥2 , (10)
For a detailed survey on Koopman operator and its applications, j=1

see Budišić, Mohr, and Mezić (2012). where

]⊤
φ(χ ) = φ1 (χ ) ... φNφ (χ )
[
3.1. Koopman operator for controlled systems
is a vector of lifting functions (or observables) φi : Rn × ℓ(U ) → R,
There are several ways of generalizing the Koopman operator to
i ∈ {1, . . . , Nφ }. Note that χ = (x, u) is in general an infinite-
controlled systems; see, e.g., Proctor et al. (in press) and Williams
et al. (2016). In this paper we present a generalization that is both dimensional object and hence the objective (10) cannot be eval-
rigorous and practical. We define the Koopman operator associated uated in a finite time unless φi ’s are chosen in a special way.
to the controlled dynamical system (1) as the Koopman operator
associated to the uncontrolled dynamical system evolving on the 3.2.1. Linear predictors
extended state-space defined as the product of the original state- In order to obtain a linear predictor (2) and a computable
space and the space of all control sequences, i.e., in our case Rn × objective function in (10) we impose that the functions φi are of
ℓ(U ), where ℓ(U ) is the space of all sequences (ui )∞i=0 with ui ∈ U .
the form
Elements of ℓ(U ) will be denoted by u := (ui )∞ . The dynamics of
the extended state
i=0
φi (x, u) = ψi (x) + Li (u), (11)

where ψi : Rn → R is in general nonlinear but Li : ℓ(U ) → R

[ ]
x
χ= , is linear. Without loss of generality (by linearity and causality) we
u
is described by
3 Note that the definition of the Koopman operator implicitly assumes that H is
f (x, u(0))
[ ]
invariant under the action of K and hence, in the controlled setting, H will typically
χ +
= F (χ ) := , (8)
Su automatically contain also functions depending on u.
152 M. Korda, I. Mezić / Automatica 93 (2018) 149–160

can assume that Nφ = N + m for some N > 0 and that the vector 4. Numerical algorithm — finding A, B, C
of lifting functions φ = [φ1 , . . . , φNφ ]⊤ is of the form
We assume that a set of data
ψ(x)
[ ]
φ(x, u) = , (12) X = x1 , . . . , xK , Y = y1 , . . . , yK , U = u1 , . . . , uK
[ ] [ ] [ ]
(16)
u(0)
satisfying the relation yi = f (xi , ui ) is available. Note that we do not
where ψ = [ψ1 , . . . ψN ]⊤ and u(0) ∈ Rm denotes the first compo-
assume any temporal ordering of the data. In particular, the data is
nent of the sequence u. Since we are not interested in predicting
not required to come from one trajectory of (1).
future values of the control sequence, we can disregard the last m
Given the data X, Y, U in (16), the matrices A ∈ RN ×N and
components of each term φ(χj+ ) − Aφ(χj ) in (10). Denoting Ā the B ∈ RN ×m in (2) are obtained as the best linear one-step predictor
first N rows of A and decomposing this matrix such that Ā = [A, B] in the lifted space in a least-squares sense, i.e., they are obtained as
with A ∈ RN ×N , B ∈ RN ×m and using the notation χj = (xj , uj ) the solution to the optimization problem
in (10), leads to the minimization problem
K
min ∥Ylift − AXlift − BU∥F , (17)
A, B
2
∑
min ∥ψ(xj ) − Aψ(xj ) − Buj (0)∥2 .
+
(13)
A, B where
j=1
Xlift = ψ (x1 ), . . . , ψ (xK ) , Ylift = ψ (y1 ), . . . , ψ (yK ) ,
[ ] [ ]
(18)
Minimizing (13) over A and B leads to the predictor of the form (2)
starting from the initial condition with
ψ1 (x)
⎡
⎤
ψ1 (x0 )
⎡ ⎤
⎢ .. ⎥
z0 = ψ (x0 ) := ⎣ . ⎦ . ψ(x) := ⎣ ... ⎦ (19)
⎢ ⎥
(14)
ψN (x0 ) ψN (x)

The matrix C is obtained simply as the best projection of x onto the being a given basis (or dictionary) of nonlinear functions. The
span of ψi ’s in a least squares sense, i.e., as the solution to symbol ∥ · ∥F denotes the Frobenius norm of a matrix. The matrix
C ∈ Rn×N is obtained as the best linear least-squares estimate of X
K given Xlift , i.e., the solution to
∑
min ∥xj − C ψ(xj )∥22 . (15)
C
j=1 min ∥X − C Xlift ∥F . (20)
C
We emphasize that (13) and (15) are linear least squares problem The analytical solution to (17) is
that can be readily solved using linear algebra.
[A, B] = Ylift [Xlift , U]† , (21)
Remark 1. Note that the solution to (15) is trivial if the set of lifting
where † denotes the Moore–Penrose pseudoinverse of a matrix.
functions {ψ1 , . . . , ψN } contains the state observable, i.e., if, after
The analytical solution to (20) is
possible reordering, ψi (x) = xi for all i ∈ {1, . . . , n}. In this case,
†
the solution to (15) is C = [I , 0], where I is the identity matrix of C = XXlift .
size n.
Notice the close relation of the resulting algorithm to the DMD
with control proposed in Proctor et al. (2016). There, however, no
The resulting algorithm for constructing the linear predictor (2)
lifting is applied and the least squares fit (17) is carried out on the
is concisely summarized in Section 4.
original data, limiting the predictive power.

3.2.2. Bilinear predictors 4.1. Practical considerations

Predictors with a more complex structure can be obtained by
imposing a structure on the functions φi different than the linear The analytical solution (21) is not the preferred method of
structure (11). In particular, bilinear predictors of the form (5) can solving the least-squares problem (17) in practice. In particular, for
larger data sets with K ≫ N it is beneficial to solve instead the
be obtained by requiring that
normal equations associated to (17). The normal equations read
φi (x, u) = ψi (x) + ξi (x)Li (u)
V = MG (22)
for some nonlinear functions ψi , ξi and linear operators Li .
with variable M = [A, B] and data
A bilinear predictor of the form (5) can be tight (in the sense [ ][ ]⊤ [ ]⊤
of convergence of predicted trajectories to the true ones as the Xlift Xlift Xlift
G= , V = Ylift .
number of basis functions tends to infinity) under the assumption U U U
that the discrete-time mapping (1) comes from a discretization
Any solution to (22) is a solution to (17). Importantly, the size of the
of a continuous-time system and the discretization interval tends
matrices G and V is (N + m) × (N + m) respectively N × (N + m) and
to zero and the underlying continuous-time dynamics is input-
hence independent of the number of samples K in the data set (16).
affine; see Section IV-C of Surana (2016) for more details. This The same considerations hold for the least-squares problem (20).
bilinearity phenomenon is well known from the classical Carleman Note that, in practice, the lifting functions ψi will typically contain
linearization in continuous time (Carleman, 1932). In this work, the state itself in which case the solution to (20) is just the selection
however, we focus on linear predictors since they are immediately of appropriate indices of Xlift , i.e., after possible reordering, C =
amenable to the range of mature linear control design techniques. [I , 0].
The use of bilinear predictors for controller design is left for further If lifting to a very high dimensional space is required, it may
investigation. be worth exploring the so called kernel methods known from
 M. Korda, I. Mezić / Automatica 93 (2018) 149–160 153

machine learning which do not require an explicit evaluation of 5.1. Eliminating dependence on the lifting dimension
the lifting mapping ψ . These methods were successfully applied to
the standard EDMD algorithm in Williams, Rowley, and Kevrekidis In this section we show that the computational complexity of
(2015), leading to substantial computational savings. solving the MPC problem (23) can be rendered independent of
the dimension of the lifted state N. This is achieved by transform-
5. Model predictive control
ing (23) in the so-called dense form
In this section we describe how the linear predictor (2) can be minimize U ⊤ HU ⊤ + h⊤ U + z0⊤ GU
used to design an MPC controller for the nonlinear system (1) with U ∈RmNp
computational complexity comparable to that of an MPC controller (24)
subject to LU + Mz0 ≤ c
for a linear system of the same state-space dimension and number parameter z0 = ψ (xk ),
of control inputs. We recall that MPC is a control strategy where
the control input at each time step of the closed-loop operation for some positive-semidefinite matrix H ∈ RmNp ×mNp and some
is obtained by solving an optimization problem where a user- matrices and vectors h ∈ RmNp , G ∈ RN ×mNp , L ∈ Rnc Np ×mNp ,
specified cost function (e.g., the energy or tracking error) is min- M ∈ Rnc Np ×N and c ∈ Rnc Np . The optimization is over the
vector of predicted control inputs U = [u⊤ 0 , u1 , . . . , uNp −1 ] .
⊤ ⊤ ⊤
imized along a prediction horizon subject to constraints on the
control inputs and state variables. Traditionally, linear MPC solves This ‘‘dense’’ formulation can be readily derived from the ‘‘sparse’’
a convex quadratic program, thereby allowing for an extremely formulation (23) by solving explicitly for zi ’s and concatenating the
fast evaluation of the control input. Nonlinear MPC, on the other point-wise-in-time stage costs and constraints; see the Appendix
hand, solves a difficult non-convex optimization problem, thereby for explicit expressions for the data matrices of (24) in terms of
requiring far more computational resources and/or relying on local those of (23).
solutions only; see, e.g., Grüne and Pannek (2011) for an overview Notice that, crucially, the size of the Hessian H or the number of
of nonlinear MPC. Just as linear MPC, the lifting-based MPC for
the constraints nc in the dense formulation (24) is independent of
nonlinear systems developed here relies on convex quadratic pro-
the size of the lift N. Hence, once the nonlinear mapping z0 = ψ (xk )
gramming, thereby avoiding all issues associated with non-convex
is evaluated, the computational cost of solving (24) is comparable
optimization and allowing for an extremely fast evaluation of the
control input. We first describe the proposed MPC controller in its to solving a standard linear MPC on the same prediction horizon,
most general form and subsequently, in Section 5.2, describe how a with the same number of control inputs and with the dimension
traditional nonlinear MPC problem translates to the proposed one. of the state-space equal to n rather than N. This comes from the
The proposed model predictive controller solves at each time fact that the cost of solving an MPC problem in a dense form is
instance k of the closed-loop operation the optimization problem independent of the dimension of the state-space, once the data
(
Np −1 Np
) matrices in (24) are formed. Importantly, these matrices are fixed
minimize J (ui )i=0 , (zi )i=0 and precomputed offline before deploying the controller (with the
ui ,zi
exception of the inexpensive matrix–vector multiplication z0⊤ G).
subject to zi+1 = Azi + Bui , i = 0, . . . , Np − 1 This is in contrast with other MPC schemes for nonlinear systems
(23)
Ei zi + Fi ui ≤ bi , i = 0, . . . , Np − 1 where these matrices have to be re-computed at each time step of
ENp zNp ≤ bNp the closed-loop operation, thereby greatly increasing the compu-
parameter z0 = ψ (xk ), tational cost.
where Np is the prediction horizon and the convex quadratic cost The closed-loop operation of the lifting-based MPC can be sum-
function J is given by marized by the following algorithm, where U1⋆:m denotes the first
( ) m components of U ⋆ :
Np −1 Np
J (ui )i=0 , (zi )i=0 = zN⊤p QNp zNp + q⊤
Np zNp
Algorithm 1 Lifting MPC – closed-loop operation
Np −1
∑ 1: for k = 0, 1, . . . do
+ zi⊤ Qi zi + u⊤ ⊤ ⊤
i Ri ui + qi zi + ri ui 2: Set z0 := ψ (xk )
i=0 3: Solve (24) to get an optimal solution U ⋆
with Qi ∈ R N ×N
and Ri ∈ R m×m
positive semidefinite. The matrices 4: Set uk = U1⋆:m
Ei ∈ Rnc ×N and Fi ∈ Rnc ×m and the vector bi ∈ Rnc define state 5: xk+1 = f (xk , uk ) [= apply uk to system (1)]
and input polyhedral constraints. The optimization problem (23) is
parametrized by the current state of the nonlinear dynamical sys- 5.2. Transforming NMPC to Koopman MPC
tem xk . This optimization problems defines a feedback controller
κ (xk ) = u⋆0 (xk ), In this section we describe in detail how a traditional nonlinear
MPC problem translates.4 to the proposed MPC (23) We assume
where u⋆0 (xk ) denotes an optimal solution to problem (23)
a nonlinear MPC problem which at each time step k of the closed-
parametrized by the current state xk .
loop operation solves the optimization problem
Several observations are in order:
Np −1
(1) The optimization problem (23) is a convex quadratic pro-
∑
minimize lNp (xNp ) + li (x̄i ) + u⊤ ⊤
i R̄i ui + r̄i ui
gramming problem (QP). ui ,x̄i
i=0
(2) At each time step k, the predictions are initialized from the
lifted state ψ (xk ). subject to x̄i+1 = f (x̄i , ui ), i = 0, . . . , Np − 1 (25)
(3) Nonlinear functions of the original state x can be penalized cxi (x̄i ) + cu⊤i ui ≤ 0, i = 0, . . . , Np − 1
in the cost function and included among the constraints cxNp (x̄Np ) ≤ 0
by including these nonlinear functions among the lifting
x̄0 = xk ,
functions ψi . For example, if one wished for some reason
∑Np −1
to minimize i=0 cos(∥xi ∥∞ ), one could simply set ψ1 =
cos(∥x∥∞ ) and q = [1, 0, 0, . . . , 0]⊤ . See Section 5.2 for 4 By translate, we do not mean to rewrite in an equivalent form. The problem (23)
more details. is of course only an approximation to (25) since the lifted linear predictor is not
exact unless the dynamical system is linear.
154 M. Korda, I. Mezić / Automatica 93 (2018) 149–160

where the notation x̄ is used to distinguish the predicted state 6.1. EDMD as L2 projection
x̄, used only within the optimization problem (25), from the true
measured state x of the dynamical system (1). Notice that the true First we give a characterization of the EDMD algorithm as an L2
nonlinear dynamics x+ = f (x, u) appears as a constraint of (25); projection. Given an arbitrary nonnegative measure µ on M, we
in addition, the functions li and cxi can be nonlinear and hence the define the L2 (µ) projection5 of a function g onto HN as
optimization problem (25) is in general nonconvex and extremely
∫
µ
hard to solve to global optimality. PN g = arg min ∥h − g ∥L2 (µ) = arg min (h − g)2 dµ
h∈HN h∈HN M
In order to translate (25) to the proposed form (23) we assume ∫
that a predictor of the form (2) with matrices A and B has been = φ arg min
⊤
(c ⊤ φ − g)2 dµ. (28)
constructed as described in Section 4, using a lifting mapping ψ = c ∈RN M
[ψ1 , . . . , ψN ]⊤ . The matrices A, B appear in the first constraint
We have the following characterization of KN ,K .
of (23) and the lifting mapping ψ is used for initialization z0 =
ψ(xk ). In order to obtain the remaining data matrices of (23) we Theorem 1. Let µ̂K denote the∑ empirical measure associated to the
assume without loss of generality that ψi (x) = li (x), i = 0, . . . , Np K
points χ1 , . . . , χK , i.e., µ̂K = K1 i=1 δχi , where δχi denotes the Dirac
and ψNp +i (x) = cxi (x), i = 0, . . . , Np . With this assumption, the
measure at χi . Then for any g ∈ HN
remaining data is given by Qi = 0, Ri = R̄i , r = r̄i , qi =
µ̂
[01×i , 1, 01×N −1−i ], Ei = [01×Np +i , 1, 01×N −Np −1−i ], Fi = cu⊤i , bi = 0, KN ,K g = PN K Kg = arg min ∥h − Kg ∥L2 (µ̂K ) , (29)
where 0i×j denotes the matrix of zeros of size i × j. Note that this h∈HN
derivation assumed that the constraint functions cxi and cui are i.e.,
scalar-valued; for vector valued constraint functions, the approach
µ̂
is analogous, setting the lifting functions ψi equal to the individual KN ,K = PN K K|HN , (30)
components of the constraint functions cxi .
where K|HN : HN → H is the restriction of the Koopman operator to
We also note that this canonical approach always leads to a
the subspace HN .
linear cost function in (23). However, in special cases, when some
of the cost functions li (xi ) are convex quadratic, one may want In words, the operator KN ,K is the L2 projection of the operator
to use the freedom of the formulation (23) and instead of setting K on the span of φ1 , . . . , φN with respect to the empirical measure
ψi = li , use the quadratic terms in the cost function of (23), thereby supported on the samples χ1 , . . . , χK .
reducing the dimension of the lift. See Section 8.2 for an example.
6.2. Convergence of EDMD
6. Theoretical analysis
Now we turn into analyzing convergence KN ,K to K as the
In this section we discuss theoretical properties of the EDMD number of samples K and the number of basis functions N tend
algorithm of Section 3.2. Full exposition of the theoretical analysis to infinity.
of EDMD is beyond the scope of this paper and therefore we only First we analyze convergence as K tends to infinity. For this we
summarize the authors’ results obtained concurrently in Korda assume a probabilistic sampling model. That is, we assume that the
space M is endowed with a probability measure µ and that the
and Mezić (2018), where the reader is referred to for proofs of the
samples χ1 , . . . , χK are independent identically distributed (iid)
theorems stated in this section and additional results. Note that the
samples from the distribution µ and we assume that H = L2 (µ).
results of Theorems 1 and 2 are not new and were, to the best of
We invoke the following non-restrictive assumption.
our knowledge, first rigorously proven in Klus, Koltai, and Schütte
(2016, Section 3.4) and alluded to already in the original EDMD
Assumption 1 (µ Independence). The basis functions φ1 , . . . , φN
paper (Williams et al., 2015); here we state them in a language
are such that
suitable for our purposes.
µ {x ∈ M | c ⊤ φ(x) = 0} = 0
( )
We work in an abstract setting with a dynamical system
χ + = F (χ ) for all nonzero c ∈ RN .

with F : M → M, where M is a given separable topological space. This is a natural assumption ensuring that the measure µ is not
This encompasses both the finite-dimensional setting with M = supported on a zero level set of a nontrivial linear combination of
Rn and the infinite-dimensional controlled setting of Section 3.1 the basis functions used.
with M = Rn × l(U ). The Koopman operator K : H → H on a Finally we define
space of observables H (with φ : M → R for all φ ∈ H) is defined µ
KN = PN K|HN , (31)
as in (9) by Kφ = φ ◦ F for all φ ∈ H. We assume the EDMD
algorithm of Section 3.2, i.e., we solve the optimization problem i.e., the L2 (µ) projection of the Koopman operator K on HN . Then
K we have the following result:
2
∑
min ∥φ(χi+ ) − Aφ(χi )∥2 , (26)
A∈RN ×N Theorem 2. If Assumption 1 holds, then with probability one
i=1

where lim ∥KN ,K − KN ∥ = 0, (32)

]⊤ K →∞
φ(χ ) = φ1 (χ ), . . . , φN (χ )
[
where ∥ · ∥ is any operator norm and
with φi ∈ H, i = 1, . . . , N, being linearly independent basis func-
lim dist σ (KN ,K ), σ (KN ) = 0,
( )
(33)
tions. Denoting HN the span of φ1 , . . . , φN , the finite-dimensional K →∞
approximation of the Koopman operator KN ,K : HN → HN where σ (·) ⊂ C denotes the spectrum of an operator and dist(·, ·) the
obtained from (26) is defined for any g = c ⊤ φ ∈ HN by Hausdorff metric on subsets of C.
KN ,K g = c ⊤ AN ,K φ, (27)
5 The Hilbert space L (µ) is the space of all square integrable functions with
2
where AN ,K is the optimal solution of (26). respect to the measure µ.
 M. Korda, I. Mezić / Automatica 93 (2018) 149–160 155

Theorem 2 says that the EDMD approximations KN ,K converge 7.1.1. Dynamics (35) is known
in the operator norm to the L2 (µ) projection of the Koopman If the dynamics (35) is known, one can construct a predictor
operator onto the span of the basis functions φ1 , . . . , φN . of the form (2) by applying the algorithm of Section 4 to data
Having established convergence of KN ,K to K we turn to study- obtained from simulation of the dynamical system (35). In closed-
ing convergence of KN to K. Since the operator KN ,K is defined only loop operation, the predictor (2) is then used in conjunction with a
on HN we extend it to all of H by precomposing with the projection state-estimator for the dynamical system (35). Alternatively, one
µ
on HN , i.e., we study convergence of KN PN : H → H to K : H → H. can design, using linear observer design methodologies, a state
We have the following result: estimator directly for the linear predictor (2). Interestingly, doing
so obviates the need to evaluate the lifting mapping z = ψ (x)
Theorem 3. If (φi )∞ i=1 forms an orthonormal basis of H = L2 (µ)
in closed loop, as the lifted state is directly estimated. This idea
and if K : H → H is continuous, then the sequence of operators is closely related to the use of the Koopman operator for state
µ µ µ
KN PN = PN KPN converges to K as N → ∞ in the strong operator estimation (Surana, 2016; Surana & Banaszuk, 2016).
topology, i.e., for all g ∈ H
µ 7.1.2. Dynamics (35) is not known
lim ∥KN PN g − Kg ∥L2 (µ) = 0. If the dynamics (35) is not known and only input–output data
N →∞
is available, one could construct a predictor of the form (2) or
In particular, if g ∈ HN0 for some N0 ∈ N, then
(5) by taking as lifting functions only functions of the output y,
lim ∥KN g − Kg ∥L2 (µ) = 0. i.e., ψi (x) = φi (h(x)). This, however, would be extremely restrictive
N →∞
as this severely restricts the class of lifting functions available.
µ
Theorem 3 tells us that the sequence of operators KN PN con- Indeed, if, for example, h(x) = x1 , only functions of the first com-
verges strongly to K. For additional results on spectral convergence ponent are available. However, this problem can be circumvented
of KN to K, weak spectral convergence of KN ,N (i.e., with K = N) to by utilizing the fact that subsequent measurements, h(x), h(f (x)),
K and for a method to construct KN directly, without the need for h(f (f (x))), etc., are available and therefore we can define observ-
sampling, see Korda and Mezić (2018). ables depending not only on h but also on repeated composition of
Now we use Theorem 3 to establish convergence of finite- h with f . In practice this corresponds to having the lifting functions
ng
horizon predictions of a given vector-valued observable g ∈ HN . depend not only on the current measured output but also on
0
For our purposes, the most pertinent situation is when g is the state several previous measured outputs (and inputs, in the controlled
observable, i.e., g(x) = x in which case the following result pertains setting). The use of time-delayed measurements is classical in
to predictions of the state itself. We let AN ,K denote the solution system identification theory (see, e.g., Ljung, 1998) but has also
ng
to (26) and set AN = limK →∞ AN ,K . Since g ∈ HN , it follows that appeared in the context of Koopman operator approximation (see,
0
for every N ≥ N0 there exists a matrix CN ∈ Rng ×N such that e.g., Brunton, Brunton, Proctor, Kaiser, & Kutz, 2017; Tu, Rowley,
g = CN φN , where φN = [φ1 , . . . , φN ]⊤ . With this notation the Luchtenburg, Brunton, & Kutz, 2014).
following result holds: Assume therefore that we are given a collection of data

ng X̃ = [ζ 1 , . . . , ζ K ], Ỹ = [ζ +
1 , . . . , ζ K ], Ũ = [u1 , . . . , uK ]
+
Corollary 1. If the assumptions of Theorem 3 hold and g ∈ HN for
0
some N0 ∈ N, then for any finite prediction horizon Np ∈ N where
]⊤
ζ i = y⊤ ū⊤ y⊤ ... ū⊤ y⊤
∫ [
)2 i,nd i,nd −1 i,nd −1 i,0 i,0
CN AiN φN − g ◦ F i dµ = 0.
(
lim sup (34)
N →∞ i∈{0,...,Np } M ∈ R(nd +1)nh +nd m (36)
⊤ ⊤
ζ+ ...
[ ⊤ ⊤
y⊤ ū⊤
]
In words, predictions over any finite horizon converge in the i =
yi,nd +1 ūi,nd i,nd i,1 yi,1
L2 (µ) norm. Unfortunately, Corollary 1 does not immediately apply (nd +1)nh +nd m
∈ R
to the linear predictors designed in Section 3.2 as the set of basis
functions (12) does not form an orthonormal basis of H because of ui = ūi,nd
the special structure of these basis functions which ensures linear- n +1
with (yi,j )j=d 0 being a vector of consecutive output measurements
ity of the resulting predictors. In this setting, one can only prove n
µ µ generated by (ūi,j )j=d 0 consecutive inputs. We note that there does
convergence of KN to P∞ K|H∞ , where P∞ is the L2 (µ) projection
not need to be any temporal relation between ζ i and ζ i+1 . If,
onto H∞ := {φi | i ∈ N}.
however, ζ i and ζ i+1 are in fact successors, then the matrices X̃ and
7. Extensions Ỹ have the familiar (quasi)-Hankel structure known from system
identification theory.
In this section, we describe extensions of the proposed ap- Computation of a linear predictor then proceeds in the same
proach to input–output dynamical systems and to systems with way as for the full-state measurement: We lift the collected data
disturbances/noise. and look for the best one-step predictor in the lifted space. This
leads to the optimization problem
7.1. Input–output dynamical systems min Ỹlift − AX̃lift − BŨF ,
 
(37)
A,B

In this section we describe how the approach can be generalized where

to the case when full state measurements are not available, but
X̃lift = ψ (ζ 1 ), . . . , ψ (ζ K ) , Ỹlift = ψ (ζ +
1 ) , . . . , ψ (ζ K )
+
[ ] [ ]
rather only certain output is measured. To this end, consider the (38)
dynamical system
and
x+ = f (x, u), (35) ψ1 (ζ )
⎡ ⎤
y = h(x), ψ(ζ ) = ⎣ ... ⎦
⎢ ⎥
where y is the measured output and h : Rn → Rnh . ψN (ζ )
156 M. Korda, I. Mezić / Automatica 93 (2018) 149–160

is a vector of real or complex-valued, possibly nonlinear, lifting 8. Numerical examples

functions. This leads to a linear predictor in the lifted space
In this section we compare the prediction accuracy of the
z + = Az + Bu (39)
linear lifting-based predictor (2) with several other predictors
ŷ = Cz , and demonstrate the use of the lifting-based MPC proposed
where ŷ is the prediction of y and C is the solution to in Section 5 for feedback control of a bilinear model of a
  motor and of the Korteweg–de Vries nonlinear partial differ-
min [y1,nd , . . ., yK ,nd ] − C X̃lift  . (40) ential equation. The source code for the numerical examples
 
C F is available from https://github.com/MilanKorda/KoopmanMPC/
The predictor (39) starts from the initial condition raw/master/KoopmanMPC.zip.
z0 = ψ (ζ 0 ), 8.1. Prediction comparison
where
]⊤ In order to evaluate the proposed predictor, we compare its
ζ 0 = y⊤ ū⊤ y⊤ ... ū⊤ y⊤
[
0 −1 −1 −nd −nd prediction quality with that of several commonly used predictors.
is the vector of nd + 1 most recent output measurements and nd The system to compare the predictors on is the classical forced Van
input measurements. We remark that the solution to (40) is trivial der Pol oscillator with dynamics given by
provided that the outputs and its delays are included among the ẋ1 = 2x2
lifting functions (e.g., if ψj (ζ ) = ζ (2j), j = 0, . . . , nd ); see Remark 1.
ẋ2 = −0.8x1 + 2x2 − 10x21 x2 + u.
Remark 2 (Closed-Loop Operation). The closed-loop operation of The predictors compared are:
the resulting MPC controller follows the steps of Algorithm 1, only
the initialization z0 = ψ (xk ) in line 2 is replaced by z0 = ψ (ζ k ), (1) Predictor based on local linearization of the dynamics at the
where ζ k = [y⊤ k , uk−1 , yk−1 , . . . , uk−n , yk−n ] .
⊤ ⊤ ⊤ ⊤ ⊤
origin,
d d
(2) Predictor based on local linearization of the dynamics at a
7.2. Disturbance/Noise propagation given initial condition x0 ,
(3) Carleman linearization predictor (Carleman, 1932),
The approach can be readily extended to systems affected by a (4) The proposed lifting-based predictor (2).
disturbance or noise of the form
In order to obtain the lifting-based predictor, we discretize the
x+ = f (x, u, w ), (41) dynamics using the Runge–Kutta four method with discretiza-
tion period Ts = 0.01 s and simulate 200 trajectories over 1000
where w is the disturbance or process noise. The goal is to construct sampling periods (i.e., 20 s per trajectory). The control input for
a predictor for (41) of the form each trajectory is a random signal with uniform distribution over
z + = Az + Bu + Dw (42) the interval [−1, 1]. The trajectories start from initial conditions
generated randomly with uniform distribution on the unit box
x̂ = Cz , [−1, 1]2 . This data collection process results in the matrices X and
starting from the initial condition z0 = ψ (x0 ), where ψ (·) is the Y of size 2 × 2 · 105 and matrix U of size 1 × 105 . The lifting
lifting mapping defined in (19). For example, if w is a stochastic functions ψi are chosen to be the state itself (i.e., ψ1 = x1 , ψ2 =
disturbance with known distribution, the linear predictor of the x2 ) and 100 thin plate spline radial basis functions6 with centers
form (42) can be used to approximately compute the distribution selected randomly with uniform distribution on the unit box. The
of the state x at a future time instance, given a sequence of control dimension of the lifted state-space is therefore N = 102.
inputs up to that time. The degree of the Carleman linearization is set to 14, resulting
In order to obtain the matrices of the predictor (42), we assume in the size of the Carleman linearization predictor of 120 (= the
that we are given data of the form number of monomials of degree less than or equal to 14 in two
variables). The B matrix for Carleman linearization predictor is set
X = x1 , . . . , xK , Y = y1 , . . . , yK , to [0, 1, 0, . . . , 0]⊤ .
[ ] [ ]
(43a)
Fig. 2 compares the predictions starting from two initial condi-
U = u1 , . . . , uK , W = w1 , . . . , w K tions x10 = [0.5, 0.5]⊤ , x20 = [−0.1, −0.5]⊤ generated by a control
[ ] [ ]
(43b)
signal u(t) being a square wave with unit magnitude and period
satisfying yi = f (xi , ui , wi ) for all i = 1, . . . , K . As in Section 4, the
0.3 s. Table 1 reports the relative root mean squared errors (RMSE)
matrices are then obtained using least-squares regression: √∑
∥xpred (kTs ) − xtrue (kTs )∥22
min ∥Ylift − AXlift − BU − DW∥F , min ∥X − C Xlift ∥F . (44) RMSE = 100 ·
k
(45)
A,B,D C √∑
2
If measurements of the disturbance w are not available, then these k ∥xtrue (kTs )∥2

must best estimated from the available data, either using one of the for each predictor averaged over 100 randomly sampled initial
nonlinear estimation techniques or using the Koopman operator- conditions with the same square wave forcing. We observe that
based estimator proposed in Surana (2016) and Surana and Ba- the lifting-based Koopman predictor is far superior to the remain-
naszuk (2016). Alternatively, if the mapping f is known (either ing predictors. Finally, Table 2 reports the prediction accuracy
analytically or in the form of an algorithm) and an algorithm to of the lifting predictor in terms of the average RMSE error as
draw samples from the distribution of w is available, one can obtain a function of the dimension of the lift N; we observe that, as
data (43) by simulation. expected, the prediction error decreases with increasing N, albeit
not monotonously.
Remark 3. If full-state measurements are not available, the ap-
proach can be readily combined with the approach of Section 7.1.1
6 Thin plate spline radial basis function with center at x is defined by ψ (x) =
or 7.1.2. 0
∥x − x0 ∥2 log(∥x − x0 ∥).
 M. Korda, I. Mezić / Automatica 93 (2018) 149–160 157

Fig. 2. Prediction comparison for the forced Van der Pol oscillator — Top: initial condition x0 = [0.5, 0.5]⊤ . Bottom: initial condition x0 = [−0.1, −0.5]⊤ . The forcing u(t) is
in both cases a square wave with unit amplitude and period 0.3 s.

Table 1 bilinearity between the state and the control input. The physical
Prediction comparison — average RMSE (45) over 100 randomly sampled initial
constraints on the control input are u ∈ [−4, 4], which we scale to
conditions: comparison among different predictors.
[−1, 1].
x0 Average RMSE
The goal is to design an MPC controller based on Section 7.1.2,
Koopman 24.4 %
i.e., assuming only input–output data available and no explicit
Local linearization at x0 2.83 · 103 %
Local linearization at 0 912.5 % knowledge of the model. In order to obtain the lifting-based pre-
Carleman 5.08 · 1022 % dictor (39), we discretize the scaled dynamics using the Runge–
Kutta four method with discretization period Ts = 0.01 s and
Table 2
simulate 200 trajectories over 1000 sampling periods (i.e., 20 s
Prediction comparison – lifting-based Koopman predictor – average prediction per trajectory). The control input for each trajectory is a random
RMSE over 100 randomly sampled initial conditions as a function of the dimension signal uniformly distributed on [−1, 1]. The trajectories start from
of the lift.
initial conditions generated randomly with uniform distribution
N 5 10 25 50 75 100 on the unit box [−1, 1]2 . We choose the number of delays nd = 1.
Average RMSE 66.5% 44.9% 47.0% 38.7% 30.6% 24.4% The lifting functions ψi are chosen to be the time-delayed vector
ζ ∈ R3 , defined in (36), and 100 thin plate spline radial basis
functions (see Footnote 6 ) with centers selected randomly with
8.2. Feedback control of a bilinear motor uniform distribution over [−1, 1]3 . The dimension of the lifted
state-space is therefore N = 103. First, in Fig. 3, we compare
In this section we apply the proposed approach to the control of the output predictions for two different, randomly chosen, initial
a bilinear model of a DC motor (Daniel-Berhe & Unbehauen, 1998). conditions against the predictor based on local linearization at a
The model reads given initial condition. The prediction accuracy of the proposed
ẋ1 = −(Ra /La )x1 − (km /La )x2 u + ua /La predictor is superior, especially for longer prediction times. This is
documented further in Table 3 by the relative root mean-squared
ẋ2 = −(B/J)x2 + (km /J)x1 u − τl /J ,
errors (45) over a one-second prediction horizon averaged over
y = x2
one hundred randomly sampled initial conditions. Both in Fig. 3
where x1 is the rotor current, x2 the angular velocity and the control and Table 3, the control signal was a pseudo-random binary signal
input u is the stator current and the output y is the angular velocity. generated anew for each initial condition.
The parameters are La = 0.314, Ra = 12.345, km = 0.253, J = The control objective is to track a given angular velocity refer-
0.00441, B = 0.00732, τl = 1.47, ua = 60. Notice in particular the ence yr , which translates into the objective function minimized in
158 M. Korda, I. Mezić / Automatica 93 (2018) 149–160

Fig. 3. Feedback control of a bilinear motor — predictor comparison. Left: initial condition x0 = [0.887, 0.587]⊤ . Right: initial condition x0 = [−0.404, −0.126]⊤ .

Table 3 L-MPC controller. This discrepancy is due to the fact that the local
Feedback control of a bilinear motor — prediction RMSE (45) for 100 randomly linearization and all data defining the underlying optimization
generated initial conditions.
problem that depend on it have to be re-computed at every it-
Koopman Local linearization at x0
eration, which is costly on its own and also precludes efficient
Average RMSE 32.3 % 135.5 % warm-starting; on the other hand, all data (except for the initial
condition) of the underlying optimization problem of K-MPC are
precomputed offline. In both cases, the computation times could
the MPC problem be significantly reduced with a more efficient implementation.
J = (CzNp − yr )⊤ QNp (CzNp − yr ) However, we believe, that the proposed approach would still be
superior in terms of computational speed.
Np −1
∑
+ (Czi − yr )⊤ Q (Czi − yr ) + u⊤
i Rui (46) 8.3. Nonlinear PDE control
i=0

with C = [1, 0, . . . , 0]. This tracking objective function readily In order to demonstrate the scalability and versatility of the
translates to the canonical form (23) by expanding the quadratic approach, we use it to control the nonlinear Korteweg–de Vries
forms and neglecting constant terms. The cost function matrices (KdV) equation modeling the propagation of acoustic waves in a
were chosen as Q = QNp = 1 and R = 0.01. The prediction horizon plasma or shallow-water waves (Miura, 1976). The equation reads
was set to one second, which results in Np = 100. We compare
the Koopman operator-based MPC controller (K-MPC) with an MPC ∂ y(t , x) ∂ y(t , x) ∂ 3 y(t , x)
+ y(t , x) + = u(t , x),
controller based on local linearization (L-MPC) in two scenarios. ∂t ∂x ∂ x3
In the first one we do not impose any constraints on the output where y(t , x) is the unknown function and u(t , x) the control input.
and track a piecewise constant reference. In the second one, we We consider a periodic boundary condition on the spatial variable
impose the constraint y ∈ [−0.4, 0.4] and track a time-varying
x ∈ [−π, π]. The nonlinear PDE is discretized using the split-
reference yr (t) = 0.5 cos(2π t /3), which violates the output con-
stepping method with spatial mesh of 128 points and time dis-
straint for some portion of the simulated period. The simulation
cretization of ∆t = 0.01 s, resulting in a computational state-space
results are shown in Fig. 4. We observe a virtually identical tracking
of dimension n = 128. ∑3 The control input u is considered to be of
performance in the first case. In the second case, however, the
local-linearization controller becomes infeasible and hence cannot the form u(t , x) = i=1 ui (t)vi (x), where the coefficients ui (t) are
complete the entire simulation period.7 This infeasibility occurs to be determined by the controller and vi are fixed spatial profiles
2
due to the inaccurate predictions of the local linearization pre- given by vi (x) = e−25(x−ci ) with c1 = −π/2, c2 = 0, c3 = π /2.
dictor over longer prediction horizons. The proposed K-MPC con- The control inputs are constrained to ui (t) ∈ [−1, 1]. The lifting-
troller, on the other hand, does not run infeasible and completes based predictors are constructed from data in the form of 1000
the simulation period without violating the constraints. trajectories of length 200 samples. The initial conditions of the
Note that, even in the first scenario where the two controllers trajectories are random convex combinations of three fixed spatial
2 2
perform equally, the K-MPC controller has the benefit of being profiles given by y10 = e−(x−π/2) , y20 = − sin(x/2)2 , y30 = e−(x+π/2) ;
completely data-driven and requiring only output measurements, the control inputs ui (t) are distributed uniformly in [−1, 1] . The 3
whereas the L-MPC controller requires a model (to compute the lifting mapping ψ is composed of the state itself, the elementwise
local linearization) and full state measurements. In addition, the square of the state, the elementwise product of the state with its
average computation time8 required to evaluate the control in- periodic shift and the constant function, resulting in the dimension
put of the K-MPC controller was 6.86 ms (including the evalua-
of the lifted state N = 3 · 128 + 1 = 385. The control goal is to track
tion of the lifting mapping ψ (ζ )), as opposed to 103 ms for the
a constant-in-space reference that varies in time in a piecewise
constant manner. In order to do so we design the lifting-based
7 Infeasibility of the underlying optimization problem is a common problem
Koopman MPC (23) with the reference tracking objective (46) with
encountered in predictive control with various heuristic (e.g., soft constraints) or
theoretically substantiated (e.g., set invariance) approaches trying to address them.
Q = QNp = I, R = 0, C = [I128 , 0] and prediction horizon Np = 10
See, e.g., Grüne and Pannek (2011) and Rawlings and Mayne (2009) for more details. (i.e., 0.1 s). The results are depicted in Fig. 5; we observe a fast and
8 The optimization problems were solved by qpOASES (Ferreau et al., 2014) accurate tracking of the reference profile. The average computation
running on Matlab and 2 GHz Intel Core i7 with 8 GB RAM. time to evaluate the control input was 0.28 ms (using the dense
 M. Korda, I. Mezić / Automatica 93 (2018) 149–160 159

Fig. 4. Feedback control of a bilinear motor — reference tracking for a bilinear model of a DC motor. Top: piecewise constant reference, x0 = [0, 0.6]⊤ , no state constraints.
Right: time-varying reference, x0 = [−0.1, 0.1]⊤ , constraints on the output imposed.

Fig. 5. Nonlinear PDE control — Tracking of a time-varying constant-in-space reference profile for the Korteweg–de Vries equation. Left: closed-loop solution. Middle: spatial
mean of the solution. Right: control inputs.

form (24) and the hardware configuration described in Footnote 8 ), without resorting to non-linear numerical optimization schemes.
allowing for deployment in real-time applications requiring very Linear inequality constraints on the states and control inputs as
fast sampling rates. well as nonlinear constraints on the states can be imposed in a
linear fashion; in addition cost functions nonlinear in the state can
9. Conclusion and outlook be handled in a linear fashion as well. Computational complexity
of the underlying optimization problem is comparable to that of an
In this paper, we described a class of linear predictors for MPC problem for a linear dynamical system of the same size. This
nonlinear controlled dynamical systems building on the Koopman is achieved by using the so-called dense form of an MPC problem
operator framework. The underlying idea is to lift the nonlinear whose computational complexity depends only on the number of
dynamics to a higher dimensional space where its evolution is control inputs and is virtually independent of the number of states.
approximately linear. The predictors exhibit superior performance Importantly, the entire control design procedure is data-driven,
on the numerical examples tested and can be readily used for requiring only input–output measurements.
feedback control design using linear control design methods. In Future work should focus on imposing or proving closed-loop
particular, linear model predictive control (MPC) can be readily guarantees (e.g., stability or degree of suboptimality) of the con-
used to design controllers for the nonlinear dynamical system troller designed using the presented methodology and on optimal
160 M. Korda, I. Mezić / Automatica 93 (2018) 149–160

selection of the lifting functions given some prior information on Koopman, B. (1931). Hamiltonian systems and transformation in Hilbert space.
the dynamical system at hand. Proceedings of the National Academy of Sciences of the United States of America,
17(5), 315–318.
Koopman, B., & von Neuman, J. (1932). Dynamical systems of continuous spectra.
Acknowledgments Proceedings of the National Academy of Sciences of the United States of America,
18(3), 255–263.
The first author would like to thank Colin N. Jones for initial Korda, M., & Mezić, I. (2018). Journal of Nonlinear Science, 28(2), 687–710.
discussions on the topic and for comments on the manuscript, Ljung, L. (1998). System identification. In Signal analysis and prediction
(pp. 163–173). Springer.
as well as to the three anonymous referees for their constructive Mauroy, A., & Goncalves, J. (2016). Linear identification of nonlinear systems: A
comments that helped improve the manuscript. The authors would lifting technique based on the Koopman operator. In Conference on decision and
also like to thank Péter Koltai for bringing to their attention (Klus control.
et al., 2016). Mayne, D. Q., Rawlings, J. B., Rao, C. V., & Scokaert, P. O. (2000). Constrained model
predictive control: Stability and optimality. Automatica, 36(6), 789–814.
This research was supported in part by the ARO-MURI grant
Mezić, I. (2005). Spectral properties of dynamical systems, model reduction and
W911NF-14-1-0359 and the DARPA grant HR0011-16-C-0116. The decompositions. Nonlinear Dynamics, 41(1–3), 309–325.
research of M. Korda was supported by the Swiss National Science Mezić, I., & Banaszuk, A. (2004). Comparison of systems with complex behavior.
Foundation under grant P2ELP2_165166. Physica D, 197(1–2), 101–133.
Miura, R. M. (1976). The Korteweg–deVries equation: A survey of results. SIAM
Review, 18(3), 412–459.
Appendix Proctor, J. L., Brunton, S. L., & Kutz, J. N. (2016). Dynamic mode decomposition with
control. SIAM Journal on Applied Dynamical Systems, 15(1), 142–161.
This appendix expresses explicitly the matrices in the ‘‘dense- Proctor, J. L., Brunton, S. L., & Kutz, J. N. (2018). Generalizing Koopman theory to
form’’ MPC problem (24) as a function of the data defining the allow for inputs and control. SIAM Journal on Applied Dynamical Systems, in press.
ArXiv preprint arXiv:1602.07647.
‘‘sparse-form’’ MPC problem (23). The matrices are given by
Rawlings, J. B., & Mayne, D. Q. (2009). Model predictive control: Theory and design (1st
H = R + B⊤ QB, h = r + B⊤ q, G = 2A⊤ QB, ed.). Nob Hill Publishing.
Surana, A. (2016). Koopman operator based observer synthesis for control-affine
nonlinear systems. In Conference on decision and control.
L = F + EB, M = EA, 0 , . . . , bNp ] ,
c = [b⊤ ⊤ ⊤
Surana, A., & Banaszuk, A. (2016). Linear observer synthesis for nonlinear systems
using Koopman operator framework. In IFAC symposium on nonlinear control
where systems.
Tu, J. H., Rowley, C. W., Luchtenburg, D. M., Brunton, S. L., & Kutz, J. N. (2014). On dy-
...
⎡ I ⎤
0 0 0
⎡ ⎤
namic mode decomposition: Theory and applications. Journal of Computational
A
⎢ ⎥ ⎢ B 0 ... 0⎥ Dynamics, 1(2), 391–421.
⎢ A2 ⎥ ⎢ AB B ... 0⎥ Williams, M. O., Hemati, M. S., Dawson, S. T. M., Kevrekidis, I. G., & Rowley, C. W.
⎢ . ⎥, B = ⎢ .
A=⎢ ⎥,
⎥ ⎢ ⎥
⎣ .. ⎦ ⎣ .. .. .. (2016). Extending data-driven Koopman analysis to actuated systems. In IFAC
. . ⎦ symposium on nonlinear control systems (NOLCOS).
Williams, M. O., Kevrekidis, I. G., & Rowley, C. W. (2015). A data–driven approxima-
ANp ANp −1 B ... AB B
tion of the Koopman operator: Extending dynamic mode decomposition. Journal
F0 0 ... 0 of Nonlinear Science, 25(6), 1307–1346.
⎡ ⎤
⎢0 F1 ... 0 ⎥ Williams, M. O., Rowley, C. W., & Kevrekidis, I. G. (2015). A kernel-based method
⎢. for data-driven Koopman spectral analysis. Journal of Computational Dynamics,
.. .. ⎥
⎢ ..
F=⎢ . . ⎥ 2(2), 247–265.
⎥
⎣0 0 ... FNp −1
⎦
0 0 ... 0 Milan Korda received the Bachelor’s degree in electrical
engineering and the Master’s degree in cybernetics and
robotics from the Czech Technical University in Prague
Q = diag(Q0 , . . . , QNp ), R = diag(R0 , . . . , RNp −1 ), in 2009 and 2011, respectively. In 2016 he received the
Ph.D. degree for his work on polynomial optimization and
E = diag(E0 , . . . , ENp ), q = [q⊤
0 , . . . , qNp ] , r = [r0 , . . . , rNp −1 ] ,
⊤ ⊤ ⊤ ⊤ ⊤ model predictive control at the Automatic Control Labo-
ratory, École Polytechnique Fédérale de Lausanne (EPFL),
with diag(·, . . . , ·) denoting a block-diagonal matrix composed of Switzerland. Currently he is an SNSF postdoctoral fellow
at the University of California, Santa Barbara (UCSB).
the arguments.

References
Igor Mezić works on operator-theoretic methods in non-
Brunton, S. L., Brunton, B. W., Proctor, J. L., Kaiser, E., & Kutz, J. N. (2017). Chaos as linear dynamical systems and control theory and their
an intermittently forced linear system. Nature communications, 8(1), 1. applications in fluid dynamics, energy efficient design and
Brunton, S. L., Brunton, B. W., Proctor, J. L., & Kutz, J. N. (2016). Koopman invariant operations and complex systems dynamics. He did his
subspaces and finite linear representations of nonlinear dynamical systems for Dipl. Ing. in Mechanical Engineering in 1990 at the Univer-
control. PloS One, 11(2), e0150171. sity of Rijeka, Croatia and his Ph.D. in Applied Mechanics
Budišić, M., Mohr, R., & Mezić, I. (2012). Applied koopmanism. Chaos. An Interdisci- at the California Institute of Technology. Dr. Mezic was
plinary Journal of Nonlinear Science, 22(4), 047510. a postdoctoral researcher at the Mathematics Institute,
Carleman, T. (1932). Application de la théorie des équations intégrales linéaires University of Warwick, UK in 1994–95. From 1995 to 1999
aux systémes d’équations différentielles non linéaires. Acta Mathematica, 59(1), he was a member of Mechanical Engineering Department
63–87. at the University of California, Santa Barbara where he is
Daniel-Berhe, S., & Unbehauen, H. (1998). Experimental physical parameter estima- currently a Professor. In 2000–2001 he has worked as an Associate Professor at
tion of a thyristor driven DC-motor using the HMF-method. Control Engineering Harvard University in the Division of Engineering and Applied Sciences. He won
Practice, 6(5), 615–626. the Alfred P. Sloan Fellowship, NSF CAREER Award from NSF and the George S.
Ferreau, H. J., Kirches, C., Potschka, A., Bock, H. G., & Diehl, M. (2014). qpOASES: Axelby Outstanding Paper Award on ‘‘Control of Mixing’’ from IEEE. He also won
A parametric active-set algorithm for quadratic programming. Mathematical the United Technologies Senior Vice President award for Science and Technology
Programming Computation, 6(4), 327–363. Special Achievement Prize in 2007. He was an Editor of Physica D: Nonlinear
Grüne, L., & Pannek, J. (2011). Nonlinear model predictive control. In Nonlinear Phenomena and an Associate Editor of the Journal of Applied Mechanics and SIAM
model predictive control. Springer. Journal on Control and Optimization and is a Fellow of the American Physical
Klus, S., Koltai, P., & Schütte, C. (2016). On the numerical approximation of the Society and SIAM. Dr. Mezic is the Director of the Center for Energy Efficient Design
Perron-Frobenius and Koopman operator. Journal of Computational Dynamics, and Head of Buildings and Design Solutions Group at the Institute for Energy
3(1), 51–79. Efficiency at the University of California, Santa Barbara.