Proceedings of the 19th World Congress

The International Federation of Automatic Control

Cape Town, South Africa. August 24-29, 2014

Invariant Manifold Approach for

Time-Varying Extremum-Seeking Control
Problem ?
E. Moshksar and M. Guay ∗
∗
Chemical Engineering Department, Queen’s University, Kingston,
Canada, (e-mail: [email protected])

Abstract: In this paper, the minimization of an unknown but measurable cost function using
extremum-seeking control is considered. The main contribution of the paper is to formulate
the extremum-seeking problem as a time-varying estimation problem. The concepts of invariant
manifold and adaptive parameter update law are used for adaptive estimation of the time-
varying gradient of the unknown cost function. The proposed approach is shown to avoid the
need for averaging analysis which minimizes the sensitivity of the closed-loop performance to
the choice of dither signal.

Keywords: Extremum-seeking control; Parameter estimation; Invariant manifold.

1. INTRODUCTION the amplitude and frequency of the dither signal, must

be chosen very carefully to guarantee convergence to a
Optimization has a significant rule in control applications neighbourhood of the unknown optimum. More precise
due to the increasing needs of products with specific statements concerning the dependence of the stability
qualities. The optimization techniques can be used to properties on the choice of different dither signals is
achieve optimal plant operating condition and to reduce provided by Tan et al. [2008].
the final cost. In many optimization problems, off-line In Gelbert et al. [2012], extended Kalman filters are used
methods are applied to reach the optimal conditions of instead of the low and high pass filters to estimate the
the plant. However, most of the systems in the real world gradient of the input to reference map. The main advan-
applications are subject to different initial conditions, tages of this nonlinear filter are the faster response and
dynamics uncertainties and external disturbances. Under the extension of the algorithm to more than one input. On
these conditions, the off-line approaches cannot guarantee the other hand, the gain of the gradient update cannot be
the acceptable performance of the system and real time adjusted freely since the convergence of the ESC depends
optimization is crucial for superior performance (Guay and largely on the magnitude of the unknown Hessian of the
Adetola [2013]). Extremum seeking control (ESC) is an steady-state measured output. A Newton-based approach
adaptive control method that is design to solve real-time is reported in Ghaffari et al. [2012], which provides an esti-
optimization problems. mate for the inverse of the Hessian matrix of the unknown
The ESC is a classical control approach that is used to cost function. This technique can effectively alleviate the
steer a control system to the optimum of a measured objec- convergence problems associated with the gradient-based
tive function of interest (Tan et al. [2010]). The first precise approach. The recent work of Nesic et al. [2013] is another
stability proof of feedback ESC, based on the averaging example of non-gradient approach to the ESC problem.
and singular perturbations techniques has been provided In Guay and Zhang [2003], an adaptive ESC algorithm is
in Krstic and Wang [2000]. Over the last few years, many considered for a system with a known objective function
researchers have considered various approaches to over- that depends on the system states and uncertain plant
come the limitations of ESC. In Krstic [2000], some of these parameters. The proposed adaptive extremum seeking
conditions were removed by using dynamic compensators, control has lots of applications in (bio)chemical processes
while the measurement noise rejection was also achieved. (see Dochain et al. [2011], and the references therein).
The non-local and semi-global stability analysis of ESC
is established in Tan et al. [2006] and Tan et al. [2009]. In this paper, we provide an alternative extremum-seeking
The main contribution of these works was to find explicit technique which is based on the estimation of the gradient
expressions for the domain of attraction in the closed-loop as a time-varying parameter. The estimation scheme is
system. based on the geometric concepts of invariant manifolds.
For this purpose, a number of high gain estimators and fil-
As highlighted in Krstic and Wang [2000], the stability ters are provided and an almost invariant manifold is gen-
analysis relies on a time-scale separation between the fast erated from the filters. This allows to exploit an implicit
transients of the system dynamics and the slow quasi function relating the known variables and the unknown
steady-state condition. This analysis demonstrates that variables. Then a parameter update law is presented using
? This work was supported by the NSERC. the almost invariant manifold and an adaptive estimator.

978-3-902823-62-5/2014 © IFAC 9129

19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

The main contribution of this work is to remove the need If one has access to the gradient θ(t), then it follows
for averaging of the quasi steady-state system, a crucial that the gradient descent u̇ = −k1 θ(t) with k1 > 0, will
element of conventional ESC approach. It also avoids the converge the optimum u∗ .
need to use the frequency of the dither signal as a singu- Lemma 1. Consider the cost function subject to the all
lar perturbation parameter. The proposed ESC algorithm assumptions. The gradient descent update u̇ = −k1 θ(t) is
can improve the transient performance of the closed-loop such that the objective function decreases monotonically
system. and reaches to its minimum at u∗ .
The paper is organized as follows. The problem description
is given in Section 2. In Section 3, the proposed ESC Proof. By convexity of the cost, it is shown in the
controller is presented for the case of process described following that the cost will decrease until a value of u is
by a static map. The extension of the algorithm to an reached such that the gradient of the cost be zero. Let
unknown dynamical system is presented in Section 4. This ũ = u − u∗ and consider the following Lyapunov function
is followed by a simulation example in Section 5 and a brief candidate for the input dynamics:
conclusion in Section 6. Vũ = 12 ũT ũ
2. PROBLEM DESCRIPTION By considering Assumption 1, and differentiation with
respect to t, the following inequality is obtained
Consider a nonlinear system 2
V̇ũ = −k1 ũT ∂`(u)
∂u ≤ −k1 α2 kũ(t)k
ẋ = f (x, u) (1)
y = h(x) (2) As a result, the system converges to the unknown mini-
where x ∈ Rn is the state, u is the control input taking mizer u∗ .
values in U ⊂ Rm and y ∈ R is the unknown and
The design of the extremum seeking scheme is based on
measurable cost function to be minimized. The functions
the unknown dynamics (4). The first step consists in
f (x, u) and h(x) are assumed to be C ∞ in all of their
the estimation of the time-varying parameter θ(t). In the
arguments.
second step, we define a suitable adaptive controller that
The objective of ESC is to bring the closed-loop control accomplishes the extremum seeking task.
system to the unknown equilibrium x∗ and u∗ that min-
imizes the cost function y. The equilibrium (or steady- 3.1 Parameter Estimation
state) map is the n dimensional vector-valued function
π(u) which is such that f (π(u), u) = 0. The steady-state In this section, an alternative technique based on the idea
cost function is given by y = h(π(u)) = `(u). Thus, of invariant manifolds is proposed for adaptive estimation
at steady-state, the problem is reduced to finding the of the unknown time-varying parameters. The geometric
minimizer u∗ of the y = `(u). The following assumptions concept of invariance has been widely used in nonlinear
are required. control theory (see Tian and Yu [2000], Astolfi and Ortega
Assumption 1. The equilibrium cost is such that [2003]). This algorithm utilizes a number of high gain
∂`(u∗ ) ∂ 2 `(u) estimators and filters. An almost invariant manifold is
= 0, > α1 I ∀u ∈ U generated from the filters, which allows to exploit implicit
∂u ∂u∂uT functions relating implicitly the known variables and the
∂`(u) 2
(u − u∗ )T ≥ α2 ku − u∗ k , ∀u ∈ U unknown variables. A parameter update law is assigned
∂u using the almost invariant manifold.
where matrix I is an identity matrix with suitable dimen-
sion, and α1 , α2 are strictly positive constants. Invariant Manifold Design: The basic idea is to find a
Assumption 2. The steady-state map is such that mapping from known variables to the unknown variables,
which has an almost invariance property for sufficiently
∂`(u) ∂ 2 `(u)
kyk ≤ Y, ∂u ≤ L1 , ∂u∂uT
≤ L2 large value of an assignable design gain. This implicit
mapping is used to facilitate the estimation the unknown
∀u ∈ U with positive constants Y , L1 and L2 . parameters.
3. STATIC MAP The estimator model for (4) is defined as
ŷ˙ = −k 2 (ŷ − y), k > 0 (5)
Since the minimization of y is performed in real-time, the
input u is taken as a time-varying signal. That is, and the filter is described along with the structure of the
system by
y(t) = `(u(t)) (3)
φ̇ = −k 2 (φ − u̇) (6)
Assumption 3. The input signal u(t) is such that u(t) ∈ U
and ku̇(t)k ≤ α3 , ∀t ≥ t0 ≥ 0. Assumption 4. The vector-valued function φ(t) is bounded
with a known bound as kφk ≤ λ, ∀t ≥ t0 ≥ 0.
The differentiation of (3) with respect to time results in Proposition 2. Consider the estimator (5) and filter (6),
ẏ = ( ∂`(u) T
∂u ) u̇. Following the time-varying approach in the following implicit manifold
Guay et al. [2013], we define the unknown gradient and lim [k 2 (ŷ − y) + φT θ(t)] = 0 (7)
the cost dynamic as k→∞
∂`(u) is such that the manifold is invariant and internally ex-
θ(t) = , ẏ = θ(t)T u̇(t) (4) ponentially stable. Furthermore, the desired manifold is
∂u

9130
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

bounded and almost invariant for bounded and sufficiently An adaptive estimator is considered as
large values of k. Σ̇(t) = k[bI − φφT (Σ(t))], Σ(t0 ) = bI (15)
Proof. First, it is shown that the manifold (7) has an in- Based on (14) and (15), the proposed parameter update
variance feature. The off-the-manifold coordinate variable law is given by
z(t) is defined as ˙
θ̂(t) = k 2 Σ(t)[δ̇ + kδ] (16)
z(t) = k 2 (ŷ − y) + φT θ(t). (8) where b > 1 and δ = p − q.
The derivative of z(t) along the trajectories of the system Assumption 5. There exists constants α > 0 and T > 0
is given by such that
Z t+T
ż(t) = k 2 (ŷ˙ − ẏ) + φ̇T θ(t) + φT θ̇(t). (9)
φ(τ )φT (τ )dτ ≥ αI, ∀t > t0 . (17)
It follows from (4)-(6) that t
T
φ θ̇(t) The condition of the Assumption 5 is equivalent to the
ż(t) = −k 2 z(t) + φT θ̇(t) = −k 2 (z(t) − ). (10)
k2 standard persistent excitation (PE) condition required for
∂ 2 `(u) parameter convergence.
By the fact that θ̇ = ∂u∂u T u̇, and in view of Assumptions 2
φT θ̇(t)
and 3, it follows that θ̇ ≤ L2 α3 = Lθ , so k2 → 0 as 3.2 Controller Design
k → ∞. Suppose there exists a time tc such that z(tc ) = 0,
The proposed extremum seeking control is given by
then
u̇ = −k1 θ̂(t) + d(t) (18)
z(tc ) = 0 ⇒ ż(t) ≡ 0 ⇒ z(t) ≡ 0, ∀t ≥ tc
where d(t) is a bounded dither signal with kd(t)k ≤ D. The
which shows that z(t) = 0 is an invariant manifold for dither signal d(t) can be chosen arbitrary, and it only need
nonlinear dynamics (4)-(6) as k → ∞, and ż(t) = −k 2 z(t) to satisfy the PE condition of the Assumption 5. A one
is globally exponentially stable. proper choice of d(t) signals are sinusoidal waves, because
For the case where k is bounded, a quadratic Lyapunov of their orthogonality feature (Tan et al. [2008]). Although,
function can be defined as in our algorithm convergence to the optimal value is not
1 sensitive to the choice of amplitude and frequency of
Vz(t) = z 2 (t) (11)
2 the sinusoidal signal. Note that, since θ̂(t) and d(t) are
The first time derivative of Vz(t) along its trajectories, assumed to be bounded then the controller (18) is such
result in that ku̇k ≤ k1 L1 + D ≤ α3 .
V̇z(t) = −k 2 z 2 (t) + z(t)φT θ̇(t) Theorem 3. Let Assumptions 1 to 5 hold. Then the pa-
rameter update law (16) and the control law (18) are
As a result of Assumption 4, and applying Young’s In- such that the closed-loop extremum seeking control system
equality, one obtains the following inequality: converges to a neighborhood of the minimizer u∗ of the
λk 2 λ T static nonlinear optimization problem (4). The size of this
V̇z(t) ≤ −k 2 z 2 (t) + z (t) + θ̇ (t)θ̇(t) neighborhood is adjustable by the gains k and k1 .
 2 2k (12)
λk λ 2
≤ − k2 − Vz(t) + Lθ Proof. By construction of parameter estimation error as
2 2k
θ̃(t) = θ(t) − θ̂(t), the parameter update law (16) can be
where k  λ2 . Hence, the manifold normal coordinate approximated implicitly in the form
variables z(t) enter a small neighborhood of the origin. ˙
The size of this neighborhood depends on the choice of θ̂(t) = k 2 Σ(t)φφT θ̃(t). (19)
the high gain k. Therefore, an almost invariant manifold The quadratic Lyapunov function is defined as
is obtained by considering sufficiently large value for k. 1
V(θ̃,ũ) = θ̃T (t)θ̃(t) + Vũ (20)
Remark 1. It will be shown that the parameter estima- 2
tion can be achieved by considering the proposed almost By differentiating of (20) along (19) and (18), we have
invariant manifold. Thus, unlike the sliding mode-based
techniques, satisfactory performance can be attained with- V̇(θ̃,ũ) = −k 2 θ̃T (t)(Σ(t)φφT )θ̃(t) + θ̃T (t)θ̇(t)
out requiring that the system states reach the reference −k1 ũT θ(t) + k1 ũT θ̃(t) + ũT d(t)
manifold.
By Assumption 1, one can write the following inequality
Adaptive Estimation: The invariant manifold (7) can be
V̇(θ̃,ũ) ≤ −k 2 θ̃T (t)(Σ(t)φφT )θ̃(t) + θ̃T (t)θ̇(t)
re-written as
k 2 (ŷ − y) = −φT θ(t). (13) −k1 α2 ũT ũ + k1 ũT θ̃(t) + ũT d(t)
This equation provides an implicit relationship from Applying Young’s inequality to all indefinite terms of the
known variables (φ, ŷ, y) to the unknown variable θ(t). last inequality, there exists a positive constant k2 such that
This mapping provides direct information about the pa- k
rameter estimation error without requiring a priori knowl- V̇(θ̃,ũ) ≤ −k 2 θ̃T (t)(Σ(t)φφT )θ̃(t) + θ̃T (t)θ̃(t)
edge of the time-varying parameters. This is achieved by 2
1 kk1 T
defining the auxiliary variables p and q with the dynamics + θ̇T (t)θ̇(t) − k1 α2 ũT ũ + θ̃ (t)θ̃(t)
2k 2
ṗ = −kp − φφT θ̂(t) k1 k2 1 T
(14) + ũT ũ + ũT ũ + d (t)d(t)
q̇ = −kq + φ(k 2 (ŷ − y)) 2k 2 2k2

9131
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

With collecting the similar terms, one can rewrite the depends on the choice of gains k, k1 and the magnitude of
above inequality as follows the dither signal. This completes the proof of Theorem 3.
 
kk1 + k Remark 2. By construction, one can decrease the contri-
V̇(θ̃,ũ) ≤ − k 2 Σ(t)φφT − θ̃T (t)θ̃(t) bution from d(t) and θ̇(t) by increasing the optimization
2
gain k1 and the estimation gain k, respectively. Since,
 
k1 + kk2 1 T 1 T
− k1 α2 − ũT ũ + θ̇ (t)θ̇(t) + d (t)d(t) the rate of change of the parameter
2k 2k 2k2  is proportional
 to
Next we claim the boundedness of the matrix Σ(t) as the optimization gain, θ̇(t) ≤ L2 k1 θ̂(t) + D , the
follows. By integration, one gets convergence to the small neighborhood is achieved by
Z t  ensuring that k  k1 .
T
Σ(t) = exp −kφ(τ )φ (τ )dτ Σ(t0 )
Z t Zt0 t  4. OPTIMIZATION IN DYNAMICAL SYSTEMS
+kb exp −kφ(ξ)φT (ξ)dξ dτ >
In this section, we consider the initial extremum control
Z t t0 Z t τ (21) system which consists in steering the unknown dynamical

kb exp −kφ(ξ)φT (ξ)dξ dτ ≥ system (1) to the equilibrium that minimizes the measure
 tZ0 t τ
cost function (2). The closed-loop extremum seeking con-
b(1 − e−kβ(t−t0 ) )

−kβ(t−τ ) trol system is given by (Guay et al. [2013]):
kb e dτ I = I
t0 β
1 ẋ = f (x, u)
where the last inequality is achieved from the boundedness u̇ = −k1 θ̂(t) + d(t)
of matrix φφT . As a result of Assumption 5, we obtain ˙
Z t Z t θ̂(t) = k 2 Σ(t)[δ̇ + kδ]
(26)

Σ(t) ≤ Σ(t0 ) + kb exp −kφ(ξ)φT (ξ)dξ dτ ≤ ŷ˙ = −k 2 (ŷ − y)
Z t t0  τ φ̇ = −k 2 (φ − u̇)
bα + b(1 − e−kα(t−t0 ) )

b + kb e −kα(t−τ )
dτ I ≤ I Σ̇(t) = k[bI − φφT (Σ(t))]
t0 α
As in other works on extremum-seeking control, the closed-
(22) loop dynamics of the system are written in error form in
It follows from (21) and (22) that a two time-scale system where t is the slow time-scale and
b(1 − e−kβ(t−t0 ) ) T the system’s dynamics are assumed to evolve over a fast
φφ ≤ Σ(t)φφT time-scale τ = t1 . The parameter 1 > 0 is a small strictly
β (23)
bα + b(1 − e−kα(t−t0 ) ) T positive parameter to be assigned.
≤ φφ
α Let us define the deviation variables x̃ = x − π(u) and ũ =
and u − u∗ where u∗ is the local minimizer of the steady-state
b(1 − e−kβ(t−t0 ) ) ≤ Σ(t)φφT map y = `(u). The auxiliary variable is defined as above.
bα + b(1 − e−kα(t−t0 ) ) (24) However, one must take into account the measurement of
≤ β. the cost function over the fast and the slow time-scale, as
α
Consider the function V̇(θ̃,ũ) and inequality (24), the gain h(x̃ + π(ũ + u∗ )). Therefore, new dynamics are defined for
the off-the-manifold coordinate variable z(t), as
should be chosen such that   
k1 + 1 2 ˙ 1 ∂h(x) ∂h(x) ∂π(u)
(1 − e−kβ(t−t0 ) )k ≥ ż = k ŷ − f (x, u) + u̇
2 1 ∂x ∂x ∂u (27)
k1 + kk2 T T
+φ̇ θ(t) + φ θ̇(t)
k1 α2 − >0
2k Substituting for ŷ˙ and φ̇, one obtains
or equivalently  
2 2 ∂h(π(u)) ∂π(u)
N ż(t) = −k z(t) + k u̇
X k1 + 1 ∂x ∂u 
kβ(t − t0 ) = kβ (ti − ti−1 ) ≥ ln( )  (28)
2k 1 ∂h(x) ∂h(x) ∂π(u)
i=1 −k 2 f (x, u) + u̇ + φT θ̇(t)
kk2 1 ∂x ∂x ∂u
k1 >
2kα2 − 1 As a result, the z(t) dynamic is affected by the fast and
Based on the last inequality, kβN T 0 ≥ ln(k1 + 1/2k), slow dynamics. One can write the system (26) and implicit
where T 0 is the sampling time and N is an integer. If the manifold (28) in deviation form as follows
estimation gain is chosen large enough as k ≥ ln(k1βT +1/2k)
, ∂π(u) ˙
0 1 x̃˙ = f (x̃ + π(ũ + u∗ ), ũ + u∗ ) − 1 ũ
then (1 − e −kβ(t−t0 ) k1 +1
)k ≥ 2 , ∀t > t0 ≥ 0. For the ∂u
∂h(x)
given k, there exist strictly positive constants ka , kb and 1 ż(t) = −k 2 f (x, u) − k 2 1 z(t) + 1 φT θ̇(t)
k 0 = min {ka , kb } such that  ∂x 
2 ∂h(π(u)) ∂π(u) ∂h(x) ∂π(u) (29)
1 2 1 2 +k 1 u̇ − u̇
V̇(θ̃,ũ) ≤ −ka θ̃T (t)θ̃(t) − kb ũT ũ + L + D ∂x ∂u ∂x ∂u
2k θ 2k2 (25) ũ˙ = −k1 θ̂(t) + d(t)
1 1 ˙
≤ −k 0 V(θ̃,ũ) + L2θ + D2 θ̃(t) = −k 2 Σ(t)[δ̇ + kδ] + θ̇(t)
2k 2k2
It is assumed that the gain k is such that
It follows that θ̃ and ũ converge exponentially to a small
1 2
neighborhood of the origin. The size of this neighborhood k 2 = ks2 + 1 kf

9132
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

This definition of k indicates that the predictor equation 2.6

Estimated value
must be operated as a low-pass in the fast time-scale, by 2.4
Optimal value

fixing kf to be large. The parameter estimation routine

u1
operates following the slower time-scale with gain ks . 2.2

Following the standard singular perturbation technique, 2

the reduced system (x̃ = π(u)) is given by 1.8

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
ż = −k 2 z(t) + φT θ̇(t) 5
Estimated value

ũ˙ = −k1 θ̂(t) + d(t) (30) 4.5

Optimal value

˙ 2
θ̃(t) = −k Σ(t)[δ̇ + kδ] + θ̇(t)

u2
4
The boundary layer system is given by
dx̃ 3.5
= f (x̃ + π(ũ + u∗ ), ũ + u∗ ) 0 0.5 1 1.5 2 2.5
Time (sec)
3 3.5 4 4.5 5

dτ
dz ∂h(x̃ + π(ũ + u∗ ))
= −kf2 z − k 2 Fig. 1. Trajectories of the control inputs
dτ ∂x (31)
∂h(π(ũ + u∗ ))
f (x̃ + π(ũ + u∗ ), ũ + u∗ ) + kf2 ( optimization problem. The size of the neighborhood de-
∂x pends on the choice of the gains k, k1 and the magnitude
∂π(ũ + u∗ ) ˙ ∂h(x̃ + π(ũ + u∗ )) ∂π(ũ + u∗ ) ˙
ũ − ũ) of the dither signal d(t).
∂u ∂x ∂u
Assumption 6. The origin of the nonlinear system (1) is The proof is given in Appendix A.
locally exponentially stable ∀u ∈ U. Let χr = {x̃ ∈
Rn | kx̃k ≤ r} for r > 0, a positive constant. Similarly, 5. SIMULATION EXAMPLE
let Er = {z ∈ R | |z| ≤ r}.
Assumption 7. The vector field f (x̃ + π(ũ + u∗ ), ũ + u∗ ) is The unknown static input-output map is given by Ghaffari
such that et al. [2012] as
kf (x̃ + π(ũ + u∗ ), ũ + u∗ )k ≤ Lf kx̃k
  T    
1 2 100 30 2
y = 100 + u− u−
∀x̃ ∈ χ and ∀u ∈ U where Lf > 0 is a positive constant. 2 4 30 20 4
Assumption 8. The cost function h(x) is such that The objective is to minimize the output y with respect to
∂h(x̃ + π(ũ + u∗ )) input u. We apply the proposed extremum-seeking
√ control
≤ Lh algorithm with b = 1.1, k = 10 10, k1 = 0.3, and
∂x arbitrary dither signal
∂h(x̃ + π(u)) ∂h(ỹ + π(u))  
− ≤ Lh kx̃ − ỹk sin(10t)
∂x ∂x d(t) = 0.002 .
sin(20t)
∀x̃, ỹ ∈ χ and ∀u ∈ U where Lh > 0 is a positive constant.
The initial conditions are chosen as u(0) = [2.5, 5]T and
Finally, we make the following assumption concerning the θ̂(0) = [10, 10]T , where θ̂(t) is estimation of the unknown
steady-state map, π(u). gradient.
Assumption 9. The steady-state map π(u) is such that
The simulation results are shown in Figs. 1-3. The
∂π(u) extremum-seeking control input trajectories re shown in
≤ Lπ
∂u Fig. 1. The changes of the unknown cost function y
∀u ∈ U where Lπ > 0 is a positive constant. is depicted in Fig. 2. The results demonstrate that the
proposed extremum-seeking control algorithm provides a
By Assumptions 7, 8 and 9, it follows that there exists rapid progression to the unknown minimizer of the opti-
a kf such that the origin of the boundary layer (31) is mization problem. In addition, the control system provides
locally exponentially stable. It then follows that there satisfactory transient behaviour for both the inputs and
exists a Lyapunov function and positive constants, βi for the objective function.
i = 1, ..., 6 such that
2 2 2 2 The normal manifold coordinate variable z(t) is shown in
β1 (kx̃k + |z| ) ≤ V (x̃, z) ≤ β2 (kx̃k + |z| ) Fig. 3. As confirmed in this figure, the implicit manifold
dV dV dx̃ dV dz 2 2 z(t) converges to the small neighbourhood of zero. It is
= + ≤ −β3 kx̃k − β4 |z| important to point out that the value of this variable is
dτ dx̃ dτ dz dτ
  not known.
dV dV 2 2
, ≤ β5 kx̃k + β6 |z|
dx̃ dz 6. CONCLUSION
∀u ∈ U, x̃ ∈ χr .
Theorem 4. Consider the nonlinear system (1) and the In this paper, an alternative time-varying ESC technique
cost function (2). Let Assumptions 1 to 9 be fulfilled then was proposed. The technique is based on the time-varying
the time-varying parameter estimation scheme and the estimation of the unknown gradient which relies on defi-
extremum-seeking controller (18) is such that for every nition of an invariant manifold principle. The ESC algo-
1 ∈ (0, ∗ ), the closed-loop system converges to a neigh- rithm is shown to provide local exponential convergence
borhood of the unknown local minimizer of the nonlinear of the closed-loop system to the unknown optimum. The

9133
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

140 M. Krstic and H. H. Wang. Stability of extremum

135
seeking feedback for general nonlinear dynamic systems.
Automatica, 36:595–601, 2000.
130
D. Nesic, T. Nguyen, Y. Tan, and C. Manzie. A non-
125
gradient approach to global extremum seeking: An adap-
120 tation of the Shubert algorithm. Automatica, 49:809–
y

115 815, 2013.

110
Y. Tan, W. Moase, C. Manzie, D. Nesic, and I. Mareels.
Extremum seeking from 1922 to 2010. In Proceedings of
105
Chinese Control Conference, pages 14–26, 2010.
100 Y. Tan, D. Nesic, and I. Mareels. On non-local stability
95 properties of extremum seeking control. Automatica, 36:
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Time (sec) 889–903, 2006.
Y. Tan, D. Nesic, and I. Mareels. On the choice of dither in
Fig. 2. The unknown cost function versus time extremum seeking systems: A case study. Automatica,
44:1446–1450, 2008.
120 Y. Tan, D. Nesic, I. Mareels, and A. Astolfi. On global
extremum seeking in the presence of local extrema.
100
Automatica, 45:245–251, 2009.
80
Y. P. Tian and X. Yu. Adaptive control of chaotic systems
using invariant manifold approach. IEEE Transactions
60 on Automatic Control, 47:1537–1542, 2000.
z(t)

40 Appendix A. PROOF OF THEOREM 4

20
Proof follows standard singular perturbation technique
0
based on the Lyapunov function candidate
0 0.02 0.04 0.06 0.08 0.1
Time (sec)
0.12 0.14 0.16 0.18 0.2 W = µV(θ̃,ũ) + ηVz(t) + (1 − µ − η)V (x̃, z) (A.1)
where µ, η ∈ (0, 1) and µ + η < 1.
Fig. 3. Trajectory of the implicit manifold Differentiating with respect to t, one obtains
µ 2 µ 2 λk
technique simplifies the tuning of the designed gains by Ẇ ≤ −k 0 µV(θ̃,ũ) + Lθ + D − η(k 2 − )Vz(t)
avoiding the limitations associated with choice of dither. 2k 2k2 2 
ηλ 2 2 ∂h(π(u)) ∂π(u) ∂h(x) ∂π(u)
+ Lθ + ηks z(t) u̇ − u̇
2k ∂x ∂u ∂x ∂u
REFERENCES (1 − µ − η) 2 (1 − µ − η) 2
− β3 kx̃k − β4 |z|
A. Astolfi and R. Ortega. Immersion and invariance: a new 1 1
tool for stabilization and adaptive control of nonlinear By Assumptions 7, 8 and 9, the inequality becomes
systems. IEEE Transaction on Automatic Control, 48: 
kx̃k

590–606, 2003. Ẇ ≤ − [kx̃k , |z|] A − k 0 µV(θ̃,ũ)
|z|
D. Dochain, M. Perrier and M. Guay. Extremum seeking (A.2)
λk µ + ηλ 2 µ 2
control and its application to process and reaction −η(k 2 − )Vz(t) + Lθ + D
systems: A survey. Mathematics and Computers in 2 2k 2k2
Simulation, 82:369–380, 2011. where
(1 − µ − η) 1
 
G. Gelbert, J. P. Moeck, C. O. Paschereit, and R. King. β3 − ηks2 Lπ Lh (k1 L1 + D)
Advanced algorithms for gradient estimation in one-and A= 1
 1 2 
two-parameter extremum seeking controllers. Journal of 2 (1 − µ − η) 
− ηks Lπ Lh (k1 L1 + D) β4
Process Control, 22:700–709, 2012. 2 1
A. Ghaffari, M. Krstic, and D. Nesic. Multivariable The matrix A is positive definite if 1 is chosen such that
Newton-based extremum seeking. Automatica, 48:1759– √
2 β3 β4 (1 − µ − η)
1767, 2012. 1 < 2 = ∗
M. Guay and V. Adetola. Adaptive economic optimising ηks Lπ Lh (k1 L1 + D)
model predictive control of uncertain nonlinear systems. It then follows that ∀1 ∈ (0, ∗ ),
International Journal of Control, 86:1425–1437, 2013. 
λk

0 2
M. Guay, S. Dhaliwal, and D. Dochain. A time-varying Ẇ ≤ −k µV(θ̃,ũ) − kc + η(k − ) Vz(t)
2 (A.3)
extremum seeking control approach. In Proceedings of µ + ηλ 2 µ 2
2
American Control Conference, pages 2643–2648, 2013. −kc kx̃k + Lθ + D
M. Guay and T. Zhang. Adaptive extremum seeking 2k 2k2
control of nonlinear dynamic systems with parametric where kc is a strictly positive constant. This completes the
uncertainties. Automatica, 39:1283–1293, 2003. proof.
M. Krstic. Performance improvement and limitations in
extremum seeking control. Systems and Control Letters,
39:313–326, 2000.

9134