Assessment of Model Predictive and Adaptive Glucose Contr - 2014 - IFAC Proceedi
Assessment of Model Predictive and Adaptive Glucose Contr - 2014 - IFAC Proceedi
Abstract: This paper addresses overnight blood glucose stabilization in people with type 1
diabetes using a Model Predictive Controller (MPC). We use a control strategy based on
an adaptive ARMAX model in which we use a Recursive Extended Least Squares (RELS)
method to estimate parameters of the stochastic part. We compare this model structure with
an autoregressive integrated moving average with exogenous input (ARIMAX) structure, and
with an autoregressive moving average with exogenous input (ARMAX) model, i.e. without
an integrator. Additionally, safety layers improve the controller robustness and reduce the risk
of hypoglycemia. We test our control strategies on a virtual clinic of 100 randomly generated
patients with a representative inter-subject variability. This virtual clinic is based on the Hovorka
model. We consider the case where only half of the meal bolus is administered at mealtime, and
the case where the insulin sensitivity varies during the night. The simulation results demonstrate
that the adaptive control strategy can reduce the risks of hypoglycemia and hyperglycemia
during the night.
1. INTRODUCTION
6
Parameter Value
τsub 15 min 5.9
Glucose (mmol/L)
λ 15.96
5.8
ξ -5.471
δ 1.6898 5.7
γ -0.5444
5.6
structures for the stochastic part. The first one is an 5.5
Blood Glucose
autoregressive integrated moving average with exogenous Subcutaneous Glucose
input (ARIMAX) structure. The second one is an autore- 5.4 Second order approximation
gressive moving average with exogenous input (ARMAX) 5.3
model, i.e. without integrator. The third one is an adaptive 0 5 10 15 20
Time (hours)
ARMAX model in which we use a Recursive Extended
Least Square (RELS) method to estimate parameters of Fig. 2. Impulse responses for a second order model and
the stochastic part (i.e. the MA part). The controller is the nonlinear Hovorka model. The bolus size is 0.1U
tested on a cohort of 100 virtual patients. and the parameters for the second order model are:
τ =4 hours and ISF = 0.4 mmol/L/0.1 U = 4.0
2. PHYSIOLOGICAL MODELS FOR PEOPLE WITH mmol/L/U.
TYPE 1 DIABETES
3.1 Choice of the deterministic model
In this paper, we use the Hovorka model to simulate people
with type 1 diabetes. Using the parameters and distribu- The physiological models listed in Section 2 contain a large
tions provided in Hovorka et al. (2002) and Wilinska et al. number of parameters, and Pillonetto et al. (2003) estab-
(2010), we generate a cohort of 100 virtual patients. lished that even the minimal model developed by Bergman
et al. (1981) may be difficult to identify. To overcome this
In addition, we use a CGM for glucose feedback in our con- issue, we use a low-order linear model to describe the
troller setup. For the numerical simulations, we generate glucose-insulin dynamics. Similar approaches have been
noisy CGM data based on the model and the parameters investigated previously. Kirchsteiger et al. (2011) used
determined by Breton and Kovatchev (2008). This model a third order transfer function with an integrator, van
consists of two parts. The first part describes the glucose Heusden et al. (2012) used a third order discrete transfer
transport from blood to interstitial tissues, which is function model and Percival et al. (2010) applied a first
order transfer function with a time delay. In this paper we
dGsub 1 use a continuous-time second order transfer function
= (G(t) − Gsub (t)) (1) Y (s) Ku
dt τsub G(s) = = (6)
Gsub (t) is the subcutaneous glucose and G(t) is the blood U (s) (τ s + 1)2
glucose. The time constant τsub is associated to glucose to model the effect of sc injected insulin on sc glucose. The
transport from blood to subcutaneous tissues. gain, Ku , and the time constant, τ , are computed from
known subject-specific parameters; the insulin action time
The second part models non-Gaussian sensor noise. It is
and the insulin sensitivity factor (ISF).
given by
ek = 0.7(ek−1 + vk ), k > 0 (2) The insulin action time and the insulin sensitivity factor
vk ∼ Niid (0, 1) (3) are related to the response of blood glucose to an insulin
bolus. If we assume that blood glucose is approximately
ek − γ identical to sc glucose, this is the impulse response of (6).
ηk = ξ + λ sinh (4)
δ The insulin action time is the time for blood glucose to
with the initial condition e0 ∼ Niid (0, 1). reach its minimum. The ISF corresponds to the maximum
decrease in blood glucose per unit of insulin bolus. These
The glucose value returned by the CGM is parameters are empirically estimated by the patient and
GCGM (tk ) = Gsub (tk ) + ηk (5) his/her physician. They may vary from day to day for a
given patient but gives an estimate of the effect of insulin
The numerical values used in this paper for τsub , λ, ξ, δ on blood glucose and sc glucose.
and γ are shown in Table 1.
Fig 2 depicts the impulse response for a virtual patient
3. MODELING OF GLUCOSE-INSULIN DYNAMICS with type 1 diabetes and its second order approximation
(6). This patient is simulated using the model developed
by Hovorka et al. (2004). The figure demonstrates that a
In this section, we derive a prediction model for subcu-
second order model provides an acceptable approximation
taneous glucose, y(t). The model has a deterministic part
of a virtual patient with type 1 diabetes.
describing the effect of sc. injected insulin, u(t), and a
stochastic part describing the effect of other unknown In the temporal domain, the impulse response of (6) is
factors. This model identification technique turns out to described by
give a good compromise between data requirements, per- t
formance and robustness of the resulting controller. y(t) = Ku 2 exp(−t/τ ) (7)
τ
232
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014
The insulin action time corresponds to the time to reach We use the same procedure as in Section 4.1 for computing
the minimum blood glucose. Consequently, this insulin β1 and β2 , i.e. β1,2 = 0.81 ± 0.16i. This yields
action time is equal to τ . We determine Ku using (7) and C(q −1 ) = 1 − 1.62q −1 + 0.68q −2 (15)
the fact that the insulin sensitivity factor is equal to the
minimal blood glucose (sc glucose), y(τ ) = −ISF , such Unlike the ARIMAX model structure described in Sec-
that tion 4.1, this model structure does not ensure offset-free
Ku = −τ exp(1)ISF (8) control. On the other hand, it does not introduce a sup-
plementary model-plant mismatch.
We discretize the transfer function (6) in the form
B(q −1 ) 4.3 Adaptive control
y(t) = u(t) (9)
A(q −1 )
The sampling time is 5 minutes. Here, we consider again the ARMAX model structure (13).
A similar approach has been proposed by Eren-Oruklu
4. STOCHASTIC MODEL et al. (2009).
We take into account the process and measurement noise The parameters c1 and c2 are estimated at each iteration
by adding a term describing the effect of unknown factors using the RELS method
to the discrete-time model (9). We assume the model εk = yk − φ0k θ̂k|k−1 (16a)
describing the glucose-insulin dynamics to be in the form Pk−1 φk
C(q −1 ) Kk = (16b)
A(q −1 )y(t) = B(q −1 )u(t) + ε(t) (10) µ + φ0k Pk−1 φk
D(q −1 )
θ̂k+1|k = θ̂k|k−1 + Kk yk − φ0k θ̂k|k−1 (16c)
The model (10) has a deterministic part describing the
0
effects of insulin injections u(t) and a stochastic part. We 1 Pk−1 φk φk Pk−1
Pk = Pk−1 − (16d)
assume either D(q −1 ) = 1 − q −1 , which turns the model µ µ + φ0k Pk−1 φk
(10) into an ARIMAX model or D(q −1 ) = 1, which turns φk is a vector of past observations
the model (10) into an ARMAX model.
φk = [yk−1 yk−2 uk−1 uk−2 εk−1 εk−2 ] (17)
In this section we propose and discuss three different θk is a vector of model parameters
choices for the stochastic model in (10). The two first 0
choices estimate the C(q −1 ) based on the data from a θk = [−a1 −a2 b1 b2 c1 c2 ] (18)
previous clinical study, while the last method estimate Pk is the model parameters covariance matrix. Since we
it recursively using a Recursive Extended Least Squares want to estimate c1 and c2 only, we initialize it with
(RELS) algorithm. P0 = diag(0, 0, 0, 0, 100, 100) (19)
4.1 ARIMAX modeling Finally, µ is the forgetting factor. This parameter has an
influence on the weight of previous observations. When
The stochastic part, C(q −1 ), of the ARIMAX model µ = 1, all the past observations are equally weighted.
Smaller values of µ give more importance to recent ob-
C(q −1 ) servations. In this paper, we chose µ = 0.95, i.e. the
A(q −1 )y(t) = B(q −1 )u(t) + ε(t) (11)
1 − q −1 corresponding memory length is approximately 1/(1 −
is assumed to be a third order polynomial of the form 0.95) = 20 time samples, or 100 minutes. This model
C(q −1 ) = 1 + c1 q −1 + c2 q −2 + c3 q −3 structure allows for a personalized and localized stochastic
(12) model description.
= (1 − αq −1 )(1 − β1 q −1 )(1 − β2 q −1 )
α = 0.99 is a fixed parameter. α has been determined 4.4 Realization and predictions
based on performance studies of the resulting MPC. The
choice of α is discussed in Huusom et al. (2012). β1 and The ARIMAX model (11) and the ARMAX model (13)
β2 are determined from clinical data for one real patient may be represented as a discrete-time state space model
(Duun-Henriksen et al., 2012; Boiroux et al., 2012). They in innovation form
are β1,2 = 0.81 ± 0.16i. xk+1 = Axk + Buk + Kεk (20a)
The main drawback of this specification is the model-plant yk = Cxk + εk (20b)
mismatch. However, this model-plant mismatch enables to The observer canonical realization for the ARMAX model
have offset free control in the resulting predictive control (13) is
system.
−a1 1 b c − a1
A= ; B= 1 ; K= 1
4.2 ARMAX modeling −a2 0 b2 c2 − a2
C = [1 0]
The stochastic part, C(q −1 ), of the ARMAX model and the observer canonical realization for the ARIMAX
A(q −1 )y(t) = B(q −1 )u(t) + C(q −1 )ε(t) (13) model (11) is
is now assumed to be a second order polynomial of the "
1 − a1 1 0
# "
b1
# "
c1 + 1 − a1
#
form A = a1 − a2 0 1 B = b2 − b1 K = c2 + a1 − a2
C(q −1 ) = 1 + c1 q −1 + c2 q −2 a2 0 0 −b2 c3 + a2
(14)
= (1 − β1 q −1 )(1 − β2 q −1 ) C = [1 0 0]
233
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014
Cost function
x̂k+1|k = Ax̂k|k−1 + B ûk|k + Kεk (22a)
x̂k+1+j|k = Ax̂k+j|k + B ûk+j|k , j = 1, . . . , N − 1 (22b) 40
ŷk+j|k = C x̂k+j|k , j = 1, . . . , N (22c)
The innovation (21) and the predictions (22) constitute 20
the feedback and the predictions in the model predictive
controller.
0
4 6 8 10
5. MODEL PREDICTIVE CONTROL Glucose (mmol/L)
Control algorithms for glucose regulation in people with Fig. 3. The penalty function ρ = ky − rk22 + κk min{y −
type 1 diabetes must be able to handle intra- and inter- ymin , 0}k22 .
patient variability. In addition, the controller must ad- λ = 100/u2ss . The soft output constraint (23d) penalizes
minister insulin in a safe way to minimize the risk of glucose values below 4 mmol/L. Since hypoglycemia is
hypoglycemia. Due to the nonlinearity in the glucose- highly undesirable, we choose the weight on the soft output
insulin interaction, the risk of hypoglycemic episodes as constraint to be rather high, i.e. κ = 100. The penalty
consequence of too much insulin is particularly prominent. function profile is illustrated in Fig. 3.
In this section we describe an MPC formulation with soft To prevent model-plant mismatch, we modify the maximal
output constraints and hard input constraints. This for- allowable insulin injection, umax , and let it depend on the
mulation is based on the individualized prediction model current glucose concentration. If the glucose concentration
for glucose computed in Section 4.2. Along with other is low (below the target of 6 mmol/L), we prevent the
features, we introduce a modified time-varying reference controller from taking future hyperglycemia into account
signal to robustify the controller and mitigate the effect of by restricting the maximal insulin injection. If the glucose
glucose-insulin nonlinearities and model-plant mismatch concentration is high (4 mmol/L above the target) we
in the controller action. increase the maximal allowable insulin injection rate. In
The MPC algorithm computes the insulin dose by solution the range 0 - 4 mmol/L above target we allow the con-
of an open-loop optimal control problem. Only the control troller to double the basal insulin injection rate. These
action corresponding to the first sample interval is imple- considerations lead to
mented and the process is repeated at the next sample 1.5uss 4 ≤ yk ≤ ∞
interval. This is called a moving horizon implementation. umax = uss 0 ≤ yk ≤ 4 (25)
The innovation (21) provides feedback from the CGM, yk ,
0.5uss −∞ ≤ yk ≤ 0
and the open-loop optimal control problem solved in each
sample interval is the convex quadratic program in which uss is the basal insulin injection rate. Due to
pump restrictions, the minimum insulin injection rate,
min φ (23a) umin , is a low value but not exactly zero.
N −1
{ûk+j|k ,v̂k+j+1|k }j=0
Garcia-Gabin et al. (2008) and Eren-Oruklu et al. (2009)
s.t. (22) (23b) use a time-varying glucose reference signal to robustify the
controller and reduce the risk of hypoglycemic events. In
umin ≤ ûk+j|k ≤ umax (23c) this paper, we use an asymmetric time-varying glucose
ŷk+j+1|k ≥ ymin − v̂k+j+1|k (23d) reference signal. The idea of the asymmetric reference
v̂k+j+1|k ≥ 0 (23e) signal is to induce safe insulin injections in hyperglycemic
with the objective function φ defined as periods and fast recovery in hypoglycemic and below
target periods.
N −1
1 X
φ= kŷk+j+1|k − r̂k+j+1|k k22 The asymmetric time-varying setpoint is given by
2 j=0 (24)
yk exp −tj /τr+
yk ≥ 0
r̂k+j|k (t) = (26)
+ λk∆ûk+j|k k22 + κkv̂k+j+1|k k22 yk exp −tj /τr−
yk < 0
N is the control and prediction horizon. We choose a Since we want to avoid hypoglycemia, we make the con-
prediction horizon equivalent to 10 hours, such that the troller react more aggressively if the blood glucose level is
insulin profile of the finite horizon optimal control prob- below 6 mmol/L, so we choose τr− = 15 min and τr+ = 90
lem (23) is similar to the insulin profile of the infinite min.
horizon optimal control problem, (23) with N → ∞.
kŷk+j+1|k − r̂k+j+1|k k22 penalizes glucose deviation from 6. COMPARISON BETWEEN ARIMAX, ARMAX
the time-varying glucose setpoint and aims to drive the AND ADAPTIVE ARMAX MODEL STRUCTURES
glucose concentration to 6 mmol/L. λk∆uk+j|k k22 is a
regularization term that prevents the insulin infusion rate In this section we compare three different versions of our
from varying too aggressively. For the simulations, we set Model Predictive Controller on a cohort of 100 virtual
234
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014
patients. These three versions are the ARIMAX formu- Fig. 4. Control Variability Grid Analysis (CVGA) plot for
lation presented in Section 4.1, the ARMAX formulation the three different stochastic model structures. 50%
presented in Section 4.2 and the adaptive ARMAX model of the meal bolus is administered at mealtime. Black:
formulation presented in Section 4.3. We compare the ARIMAX. Red: ARMAX. White: Adaptive ARMAX.
performance of the controllers for the case where the meal
is underbolused and the case where the insulin sensitivity 22.2
is increased by 30% during the night. The change in insulin
sensitivity is simulated by a step change in the insulin
235
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014
Table 3. Evaluation of the controller for the R. Hovorka, F. Shojaee-Moradie, P. V. Caroll, L. J. Chas-
different control strategies in the case where sin, I. J. Gowrie, N. C. Jackson, R. S. Tudor, A. M.
the insulin sensitivity is increased by 30% Umpleby, and R. H. Jones. Partitioning glucose distri-
during the night. The numbers show the total bution/transport, disposal, and endogenous production
percentage of time spent in different glucose during IVGTT. American Journal of Physiology, 282:
ranges for the 100 virtual patients during the 992–1007, 2002.
period 22:00 - 08:00. R. Hovorka, V. Canonico, L. J. Chassin, U. Haueter,
M. Massi-Benedetti, M. O. Federici, T. R. Pieber, H. C.
Glucose (mmol/L) ARIMAX ARMAX Adaptive ARMAX
Schaller, L. Schaupp, T. Vering, and M. E. Wilinska.
G > 10 <0.1 <0.1 <0.1
Nonlinear model predictive control of glucose concen-
G>8 3.2 2.5 2.2
3.9 ≤ G ≤ 10 99.1 99.4 99.7 tration in subjects with type 1 diabetes. Physiological
3.9 ≤ G ≤ 8 95.9 96.9 97.5 Measurement, 25:905–920, 2004.
G < 3.9 0.9 0.6 0.3 R. Hovorka, J. M. Allen, D. Elleri, L. J. Chassin, J. Harris,
G < 3.5 0.2 0 0 D. Xing, C. Kollman, T. Hovorka, A. M. F. Larsen,
M. Nodale, A. De Palma, M. E. Wilinska, C. L. Acerini,
the adaptive ARMAX formulations presented in this paper and D. B. Dunger. Manual closed-loop insulin delivery
have the potential to improve the controller performance, in children and adolescents with type 1 diabetes: a phase
but the method would need a further investigation before 2 randomised crossover trial. Lancet, 375:743 – 751,
being tested on real patients. 2010.
J. K. Huusom, N. K. Poulsen, S. B. Jørgensen, and J. B.
REFERENCES Jørgensen. Tuning SISO offset-free model predictive
control based on ARX models. Journal of Process
R. N. Bergman, L. S. Phillips, and C. Cobelli. Physiologic Control, 22(10):1997–2007, 2012.
evaluation of factors controlling glucose tolerance in J. B. Jørgensen, J. K. Huusom, and J. B. Rawlings.
man: measurement of insulin sensitivity and beta-cell Finite horizon MPC for systems in innovation form.
glucose sensitivity from the response to intravenous In 50th IEEE Conference on Decision and Control and
glucose. Journal of Clinical Investigation, 68(6):1456 European Control Conference (CDC-ECC 2011), pages
– 1467, 1981. 1896 – 1903, 2011.
D. Boiroux, D. A. Finan, J. B. Jørgensen, N. K. Poulsen, H. Kirchsteiger, G. C. Estrada, S. Pölzer, E. Renard, and
and H. Madsen. Strategies for glucose control in people L. del Re. Estimating interval process models for type
with type 1 diabetes. In Proceedings of the 18th World 1 diabetes for robust control design. In Preprints of the
Congress, The International Federation of Automatic 18th IFAC World Congress, pages 11761 – 11766, 2011.
Control, pages 3765–3770, 2011. M. W. Percival, W. C. Bevier, Y. Wang, E. Dassau,
D. Boiroux, A. K. Duun-Henriksen, S. Schmidt, H Zisser, L. Jovanovič, and F. J. Doyle III. Modeling
K. Nørgaard, S. Madsbad, O. Skyggebjerg, P. R. Jensen, the effects of subcutaneous insulin administration and
N. K. Poulsen, H. Madsen, and J. B. Jørgensen. carbohydrate consumption on blood glucose. Journal of
Overnight control of blood glucose in people with type Diabetes Science and Technology, 4(5):1214–1228, 2010.
1 diabetes. In 8th IFAC Symposium on Biological and G. Pillonetto, G. Sparacino, and C. Cobelli. Numerical
Medical Systems, Budapest, Hungary, 2012. BMS 2012. non-identifiability regions of the minimal model of glu-
M. Breton and B. Kovatchev. Analysis, modeling, and cose kinetics: superiority of bayesian estimation. Math-
simulation of the accuracy of continuous glucose sensors. ematical Biosciences, 184:53 – 67, 2003.
Journal of Diabetes Science and Technology, 2:853–862, S. Schmidt, D. Boiroux, A. K. Duun-Henriksen,
2008. L. Frøssing, O. Skyggebjerg, J. B. Jørgensen, N. K.
C. Cobelli, C. Dalla Man, G. Sparacino, L. Magni, Poulsen, H. Madsen, S. Madsbad, and K. Nørgaard.
G. De Nicolao, and B. P. Kovatchev. Diabetes: Mod- Model-based closed-loop glucose control in type 1 di-
els, signals, and control. IEEE Reviews in Biomedical abetes: The DiaCon experience. Journal of Diabetes
Engineering, 2:54–96, 2009. Science and Technology, 7(5):1255–1264, 2013.
A. K. Duun-Henriksen, D. Boiroux, S. Schmidt, K. van Heusden, E. Dassau, H. C. Zisser, D. E. Seborg,
K. Nørgaard, S. Madsbad, O. Skyggebjerg, P. R. Jensen, and F. J. Doyle III. Control-relevant models for glucose
N. K. Poulsen, J. B. Jørgensen, and H. Madsen. Tuning control using a priori patient characteristics. IEEE
of controller for type 1 diabetes treatment with stochas- transactions on biomedical engineering, 59(7):1839 –
tic differential equations. In 8th IFAC Symposium on 1849, 2012.
Biological and Medical Systems, Budapest, Hungary, M. E. Wilinska, L. J. Chassin, C. L. Acerini, J. M. Allen,
2012. BMS 2012. D. B. Dunger, and R. Hovorka. Simulation environment
M. Eren-Oruklu, A. Cinar, L. Quinn, and D. Smith. to evaluate closed-loop insulin delivery systems in type
Adaptive control strategy for regulation of blood glucose 1 diabetes. Journal of Diabetes Science and Technology,
levels in patients with type 1 diabetes. Journal of 4(1):132 – 144, 2010.
Process Control, 19:1333 – 1346, 2009.
W. Garcia-Gabin, J. Vehı́, J. Bondia, C. Tarı́n, and
R. Calm. Robust sliding mode closed-loop glucose con-
trol with meal compensation in type 1 diabetes mellitus.
In Proceedings of the 17th World Congress, The Inter-
national Federation of Automatic Control, pages 4240 –
4245, 2008.
236