Data-driven ODE modeling of the high-frequency complex dynamics via a low-frequency dynamics

model
Natsuki Tsutsumi
Faculty of Commerce and Management, Hitotsubashi University, Tokyo 186-8601, Japan

Kengo Nakai
The Graduate School of Environment, Life, Natural Science and Technology, Okayama University, Okayama 700-0082, Japan

Yoshitaka Saiki
Graduate School of Business Administration, Hitotsubashi University, Tokyo 186-8601, Japan
arXiv:2409.00668v2 [nlin.CD] 10 Nov 2024

(Dated: November 12, 2024)

In our previous paper [N. Tsutsumi, K. Nakai and Y. Saiki, Chaos 32, 091101 (2022)], we proposed a method
for constructing a system of differential equations of chaotic behavior from only observable deterministic time
series, which we call the radial function-based regression (RfR) method. However, when the targeted variable’s
behavior is rather complex, the direct application of the RfR method does not function well. In this study, we
propose a novel method of modeling such dynamics, including the high-frequency intermittent behavior of a
fluid flow, by considering another variable (base variable) showing relatively simple, less intermittent behavior.
We construct an autonomous joint model composed of two parts: the first is an autonomous system of a base
variable, and the other concerns the targeted variable being affected by a term involving the base variable to
demonstrate complex dynamics. The constructed joint model succeeded in not only inferring a short trajectory
but also reconstructing chaotic sets and statistical properties obtained from a long trajectory such as the density
distributions of the actual dynamics.

Introduction. We are often eager to identify a dynami- estimated a system of ODEs from the time series of all vari-
cal system model that generates observable time series data. ables of the background system. Other investigations [21–23]
Many systems that describe physical phenomena have been have derived a system of ODEs using an observable variable
found since the finding of the equation of motion. In decades, and its time derivatives as the model variables. In these stud-
various methods have been proposed that estimate a dy- ies, prior knowledge of the background dynamics was used
namical system describing observed data combining physi- to choose basis functions for the regression; however, this
cal knowledge and machine learning techniques. Chorin et method is not appropriate for some practical purposes 1 .
al [1] proposed a method to describe unknown partial dynam- Recently, we proposed a simple method for constructing
ics that are not described by equations of physical systems. a system of ODEs for chaotic behavior based on regres-
Physics-informed machine learning methods [2–4] construct sion using only observable deterministic scalar time series
a model by supposing physical conditions that a system gen- data [24, 25], referred to as the radial function-based regres-
erating time series is expected to have. Sometimes, we do not sion (RfR) method. The RfR method constructs a model de-
have any physical knowledge of a system that generates time scribing the dynamics of a time series in a space made only
series. Several approaches have been proposed concerning from understandable variables without background knowl-
modeling dynamics using machine learning from given time edge. For the regression, we employ the delay coordinate
series data without physical knowledge. The dynamic mode of the observable variable and spatially localized radial basis
decomposition [5] is a method to derive macroscopic features functions in addition to polynomial basis functions. The in-
and estimate a dynamical system of the variables. Reservoir troduction of the localized functions leads to higher computa-
computing [6–9] is a recurrent neural network with low com- tional costs than that of some other methods such as reservoir
putational cost. These methods usually model a continuous computing. The method’s forecasting capability is not limited
dynamical system as discrete-time, not differential equations. to a short time series but also to a density distribution created
Note that a model constructed using reservoir computing is using a long time series. We can generate a long trajectory
commonly a discrete-time system, but there are some types of of the model using the Stagger-and-Step method [26] when
reservoir computing using differential equations to describe the appropriate model trajectory is realized on a chaotic set of
the dynamics of the reservoir states [12]. the model that is not an attractor. This RfR method functions
In some studies, a system of ordinary differential equations even when observable variables are obtained from an infinite-
(ODEs) is constructed to describe the dynamics of determin- dimensional system such as partial differential equations and
istic time series. Neural ODE [13] recently attracted much at- delay differential equations [25].
tention for using a neural network to model ODE. The method Regardless of the modeling method, some types of complex
based on the theory of the Koopman operator [14–16] projects time series governed by dynamics with multiscale structures
the phase space to an infinite dimensional space to construct
a model as a linear system. There are studies to make a sim-
1
ple and understandable model of ODEs in a finite dimensional In some situations, prior knowledge of the background dynamics can be
space without a neural network. Several studies [17–20] have obtained and the modeling with the knowledge is useful [10, 11].
 2

and/or high Lyapunov dimensions are difficult to model. In a system of ODEs, which is referred to as the fiber model.
the case of a fluid flow, direct modeling of high-frequency In this section, we briefly review the RfR method for con-
variable dynamics is not practically easy, even if the delay structing the base model and explain the method used for the
coordinate of the observable variable is used. To model high- targeted variable in detail. Finally, a method to generate a nu-
frequency variable dynamics, a method using a low-frequency merical trajectory from the estimated system is explained. To
variable is effective [7]. In this study, we first model the dy- describe the dynamics of a targeted observation y(t), we con-
namics of a low-frequency variable using only the time se- struct a joint model composed of base and fiber models.
ries data of the variable and its delay coordinate. Next, we Base model. The first part of the joint model describes
construct a model that predicts the dynamics of the high- the dynamics of the base variable x. It is constructed from
frequency variable from that of the low-frequency variable. scalar time series data of x(t) using the RfR method [24].
The set of the constructed models succeeds in describing the In the RfR method, the delay coordinate is employed to de-
dynamics of the high-frequency variable. This approach in- scribe chaotic behavior using only a scalar observable time
volving modeling via other variables is effective for dealing series. We estimate a D-dimensional system of ODEs of the
with complex dynamics. Abarbanel et al. [27] introduced a variable X = (X1 , X2 , . . . , XD ), which almost satisfies the
method to construct a function describing the targeted vari- delay structures among components of the variable:
ables depending on the delay coordinate of another variable.
By applying the proposed method and adopting spatially lo- X1 (t) ≈ X2 (t + τ ) ≈ · · · ≈ XD (t + (D − 1)τ ), (1)
calized functions, we construct ODEs that describe the tar-
geted variable using mainly the delay coordinate of a different where X1 is considered to describe the dynamics of x and τ
variable. is the delay time. See [24] for details concerning the choice
of τ . The model
We introduce a variant of the RfR method, which first mod-
els the simple coherent dynamics of a certain variable using dX
the RfR method, and this system is then used to model the = F (X), (2)
dt
complex and sometimes intermittent dynamics of a targeted
variable. The proposed method is referred to as the joint is constructed, where F (X) is formulated by
RfR method. This method enables the construction of a sys- X X
tem of ODEs describing the complex time series using only F (X) = β 0 + β d Xd + β D+j φj (X), (3)
physically understandable variables, and it dramatically en- d=1,··· ,D j=1,··· ,J
larges the capabilities of methods for constructing data-driven  
−||X−cj ||2
ODEs. We describe the joint RfR method and demonstrate where φj (X) = exp σ2 , and the paramters cj (j =
its validity by applying it to two examples. The first involves 1, · · · , J) are distributed as lattice points and σ is the standard
the time series of the Rössler equation, which is a well-known deviation of the Gaussian distribution. The model coefficients
chaotic system. The second concerns the time series of the βi (i = 1, · · · , D+J) are determined using the time derivative
energy function of a chaotic fluid flow. The proposed method at each sample point estimated by the Taylor approximation.
succeeds in describing the connection between the two types See [24] for more details concerning this process.
of time series of a single dynamical system, although their It should be noted that if we can observe a time series of the
behaviors are quite different. base variable, the base modeling process is not required, and
Modeling Method. We aim to achieve a data-driven con- a fiber model may be directly constructed to predict the time
struction of a system of ODEs for describing the relatively series of the targeted variable.
complex deterministic dynamics of a targeted variable. For Fiber model. With the knowledge of the variable X ob-
this purpose, we assume that we can observe a time series of tained from the base model, we next construct a model de-
two variables showing simple and relatively complex dynam- scribing the dynamics of the targeted variable y. For a given
ics. Because directly modeling the targeted variable’s dynam- delay time τ̂ (> 0), we construct a system of two-dimensional
ics is difficult, we first construct a relatively simple dynamics ODEs of the variable Y = (Y1 , Y2 ), which is expected to sat-
model and use it to model the targeted variable’s dynamics as isfy the delay structures among components of the variable:
the skew-product-type ODE, as described in figure 1. When Y1 (t) ≈ Y2 (t + τ̂ ). The relationship will be used for gener-
the dimension of the delay coordinate for a variable show- ating an appropriate numerical trajectory. Using the variable
ing relatively moderate dynamics is sufficiently high, invari- X of the base model, we form the ODEs describing Y as
ant sets of the original system are reconstructed in the base follows:
model [28]. The relatively complex dynamics of the origi-
nal system are almost completely reconstructed by the base dY
= aY1 + G(X), (4)
model. The fiber model picks up the dynamics of the targeted dt
variable from the base model. The skew-product-type ODE X X
is expected to demonstrate relatively complex dynamics. We where G(X) = γ 0 + γ d Xd + γ D+j φj (X),
use the RfR method [24] to estimate the base model describ- d=1,...,D j=1,...,J
ing the relatively simple variable x(t) (i.e., the base variable), and γ i and a is the model coefficients and φj is the same as
and the new method is applied to modeling the relatively com- that in Eq. (3), which means that cj of φj are distributed as
plex dynamics of a variable y(t) (i.e., the targeted variable) as lattice points for X. The model coefficients γ i and a are de-
 3

FIG. 1. Outline of the joint RfR method for constructing a joint model composed of the base and fiber models. The proposed method
constructs a system of ODEs that infer the time series of two variables x(t) and y(t) from only their observable time series data, where
x(t) = g1 (p) and y(t) = g2 (p) for some functions g1 and g2 , and p is determined by some unknown dynamical system dp dt
= f (p). Using
the time series of x(t) that shows relatively simple dynamics, we construct a “base model” using a system of ODEs of a variable X composed
of x and its time delay variables employing the RfR method [24, 25]. The relatively complex behavior of the y variable is described as fiber
dynamics using a system of skew-product-type ODEs of a variable Y composed of y, its time delay variables, and the variable X of the “base
model.” The constructed model for the fiber dynamics is called the “fiber model”. Here the first component of the variables X and Y are X1
and Y1 , respectively.

termined via the time series data using ridge regression. This on a chaotic saddle of the model constructed using the RfR
formulation and the introduction of the linear term Y1 into (4) method employing the Stagger-and-Step method [26]. The ap-
are important for modeling complex dynamics with restricted propriateness of a trajectory is determined by the delay struc-
computational resources. In the above formulation, we can ture (1) along a trajectory. To generate a long trajectory of the
avoid distributing center points in the delay coordinate of the base model, we employ the Stagger-and-Step method when
targeted variable. See Supplemental Material 1 for the ex- the chaotic invariant set of the model is a saddle. We also ap-
planation of the linear term. When we estimate the model ply the idea of this Stagger-and-Step method to the given fiber
coefficients, we use the time series of the delay coordinate model (4).
(y(t), y(t − τ̂ )) as that of Y (t). See [24] for more details con- To generate a trajectory from the estimated fiber model (4),
cerning this estimation. The appropriate delay time depends The time series of the variable of the base model X(t) is
on the frequency of the time series, and the delay time τ for required. When the time series X(t) is given, we can in-
the base model and the delay time τ̂ for the fiber model need terpret the fiber model (4) as a linear system of Y with a
not be the same. In our examples, τ̂ should be chosen to be time depending external force G(X(t)). Using the descrip-
smaller than τ . tion of a solution of a linear system, the solutions of the
Generation of a model trajectory. Here, we introduce a fiber model (4) can be analytically described. The appro-
method for estimating a numerical trajectory without detailed priateness of a model trajectory is quantified by the delay
information for the variable y. In some data-driven models, an structure ∆(t) := Y1 (t) − Y2 (t + τ̂ ), which is written as
appropriate trajectory is not on its chaotic attractor but on its ∆(t) = C(t)Y1 (0) − Y2 (0) + M (t), where C(t) and M (t)
chaotic saddle. A chaotic saddle is a chaotic invariant set that do not depend on Y1 (0) and Y2 (0). Note that this form is a
does not attract some neighborhoods. The bias and the short- linear equation for Y1 (0) and Y2 (0). The appropriate trajec-
age of learning data can easily cause the situation. Even when tory of the fiber model (4) should satisfy ∆(t) ≈ 0 for any t.
a model trajectory is realized on a chaotic saddle, a short- Suppose ∆(0) = 0 and ∆(t∗ ) = 0 for some given t∗ > 0,
term numerical trajectory via the forward integration of the then Y1 (0) and Y2 (0) can be obtained only from the ODEs
model mimics that of the actual well, but a long-term trajec- of Eq. (4) (see Supplemental Material 2 for more detail). By
tory diverges. In [25], instead of using the simple forward applying this approach, we can estimate a model trajectory
integration, we generate an appropriate numerical trajectory corresponding to the targeted observation y(t).
 4

D τ J τ̂ t∗
Rössler 3 0.60 7,716 0.40 0.60
Fluid flow 7 2.00 832,291 1.00 1.50

TABLE I. Sets of parameters. These parameters are used for mod-

eling the dynamics of the Rössler equation and fluid flow. a
a We set (δgrid , n, T, ∆t, m, p) = (0.25, 5 · 104 , 105 , 0.10, 3, 0.1) in the
case of the Rössler equation, and
(δgrid , n, T, ∆t, m, p) = (0.40, 5 · 104 , 5 · 104 , 0.05, 3, 0.1) in the
case of the fluid flow, where the parameter δgrid is the grid size used to
determine the Gaussian radial basis function, n is the number of
regression data points, T is the time length of the time series of the base
and targeted variable, and ∆t is the time step of the time series data. The
parameter σ of φj is determined from m and p (see [25] for more details).
FIG. 2. Short time series for the Rössler dynamics. The estimated
time series of the targeted variable z (Y1 of the fiber model) (red) is
The settings of the parameters for the example dynamics shown together with that of the actual time series (blue). The time
are given in Table I. The following results can be obtained series including noise used for the estimation is also plotted (yellow).
robustly with different sets of parameters. We added Gaussian noise, with a standard deviation of 1% of the raw
Example 1: Rössler equation. We model a system of dif- data, to the time series of the z variable.
ferential equations using the time series of the Rössler equa-
dy
tion [29]: dx dz
dt = −y −z, dt = x+0.2y, dt = 0.2+xz −5.7z.
It is known that a z variable behaves less coherently than x and
y variables. Therefore, the direct modeling of z dynamics is
more difficult than that of x or y dynamics. As an example,
we first model the dynamics of a variable x by considering it
to be the base variable and using it to model the z dynamics.
We assume that we can observe discrete time series of x
and z variables with Gaussian noise whose standard deviation
is 1% of the standard deviation for each variable. See Fig. 2
for the time series of the z variable with noise used for the
modeling together with that without noise. Here, the size of
the noise is large enough to hide small amplitude fluctuations.
We construct a base model (2) for X := (X1 , X2 , X3 ) us-
ing the time series of the x variable and a fiber model (4) for FIG. 3. Projection of a chaotic set for the Rössler dynamics. The
Y := (Y1 , Y2 ) using the time series of z together with the dy- projection of a long trajectory generated from the original system
namics of the base model. Refer to Fig. 2 for the short time onto the (x, z) plane (actual) is shown in the left panel and that of
series of the Y1 variable obtained from the joint model, the the model (model) in the right panel. The two trajectories exhibit a
similar shape.
skew-product-type model that joins the fiber and base mod-
els. It is shown that the joint model well predicts the short
time series. Figure 3 shows a projection of long trajectory
points to a (X1 , Y1 )-plane. The results imply the successful to preserve the energy of the low-frequency part. The settings
reconstruction of the connection between the base and tar- of the numerical simulation are the same as those used by [7]
geted variables. Note that the positive Lyapunov exponent of with a viscosity parameter of 0.0585. We exemplify a time se-
the base model (λ1 = 0.07124) agree with that of the original ries of energy functions Ek (t) for wavenumber k ∈ N, which
P P3
Rössler equation (λ1 = 0.07123), and the second Lyapunov is defined by Ek (t) := 12 Dk ζ=1 |F[vζ ] (κ, t)|2 , where
exponents λ2 of both systems are neutral. The negative Lya- Dk := {κ ∈ Z3 |k − 0.5 ≤ |κ| < k + 0.5}, vi is the ith
punov exponent of the model is −0.27506, and that of the component of the velocity, and F[vi ] is the Fourier coefficient
original is −5.21974. This discrepancy of the negative Lya- of the variable vi . We assume that the time series of the E1 (t)
punov exponents is commonly observed in data-driven mod- and E10 (t) variables is known. In this section, we construct a
elings [24, 30]. base model (2) for X := (X1 , . . . , X7 ) using the time series
Example 2: Fluid system. Next, we model the complex of the E1 (t) variable and a fiber model (4) for Y := (Y1 , Y2 )
dynamics of a high-frequency fluid variable using a model using the time series of the E10 (t) variable. Note that the tur-
of the simpler dynamics of a low-frequency variable. It is bulent dynamics of the high-wavenumber variable E10 (t) is
known that the high-frequency dynamics of fluid flow show much more complex than that of the low-wavenumber vari-
intermittency and complex behavior. To obtain time series able E1 (t). We succeeded in achieving the short-term predic-
data for training, we consider the direct numerical simulation tion of the E1 (t) variable using the base model (Fig. 4 (top))
of the incompressible three-dimensional Navier–Stokes equa- and E10 (t) variable using the joint model (Fig. 4 (bottom)).
tion under periodic boundary conditions, and the forcing is Although the long time series prediction failed because of the
input into the low-frequency variables at each time step so as chaotic property, the density distribution of E10 can be repro-
 5

Lyapunov exponents using a governing equation, because a

closed-form equation for the energy variable cannot be de-
rived from the Navier–Stokes equations. Since the attractor
dimension is high, it is hard to estimate Lyapunov exponents
directly from time series [31].

FIG. 4. Short time series of two different variables for the

macroscopic behavior of the fluid dynamics. The relatively
moderate estimated time series of the base variable E1 (X1 of the
base model) (red) is shown in the upper panel. The relatively com-
plex estimated time series of the targeted variable E10 (Y1 of the fiber
model) (red) is shown in the lower panel. The actual trajectories are FIG. 6. Projection of a chaotic set for the macroscopic behavior
plotted together with the estimations, which are shown by the blue of the fluid dynamics. The projection of a long trajectory generated
lines. from the original system onto the (E1 , E10 ) coordinate (actual) is
shown in the left panel and that of the model (model) is shown in the
right panel. These two systems exhibit similar shapes. Note that the
original system has an attractor, whereas the corresponding chaotic
set in the model is a saddle.

Concluding remarks. The proposed method of constructing

a system of ODEs from an observable time series is useful
for modeling complex and multiscale dynamics in time and
space physically only using understandable variables without
background knowledge. The computational cost to model be-
haviors of two or more variables with different time scales
is high. In such a situation, the effective strategy that we
employ is to model moderate dynamics and use it to model
complex dynamics through a fiber model. The computational
FIG. 5. Density distribution of a targeted variable Y1 for the cost to model moderate dynamics with the delay coordinate
macroscopic behavior of the fluid dynamics. The density distri- is lower, and describing the major part of the complex dy-
bution created from a long model trajectory (T = 50, 000) of Y1 namics with the variable of the base model reduces the re-
(estimation of E10 ) (red) is shown together with the actual trajec- quired computational resource. This skew-product-type mod-
tory (blue). The distributions exhibit similar shapes. eling succeeds because the invariant sets of the original system
are reconstructed in the base model when the dimension of the
delay coordinate is sufficiently high. The developed method
duced from the time series of the model (Fig. 5). These results will be used to model multiscale dynamics, such as those of
imply that the constructed joint model can well describe the weather, beginning with the construction of a base model of
dynamics of E10 . low-frequency dynamics.
The joint model reconstructs the connection between E1 Acknowledgement. YS was supported by JSPS KAK-
and E10 (Fig. 6). This imitation of the connection between ENHI Grant Nos. 19KK0067, 21K18584, 23H04465, and
the two energy variables contributes to the success of describ- 24K00537. KN was supported by JSPS KAKENHI Grant
ing the high-frequency variable, which is difficult to model. No.22K17965. The computations were carried out using the
The positive Lyapunov exponents of the base model are λ1 = JHPCN (jh230028 and jh240051) and the Collaborative Re-
0.2157, λ2 = 0.0718, λ3 = 0.0200, although we do not have search Program for Young · Women Scientists of ACCMS and
the actual exponents for comparison. We cannot obtain the IIMC, Kyoto University.

[1] A. J. Chorin and F. Lu, Proceedings of the National Academy S. Wang, and L. Yang, Nature Reviews Physics 3, 422 (2021).
of Sciences 112, 9804 (2015). [3] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Journal of Com-
[2] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, putational physics 378, 686 (2019).
 6

[4] S. Greydanus, M. Dzamba, and J. Yosinski, Advances in neural

information processing systems 32 (2019).
[5] P. J. Schmid, Journal of fluid mechanics 656, 5 (2010).
[6] J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, and E. Ott, Chaos 27,
121102 (2017).
[7] K. Nakai and Y. Saiki, Phys. Rev. E 98, 023111 (2018).
[8] K. Nakai and Y. Saiki, Discrete Contin. Dyn. Syst. S 14, 1079
(2021).
[9] M. U. Kobayashi, K. Nakai, Y. Saiki, and N. Tsutsumi, Phys.
Rev. E 104, 044215 (2021).
[10] J. Pathak, A. Wikner, R. Fussell, S. Chandra, B. R. Hunt,
M. Girvan, and E. Ott, Chaos: An Interdisciplinary Journal of
Nonlinear Science 28 (2018).
[11] A. Wikner, J. Pathak, B. Hunt, M. Girvan, T. Arcomano,
I. Szunyogh, A. Pomerance, and E. Ott, Chaos: An Interdis-
ciplinary Journal of Nonlinear Science 30 (2020).
[12] Z. Lu, B. R. Hunt, and E. Ott, Chaos 28, 061104 (2018).
[13] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud,
Advances in Neural Information Processing Systems 31, 6571
(2018).
[14] Y. T. Lin, Y. Tian, D. Livescu, and M. Anghel, SIAM Journal
on Applied Dynamical Systems 20, 2558 (2021).
[15] I. Mezić, Annual review of fluid mechanics 45, 357 (2013).
[16] T. Berry, D. Giannakis, and J. Harlim,
Phys. Rev. E 91, 032915 (2015).
[17] E. Baake, M. Baake, H. G. Bock, and K. M. Briggs, Phys. Rev.
A 45, 5524 (1992).
[18] W.-X. Wang, R. Yang, Y.-C. Lai, V. Kovanis, and C. Grebogi,
Phys. Rev. Lett. 106, 154101 (2011).
[19] S. L. Brunton, J. L. Proctor, and J. N. Kutz, Proc. Natl. Acad.
Sci. 113, 3932 (2016).
[20] K. Champion, B. Lusch, J. N. Kutz, and S. L. Brunton, Proc.
Natl. Acad. Sci. 116, 22445 (2019).
[21] G. Gouesbet, Physical Review A 43, 5321 (1991).
[22] G. Gouesbet and J. Maquet, Physica D: Nonlinear Phenomena
58, 202 (1992).
[23] G. Gouesbet, L. Le Sceller, C. Letellier, R. Brown, J. Buckler,
and Z. Kollath, Annals of the New York Academy of Sciences
808, 25 (1997).
[24] N. Tsutsumi, K. Nakai, and Y. Saiki, Chaos: An Interdiscip. J.
Nonlinear Sci. 32, 091101 (2022).
[25] N. Tsutsumi, K. Nakai, and Y. Saiki, Phys. Rev. E 108, 054220
(2023).
[26] D. Sweet, H. E. Nusse, and J. A. Yorke, Phys. Rev. Lett. 86,
2261 (2001).
[27] H. D. Abarbanel, T. Carroll, L. M. Pecora, J. J. Sidorowich, and
L. S. Tsimring, Physical Review E 49, 1840 (1994).
[28] F. Takens (Springer Berlin Heidelberg, Berlin, Heidelberg,
1981) pp. 366–381.
[29] O. E. Rössler, Physics Letters A 57, 397 (1976).
[30] J. D. Hart, Chaos: An Interdisciplinary Journal of Nonlinear
Science 34 (2024).
[31] A. Wolf, J. B. Swift, H. L. Swinney, and J. A. Vastano, Physica
D: nonlinear phenomena 16, 285 (1985).
 7

SUPPLEMENTAL MATERIAL 1: NECESSITY OF THE model (4) can be described as follows:

LINEAR TERM
dY
= aY1 + G(t),
dt
In this section, we explain the reasons why the linear term
of Y1 in the fiber model (4) is necessary to construct a model which is a linear system of Y . The solution of this linear
practically. When the dimension of the base model D is high, system can be described as
the dynamics of the targeted variable may be described with-
out a linear term as follows: Y1 (t) = Y1 (0) ea1 t + ea1 t K(t),
a 2 a1 t a2 1
e − 1 Y1 (0) + ea1 t K(t) + L(t),


dy Y2 (t) = Y2 (0) +
= Ĝ(X), (5) a1 a1 a1
dt
where
where y(t) := (y(t), y(t − τ̂ )) and y(t) is the targeted vari-
able. To construct this system using numerical data, the con-
Z t

structed model includes the estimation and numerical errors, K(t) = e−a1 s G1 (s)ds,
0
which means that the estimated model can be described as Z t

dY L(t) = a1 G2 (s) − a2 G1 (s)ds.

= Ĝ(X) + δ(X). (6) 0
dt The appropriateness of a model trajectory is quantified by the
By using an adequate modeling method and sufficient data, delay structure ∆(t) := Y1 (t) − Y2 (t + τ̂ ), which is written
δ(X) is determined to be small. A trajectory of the model (6) as follows using the above forms:
can be described as
∆(t) = C(t)Y1 (0) − Y2 (0) + M (t),
Z t
Y (t) = y(t) + δ(X(s))ds. where
0  
a2 a2
This implies that a long-generated trajectory includes accu- C(t) = 1 − ea1 τ̂ ea1 t + ,
a1 a1
mulated errors, and the effect of these errors gradually in-  
creases. This deviation is caused by the independence of the a 2 1
M (t) = ea1 t K(t) − ea1 τ̂ K(t + τ̂ ) − L(t + τ̂ ).
right-hand side of the fiber model (6) from Y . In the joint RfR a1 a1
method, the linear term of Y1 is employed to avoid this issue.
Note that this form is a linear equation for Y1 (0) and Y2 (0).
The appropriate trajectory of the fiber model (4) should satisfy
∆(t) ≈ 0 for any t. Suppose ∆(0) = 0 and ∆(t∗ ) = 0 for
some given t∗ > 0; Y1 (0) and Y2 (0) can then be obtained
using only the fiber model (4) as follows:
SUPPLEMENTAL MATERIAL 2: GENERATION OF A
MODEL TRAJECTORY FROM THE FIBER MODEL M (t∗ ) − M (0)
Y1 (0) = − ,
C(t∗ ) − C(0)
Here, we explain the method used to generate a model tra- C(t∗ )M (0) − C(0)M (t∗ )
jectory of the fiber model in detail. To simplify the descrip- Y2 (0) = .
C(t∗ ) − C(0)
tions, we denote G(X(t)) of the fiber model (4) as G(t)
and the i-th component of the vector G(t) and a of the fiber By applying this approach, we can estimate a model trajectory
model (4) as Gi (t) and ai , respectively. The estimated fiber corresponding to the targeted observation y(t).