PHYSICAL REVIEW E VOLUME 49, NUMBER 3 MARCH 1994

Predicting physical variables in time-delay embedding

Henry D. I. Abarbanel*
Department of Physics and Marine Physical Laboratory, Scripps Institution of Oceanography, University of California, San Diego,
Mail Code 0402, La Jolla, California 92093-0402

T. A. Carroll and L. M. Pecora

U. S. Naual Research Laboratory, Code 6341, Washington, D. C. 20375-5000

J. J. Sidorowich and L. S. Tsirnring

Institute for Nonlinear Science, University of California, San Diego, Mail Code 0402, La Jolla, California 92093-0402
(Received 30 September 1993)
In time-delay reconstruction of chaotic attractors we can accurately predict the short-term future
behavior of the observed variable x (t) =x (n) =x (to+7., n) without prior knowledge of the equations of
motion by building local or global models in the state space. In many cases we also want to predict vari-
ables other than the one which is observed and require methods for determining models to predict these
variables in the same space. We present a method which takes measurements of two variables x (n } and
z(n) and builds models for the determination of z(n) in the phase-space made out of the x(n) and its
time lags. Similarly we show that one may produce models for x (n) in the z(n) space, except where spe-
cial symmetries prevent this, such as in the familiar Lorenz model. Our algorithm involves building lo-
cal polynomial models in the reconstructed phase space of the observed variable of low order (linear or
quadratic) which approximate the function z(n) =E(x(n)) where x(n) is a vector constructed from a se-
quence of values of observed variables in a time delay fashion. We train the models on a partial data set
of measured values of both x (n) and z(n) and then predict the z(n) in a recovery set of observations of
x (n) alone. In a11 of our analyses we assume that the observed data alone are available to us and that we
possess no knowledge of the dynamical equations. We test this method on the numerically generated
data set from the Lorenz model and also on a number of experimental data sets from electronic circuits.

PACS number(s): 05.45. + b, 84.40. —x

I. INTRODUCTION are locally dL dimensional [7]. The reason for tnaking

models with di degrees of freedom, but in a dE~dL-
The analysis of data from chaotic systems rests on the dimensional space, is so that the trajectory is unfolded
use of the embedding theorem which allows the recon- from its projection on the observation axis x (n }, and we
struction of the system phase space (or state space) from can unambiguously identify points on the orbit which are
scalar observations. In this method [1 —5] a scalar mea- dynamical neighbors. These models are of the form
surement x(n)=x(to+nr, ) and its time delays y(n +1)=F(y(n) } with F( ) a map from R to itself.
x(n +kT); k =1,2, . . . , d —1 are used to make d In physical settings one is almost always interested in
dimensional vectors additional information beyond the measured variable
—1)T) x (n), though the ability to measure other variables along
y(n)= x(n), x(n+T), . . . , x(n+(d with the measurements of x(n) may often be quite cir-
cumscribed. An example which comes readily to mind is
whose components provide a coordinate system in which
the measurement of the atmospheric pressure at some lo-
one can identify the attractor structure associated with
cation from which one would like to be able to infer the
the observations. The delay Tv, is an integer multiple of
horizontal wind velocity at that location or perhaps at
the sampling time ~, . We choose the integer T by con- another location. Other examples are easy to construct.
siderations of average mutual information [6]. If one has measured only the x (n), then absent any
It is now quite a routine effort to make models of the knowledge of the connection between this variable and
future behavior of the variable x(n) by working within the others there seems no clear way to estimate the
the d-dimensional embedding space of the vectors y(n) behavior of another dynamical variable, call it z(n), of
These models are either global in phase space or consist the system. In typical experimental situations the equa-
of local neighborhood-to-neighborhood maps in the space tions of motion are either not available or not reliable.
R". If one knows that the global space is larger than the Even if given, they usually have a form of some set of
dynamical space of the local dynamics dL, then the maps differential equations, ordinary or partial, which de-
scribes the evolution in a space which is often much
larger than that required to embed the observed data, and
'Also at Institute for Nonlinear Science, University of Califor- therefore is difficult to deal with.
nia, San Diego, La Jolla, CA 92093-0402. In this paper we explore the idea that if one were to

1063-651X/94/49(3)/1840(14)/$06. 00 49 1840 1994 The American Physical Society

49 PREDICTING PHYSICAL VARIABLES IN TINE-DELAY EMBEDDING 184l

measure another dynamical variable or dynamical vari- evolution of z(n } .This is just it would be if we knew the
ables, call them z(n), in addition to the single scalar underlying difFerential equations or map.
x (n), then one could make a model of the connection be- The nonlinear functional reconstruction techniques
tween z(n) and the time-lagged vector y(n), which are which can be used for this purpose are essentially the
the data as seen in the global reconstructed embedding same as those employed for conventional nonlinear model
space. This possibility follows from the natural connec- building [5,8]. They are roughly divided into local and
tion among all the variables in a nonlinear system, and global techniques. Local methods represent a nonlinear
the embedding theorem [2 —4] then suggests that if we function g(y(n)) as a collection of local polynomial (or
have used the proper number of time delays to establish other) maps, different for difFerent neighborhoods in the
coordinates for the attractor, then other variables should phase space [9]. The parameters of the individual local
be dependent on the y(n) in that coordinate system. Thus maps are determined by locally fitting the points in a data
we are led to consider a connection set contained within a small region in the phase space.
More precisely, we use the phase-space neighbors of the
z(n)=g(y(n)), (2)
orbit points y(n) and make maps from the neighborhood
which we can deduce from the data by working in the d- of y(n) to the neighborhoods of y(n +1).The parameters
dimensional space of the y(n). Since we typically do not in the local map are determined by a least-squares fit to
have the differential equations of the system or, more to the evolution of whole neighborhoods of points. There
the point, we may be trying to learn de'erential equations are now well-developed methods for finding neighbors in
from the data, we have no other path to the connection multidimensional phase space [5, 10]. In contrast with
between one physical variable, the z(n), and measure- these local neighborhood-to-neighborhood maps, global
ments of the y(n). methods provide a single nonlinear function for the entire
To establish the connection z(n)=g(y(n)) we must data set [11,12]. Global models are smaller and thus
have some measurements of z(n) along with x(n). So more convenient to deal with; however, local models are
our scheme will be based on the scenario where we initial- usually more accurate.
ly have some measurements of both x (n) and z(n) From. In this paper we use a local polynomial representation
these measurements we establish the phase space using of the function z(n)=g(y(n)). Further, instead of vec-
the x (n), and in that space we determine the function tors built on time delays solely from the future or the
g(y(n)) which gives z(n). With this construction we may past, we take advantage of the knowledge we have of
predict from future measurements of the x(n) alone the both past and future. For d even, d =2D, we use vectors
I

y(n) =(x (n DT), x(n (D —1)T)—

— —
, . . . , x (n), . . . , x(N +(D 1)T}},

while for d odd, d = 2D + 1, we use

y(n)=(x(n DT), x(n —(D —1)T), —
. . . , x (n), . . . , x (n+DT)) . (4)

Although the reconstruction theorem is insensitive to this or quadratic maps. For a linear map we write the rela-
choice, in practice time-symmetric einbedding gives more tion z (n) =g(y(n })in the form
accurate results as compared to a standard backward
time delay embedding in forecasting models. The reason
z(n)=ao(n)+a(n) y(n),
for this is that in the time-symmetric embedding, the loss emphasizing that the local constants depend on the
of correlation between z(n) and the last components of phase-space location. For this local linear model we have
the vector y(n } is reduced. d+1 parameters at each local neighborhood in phase
Denoting each of these vectors as space. We typically would use Ns =2(d + 1) total neigh-
y(n)=(yi(n), y2(n), . . . , yz(n) }, we write the local poly- bors to make the determination of the constants ao and a.
nomial relationship as trade-off

Our algorithm works as follows. We present the pro-

z(n)= g a;, . yi'(n). . . yz (n), (5) gram with two parallel streams of scalar measurements
x(n) and z(n). N pairs of d-dimensional vectors y(n) and
corresponding z(n) are made from the x (n) as just de-
where g~ ii ~M, and M is the maximum order of the scribed. First Nr of the total pairs [y(n), z(n) j are used
polynomial. The number of parameters to be found from for "training" the local models as we describe in a mo-
the data is -d . There is always a betwee the ment. A kd tree [10] for these Nz is now made so the
order of the polynomials and the state space locality for search for nearest neighbors can be done quickly. On the
the maps. Indeed, the larger is M, the more parameters —
remaining Nz =N Nz pairs we wish to check recovery
must be determined by the local fit; hence with a fixed of the value of the z(j );j =NT+1, NT+2, . . . , N from
number of data points in the training set the larger the the data stream of x(j );j =NT+1, NT+2, . . . , N alone.
neighborhoods around the orbit points y(n} must be to To start this recovery for point z(j), we search our kd
accurately establish the coefBcients a; tree to find the Nz nearest neighbors
In fact, for loca/ reconstruction there is no need for y'"'(j);k =1,2, . . . , N~ of y(j) which come froin the first
very high-order polynomials, and we will use only linear WT data. Using these neighbors, a local linear or quadra-
1842 HENRY D. I. ABARBANEL et al.

tic map is constructed by a least-squares fit employing all will show how one can recover the behavior of the vari-
N~ required neighbors to find a map of the form able z(t) from a model constructed in a state space
z (- ) =g(y(. ) ) which minimizes residuals over all N~ spanned by vectors with components made from values of
neighbors from the training data. This means minimize x (t).
We proceed by taking measured values of both x (n)
and z(n), with the sampling time being the time step in-
troduced in the integration of the Lorenz system of mag-
nitude r, =0.02 (see Fig. 1). Using average mutual infor-
at each "time" j.
Nz is taken to be twice the number of mation we easily establish that T=5 is appropriate.
parameters, so Nz = 2(d + 1) for a linear fit, and With this value of the time lag, we learn from a global
Ns=(i+1)(d+2) for a quadratic fit. From this local false nearest-neighbor calculation that a global embed-
map we determine the recovered z„(j) by ding dimension of dE=3 will unfold the attractor with
time-lagged values of x (n ) as coordinates in the space.
Now taking simultaneous data from x (n) and z (n), we
Since we also have recorded the true value of z( j) for make a quadratic local neighborhood to neighborhood
each of the Nz values of j, direct comparison of our accu- map in dE=5. Of course, dE=3 would do, but it is
racy in recovery is possible. Let us emphasize again that known that taking an embedding dimension larger than
the coefficients a0 and a constituting the model g are the minimum suggested by false nearest neighbors can
different for different data vectors y( j). often improve the prediction, especially if noise is present
If we can measure both variables all the time, then it is [16]. This is also true in our case of predicting one physi-
sometimes more useful to use time delays of both vari- cal variable from another (see text below and Fig. 3). So
ables for the coordinates of the state space [1]. This ap- the map z(n) =g {y(n) } has five-dimensional vectors
proach was tested by Casdagli and Weigend [13], who y(n) ={x(n 2T), x (n— —T), x (n), x (n + T), x (n +2T) }
used previous values of chest volume and heartbeat rate as its argument. For test purposes we split our data set of
to forecast future values of the heart rate. The main N =10000 (x, z) pairs into a training set of NT=9000
difference in our algorithm is that at the stage of forecast- and a recovery set of Nz =1000 pairs. First we build a
ing no use is made of information about the previous dy- kd tree in the five-dimensional space. For each x ( j) from
namics of the variable to be predicted. We envision situa- the recovery set we construct a vector y(j) and find the
tions where in a laboratory or other controlled setting N~ nearest neighbors [y'"'( j), k = 1, . . . , Ns I among the
one can measure both x (n) and z(n). However, in some training set. Using the yI"'(j) and their images z'"'(n),
application in the field or other environment, only one we fit the coefficients of the local map (5). Finally we sub-
variable can accurately or efficiently or reliably be mea- stitute our test vector y(n) into (5) and obtain the
sured. The connection between the variables determined recovered value z„(j) of the variable z ( ). j
in the controlled setting then allows us to determine the We varied the time delay T, always keeping dE = 5, and
other dynamical quantity from the first one in other situ- determined how well our reconstructed values z„(j)
ations where we may not be able to observe both quanti- match the known values of z ( j). T = 1 is used in Fig. 2(a),
ties. where we have the values of the direct error z(n) — z„(n)
For example, we can suggest a setting where in a labo- over the recovery set of 1000 points. Note the change in
ratory we are able to measure the pressure at the wall of a scale as compared to Fig. 1. Figure 2(b) shows the
turbulent boundary layer How and the horizontal velocity recovery error for T=2; note the substantial improve-
magnitude at some interesting point just above the wall.
If we measure both and establish the dynamical connec- 20.0
tion between them on the attractor, then in future obser-
vations where only the wall pressure may be available, we
can deduce the How velocity without further measure-
ment and with some confidence [14].
This paper deals with the implementation of these -10 0 )-

ideas on the Lorenz attractor taken as a standard model -20. 0

0 200 4oo 6oo Boo 1000
of low-dimensional dynamics, and on data from several
nonlinear circuits built and run in our laboratories.
50. 0 (-
II. LORENZ MODEL 40.0
b

The Lorenz model [15]

x =o (y —x), 20.tl
1( i. o

y = —xz+rx —y, 0.0 I

200 400 600 800 1000

z =xy —bz,
with parameter values o. =16, b =4, and r =45. 92 serves FIG. 1. Time series for two variables from the Lorenz system
as a paradigm for low-dimensional chaotic systems. We (9); ~, =0.02 s: (a) x(t), and (b) z(t).
49 PREDICTING PHYSICAL VARIABLES IN TIME-DELAY EMBEDDING 1843

ment in recovery with a larger T. Big glitches in the pre- 10 ))

0

diction error of Fig. 2(a) are clearly correlated with the +

points where the local derivative Bz/Bx becomes large. + local linear fit
This effect is especially significant near x =0. In these 10
i local quadratic fit

points small variations of x are accompanied by large

variations of z, which is known to cause a noise
amplification phenomenon [16]. As one can see from Fig. 10
2, the best way to diminish this efFect is to choose the op-
timal (nonsmall) value of tiine delay. At T =5, as shown
in Fig. 2(c), we have still better predictability note the — 10
change in scale. In Figs. 2(d) and 2(e), the values of T are
T=7 and 12, respectively, and we see the quality of the
local map predicting z (n) degrade slowly. In a sense this 10
is another method for determining a time delay for the
reconstructions of the state space. It rests on a quite
different aspect of the data from mutual information, and
while good predictability sounds attractive, it seems to us
E
to have little fundamental meaning in terms of the prop-
erties of dynamical systems. FIG. 3. Root-mean-square (rms) error from modeling z(n)
on x (n) using data from the Lorenz system as a function of the
In Fig. 3 the root-mean-squared error cr averaged over
embedding dimension dE=5. In each analysis T =5. Results
all data points
are shown for both local linear and local quadratic maps.
((z —z„) ) —((z —„))
02
(z') —(z)' sharp decrease in 0 as we pass through dE =3, as is con-
is shown for a fixed time delay, T=5 — the one which sistent with the false nearest-neighbor determination of
worked best —as a function of the embedding dimension. dE. For the local quadratic map the error continues to
decrease with dE, until we run out of data points in
We show this result both for the local quadratic map just
discussed and for a local linear map. For each there is a higher dimensions with which to determine a good map
y(n) ~z (n). Figure 4 looks at the method from the point
0.1 of view of variation in time delay for fixed dz. A
minimum in 0 is seen with both linear and quadratic lo-
0.0 ill( . I. . i. , il .i t.
w
~l i[, . i g l
cal models. The rms error in each is quite small, and the
message is that while quadratic models provide a substan-
-0.1 I I I I

tial improvement in predictability, for many purposes

0 200 400 600 800 1000
0.1 they may not be required if prediction at the 1% level is
b sufBcient.
0.0 It is noteworthy that the inverse problem, namely
reconstructions of z(n) from x(n), is impossible for
-0.1
0 200
I

400
I

600 800
I

1000
Lorenz system because of its symmetry x — x, ~
0.01
10
opo- 'i 'i( ' )~r IF W
+ local linear fit
-0.01 I I I I + local quadratic fit
0 200 400 600 800 1000
0.1
.d
PP ~Li. u a » . ii. ~ lj I UA
1-er ~- ~
P
10

-0.1 I I I I

0 200 400 600 800 1000

10
1.0

0.0 l. l.

10
-1.0 I I I I

0 200 400 600 800

I I

FIG. 2. Recovery error for modeling the z variable of the 9 11

Lorenz system in a phase space composed of vectors built from
time delays of the x variable. Each analysis used dE = 5 and a lo- FIG. 4. rms error from modeling z(n) on x(n) using data
cal quadratic fit. The time delay used in the phase-space vectors from the Lorenz system as a function of the time delay T. In
was varied in these calculations: (a) T =1; (b) T =2; (c) T=5; each analysis dE =5. Results are shown for both local linear and
=
(d) T 7; and (e) T = 12. local quadratic maps.
1844 HENRY D. I. ABARBANEL et al. 49

y ~— v, z~z. In~e
ndeed, each z can correspond to two
—
4000.0 I I

different values x and x , so th ere is no way to deter- 3000.0

mine the sign of x from the knowledge of z. However, we
believe that this sort of ssymmetry
mm is rather exceptional ) 2000.0
an d'in most casess the
' '
th reconstruction is possible i e in b ot h
directions.
0.0 I I I

0 200 400 600 800 1000

III. HYSTERETIC CIRCUIT
A. Description of the circuit 4000.0 I I

.b
3000.0
The hysteretic circuit used for our first rs sett of experi-
mental data
d has been described earlier'er [17]~ an
and consists 2000.0
in essence of an unstable
un second degree oscillator which
operates in the range of severala h un d re d H z connected to
a h ysteretic element. Diagrams for th e circuit can be 0.0 I I I I

~ ~ ~
0 200 400 600 800 1000
found in the earlier p a p er.. The circuit is similar to some
~ ~

described by Newcomb and co-workers [18].

The hysteretic circuit can be modeled b y th ree ordi- FI . S. Two volta g es V& an and Vz taken at the same time
nary difFerential equations: rom the hysteretic circuit described in the text. H
oth d t et s bootth voltages were offset and normal' ize d to b- o-
dx, (t) tain a better resolution
re upon digitizing with a 12-bit ana
=x2(t)+yxi(t)+cx&(t), , so the relative scales are not preserved
or our predicting
(for redi algorithm is does not ma er. Thee sampling
matter). s
dx2(t)
=tox, (t) —5~x~(t),
differ

(10)

e „[1
dxi(t) —x, (t)' ][S x, (t) —D+x, (t)] 5,x,—
(t), data set. The V s(n) data are certainly embeddable in
dz =3, while V„(n) data appear to require dE = 5.
Weeseet h at
with y=0. 2, c =2.2, 52=0. 001, 5 =0. This result in itself is quite illuminating.
'
co= . , =1667.0, and D =0.0. Thee constants in the usin g e coordinates can result in ra
ent ratherer d'ff
i erent
or x
equation for x3&t' oop..
e ic loo
&t',, are set to give a square hysteretic choices for global embedding dimension. Th'is is coIl-
~ ~

We present these equations for illustration purposesesony; l sistent with the embedding theoremrn, w h'ich gives only a
the data we will be using was not generated from the sufficient condition for the glob a l em b edd'ing dimension
m thee experimental cir- E in terms of the fractal (box counting) dimension d„;
equations, but taken directly from
cui.
cuit. Two volta es were observed in the experiment 7
wo voltages namely the integer dz must be d 2d . In )
The voltage V„(t) corresponds to the we wi argue below that the fractal dimension (to b
(!
variable x , ~t& above,
abo and V~(t) is described by x2(t). precise the Ly a punov dimension) of this system is about
2.4, so an inte er
Both voltages are in the order of 0 —10 V , and were digi-
twice this, namely dE =5, is sufficient. The false nearest-
tized at
a ~~, =
=0.. 1 ms. Samples of digitized and rescaled
voltages are shown in Figs. 5(a) and an 5(b).. A s always in
an experiment, there is some noise present in the data. n 8.0

order of a few mV; that is, about 0. 1% noise.

0 Voltage A
6.0— ~ Voltage B
B. Analysis of the data:
two observed scalars

40
the voltages from the hysteretic circuit [5], and repeat
part of that for completeness. Th F
or these
for th data are not displayed here both because they
appear elsewhere and little is learned from them exce t 2.0 -0 I

~ ~
00 o ~ 0 ~
~ 00~ ~ 0~ ~ ~~0~~
p ~0~ 0 ~~
'

een ca e V„(n) and Vs(n); each was 64000 OpppP Op OOP pPpppp 0 00000
mpu-t-
points in len g th.. In Fig. 6 we show the results of corn 0.0 1 i, . . . \ ~ I

in g th e average mutual information in each d 0 5 10 15 20 25 30 35 40

Timelag
ime ag; the physical units of the sarnplinp ing
time are ~ =0. 1 mss. We see that each data set has its FIG. 6. Avera ge mutual information as a function of time de-
first minimum at T =6, and we use that value throughout nalss V „and
lay for siigna an V& from the hysteretic circuit. 64000
'
our anal y sis of thesese data. In Fig. 7 we present th e resu l t data points were used. The units of the time lags are 0.1 ms.
of the g loobala false nearest-neighbor computation for each Each signal has its 6rst minimum at T =6.
49 PREDICTING PHYSICAL VARIABLES IN TIME-DELAY EMBEDDING 1845

100.0:: 0.160

90.0-
0.140
GN8=10
80.0- DN =25
0.120 NB=5
70.0-
0 Voltage A
NB=75
~ Voltage B
X 60.0- 0.100 &

Cl

50.0-
4l
CO o" 0.080
C
4o.o—
Q. 0.060
0-
L'

30.
0.040
20.0-

100- O. O2O(.

4E
0.0 I

0.000 I

1 2 3 4 5 6 7 8 9 10 11 12 1 4 5
Dimension Local Dimension; d,

FIG. 7. Percentage of global false nearest neighbors as a

FIG. 9. Local false nearest-neighbor test for Vz from the
function of the embedding dimension for the signals V& and V&
hysteretic circuit, the same parameter values as in Fig. 8. dL = 3
from the hysteretic circuit, T =6, 47000 points. V& requires di-
is identified here as in Fig. 8.
mension dE=5 to unfold the attractor, while V& only requires
dE =3.

false nearest neighbors [7] for these two data sets. The lo-
neighbor method determines for a given data set what the
cal dimension must be the same for each data set regard-
necessary embedding dimension must be for time-delay
less of which global embedding dimension is chosen.
reconstruction of the attractor using a given set of time-
This local dimension is the (integer) number of degrees of
delayed measurements. It need not be the same necessary freedom active in the orbits making up the attractor.
embedding dimension for all choices of reconstruction
Whatever the global twisting of the attractor in one coor-
coordinates, since one set is some unknown nonlinear
dinate system or another, we must have the same local
transformation from another, and this can twist the at-
dynamical dimension. This number also determines what
tractor in such a fashion as to require a higher dimension
dimension model will be required locally as well as the
to unfold it via time-delay coordinates. The embedding number of Lyapunov exponents are to be considered true
theorem tells us with certainty when we can stop unfold-
and not artifacts of the choice of global embedding. We
ing, as additional dimensions could not possibly achieve
see in these two figures that both V„(n) and V~(n} indi-
any further unfolding of the attractor.
cate that dL =3 for this system. Using data from the
In Figs. 8 and 9 we show the result of evaluating local
V~(n) data set, we have evaluated the average local
Lyapunov exponents [19,20] in global dimensions dE =4,
5, and 6, with dr =3 in each case. The analyses are con-
o.44o»
sistent regardless of dz. It is from this that we are able to
O. 42O'
Ne=10 deduce that the Lyapunov dimension is DL =2.4. One of
0.400 ON8=25
QN =50 the local exponents goes rapidly to zero, with the number
0.380 Ne=75 of time steps along the attractor indicating that ordinary
0.360 differential equations are the appropriate model for the
0.340 dynamics producing these observations. This, of course,
L

0.320 is totally in accord with earlier studies of the hysteretic

0.300 circuit.
0.280
(
0.260
0.240 C. Modeling A on B and B on A
0.220
I I

4 5 7
Local Dimension; d, Now we turn to the discussion of how we11 we can pre-
FIG. 8. Local false nearest-neighbor test for V& from the
dict the evolution of one of the data sets from making lo-
hysteretic circuit. Here I'~ is the percentage of bad predictions, cal polynomial models for the variable we wish to predict
and N& is the number of neighbors used for local predictions. based on a phase space reconstructed using the other
We used 60000 data points, time delay T = 6 and P=0. 3, where variable. First let us see how well we can do using Vz(n)
P is the fraction of the attractor size two neighboring points to create the state space. We reconstructed the phase
must separate in defining a bad prediction (Ref. [7]}. It is im- space with time lags of V„(n} and then made local linear
portant that dL = 3 is identified here as in Fig. 9. models of Vz ( n }=g (y z ( n ) ) using various values of the
1846 HENRY D I A~AR~ANEI 49

400.0 dx, (t) = —1098[1.466x, (t) —2. 466x2(t)

00 ~ g y t y r~q~r-~ g p.
1 r (YEI

i
(t)+ q (x4(t) ) ],
+x 3( t) x4— &

-400.0 I I I I

0 200 400 600 800 1000

a5 dx2(t)
400.0 10 980x, (t),
b

0.0 dx 3(t)
= —4972x 2 (t ),
-400.0 I I l

= —10 980 —100 000 x, (t)+ A4x4(t)

0 200 400 600 800 1000
dx 4(t)
400.00 I t I

e
p pp
~le g~iJ&i. t. u ~&g JL4

-400.00
200
I I I I
+q2(x4(t))
0 400 600 800 1000

x)=0. 5(~x —2. 5~ —x +2. 5~ )

)~ 400.0 I I \

I
d q&
p. p;-"1L'-" t~ &~. —1. 3~ —x+1.
q2 x =0. 5x+0. 164(~x + 3)
'~]17't~lplY p~p ng~t~ p -rprr

-400. 0
+0. 361( ~x —2. 6~ —~x +2. 6~
I I 1

0 200 400 600 800 1000

The parameters are set to A 4 = =0. 5 and R =79 000 0;

FIIG. 10. Data from thee h ysteretic circuit. h' system. The f
R „ is the control param et er in this
& using &. (a) and (b). Re y
—6, T —6, (b)
50000 points in thee trainin
raimng set
ng z. (c) and (d). d . (a) d

se and employed local linear fits

E=, T=6. %e used
ui
h. .-. .-.,
.
ey are piecewise linear and are desi

d"' 'f 'h""'

~ matc,
o h so that the
o ic synchronization.
h The a
e circuits were se so t at a sin g le well
'
g,
or i erent settin s iit was also possible
dimension of the state space of the n . to see a chaot' ic attractor that ho ed b
analysis is done at T =6. The tr
'
t . Th dt ' tV„(t
iti d wiith 12-- it resolution at ~ =50
V (t) o t i th e stgnal x4(t), digitized at the sa
'
th e or aE =6 and 7 res Now we mi e th e ob served V (t ) and Vtt(t) data.
thou h th' '"'"ll
a per formance
in both c eries are seen in Fi s. 11 a
The time seri
e ing d'
~

ua 1 iin foormation for both data sets is s

~

using a high r embedd' dimension rem g

ery w ic are probabl y rerel ated to regions in Fi . 1
wheere we do not have unfol
the phase spacee w o ding in dE =6
'
1 t) tdfo
VA
but do have it in d =7. sing these values for th e t'ime clay, we eval-
Turning to our ability to model the behavior
4000.0
10(d) th 1 o ma k'ing local
s of 1 linear
ofV(
-neig or test now shows up u in our improved
'c Vz from observations
ability to predict tions of V~. In d =6
'
i e improvement
we see little overr thee results
res in dE=3' 0.0 I I 1 i

0 200 400 600 800 1000

indeed the efFec t s of 1'imited , possibly noisy data show up
here.
4000.0
. b
IV. OTHER CIRCUITS 3000.0
— 2000.0
A. Four-dimensional
an der Pol-Duf5ng oscillator: Circuit 1
Van
) IS

1000.0

0.0 I I I

0 200 400 600 800 1000

The data for this circuit are from two oin
y e sync ronization of
circuit dia ram ca FIG.
0
G. 11.
1 Two voltages V& and V8 tak
1 per. T}1is circuit is de ui we digitized with
as escribed in the text. Theey were
12-bit resolutioion.. ~,, =50
=50ps.
49 PREDICTING PHYSICAL VARIABLES IN TIME-DELAY EMBEDDING 1S47

0.090 q-

6.5— 0 Voltage A 0.080

6.0— o Voltage 8
5.5— 0.070
5.0
4.5—
0.060

0.050
3.0 - o
2.5 - &&

2.0-
0.040
&& o
oooooooo~@~ggg
10 oOOOOO ~8(gggggeee8%I 0.030
05-
0.0 I I I I

0.020
0 10 15 20 25 30 4
Timelag Local Dimension; d,
Nfl=2
FIG. 12. Average mutual information as a function of time FIG. 14. Local false nearest neighbors for V„ from circuit 1,
delay for signals V& and V& from circuit 1. 64000 data points T =8, P=0.4, and N =60000. It is important that dL =4 is
were used. The units of the time lags are 50 LMs. V& has its first identified here as for Vz.
minimum of I( =
1) at T 8, while V& has its first minimum at
T=7.
uated the global false nearest-neighbor percentage as a
0.075
function of embedding dimension, and found in Fig. 13
GNa = 10
that dE=4 is clearly suggested for V„(t), and possibly E)
for Vz(t) as well, although in the latter case dz =3 may 0.065
ONfi = 50
~ NB=75
do. To further examine this we look at the local false
nearest neighbors which are displayed for V„(t) in Fig. 0.055
14 for the data read forward in time, and in Fig. 15 for
the data read backward in time. Each clearly distin- 0.045
guishes dL =4, consistent with the known circuit equa-
tions. 0.035
For the data from V~(t), looking at the local false
neighbors results in Figs. 16 and 17, where dL =4 is 0.025
selected, in accord with the analysis of the voltage V„(t)
just given. 0.015
Now working in dE = 5 and using T = 8 for the
I

1 4
Local Dimension; d,
time delay, we first constructed a phase space based on
voltage V„(t) and then created a local linear map FIG. 15. The same as Fig. 14, but with the data read back-
ward in time.
100.0;:

90.0— 0.065
~ Voltage A; T = 8 0.060
0-
&

80. OVoltage B; T=7

0.055 C GN =10
70.0- 0.050 Ne =25
C Ne=50
LL 60.0- 0.045 NB=75
O
0.040
6$
50.0—
0.035
X
40.0
tu 0.030
Q
0
30.0— 0.025 [

20.0
0.020
0.015
10.0
0.010
0.0 BL (
1 5 6 10
0.005
Dimension 0.000 I

2 4 5
Local Dimension; d,
FIG. 13. Percentage of global false nearest neighbors as a
function of the embedding dimension for the signals V& and V& FIG. 16. Local false nearest neighbors for V& from circuit 1,
from circuit I, 37000 data points. V& requires dimension T=7, @=0.1, and N=60000. dL =4 is identified here as for
dE = 5 to unfold the attractor, while V& only requires dz =4.
1848 HENRY D. I. ABARBANEL et al.

0.150 ) 100.0
a
0.140
50.0
0.130 QNe = 10
Ne=25 '
~ ~
0.120 ( ONe = 50 'y ')(I I' I III' (1) ll! &,
&

II'~ I i l&tIi+llp pli& vp dipl+&lllp&f

Ne = 75
0.110 -50.0
0.100
-100.0 I I I I

0.090 200 400 600 800 1000

0.080
L

0.070
100.0
0.060
0.050
50.0

0.040
0.030
-50.0
0.020 I I

2 4 5
Local Dimension; d„ -100.0 l 1 I

0 200 400 600 800 1000

FIG. 17. The same as in Fig. 16, but with the data read back-
ward in time.
FIG. 18. Data from circuit 1. Recovery errors for modeling
V& using V„(a), and V& using Va (b). We used 50000 points in
the training set and employed local linear fits. In each case
Vs(n) =g( V„(n —2T), V„(n —T), V„(n), V„(n+ T), dE=5, T =8.
V„(n+2T}) which allows us to predict the values of
voltage B when we observe only voltage A. Similarly we pie. This Duffing circuit used analog multiplier chips for
then used voltage B to create a phase space of dimension the cubic part of the Duffing equations. The output sig-
dz =5, and made a local predictor for voltage A in that nals for x and y are contained in files Vz(t} and Vtt(t),
space. In Fig. 18 we see the results of making these pred- each digitized at r, =20 p, s (see Fig. 19). The equations
ictors a local linear map. In Tables I and II we present a describing this circuit are
summary of all our studies of circuit 1 associated with
recovery of voltage 3 from voltage B and B from A with '" =100000 ( )
different time delays and embedding dimensions. The last dt
column of both tables shows the rms error of recovery (12)
(t)
6y (.t) x(t) ], —
dy
calculated by (9). =100000[8 cos(cot) 0 25— 3
dt
B. Single wellduiBngcircuit: Circuit4 where the driving frequency (co=2rrf) was 769 Hz, and
the driving amplitude was B =6.0 V. Part of the dynam-
f
A single well Duffing equation circuit was used to gen- ics in this circuit comes from the fact that the cubic func-
erate the data for voltages V„(t) and V~(t) in this exam- tion, which is executed by analog multiplier chips, satu-

TABLE I. A summary of the various local models made to recover V& from V& using data from cir-
cuit 1. It is clear that the method is rather robust under changes of the embedding dimension dE and
time delay T, in the vicinity of the values selected by global false nearest neighbors and average mutual
information, respectively.

rms errors
Circuit 1
Recovering voltage B from voltage A
Embedding Order of local rms error
Time delay dimension maps B„-B
1 Linear 0.504
3 Linear 0. 105
5 Linear 0.0528
7 Linear 0.0545
9 Linear 0.0477
11 Linear 0.0830
13 Linear 0. 111
9 Linear 1.11
9 Linear 0. 163
9 Linear 0.0324
9 Linear 0.0294
9 Quadratic 0.0499
 49 PREDICTING PHYSICAL VARIABLES IN TIME-DELAY EMBEDDING 1849

TABLE II. A summary of the various local models made to recover V„ from V& using data from
circuit 1. It is clear that the method is rather robust under changes of the embedding dimension dE and
time delay T, in the vicinity of the values selected by global false nearest neighbors and average mutual
information, respectively.
rms errors
Circuit 1
Recovering voltage A from voltage B
Embedding Order of local rms error
Time delay dimension maps A, -A
1 5 Linear 0.0515
3 5 Linear 0.0498
5 5 Linear 0.0527
7 5 Linear 0.0533
9 5 Linear 0.0544
11 5 Linear 0.0477
13 5 Linear 0.0566
9 1 Linear 1.114
9 3 Linear 0.0643
9 7 Linear 0.0473
9 11 Linear 0.0456
9 5 Quadratic 0.0526

rates at the power supply voltage of 15 V, producing a when the fractal dimension of the attractor is close to an
clipping effect. integer, and was seen in the original analysis of the
The first step is to establish the properties of each data Lorenz model [7]. The reason is clearly numerical, as an
set, and in Fig. 20 we present the average mutual infor- attractor with fractal dimension close to an integer may
mation for voltages V„(t) and V~(t) for circuit 4. Each appear to have that integer as its local dimension when
has its first minimum at T = 5 or 100 ps. Figure 21 shows real, and thus inaccurate or noisy data, are used to estab-
that each data set can be embedded in dE =4, while the lish dL.
local false nearest-neighbor plot in Fig. 22 clearly estab- Turning to the main subject of this paper, we now ex-
lishes that the local dimension of the dynamics is dL =3, amine how well we can do using local linear maps to
totally consistent with the circuit equations suggested for determine one measured voltage in terms of the other.
the data. Data from Vtt read forward and backward in First we question our ability to model the behavior of
time are consistent with this. It is important to examine voltage Vtt(n) in a state space constructed from time lags
the data both forward and backward in time to confirm of voltage A. Using a six-dimensional embedding space,
the value for dL. This seems to be a general property we constructed a function
I

V (n) g(y(n}}y(n)=( V (n —3T) V&(n —2T), Vz(n —T), V„(n), V„(n + T), V„(n +2T}}

4000.0 I I 8.0
.a

2000.0
6. 0- o Voltage A
~ Voltage B
1ooo.o

0.0 I I I I

0 200 400 600 800 1000

4.0

4000.0
. b
3000.0 2.0 -o

2000.0 p ~~ ~~g ~ ~ ~ 0~~~~

p ~~~ Q Q j3 ts} ~~~fy
$uppo $ ~ ~0 ~p ~0 ~Q OO Q oQ Q p ~ ))
'
~
QOO Q p (s}
Qoooooop
1000.0
0.0 I I I I I

0 10 15 20 25 30 35 40
0.0 I I I I

0 200 400 600 800 1000 Timelag

FIG. 20. Average mutual information as a function of time

FIG. 19. Two voltages V„and V& taken at the same time delay for signals V& and V& from circuit 4. 64000 data points
from circuit 4 as described in the text. They were digitized with were used. The units of the time lags are 20 ps. Each signal has
12-bit resolution. v; =20 p, s. its first minimum at T = 5.
 1850 HENRY D. I. ABARBANEI. et al. 49

1000" 20.0
90.0— 1 0.0
80.0
Ih
I'I ll'UII rtrr Ul rills t Lli I rl t rlrrrr '' ~ II I' ll i& r ''lrrlg
0 Voltage A; 37,000 Points l'r srr ~
j~jj ll prpt r j"I' ]V rlr» I
,

~
I'll rr'r w~rlr
p] «I 'lrl't"
~ Voltage B; 57, 000 Points
70.0 -10.0
60.0— -20.0 I I I I

O 0 200 400 600 800 1000

50.0
C:
Qi
O 40.0—
Q)
CL
20.0
. b
300—
10.0
20.0— III, llUa jr' '

ll 'r Ii lrll'I ) Igr

ao l
' 'll i, I

'
II & I I1 j I
I .„ '
(
10.0—
I
rj
I l
1 r

-1OO—
0.0
1 5 6 10
-20.0 l l r l

Dimension 200 400 600 800 1000

FIG. 21. Percentage of global false nearest neighbors as a

function of embedding dimension for the signals V& and V& FIG. 23. Data from circuit 4. Recovery errors for modeling
from circuit 4, T=5. Both require dimension dE=5 to unfold using V& (a), and V& using
V& V& (b). We used 50000 points in
the attractor. the training set and employed local linear fits. In each case
dE=6 T=S.
with T =5. Using 50000 of the total of 64000 measured
data points as the training set, we predicted the values of
C. Square-loop hysteretic circuit: circuit 5
Vtt(n } for 1000 points in the recovery set. In Fig. 23(a} we
show the result of this operation, using a local linear fit A different type of hyseteretic circuit generated the
for the local maps. Errors in the recovery of the voltage data in this example [22]. This circuit is very similar to
Vtl(n) of order 0.5%%uo are seen. Turning the problem the first hysteretic circuit, except that the hysteresis loop
around —to recover V„(n) from measurements of is much sharper. This simple circuit is actually not sim-
Vil(n) —in Fig. 23(b) we see a similar substantial success ple to model, so for illustrative purposes we provide a set
of the local mapping method. The errors rise to about of equations that reproduces the approximate behavior of
0.7% in this case but are still quite acceptable. Note that this circuit. Most of the parameters are the same as in
the noise level in the recovered series is a few times the circuit, but the damping factors have been increased
greater than in the measured signal (0. 1%), which can to produce chaotic behavior. The values of x and y in the
easily be understood. Indeed, we use time-delayed values equations exceed those in the circuit by factors of 3 or 4.
of one signal to recover another, and since the noise is The discrepancy between the actual circuit and the equa-
amplified due to the largest positive and negative tions that describe it may be caused by capacitor leakage
Lyapunov exponents, small errors in the last components or inductance in the resistors used to provide damping in
V„(t+dsT/2)) of the embedded vector produce an the integrator portions of the circuit. The equations that
amplified effect on the predicted value V~(t), and vice have been used for modeling the circuit are
versa.
dx (t) = —1000[0.67x (t) —l. 77y (t)
0.360 dt
—3.9g (y„,y„,) —0. 77x(t)],
0.310 ON =10 (13)
DN =25 dy (t) = —100[x (t)+0. 675y (t) ],
QN =50
0.260 Ne =75 dt

0.210
where g(y„, y„, = —2. 0sgn(y„) if abs(y„) ~2. 0;—other-
)
wise it is equal to 2.0 if y„& 0. 0, and 2.0 if
0. 160 y„ l (0.0.
The function g(y„,y„ l} is used to approxi-
mate a square hysteresis loop produced by a multivibra-
0.110
tor in the circuit. The real circuit is a How, but the nu-
merical integration routines are maps, so the notation y„
0.060
and y„, refers to time steps in the numerical integration
routine, and is used to provide a history for the hysteresis
0.010 loop. The hysteresis loop provides the third degree of
1 3 4 freedom necessary for chaos in a flow. The x (t) variable
Local Dimension; d,
corresponds to V~(t), while y(t) corresponds to Vtl(t).
FIG. 22. Local false nearest neighbors for V& from circuit 4, The measured voltages seen in Fig. 24 were digitized at a
T =5, P=0. 25, and N =60000. dI =3 is identified here as for rate of v, =20 ps.
V~. The average mutual information for the two measured
 PREDICTING PHYSICAL VARIABLES IN TIME-DELAY EMBEDDING 1851

4000.0 100.0::
3000.0 90.0-

2000.0 80.0-
0
1000.0 70.0 0 Voltage A; T = 21
~ Voltage B; T = 22
0.0 I I I I
u 600
0 200 400 O
600 800 1000
50.0—

4o. o—
4000.0 Q)
CL
.b 30.0-
3000.0
20. 0-
2000.0
10.0
1000.0
IL
0.0
1 5 6 10
0.0 I I I I

0 200 400 600 800 1000 Dimension

FIG. 26. Percentage ofglobal false nearest neighbors as a

function of the embedding dimension for signals V& and V&
FIG. 24. Two voltages V& and Vz taken at the same time from circuit 5. Both require dimension dE =3 to unfold the at-
from circuit 5 as described in the text. They were digitized with
tractor.
12-bit resolution. ~, =20 ps.

0.014
voltages is displayed in Fig. 25, where close inspection re-
veals that the first minimum is achieved for T=21 (or 0.012 C GNB = 10
420)Lts) for voltage V„(n), and at T=22 for V~(n). Glo- 2NS= 25
C Ne= 50
bal false nearest neighbors shown in Fig. 26 demonstrates 0.010 N =75
that each time series can be embedded by time delays in
dE=3. This is amply confirmed by the local false 0.008
nearest-neighbor calculations shown in Figs. 27-29. For Y
CL

oversampled data sets like this one, one should be careful 0.006
implementing a global false neighbor search, because
nearest neighbors which are also close in time must be 0.004
discarded. This requirement is similar in spirit to the ob-
servations of Theiler [23] in a related context, and form a 0.002
standard part of our algorithms.
Working now in dE=5 with y(n) 0.000 '

=[ V(n — 2T), V(n —T), V(n), V(n + T), V(n +2T)] Local Dimension; dl
and using T =22, we make models of one voltage deter- false nearest neighbors for V& from circuit 5,
FIG. 27. Local
T =21, P=0.4, and N =60000. dl. =3 is identified here as for
8.0 V~.

0 Voltage A
~ Voltage B
C ON~=10
Ne= 25
0.025 C ON =50
Na=75
i= 4o -
0
o 0.020
0 0~
o~ o
2.0 pO
0 ~ ~ o 0.015 L

y 0 00
O
Oopy
~~&88888888~O
~ '~
o~oo

0.0 I I I. . . l I
0.010
0 10 15 20 25 30 35 40
Timelag
(
0.005
FIG. 25. Average mutual information as a function of time 1 4
Local Dimension; d,
delay for signals V& and V~ from circuit 5. 64000 data points
were used. The units of the time lags are 50 ps. V& has its first FIG. 28. Local false nearest neighbors for V& from circuit 5,
minimum of I(T) at T =21, while V~ has its first minimum at the same parameters are used as in Fig. 27. dL =3 is identified
T =22. here as for V~.
18S2 HENRY D. I. ABARBANEL et al. 49

0. 134
V. SUMMARY
0.131 (
0.128
0. 125 ~
'~ —
-~2 N
NS =10
=25
In this paper we have extended the usual nonlinear
modeling of the evolution of a measured scalar variable
0.122 B
-6 Ns = 50
0.119 -+ N = 75 x (n) through a function in d-dimensional space-
0. 116
y(n) =(x (n), x (n + T), . . . , x{n +(d —1)T))~F(y{n))
0. 113
0. 110
—
=y(n+1} to the representation of a second physical
0. 107 variable observed in the same dynamical system within
0. 104— that reconstructed state space. We have demonstrated
0. 101
this recovery of one physical variable from the measure-
0.098
0.095 ment of another in the Lorenz model and four examples
0.092 ( of nonlinear electronic circuits in our laboratories.
0.089 l- In the process of constructing our examples, we started
0, 086
0.083
from the scalar observations and carefully examined the
0.080 ' I phase space to be reconstructed before carrying out the
4 5 6 7
Local Dimension; d„
local mappings required. In each case we considered,
computer generated data from the Lorenz model or ex-
FIG. 29. The same as in Fig. 28, but with the data read back- perimental data from real circuits, we were able without
ward in time. Here we see the local dimension is dL = 3.
knowledge of the equations of motion to verify the num-
ber of dynamical variables requires and to accurately
mined by time delays of the other. In Fig. 30(a) we have model the evolution of one variable on another.
an error for Vs(n) modeled on voltage A with local In a sense all we have accomplished is a demonstration
linear fits. As usual the training set is 50000 points, and that the embedding theorem is correct: once one has
the recovery set is taken to be 1000 points. The errors in properly captured the state space, then all variables of a
this representation of Vs(n) are about 0.5%. In Fig. nonlinear system coupled to the variable from which the
30(b) we use local linear models to predict the other way coordinates of the state space are built can be recovered
around: Vz (n } within a time delay space of Vs (n ). The in that space. The importance of the construction ex-
results are comparable to those just shown. In Fig. 30(c) tends beyond some kind of indirect verification of the
we increase the order of the local model to quadratic, embedding theorem. Indeed, the motivation for this
thus requiring more neighbors and larger neighborhoods work was to discover how we can work with experimen-
for the local models, but we see no substantial increase in tal data on a strange attractor and recover the equivalent
our ability to predict. Local linear models seem to work of the differential equations we usually depend upon to
with great accuracy in these examples. tell us how one dynamical variable is linked to another.
20.0 Consider the familiar case of an incompressible fluid,
where knowledge of the fluid velocity allows the recovery
10.0
of the pressure by the solution of the Poisson equation.
0Ih

g$
LM.JAlNJI
I
~q) I l~ 14l..
(
w ~.M-~~llull~'f {
~1ppp'T lpprlllq'I PITT
u
~p %fhlf'
Our procedure is the direct analog of that construction,
but carried out without foreknowledge of the equations of
i

0 -10.0
motion, and done directly on the attractor.
-20.0 I I I i
We anticipate that the method we have described and
0 200 400 600 800
demonstrated will be of use in the analysis of chaotic ob-
20.0
servations when one variable is relatively easy to measure
10.0 but it is the detailed behavior of another variable which is
of direct physical interest. One would proceed in that
case to carry out experiments under a situation in which
-10.0 both variables could be measured, and deduce the
dynamical rules — of the form z(n) =g(y(n)) as done
-20.0
0 200
I I

400 600
I

800
I

1000 here— which allow the evaluation of the second variable

20.0 in another setting where only the variable x (n) is accessi-
ble.
1 0.0
There are some difhculties and unsolved problems in
) 0.0 +~u&&~
r ~q
kkIk Lw JLkl
~Pi Ie"q ~rrP-II'
jMg tL&ak I ~U using this recovery method in practical applications.
-1O O— First of all, as we saw in the example of the Lorenz at-
tractor, there are cases when due to some specific proper-
-20.0 I I I I
ties of the system at hand (symmetries, etc. }, the recovery
200 400 600 800 1000
of one variable from another is impossible in principle. It
FIG. 30. Data from circuit 5, recovering errors. We used is not clear at the moment how common these counter-
50000 points in the training set and employed local linear fits. examples are. Nevertheless, all the experimental data
(a) Recovering Vz and V&. (b) Recovering V& using V&, d&=5, series we dealt with allowed the reconstruction in both
T =22, and local linear fit. (c) The same as (b), using a local directions, therefore suggesting that generically recovery
quadratic fit. is possible. Another interesting question is about the na-
49 PREDICTING PHYSICAL VARIABLES IN TIME-DELAY EMBEDDING 1853

ture of prediction errors we observed. The time series of when the proposed equations of motion are not formally
differences between the original signal and its recovered differentiable, as in several of the examples we presented
counterpart reveal some well-separated sharp peaks. The here.
occurrence of these peaks is certainly related to some The unattractive feature of local models is that they
dynamical properties of the system under investigation, are numerical "look up" or interpolation devices which
and can possibly be explained by the noise amplification embody the evolution in local regions of phase space.
phenomenon [16]. One more problem which deserves They are not global by construction and may not allow
systematic study is the inhuence of the dynamic or mea- generalization of how differential equations in high-
surement noise on one s ability to recover physical vari- dimensional spaces collapse down to much smaller di-
ables via time-delay embedding. mensional attractors. Recognizing their limitations in
Note that since we are making local models of the dy- this way, they certainly provide a powerful tool for the
namics in phase space, we need not worry excessively analysis and prediction of actual physical experiments, as
about whether we are using polynomial basis functions, we have demonstrated.
radial basis functions, or other richer sets of functions.
Local linear and local quadratic models work exceedingly
well for the goals we set. Of course, one can be more am- ACKNOWLEDGMENTS
bitious and seek global models for the same purpose of
representing one observable scalar variable in terms of We thank the members of INLS for numerous discus-
another, but this would take us far afield of the frame- sions on this subject. This work was supported in part by
work in this study, and its success would rest heavily on the U. S. Department of Energy, Office of Basic Energy
our ability to choose a set of basis functions properly Sciences, Division of Engineering and Geosciences, under
tuned to the global dynamics of the system being ob- Contract No. DE-F603-90ER14138, and in part by the
served. Local polynomial dynamics is more accurate Army Research Office (Contract No. DAAL03-91-C-
since it is less clever: it is universal as long as the vector 052), and by the Office of Naval Research (Contract No.
field governing the evolution is differentiable, namely in N00014-91-C-0125), under subcontract to the
essentially every physical setting. Indeed, it works even Lockheed/Sanders Corporation.

[I) J.-P. Eckmann and D. Ruelle, Rev. Mod. Phys. 57, 617 (unpublished).
(1985). [13] M. Casdagli and A. S. Weigend, in Time Series Prediction:
[2) R. Matte, in Dynamical Systems and Turbulence, Warwick Forecasting the Future and Understanding the Past, edited
1980, edited by D. Rand and L. S. Young, Lecture Notes by A. S. Weigend and N. A. Gershenfeld (Addison-
in Mathematics Vol. 898 (Springer, Berlin, 1981), p. 230. Wesley, Redwood City, CA, 1993).
[3] F. Takens, in Ref. [2], p. 366. [14] Matthew B. Kennel, R. Brown, and H. D. I. Abarbanel,
[4] T. Sauer, J. A. Yorke, and M. Casdagli, J. Stat. Phys. 65, Phys. Rev. A 45, 3403 (1992).
579 (1991). [15] E. N. Lorenz, J. Atmos. Sci. 20, 130 (1963).
[5] H. D. I. Abarbanel, R. Brown, J. J. Sidorowich, and Lev [16] M. Casdagli, S. Eubank, J. D. Farmer, and J. Gibson, Phy-
Sh. Tsimring, Rev. Mod. Phys. 65, 1331 (1993). sica D 51, 52 (1991).
[6] A. M. Fraser and H. L. Swinney, Phys. Rev. 33A, 1134 [17] T. L. Carroll and L. M. Pecora, IEEE Trans. Circuits Syst.
(1986); A. M. Fraser, IEEE Trans. Inf. Theory 35, 245 38, 453 (1991).
(1989); Physica 34D, 391 (1989). [18] R. W. Newcomb and S. Sathyan, IEEE Trans. Circuits
[7] H. D. I. Abarbanel and M. B. Kennel, Phys. Rev. E 47, Syst. CAS-30, 54 (1983); R. W. Newcomb and N. El-
3057 (1993). Leithy, Circuits Syst. Signal Process. 5, 321 (1986).
[8] M. Casdagli and S. Eubank, Nonlinear Modeling and Fore [19] H. D. I. Abarbanel, R. Brown, and M. B. Kennel, J. Non-
casting (Addison-Wesley, Redwood City, CA, 1992). linear Sci. 1, 175 (1991).
[9] J. D. Farmer and J. J. Sidorowich, Phys. Rev. Lett. 59, [20] H. D. I. Abarbanel, R. Brown, and M. B. Kennel, J. Non-
845 (1987). linear Sci. 2, 343 (1992).
[10] R. F. Sproull, Algorithmica 6, 579 (1991). [21] T. L. Carroll and L. M. Pecora, Int. J. Bifur. Chaos 2, 659
[11]M. Giona, F. Lentini, and V. Cimagalli, Phys. Rev. A 44, (1992).
3496 (1991). [22] J. Neif and T. L. Carroll, Sci. Am. 269 (2), 120 (1993).
[12] R. Brown, Institute of Nonlinear Science Report, 1993 [23] J. Theiler, Phys. Rev. A 34, 2427 (1986).