PHYSICAL REVIEW A ‘VOLUME 45, NUMBER 6 15 MARCH 1952 ARTICLES ‘Determining embedding dimension for phase-space reconstruction using a geometrical construction Matthew B. Kennel Institute for Nonlinear Science and Department of Physics, University of California, San Diego, La Jolla, Califormia 92093-0402 Reggie Brown Institute for Nonlinear Science, University of California, San Diego, La Jolla, California 92093-0402 Henry D. I. Abarbanel Institute for Nonlinear Science, Department of Physics, ‘and Marine Physical Laboratory, Scripps Institution of Oceanography, University of California, San Diego, Mail Code R-002, La Jolla, California 92093-0402 (Received 24 April 1991) ‘We examine the isue of determining an acceptable minimum embedding dimension by looking at the behavior of near neighbors under changes in the embedding dimension from d—+d-+1, When the num- ber of nearest neighbors arising through projection is zero in dimension dg, the attractor has been un- folded in this dimension. The precise determination of dj is clouded by “noise” and we examine the in which noise changes the determination of de. ur criterion also indicates the error one ‘makes by choosing an embedding dimension smaller than dy. This knowledge may be useful in the prac tical analysis of observed time series. PACS numbers) 05.45.+b, 02.40.+m 1. INTRODUCTION It has become quite familiar in the analysis of observed time series from nonlinear systems to make a time-delay reconstruction of a phase space in which to view the dynamics. This is accomplished by utilizing time- delayed versions of an observed scalar quantity X(iotnAN=x(n) as coordinates for the phase space [1,2]. From the set of observations, multivariate vectors in'd-dimensional space yin) =(xn),x(n 7), sxtnHd-0T) are used to trace out the orbit of the system. Time evolu- tion of the y's is given by y(n)—>y(n +1). In practice, the natural questions of what time delay T and what embedding dimension d to use in this reconstruction have hhad a variety of answers. In this paper we provide a clean, direct answer to the question: What is the ap- propriate value of d to use as the embedding dimension? We do so by directly addressing the topological issue raised by the embedding process. Our procedure identifies the number of “false nearest neighbors,” points that appear to be nearest neighbors because the embed- ding space is too small, of every point on the attractor as- sociated with the orbit y(n), m=1,2,...,.N. When the number of false nearest neighbors drops to zero, we have unfolded or embedded the attractor in RY a d- dimensional Euclidian space. The observations, x(n), are a projection of the mul- tivariate state space of the system onto the one- 4 dimensional axis of the x(n)'s. The purpose of time- delay (or any other [3]) embedding is to unfold the projec- tion back to a multivariate state space that is representa- tive of the original system, The general topological result of Mané and Takens (4,5] states that when the attractor hhas dimension d ,, all self-crossings of the orbit (which is the attractor) will be eliminated when one chooses >2d4 [6]. These self-crossings of the orbit are a result of the ‘projection, and the embedding process seeks to undo that. The Mané and Takens result is only a sufficient condition as can be noted by recalling that the familiar Lorenz [7] attractor, d 4 =2.06, can be embedded by the time-delay method in d=3. This is in contrast to the theorem, which only informs us that d=5 will surely do the job. The open question is, given a scalar time series, what is the appropriate value for the minimum embedding dimension dg? From the point of view of the mathematics of the embedding process it does not matter whether one uses the minimum embedding dimension dy or any d>dg, since once the attractor is unfolded, the theorem's work is done. For a physicist the story is quite different. Working in any dimension larger than the minimum required by the data leads to excessive compu- tation when investigating any subsequent question (Lyapunov exponents, prediction, etc.) one wishes to ask. It also enhances the problem of contamination by round- ‘off or instrumental error since this “noise” will populate and dominate the additional d —d dimensions of the embedding space where no dynamics is operating. ‘The usual method of choosing the minimum embed- 403 (©1992 The American Physical Society3408 ding dimension dy is to compute some invariant on the attractor. By increasing the embedding dimension used. for the computation one notes when the value of the i variant stops changing. Since these invariants are geometric properties of the attractor, they become it dependent of d for d>dp (ue., after the geometry is un- folded). The problem with this approach is that it is often very data intensive and is certainly subjective. Furthermore, the analysis does not indicate the penalty ‘one pays for choosing too low an embedding dimension. ‘We have already discussed the penalty for choosing too large an embedding dimension. In this paper we present calculations based on the idea that in the passage from dimension d to dimension d++1 fone can differentiate between points on the orbit y(n) that are “true” neighbors and points on the orbit y(n) which are “false” neighbors. A false neighbor is a point in the data set that is a neighbor solely because we are viewing the orbit (the attractor) in too small an embed- ding space (d dg), all neighbors of every orbit point in the multivariate phase space will be true neighbors. A simple example of this behavior is found in, the Hénon map [8] of the plane to itself. In Fig. 1 we show the attractor for the Hénon map as a projection, onto a d=1 dimensional phase space (the x axis) as well as in a d=dg=2 dimensional embedding space. The points A and B appear to be neighbors in the projection onto the x axis. However, they are neighbors solely be- cause we are viewing the orbit of the attractor in too small an embedding space. When viewed in a d=2 di- mensional embedding space they are no longer neighbors. Points A and B are examples of false neighbors. In con- trast points A and C are true neighbors. This follows be- cause they are neighbors in d=1 dimension (x-axis) as well as d=d,=2 and all higher embedding dimensions. We will return to and fully develop this point in Sec. 11 FIG. 1. The R! and R? embeddings of the x coordinate of the Hénon map of the plane. It is known that for this map 4d; =2, The points A and Bare false neighbors while the points ‘A and C are true neighbors. KENNEL, BROWN, AND ABARBANEL 45 In the process of writing this paper we found an article by Liebert, Pawelzik, and Schuster (LPS) [9] which re- ports on the same basic idea but implements it in quite a different fashion. We find rather distinct results from, these authors. We have several comments on their obser- vations and conclusions. Their method appears to be somewhat more time consuming, but only by @ constant factor. Furthermore, their method does not yield one of the desirable features of the present work. Namely, we provide an estimate of the error encountered in using t00 small an embedding dimension. We also found an older paper by Bumeliene, Lasiene, Pyragas, and Cenys [10] (BLPC) which cites an even ear” lier paper by Pyragas and Cenys [11] that has the essen. tial geometric idea of seeking false neighbors, and again implements it in another fashion, We will comment on, these papers as well The question of what time delay T'to use in the embed- ding is logically independent of the topological question, of how large the space must be to eliminate false neigh= bors. The determination of the time lag T requires infor- mation which is independent of the topological argu- ments of Mané and Takens. This can be understood since their argument works in principle for any lag T, and so must be independent of considerations about 7.” The issue of the time lag is dynamical, not geometric. For the purpose of determining T' we use the information theoret- ic techniques of Fraser and Swinney [12,13]. 11, THE METHOD OF FALSE NEIGHBORS ‘One of the important features of an attractor is that it is often a compact object in phase space. Hence, points of an orbit on the attractor acquire neighbors in this, phase space. The utility of these neighbors, among other things, is that they allow the information on how phase~ space neighborhoods evolve to be used to generate equa- tions for the prediction of the time evolution of new points on or near the attractor [14]. They also allow ac curate computations of the Lyapunov exponents of the system [15,16] In an embedding dimension that is too small to unfold the attractor, not all points that lie close to one another will be neighbors because of the dynamics. Some will ac~ tually be far from each other and simply appear as neigh- bors because the geometric structure of the attractor has been projected down onto a smaller space (cf. Sec. I). IF we are in d dimensions and we denote the rth nearest neighbor of y(n) by y(n), then from Eq, (I), the square of the Euclidian distance between the point yirt) and this, neighbor is| Rion Stein terrains, a) In going from dimension d to dimension d +1 by time- delay embedding we add a (d + Lith coordinate onto each of the vectors yin). This new coordinate is just x(n -+Td). We now ask what is the Euclidean distance, ‘as measured in dimension d-+1, between y(n) and the same rth neighbor as determined in dimension @? After the addition of the new (d+ 1)th coordinate the distance43 DETERMINING EMBEDDING DIMENSION FOR PHASESPACE.. between y(n) and the same rth nearest neighbor we deter- mine in d dimensions is R} a (mN=Rinn+ [xin Fd)—x%n +d). @ A natural criterion for catching embedding errors is that the increase in distance between y(n) and y(n) is large when going from dimension d to dimension d +1. ‘The increase in distance can be stated quite simply from Eqs. (2) and (3). We state this erterion by designating as 4 false neighbor any neighbor for which REailmr)—Riins) |! Riis) x(n + Ta)—x"%n + TA) Ralnr) rRar O where Ria, is some threshold. We will investigate the sensitivity of our criterion to Ryo in our numerical work below, and we will find that for Ryo, = 10 the false neigh- bors are clearly identified. It is sufficient to consider only nearest neighbors (r= 1) and interrogate every point on the orbit (n=1,2,...,.N) to establish how many of the N nearest neighbors are false. We record the results of, the computations as the proportion of all orbit points which have a false nearest neighbor. Before we report on these computations we remark that this criterion, by itself, is not sufficient for determin- ing a proper embedding dimension, To illustrate this we note that when we used this criterion to examine the embedding dimension for white “noise,” the criterion er- roneously reported that this noise could be embedded in a quite small dimensional space, (By “noise” we mean very-high-dimensional attractors associated with compu- terized random number generators.) ‘The problem turns ‘out to be that even though y'"'(n) is the nearest neighbor to y(n), it is not necessarily clase to y(n). Indeed, as we moved up in embedding dimension, with finite data from noise signal, the actual values of Ry(t)=Ry(n,r=1) ‘were comparable with the size of the attractor R ,.. Thus the nearest neighbor to y(n) is not close to y(n). ‘This be- havior follows from the fact that trying to uniformly pop- ulate an object in d dimensions with a fixed number of points means that the points must move further and fur~ ther apart as d increases. We note that as we increased. the number of data points in the noise signal the embed~ ding dimension (that dimension where the number of false neighbors drops to nearly zero) systematically i creased. In the limit of an infinite amount of data we would find that the embedding dimension d also diverged to infinity. This is in contrast to the low-dimensional at- tractor common to many dynamical systems. For a low- dimensional dynamical system increasing the number of data points on the attractor will not change d. However, in practical settings the number of data points is often not terribly large. We have implemented, the following criterion to handle the issue of limited data set size: If the nearest neighbor to yin) is not close [Rein }=R , ] and it isa false neighbor, then the distance aos, Ryya(m) resulting from adding on @ (d+1)th com- ponents tothe data vectors will be Ry .,(m)=2R ,. That 4s, even distant but nearest neighbors will be stretched to the extremities of the attractor when they are unfolded from each other, if they are false nearest neighbors. ‘We writ this second eriterion as Rat) 4 ry Ry a In our work we advocate using this pair of criteria jointly by declaring a nearest neighbor (as seen in dimension d) as false if either test fails. ‘Asa measure of R , we chose the value za1 $ xP Rimy Z leini-aP where 1s we” Other choices for R4, such as the absolute deviation about ¥, did not change our results. The reader may ‘choose whatever estimate of the attractor size which ap- peals to her or him. In Fig. 2 we show the result of applying the first cri- terion alone, of applying the second criterion alone, and 1000 700 500 False NN Percentage FIG. 2. The percentage of false nearest neighbors for 24000 data points from the Lorenz-Il equations. The data were output Which isthe location ofthe first minimum inthe average mutual information for this system, was used in forming the time- delayed vectors. Three different criteria are compared for ‘detecting false nearest neighbors. First is the change in distance of the nearest neighbors in d dimensions when the component x(n +Ta) is added to the vectors, Eq. (§). These points are marked with squares. Second is the eriterion which compares Res, t0 the size of the atractor R «, Eq. (5). These points are ‘marked with triangles. The third criterion applies both of these to the data, These points are marked with circles. In this last case a point which fails either test is declared false.3406 then applying them jointly when a data set from the three-dimensional model of Lorenz (17} y a(x-F), Paaymbaz—y +6 , 6 bxytaz—z, is used to generate the data, We use the values @=0.25, 0, F=8.0, and G=1.0 where Lorenz points out ir- regular behavior is encountered. The attractor has a di mension, d , slightly greater than 2.5. We produced the data used in this figure with a variable order Adams output at Ar=0.05, The first minimum average mutual information occurs at ITAL. This is the time delay T, we used in reconstructing the phase-space vectors yim) from samples of x(n)=altg-tns), In Fig, 2 a total of 25000 data points were used. It is clear that the joint eriterion (Eqs. (4) and (5)} as well as each of the individual criteria [Eqs. (4) or (5)] mentioned above yield an embedding dimension (of dg =6 for this attractor. In this case the result is actu- ally the same as the d>2d, sufficient bound of Mané land Takens. For this computation we used the values Rug =15.0 and Aj =20. We will report on the depen- dence of the method on the number of data points Nand con the value of Rin Sec. IU. The results are quite different for noise, In Fig. 3 we show a similar comparison of false-nearest-neighbor cri- teria for noise uniformly distributed in x(n) between —1 and 1. We used a time delay T=1, and again N =25 000. Here we see that the first criterion, Eq. (4), fails to indi- cate the need for a high embedding dimension. However, the second criterion, Eq. (5), yields the expected answer. False NN Percentage Dimension FIG. 3. The same as Fig. 2 but the data come from a random set of numbers uniformly distributed in the interval [— 1.01.0} ‘The comparison between Fig. 2 and this figure is quite striking and points out clearly the difference between a low-dimensional chaotic signal and noise (a high-dimensional chaotic signal. KENNEL, BROWN, AND ABARBANEL 6 OF course, the joint criterion also works and, as one would expect tracks the second criterion as d increases. This striking difference between low-dimensional chaos and high-dimensional chaos was seen in all examples we tried, Henceforth, we quote only the result of applying ‘our eriteria jointly: a nearest neighbor which fails either test is declared false. tis important to note that in each case we are able to determine the quality of the embedding because the per- centage of false neighbors is reported. For example, in the case of the Lorenz 84 attractor noted above, if we cchose to use an embedding dimension of d=S rather than 6, there would have been 0.18% false neighbors remaining. It is likely that this is an acceptably small ‘number for many purposes. A physicist might well chose to accept this error to make more efficient any further computations performed on the data from this system. ‘Actually the error in choosing only d=4 is not substan- tial since the percentage of false neighbors is only 0.59%. ‘One may even interpret these neighborhoods as isolated, instances of very large local Lyapunov exponents. A very high rate of instability growth may cause Ry ,,/Ry to be very large, even if the embedding is correct. In other in- vestigations (18], we have found that for short-time inter~ vals, the distribution in the finite-time Lyapunov ex- ponents may be very broad. The implication for the present situation is that there may be a few neighbor- hoods, whose largest local Lyapunov exponent falls in the high extreme of the distribution, for which the false- ighbor criterion (and any other similar type of mea- sure) will fail due to inherent dynamical reasons. Care- fully disentangling false neighbors from high local ex- ponents is still an unresolved issue, but the problem does, ‘not appear to be terribly severe. III, NUMERICAL RESULTS FOR SEVERAL EXAMPLES ‘A. Clean data and “noise” We have implemented the method just described on a variety of simple models. Let us begin with the Lorenz-IT ‘model previously discussed. Using the same time delay, T=17Ar, we examined the dependence of our method on the tolerance R,.. and the length of the data set N. In Fig. 4 we show the false-neighbor percentage in d=I as a funetion of these two variables. It is clear that except for very small amounts of data NV ~100 and large values of Rig the method of false neighbors would not indicate d=1 as a good embedding dimension. The same depen- dence, false-nearest-neighbor percentage as a function of Ry and N, is shown in Fig. 5 for d=5. It is clear that when the number of data points 1 is sufficient to fill out an attractor in d= dimensions, and the tolerance level is, R215, we can with confidence select d=$ as the ‘embedding dimension. As noted above one could just as, well choose d=6, where the percentage of false nearest neighbors drops to exactly zero. However, the error as- sociated with dS is so small (approximately 0.189%) one ‘might as well use d=5, We have examined this system ford=1,2,... 10.4s DETERMINING EMBEDDING DIMENSION FOR PHASE-SPACE ... 3407 FIG. 4. Data from the Lorenz-II system in d=1. The false- nearest-neighbor percentage is evaluated as a function of R, and the length of the data set N. The time lag in forming multivariate vectors is T=0.85. d=1 is clearly a good choice for embedding this system. ‘The message here is clear. For a fixed amount of data ‘one cannot always explore the finest details of the attrac tor. So some very close near neighbors might still appear false. With large amounts of data, this appearance is re- moved, and the trueness of the neighbors is exposed. When tackling the analysis of a data set from an un- known source, a bit of judgment is called for. We see no significant harm in suggesting that whatever dimension yields a percentage of false nearest neighbors below 1% Should work fine as an effective embedding dimension. Returning to uniform noise we show in Figs. 6-8 the dependence on N and jo, of the false-nearest-neighbor percentage in d=1, d=4, and d=8. The percentage falls. to a plateau in each case and never indicates that a low dimension would qualify as a good embedding. The false-nearest-neighbor percentage stays about 25% even as both Nand R,q grow. In Figs. 9-12 we show our results for the Ikeda [19,20] ‘map of the plane to itself, FIG. 5. Data from the Lorenz-I system in d=3. The false nearest-neighbor percentage is evaluated as a function of Rs and the length of the data set N. The time lag in forming all, ‘multivariate vectors is T=0.85, d=5 is clearly a good choice {or embedding this system, FIG. 6. Data from a uniform noise distribution (—1,1} in ‘The false-nearest-neighbor percentage is evaluated as & function of R,. and the length of the data set. FIG. 7. Data from a uniform noise distribution (1,1) in 4. ‘The false-nearest-neighbor percentage is evaluated as & function of Ry and the length ofthe data set N, FIG. 8. Data from a uniform noise distribution [—1,1] in d=8. ‘The false-nearest-neighbor percentage is evaluated as @ function of R,. and the length of the data set.FIG. 9. Data from the two-dimensional Ikeda map in d ‘The false-nearest-neighbor percentage is evaluated as function Of Ryq and the length of the data set. {rom the two-dimensional Ikeda map in d =2. ‘The false-nearest-neighbor percentage is evaluated as a function of Rg and the length of the data set ¥, FIG. 11. Data from the two-dimensional Ikeds map in d ‘The false-nearest-neighbor percentage is evaluated as a function, of Rg and the length of the dataset. KENNEL, BROWN, AND ABARBANEL 45 FIG. 12. Data from the two-dimensional Ikeda map in d = 4, ‘The false-nearest-neighbor percentage is evaluated as a function, (of Riu and the length of the data set N. d=4 is clearly a good choice for embedding this system with the use of time delay coordinates. This is consistent with the observations in Ref. 15}. zin+1)=p +Be(miexplieia 1+ [2(n))°)}, 7) where p=1.0, B=0.9, x=0.4, a=60, and z(n) is com- plex. The dimension of the attractor associated with this, map is d,=1.8, It is known from other considerations [15] that an embedding dimension of d=4 is required to unfold the attractor by time-delay embedding. This property shows up quite clearly in the figures where re- sults from d=1, 2, 3, and 4 are shown. We also applied our methods to the determination of a ‘minimum embedding dimension for the Lorenz-I [7] model, the Hénon map [8], the Réssler attractor [21] and the Mackey-Glass differential-delay equation [22]. The results are just as striking as those just shown, and in the latter two cases are consistent with the minimum embed- ding dimensions found by LPS. We forbear from drown- ing the reader in data, since the point is quite clear. Perhaps we should add, however, that in going through the data set and determining which points are near neigh- bors of the point yin) we use the sorting method of a d” tree to reduce the computation time from O(n") 10 O(N logiglN}}) [23]. For this reason each of the calcula- tions required takes minutes on a SUN Sparestation 2 workstation, B, Realistic Data Data are never delivered to the user in the pristine, no-noise, state of our examples above. Time series ob- tained from real experimental apparatus always contains both signal and noise. The results shown for “pure noise” certainly indicate that the effective embedding di- mension will degrade as the signal-to-noise ratio de- creases from plus infinity. We have examined the way in which noise degrades the results one obtains for our set of, example systems, The basic trend we expect, and see, is that as the signal to noise ratio is lowered, the effective embedding dimension rises. In Fig. 13 we show the results of applying our joint false-nearest-neighbor criterion to 25000 points on the Lorentz-I attractor. The Lorenz-I equations are (7]s DETERMINING EMBEDDING DIMENSION FOR PHASE-SPACE sap s000 features of the data observed. We would surely choose woo d=a'as an effective embedding dimension and interpret the plateau at 2.5% as a noise effect. Indeed, one of the g t00 frst tasks one would perform with noise observed data is Eno to apply one of the several methods of noise reduction, oF 3 separation, that have been recently suggested [2428] £ coo ‘The techniques we have presented will give an sccuraic af starting point by choosing an embedding dimension in z Which to perform the noise eduction, After separating 2 oo the signal from the contamination, one ean easily go back 3 oo to the false-nearest-neighbor method to determine anoth- = er (probably smaller) embedding dimension to use in Dimension FIG. 13. Data from the Lorenz-I model which have been contaminated with uniformly distributed noise in the interval [=1,L}. 41000 points from the attractor are used here, The percentage of false nearest neighbors is shown for L/R 4 0.005, 0.01, 005, 0.1, 05, and 1.0. For this system Ry Even’ with’ L/R,=10%, the error in choosing d=4 as the embedding dimension is only 0.27%. The false-nearest- neighbor method is rather robust against noise contamination. ‘The result for L/R,=0 is marked with an open star; for L/R,=0.005, itis marked with a square; for L/R4=001, with a circle; for L/R 40.05, with a triangle; for L/R «=0.1, with a diamond; for L/R,=0:5, with a filled star; and for L/R,=1.0, with an open ef0s. s=oly x), Somxteny, s=xy—bz, and we use parameter values ¢=16, b=4, and r=45.92. ‘The data were generated by a variable order Adams code with output at Ar=0.02. For this system the first minimum of the average mutual information is at T~0.1=SAt, and we used tolerance values of R\= 15.0 and A, =2.0. The orbits have been contaminated with noise uniformly distributed in the interval (—L,L] with L/R4=0.0, 0.005, 0.01, 0.05, 0.1, and 0.5. The last noise level corresponds to a signal-to-noise ratio [20 og,o(R 4 /L)] of about 6 dB in power; namely, a very low signal-to-noise ratio. ‘The increase in embedding di- ‘mension for this attractor in the presence of noise is real- ly quite slow, and only for the last, very contaminated, ccase does it fail. (When the signal-to-noise ratio falls to 0 4B, the false-nearest-neighbor criterion fails, and we find the Lorenz-I signal contaminated at that level to be indis- tinguishable from noise.) In the case where L/R4=10%, and d=3, we have 7.8% false nearest, neighbors. For d=4 this number drops to 2.5% and pla- teaus for higher embedding dimensions. Figure 13 illustrates our approach to embedding di mensions. The practical issue, to say it again, is what embedding dimension should one use to capture the analyzing the signal. Actually since two of the methods require either knowing the dynamics [24-26] or observ- ing a clean trajectory of the dynamics (27), it is only in the case where no knowledge of the “clean” system is re- quired [28] that these remarks might come into play. In the probabilistic method [27] where only a clean trajecto- ry has been observed, the present false-nearest-neighbor technique will give a direct, useful answer about the embedding dimension in which to clean subsequent noise data, IV, CONCLUSIONS AND COMMENTS ‘We beain this section by commenting on the relation- ship between this paper and that of LPS [9] and the paper of BLPC {IO}. It should be mentioned that the general ideas behind all of these methods have been around for some time, for example, se Fig. 3 in Ref 29), and in fact are just generalizations to the realm of nonlinear dynam- ies of questions dealt with in state-space formulations of linear contol systems ‘Both the LPS paper and this one provide an implemen- tation of the same basic idea. We each use time-delay coordinates and attribute the disappearance of false neighbors as an indication of a minimum embedding di mension for the data. From this point forward out ap- proaches are diferent. The LPS technique uses. small heighbochoods whereas. we use individval neighbors They then compute two distances, Dy (mind) and Dyyilnind +1). Dey nin) is the distance between yin) and its rth neighbor. For this case the calculation that determines which isa near neighbor is performed in 4 dimensions, while the calculation of the distance be- tween neighbors is performed in (d-+1) dimensions Dyeilnjnd +1) is also the distance between yln) and its rth ‘neighbor. However, for this case both the aleulation that determines which isa near neighbor and the calculation ofthe distance to that near neighbor are performed in’ (d-+1) dimensions. ‘The ratio Dy yinind)/Dyinind +1)=Q, defines their quanti- 9, LPS then goon to define Qs via Q =Dainsr.d)/Dylnsr.d +1). In this case the ealeula- tions for distances between neighbors are performed in d dimensions. The calculation ‘used to identify the 7th nearest neighbor is performed in d dimensions for the rumerator and (d-+1) dimensions for the denominator. LPS then examine the geometric mean of the product, 010; over a neighborhood (in their reported case the3410 first ten nearest neighbors). Our first criterion, Eq. (4), can be seen as the ratio of part of their Q, to part of their Qs. LPS's statistic, called W, is the logarithm of the ar- ithmetical average over neighborhoods of the individual geometric means. For the purposes of finding @ minimum embedding di- mension our method and that of LPS use the same under- lying principle; however, the statistic and interpretation in our case are simpler.’ The proportion of false neigh- bors is on an absolute scale, always bounded between 0 and 1, whereas with LPS it is not immediately clear what constitutes a sufficiently small statistic unless the conver- zzence is sharp. If one sees a plateau of 0.1% false neigh- ‘bors with increasing embedding dimension, in contrast to perhaps a plateau at 10%, one could be rather confident that a good reconstruction of a clean chaotic attractor hhas been accomplished. LPS did not examine the effect of noise on their algorithm, but in @ brief comparision we found that on pure uniform noise, their statistic contin- ued to decrease with larger embedding dimension, whereas ours plateaued at a comparatively high level This is not surprising as their statistic uses the same formation as ours, but without the additional criterion that we found necessary to guard against pure noise. For moderate noise levels (10%) added to the Lorenz-II at- tractor, the LPS statistic did not have a definite plateau with increasing embedding dimension as did the false- neighbors criterion, and so we found the identification of ‘minimum embedding dimension easier to make in the latter case. LPS also use their statistic to choose the time lag for embedding. They define a new quantity W=W/T and search for a minimum in Was a function of T. We admit {0 not fully understanding the motivation behind the division by 7. The W quantity alone gives the general be- hhavior that we intuitively expect, and also observe with the false-neighbors statistic. For large T, the proportion of false neighbors and also the W statistic increase, be- cause the attractor looks more like noise, as the clements of the state-space vector become more decorrelated due {o the positive Lyapunov exponent. For suficiently small time lags, the attractor eventually collapses onto a one- dimensional object, and so the estimated embedding ‘mension will be spuriously low. The division by T ap- pears to be a way of ameliorating the latter problem, if fone then ignores new minima in W created at large T by the division. In contrast to the case for embedding dimension, the theorems do not define a “correct” time delay to use, rather one must choose a condition that simply gives reasonable results given a finite amount of data at a cer- tain sampling rate, and the LPS criterion is another one to do so. Our method, as it is attuned specifically to the ross errors created by an improper embedding, appears in practice to be less sensitive to time delay than LPS's method. This fact may be either a virtue or a viee, de- pending on one’s outlook. ‘Our example of the Ikeda map also demonstrates that the choice of embedding dimension does very litle to climinate the problem of spurious Lyapunov exponents as LPS suggest it might. With time delay embedding di- KENNEL, BROWN, AND ABARBANEL 45 mensions larger than that of the “true” dynamics, one must still identify, by dynamical means, the Lyapunov exponents which describe the system and not the artifacts of time-delay embedding by Euclidian coordinates {15}. ‘The BLPC paper is closer in spirit to ours than the LPS work. BLPC identify a quantity ele) which is the average over the attractor of the value of [x(n-tdT)—x(m +47) for all pairs of points (labeled by mand m) with y(n) and ym) having all components within a distance ¢ of each other divided by the number of such points on the orbit. This is essentially the average over the attractor of what we have called RJ (nr)=Riin,r) for all neighbors within a sphere of radius €V@ of the data point yim). They then display @ 4c) for various dimensions d with various selections of the neighborhood size € and the noise level as added to the equations of evolution of the system. When there is a sharp break in the curve of this quantity versus dimen- sion, they argue this dimension should be chosen as the embedding dimension. For dimensions above this break, the value of o(€) should be of order ©. They also state that this minimum embedding dimension dp is related to the fractal dimension of the attractor dy by dp=1+ finteger part of (d ) ‘Tn general idea ths is quite close to what we have done. BLPC must choose a neighborhood size, of course, and as far as we can see from their work they did not systemati- cally investigate the effect this size has on their results. They also always work with 4096 data points, and we suspect this is likly to be sufficient for the data they in- vestigated. We must choose a value for the tolerance lev els we use in our criteria for false nearest neighbors, but wwe have demonstrated the independence of this choice over a very wide range of values. By choosing a neigh- bothood size, apparently in a fashion unrelated to the size of the attractor, BLPC in effect determine how many neighbors will be included as they move around the at- tractor. This number will vary aS one moves about the attractor since the neighborhood size is fixed and the den: sity on the attractor is inhomogeneous. One can see from the data of BLPC that the value of a j(€) does not reach down to € above dy since they are, in some fashion, counting points with true neighbors within ball of radius eVd more than once. This is not severe really, but the estimate of when one has reached dy could be spoofed by this when there is noise in the data. Also we comment that the example of the Ikeda map we present here shows that their suggested connection between dy and d , is not correct in general. Choosing neighborhood sizes and asking when points stop moving out of those neighborhoods, and this s the essence of the BLPC algo- rithm, entails additional work and computing cost over our procedure using nearest neighbors only. This is be- cause points are likely to appear in many neighborhoods, especially when the neighborhood size is not small, as is the case in several of the BLPC examples. Finally the simplicity of the computations required for our implementation of the false-neighbor idea coupled with the ability to estimate (quantitatively and directly) the error implied by various choices of embedding dimen- sions should allow the use of our procedures in a large4s DETERMINING EMBEDDING DIMENSION FOR PHASE-SPACE, variety of settings. Indeed, we have successfully applied this method to experimental data [30] from several sources, and we have been able to determine, with no trouble and quite efficiently, minimum embedding dimen- sions even in the presence of contamination by noise. ACKNOWLEDGMENTS We thank the members of INLS and the members of the UCSD-MIT-Lockheed-Sanders Nonlinear Signal (1]J-P, Bekmann and D. Ruelle, Rev. Mod Phys. $7, 617 1985) (2)T. 8. Parker and L. ©. Chua, Practical Numerical Algo- rithms for Chaotic Systems (Springer-Verlag, New York, 1990), [3] P.S. Landa and M. Rosenblum, Physica D 48, 232 (1991) [4JR. Mané, in Dynamical Systems and ' Turbulence, Warwick, 1980, edited by D. Rand and L. S. Young, Lec ture Notes in Mathematics 898 (Springer, Berlin, 1981), p. 366. [S]F. Takens, in Dynamical Systems and Turbulence, Warvick, 1980, edited by D. Rand and L. $. Young, Lec- ture Notes in Mathematis 898 (Springer, Berlin, 1981), p 366, [6A recent revisit to the theorem of Mané and Takens is found in Tim Sauer, J. A. Yorke, and M. Casdagli (unpub- lished. (TE. N. Lorenz, J. Atmos. Sci 20, 130 (1963). We refer to this asthe Loren2-1 model. [8] M. Hénon, Commun. Math. Phys. 5, 69 (1976) [9] W. Liebert, K. Pawelzik, and H. G. Schuster, Europhys, Lett, 14, 521 (1990). [10]. Bumeliene, G. Lasiene, K. Pyragas, and A. Cenys, Li: toy. Fiz. Sbornik 28, 569 (1988) [11] K. Pyragas and A! Cenys, Litov. Fiz. Sbornik 27, 437 (1989) [12] A. M, Fraser and H. L. Swinney, Phys. Rev. A 33, 1134 (1986), A. M. Fraser, IEEE Trans. Info. Theory 35, 245 (1989); A.M. Fraser, Physica D 34, 391 (1985). [13] W. Liebert and H. G. Schuster, Phys. Lett. A 142, 107 (1989), This paper evaluates the average mutual informs tion function of the previous reference by an efficient mux ‘merical algorithm. [14] H, D. 1, Abarbanel, R. Brown, and J. B. Kadtke, Phys. Rev. A 138, 1782 (1990); M. C. Casdagli, Physica D 35, aan Processing Group for numerous discussions on this sub- ject; the comments and assistance of Steve Isabelle, Doug Mook, Cory Myers, Al Oppenheim, and J. J. Sidorowich were especially helpful. The data on the Mackey-Glass system were provided to us by Sidorowich. This work was supported in part under the DARPA-University Research Initiative, URI Contract No. N00014-86-K- 0758, in part by the U.S. Department of Energy, Office of, Basic Energy Sciences, Division of Engineering and Geosciences, under Contract No. DE-FG03-90ER 14138, 335 (1989); J. D. Farmer and J.J. Sidorowich, Phys. Rev. Lett. 59, 845 (1987, [15] R. Brown, P. Bryant, and H. D, I. Abarbanel, Phys. Re. ‘A-43, 2787 (1991); Phys, Rev. Lett 68, 1523 (1990). Also see the review article by H. D. I. Abarbanel, R. Brown, and M. B. Kennel, Int. J. Mod. Phys. B 5, 1347 (1991). [16]J-P. Eckman, $: 0. Kamphorst, D. Ruelle, and S. Cili- bberto, Phys. Rev. A 34, 4971 (1986). [IT] EN. Lorenz, Tellus 364, (1988). We refer to this as the Lorenz-II model, [18] H. D.1 Aberbanel, R. Brown, and M. B. Kennel, J. Non- linear Sei. 1, 175 (1990), [19] K. tkeda, Opt. Commun. 30,257 (1979). [20]. M. Hammel, C. K. R. T. Jones, and J. V. Moloney, J. ‘Opt. Soc. Am B 2, 52 (1985), (21] 0. E. Réssler, Phys. Lett. A 57, 397 (1976) [22] M. Mackey and L. Glass, Science 197, 287 (1977) (23]J. H. Friedman, J, L. Bentley, and R. A. Finkel, ACM ‘Trans. Math. Sofware 3, 209 (1970. (24). M. Hammel, Phys. Lett. A 148, 421 (1990) [23]J. D. Farmer and J. J. Sidorowich, Physica D 47, 373, agen, [26] H. D.1. Abarbanel, R. Brown, 8. M. Hammel, PF. Mar- teau, and J.J. Sidorowich (unpublished) [27] P-F. Marteau and H. D. I. Abarbanel, J. Nonlinear Sci 1, 313 (1990), th and J. A. Yorke, Physica D 41, 183 (1990) Shand J. A. Yorke, Physica D 41,183 (1990) [29] N. Packard er al., Phys. Rev. Lett 45, 9 (1980) (0) We are grateful t0 T, Carroll and L, Pecora from the US. Naval Research Laboratory for providing us with their data. The results of this analysis using the tool of false nearest neighbors and other analysis will appear in the near future.