Download the full version of the textbook now at textbookfull.

com

Analysis and Data Based Reconstruction of

Complex Nonlinear Dynamical Systems Using the
Methods of Stochastic Processes M. Reza
Rahimi Tabar
https://textbookfull.com/product/analysis-and-
data-based-reconstruction-of-complex-nonlinear-
dynamical-systems-using-the-methods-of-stochastic-
processes-m-reza-rahimi-tabar/

Explore and download more textbook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Complex Dynamical Systems in Education Concepts Methods

and Applications 1st Edition Matthijs Koopmans

https://textbookfull.com/product/complex-dynamical-systems-in-
education-concepts-methods-and-applications-1st-edition-matthijs-
koopmans/
textbookfull.com

Analysis and Control of Output Synchronization for Complex

Dynamical Networks Jin-Liang Wang

https://textbookfull.com/product/analysis-and-control-of-output-
synchronization-for-complex-dynamical-networks-jin-liang-wang/

textbookfull.com

Dynamic Modeling of Complex Industrial Processes Data

driven Methods and Application Research 1st Edition Chao
Shang (Auth.)
https://textbookfull.com/product/dynamic-modeling-of-complex-
industrial-processes-data-driven-methods-and-application-research-1st-
edition-chao-shang-auth/
textbookfull.com

The New Poetics of Climate Change Modernist Aesthetics for

a Warming World Matthew Griffiths

https://textbookfull.com/product/the-new-poetics-of-climate-change-
modernist-aesthetics-for-a-warming-world-matthew-griffiths/

textbookfull.com
Information and Communication Technologies in Education
Research and Industrial Applications 11th International
Conference ICTERI 2015 Lviv Ukraine May 14 16 2015 Revised
Selected Papers 1st Edition Vitaliy Yakovyna
https://textbookfull.com/product/information-and-communication-
technologies-in-education-research-and-industrial-applications-11th-
international-conference-icteri-2015-lviv-ukraine-
may-14-16-2015-revised-selected-papers-1st-edition-v/
textbookfull.com

Brute Force A Rough Sci Fi Alien Romance 1st Edition Lizzy

Bequin Bequin Lizzy

https://textbookfull.com/product/brute-force-a-rough-sci-fi-alien-
romance-1st-edition-lizzy-bequin-bequin-lizzy/

textbookfull.com

Suicide by Democracy an Obituary for America and the World

3rd Edition Michael Starks

https://textbookfull.com/product/suicide-by-democracy-an-obituary-for-
america-and-the-world-3rd-edition-michael-starks/

textbookfull.com

A Portable Cosmos: Revealing the Antikythera Mechanism,

Scientific Wonder of the Ancient World 1st Edition
Alexander Jones
https://textbookfull.com/product/a-portable-cosmos-revealing-the-
antikythera-mechanism-scientific-wonder-of-the-ancient-world-1st-
edition-alexander-jones/
textbookfull.com

Graphic Medicine Manifesto 1st Edition Mk Czerwiec

https://textbookfull.com/product/graphic-medicine-manifesto-1st-
edition-mk-czerwiec/

textbookfull.com
The Longest Run a Ski Patrol Rescue Romance Lawson Family
Book 4 1st Edition Tracey Cramer-Kelly [Cramer-Kelly

https://textbookfull.com/product/the-longest-run-a-ski-patrol-rescue-
romance-lawson-family-book-4-1st-edition-tracey-cramer-kelly-cramer-
kelly/
textbookfull.com
Understanding Complex Systems

M. Reza Rahimi Tabar

Analysis and
Data-Based
Reconstruction of
Complex Nonlinear
Dynamical Systems
Using the Methods of Stochastic
Processes
Springer Complexity
Springer Complexity is an interdisciplinary program publishing the best research and
academic-level teaching on both fundamental and applied aspects of complex systems—cutting
across all traditional disciplines of the natural and life sciences, engineering, economics,
medicine, neuroscience, social and computer science.
Complex Systems are systems that comprise many interacting parts with the ability to
generate a new quality of macroscopic collective behavior the manifestations of which are
the spontaneous formation of distinctive temporal, spatial or functional structures. Models
of such systems can be successfully mapped onto quite diverse “real-life” situations like
the climate, the coherent emission of light from lasers, chemical reaction-diffusion systems,
biological cellular networks, the dynamics of stock markets and of the internet, earthquake
statistics and prediction, freeway traffic, the human brain, or the formation of opinions in
social systems, to name just some of the popular applications.
Although their scope and methodologies overlap somewhat, one can distinguish the
following main concepts and tools: self-organization, nonlinear dynamics, synergetics,
turbulence, dynamical systems, catastrophes, instabilities, stochastic processes, chaos, graphs
and networks, cellular automata, adaptive systems, genetic algorithms and computational
intelligence.
The three major book publication platforms of the Springer Complexity program are the
monograph series “Understanding Complex Systems” focusing on the various applications of
complexity, the “Springer Series in Synergetics”, which is devoted to the quantitative theo-
retical and methodological foundations, and the “Springer Briefs in Complexity” which are
concise and topical working reports, case studies, surveys, essays and lecture notes of rele-
vance to the field. In addition to the books in these two core series, the program also incor-
porates individual titles ranging from textbooks to major reference works.

Series Editors

Henry D. I. Abarbanel, Institute for Nonlinear Science, University of California, San Diego, La Jolla, CA, USA
Dan Braha, New England Complex Systems Institute, University of Massachusetts, Dartmouth, USA
Péter Érdi, Center for Complex Systems Studies, Kalamazoo College, USA and Hungarian Academy of
Sciences, Budapest, Hungary
Karl J. Friston, Institute of Cognitive Neuroscience, University College London, London, UK
Hermann Haken, Center of Synergetics, University of Stuttgart, Stuttgart, Germany
Viktor Jirsa, Centre National de la Recherche Scientifique (CNRS), Université de la Méditerranée, Marseille,
France
Janusz Kacprzyk, Polish Academy of Sciences, Systems Research Institute, Warsaw, Poland
Kunihiko Kaneko, Research Center for Complex Systems Biology, The University of Tokyo, Tokyo, Japan
Scott Kelso, Center for Complex Systems and Brain Sciences, Florida Atlantic University, Boca Raton, USA
Markus Kirkilionis, Mathematics Institute and Centre for Complex Systems, University of Warwick, Coventry,
UK
Jürgen Kurths, Nonlinear Dynamics Group, University of Potsdam, Potsdam, Germany
Ronaldo Menezes, Department of Computer Science, University of Exeter, UK
Andrzej Nowak, Department of Psychology, Warsaw University, Warszawa, Poland
Hassan Qudrat-Ullah, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
Linda Reichl, Center for Complex Quantum Systems, University of Texas, Austin, USA
Peter Schuster, Theoretical Chemistry and Structural Biology, University of Vienna, Vienna, Austria
Frank Schweitzer, System Design, ETH Zürich, Zürich, Switzerland
Didier Sornette, Entrepreneurial Risk, ETH Zürich, Zürich, Switzerland
Stefan Thurner, Section for Science of Complex Systems, Medical University of Vienna, Vienna, Austria
Understanding Complex Systems
Founding Editor: S. Kelso

Future scientific and technological developments in many fields will necessarily

depend upon coming to grips with complex systems. Such systems are complex in
both their composition–typically many different kinds of components interacting
simultaneously and nonlinearly with each other and their environments on multiple
levels–and in the rich diversity of behavior of which they are capable.
The Springer Series in Understanding Complex Systems series (UCS) promotes
new strategies and paradigms for understanding and realizing applications of
complex systems research in a wide variety of fields and endeavors. UCS is
explicitly transdisciplinary. It has three main goals: First, to elaborate the concepts,
methods and tools of complex systems at all levels of description and in all scientific
fields, especially newly emerging areas within the life, social, behavioral, economic,
neuro- and cognitive sciences (and derivatives thereof); second, to encourage novel
applications of these ideas in various fields of engineering and computation such as
robotics, nano-technology, and informatics; third, to provide a single forum within
which commonalities and differences in the workings of complex systems may be
discerned, hence leading to deeper insight and understanding.
UCS will publish monographs, lecture notes, and selected edited contributions
aimed at communicating new findings to a large multidisciplinary audience.

More information about this series at http://www.springer.com/series/5394

M. Reza Rahimi Tabar

Analysis and Data-Based

Reconstruction of Complex
Nonlinear Dynamical
Systems
Using the Methods of Stochastic Processes

123
M. Reza Rahimi Tabar
Department of Physics
Sharif University of Technology
Tehran, Iran

ISSN 1860-0832 ISSN 1860-0840 (electronic)

Understanding Complex Systems
ISBN 978-3-030-18471-1 ISBN 978-3-030-18472-8 (eBook)
https://doi.org/10.1007/978-3-030-18472-8
© Springer Nature Switzerland AG 2019
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
In memory of Professor Rudolf Friedrich
(1956–2012) whose original and innovative
thinking with Professor Joachim Peinke on
the methods that we describe in this book
motivated a large body of work by others on
the subject.
Preface

The data analysis of physical observables has a long tradition in the field of
nonlinear dynamics and complex systems. Much effort has been devoted to
answering the question of how to extract a “deterministic” dynamical system from a
suitable analysis of experimental data, given that an appropriate analysis can yield
important information on dynamical properties of the system under consideration.
Fluctuations in these time series are usually considered as a purely random or
uncorrelated variable, which is additively superimposed on a trajectory generated
by a deterministic dynamical system. The problem of dynamical noise, i.e., fluc-
tuations that interfere with the dynamical evolution, has not been addressed in much
detail, although it is of utmost importance for the analysis of fluctuating time series.
This book focuses on a central question in the field of complex systems: Given a
fluctuating (in time or space), uni- or multi-variant sequentially measured set of
experimental data (even noisy data), how should one analyse non-parametrically
the data, assess underlying trends, uncover characteristics of the fluctuations
(including diffusion and jump contributions), and construct a stochastic evolution
equation? Here, the term “non-parametrically” exemplifies that all the functions and
parameters of the constructed stochastic evolution equation can be determined
directly from the measured data.
In recent years, significant progress has been made when addressing this ques-
tion to the classes of continuous stochastic processes and of processes with jump
discontinuities. These can be modeled by nonlinear generalized Langevin equations
that include additive as well as multiplicative diffusive and even jump parts. An
important building block for the analysis approach presented in this book is a
Markovian property, which can be detected in real systems above a certain time or
length scale. This scale is referred to as the Markov–Einstein scale, and has turned
out to be an important characteristic of complex time series. The Markov–Einstein
time scale is the minimum scale above which the data can be considered as a
Markov process, and one can estimate it directly from observations. The main
advantage of the analysis approach is that is completely data-based and thus allows
one to find all functions and parameters of the modeling directly from measured
data. Due to its feasibility and simplicity, it has been successfully applied to

vii
viii Preface

fluctuating time series and spatially disordered structures of complex systems

studied in scientific fields such as physics, astrophysics, meteorology, earth science,
engineering, finance, medicine, and the neurosciences, and has led to many
important results.
This book provides an overview of methods that have been developed for the
analysis of fluctuating time series and of spatially disordered structures.
The book also offers numerical and analytical approaches to the analysis of
complex time series that are most common in the physical and natural sciences. It is
self-contained and readily accessible to students, scientists, and researchers who are
familiar with traditional methods of mathematics, such as ordinary, and partial
differential equations. Codes for analysing continuous time series are available in an
R package developed under the supervision of Joachim Peinke by the research
group Turbulence, Wind energy, and Stochastics (TWiSt) at the Carl von Ossietzky
University of Oldenburg. This package allows one to extract the (stochastic) evo-
lution equation underlying a set of data or measurements.
The book is divided into three main parts: I (Chaps. 1–9), II (Chaps. 10–21), and
III (Chaps 22–23).
Chapter 1 provides an introduction and an overview of topics covered in this
book. Chapter 2 reviews essentials of stochastic processes, namely the statistical
description of stochastic processes, stationary processes, classification of stochastic
processes, the Chapman–Kolmogorov equation as a necessary condition for
Markov processes, statistical continuous processes, as well as stochastic processes
in the presence of jump discontinuities. In Chap. 3, we present details of the
Kramers–Moyal expansion, the Pawula theorem, the Fokker–Planck equation and
its short-term propagator, and derive the master equation from the Chapman–
Kolmogorov equation. In Chap. 4, we provide Lindeberg’s condition for the con-
tinuity of stochastic trajectories. It is shown that the Fokker–Planck equation
describes a continuous stochastic process. We derive the stationary solutions of the
Fokker–Planck equation and define a potential function for dynamics. In Chap. 5,
we introduce the Langevin equation and Wiener processes along with their statis-
tical properties. Chapter 6 reviews the Itô and the Stratonovich calculus. We prove
Itô’s lemma and describe the Itô calculus for multiplicative noise. In Chap. 7, we
show the equivalence between the Langevin approach and the Fokker–Planck
equation and derive equations for statistical moments of a process whose dynamics
is given by the Langevin equation. In Chap. 8, we provide examples for stochastic
calculus using the Kubo–Anderson process, the Ornstein–Uhlenbeck process, and
the Black–Scholes process (or geometric Brownian motion). Chapter 9 covers the
following topics: Langevin dynamics in higher dimension, the Fokker–Planck
equation in higher dimension, finite-time propagators of a d-dimensional Fokker–
Planck equation, as well as discrete time evolution and discrete time approximation
of stochastic evolution equations. Chapters 1–9 can be skipped by readers who are
familiar with the standard notions of stochastic processes, or they may be useful for
examples and applications.
Preface ix

In Chap. 10, we introduce the Lévy noise-driven Langevin equation and the
fractional Fokker–Planck equations, derive the short-time propagator of Lévy
noise-driven processes, and provide limit theorems for Wiener and Lévy processes.
Finally, a non-parametric determination of Lévy noise-driven Langevin dynamics
from time series will be described. In Chap. 11, we study stochastic processes with
jump discontinuities and discuss the meaning of nonvanishing higher-order
Kramers–Moyal coefficients. We address in detail the physical meaning of
non-vanishing fourth-order Kramers–Moyal coefficients, stochastic processes with
jumps, as well as stochastic properties and statistical moments of Poisson jump
processes. In Chap. 12, we introduce the jump-diffusion processes with Gaussian
and mixed-Gaussian jumps. In Chap. 13, we introduce bi-variant jump-diffusion
equations and in Chap. 14, we describe different numerical schemes for the inte-
gration of Langevin and jump-diffusion stochastic differential equations, such as the
Euler–Maruyama scheme, the Milstein scheme, and Runge-Kutta-like methods.
This chapter closes with an introduction of packages in R and Python for the
numerical integration of stochastic differential equations. In Chap. 15, we discuss
the analysis of spatially disordered structures and provide a physical picture
of the fluctuation cascade from large to small scales. Moreover, this section
introduces the multipliers in cascade processes, and we derive a scale-dependent
solution of the Fokker–Planck equation and present the Castaing equation.
An answer to the question of how to set up stochastic equations for real-world
processes is presented in Chaps. 16–21. In Chap 16, the reader is familiarized with
the methods for estimating the Kramers–Moyal coefficients, and we introduce the
Markov–Einstein time (length) scale of a data set. This chapter also contains
important technical aspects of the method for estimating drift and diffusion coef-
ficients as well as higher-order Kramers–Moyal coefficients from time series. In
Chap. 17, we explain how to derive the Kramers–Moyal coefficients from
non-stationary time series using the Nadaraya–Watson estimator and we investigate
Kramers–Moyal coefficients in the presence of microstructure (measurement) noise.
In Chap. 18, we study the influence of a finite time step on the estimation of the
Kramers–Moyal coefficients from diffusive and jumpy data. In Chap. 19, we ana-
lytically derive a criterion (as a necessary condition) that allows one to check
whether for a given, even noisy time series the underlying process has a continuous
(diffusive) or a discontinuous (jumpy) trajectory. In Chap. 20, the steps of deriving
a Langevin equation from diffusive experimental time series are given, and we
finish the chapter with an introduction of an R Package for the modeling of one- and
two-dimensional continuous stochastic processes. In addition, the steps for deriving
a jump-diffusion stochastic equation from experimental time series with jumps are
presented. Also, two other methods for a reconstruction of time series will be
reviewed shortly. In Chap. 21 we reconstruct, as examples, some stochastic
dynamical equations from various synthetic continuous time series, from time series
with jump discontinuities and from time series generated by Lévy noise-driven
Langevin dynamics.
x Preface

Chapter 22 briefly reviews applications of the presented method (Chaps. 16–21)

to the analysis of real-world time series and ends with an outlook. As an example
of the analysis methods to real-world time series, we present in Chap. 23 results
derived from analyses of electroencephalographic time series.
I would like to thank N. Abedpour, M. Anvari, A. Barhraminasab, D. Bastine,
F. Böttcher, J. Davoudi, F. Ghasemi, J. Gottschall, Z. Fakhraai, S. M. Fazeli,
J. Friedrich, T. Jadidi, G. R. Jafari, A. Hadjihosseini, N. Hamedai-Raja,
A. M. Hashtroud, H. Heibelmann, J. Heysel, M. Hölling, C. Honisch, O. Kamps,
D. Kleinhans, M. Kohandel, P. G. Lind, G. Lohmann, St. Lück, P. Manshour,
P. Milan, E. Mirzahossein, S. Moghimi, S. M. Mousavi, S. M. S. Movahed,
I. Neunaber, M. D. Niry, J. Puczylowski, N. Reinke, Ch. Renner, P. Rinn,
V. Rezvani, F. Shahbazi, A. Sheikhan, M. Siefert, S. Siegert, F. T. Shahri,
S. M. Vaez Allaei, M. Wächter, L. Zabawa, and F. Zarei for useful discussions and
whose Ph.D. theses have contributed to this book. I am also thankful to Daniel
Nickelsen and Adrian Baule for sharing their ideas presented in Sects. 22.2.3
and 22.2.5.
Special acknowledgments should be given to Uriel Frisch (Observatoire de la
Côte d’Azur, Nice), Joachim Peinke (Carl-von-Ossietzky University Oldenburg),
Muhammad Sahimi (University of Southern California), and Holger Kantz (Max–
Planck Institute for Physics of Complex Systems, Dresden). I would also like to
thank the Alexander von Humboldt Foundation for financial support and Klaus
Lehnertz (Department of Epileptology, University of Bonn) for many detailed
discussions, the kind hospitality, and proofreading of Chap. 23.
It is greatly appreciated if readers could forward any errors, misprints, or
suggested improvements to: [email protected] or [email protected].

Oldenburg, Bonn M. Reza Rahimi Tabar

2019
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Time Series of Complex Systems . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Stochastic Continuous Time Series . . . . . . . . . . . . . . . . . . . . . 2
1.3 Time Series with Jump Discontinuity . . . . . . . . . . . . . . . . . . . . 4
1.4 Microstructural (Measurement) Noise . . . . . . . . . . . . . . . . . . . . 5
1.5 Intermittency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Introduction to Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Statistical Description of Time Series . . . . . . . . . . . . . . . . . . . . 10
2.2.1 The Probability Density . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.2 Joint and Conditional Probability Distribution
Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Classification of Stochastic Processes . . . . . . . . . . . . . . . . . . . . 12
2.3.1 Purely Random Processes . . . . . . . . . . . . . . . . . . . . . . 13
2.3.2 Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.3 Higher Order Processes . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 Stationary Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 The Chapman–Kolmogorov Equation and the Necessary
Condition for a Process to Be Markov . . . . . . . . . . . . . . . . . . . 15
2.6 Continuous Stochastic Markov Processes . . . . . . . . . . . . . . . . . 16
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 Kramers–Moyal Expansion and Fokker–Planck Equation ....... 19
3.1 Kramers–Moyal Expansion . . . . . . . . . . . . . . . . . . . . ....... 19
3.2 Pawula Theorem and Fokker–Planck Equation . . . . . . ....... 21
3.3 Short-Time Propagator of Fokker–Planck Equation
in One Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... 22

xi
xii Contents

3.4 Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Continuous Stochastic Processes . . . . . . . . . . . . . . . . . . . . ....... 31
4.1 Stochastic Continuity . . . . . . . . . . . . . . . . . . . . . . . . ....... 31
4.1.1 Stochastic Mean-Square Continuity . . . . . . . . ....... 31
4.1.2 Lindeberg’s Continuity Condition for Markov
Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... 32
4.2 Stochastic Differentiability . . . . . . . . . . . . . . . . . . . . . ....... 33
4.2.1 Mean-Square Differentiability of Stochastic
Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... 33
4.2.2 General Condition for Non-differentiability
of Stochastic Processes . . . . . . . . . . . . . . . . . ....... 34
4.3 Description of a Continuous Stochastic Process
by a Fokker–Planck Equation . . . . . . . . . . . . . . . . . . ....... 34
4.4 Stationary Solution of the Fokker–Planck Equation
and the Potential Function . . . . . . . . . . . . . . . . . . . . . ....... 35
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... 37
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... 37
5 The Langevin Equation and Wiener Process . . . . . . . . . . . . . . ... 39
5.1 The Langevin Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 39
5.2 The Kramers–Moyal Coefficients of Wiener Process . . . . . . ... 40
5.3 Conditional Probability Distribution Function of the Wiener
Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.4 Statistical Moments of the Wiener Process . . . . . . . . . . . . . . . . 41
5.5 Markov Property of the Wiener Process . . . . . . . . . . . . . . . . . . 42
5.6 Independence of Increments of the Wiener Process . . . . . . . . . . 43
5.7 The Correlation Function of the Wiener Process . . . . . . . . . . . . 43
5.8 Wiener Process Is Not Differentiable . . . . . . . . . . . . . . . . . . . . 44
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6 Stochastic Integration, Itô and Stratonovich Calculi . . . . . . . . . . . . 49
6.1 Stochastic Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.2 Nonanticipating Function and Itô Lemma . . . . . . . . . . . . . . . . . 53
6.2.1 Itô or Stratonovich . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.3 Integration of Polynomial and Examples of Itô Calculus . . . . . . 54
6.4 Itô Calculus for Multiplicative Noise and Itô-Taylor
Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...... 56
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...... 57
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...... 60
Contents xiii

7 Equivalence of Langevin and Fokker–Planck Equations . . . . . .... 61

7.1 Probability Distribution Functions of Langevin Dynamics . .... 61
7.2 Equation for Statistical Moments Based on the Langevin
Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.3 Existence of Solutions to Langevin Equation . . . . . . . . . . . . . . 64
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
8 Example of Stochastic Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
8.1 Anderson–Kubo Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
8.2 Ornstein–Uhlenbeck Process . . . . . . . . . . . . . . . . . . . . . . . . . . 71
8.3 Black–Scholes Process, or Geometric Brownian Motion . . . . . . 73
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
9 Langevin Dynamics in Higher Dimensions . . . . . . . . . . . . . . . . . . . 79
9.1 d-Dimensional Langevin Dynamics . . . . . . . . . . . . . . . . . . . . . 79
9.2 The Fokker–Planck Equation in Higher Dimensions . . . . . . . . . 80
9.3 The Kramers–Moyal Expansion in Higher Dimensions . . . . . . . 81
9.4 Discrete Time Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
9.4.1 Proper Langevin Equations: White Noise . . . . . . . . . . . 81
9.5 Discrete-Time Approximation of Stochastic Evolution
Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 82
9.6 Short-Time Propagators of d-Dimensional Fokker–Planck
Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 83
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 84
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 86
10 Lévy Noise-Driven Langevin Equation and Its Time Series–Based
Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
10.1 Langevin Equation with Lévy Noise . . . . . . . . . . . . . . . . . . . . 87
10.1.1 Lévy Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
10.1.2 Fractional Fokker–Planck Equations . . . . . . . . . . . . . . 91
10.2 Discrete Time Approximation of Langevin Equations:
Lévy Noise-Driven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 91
10.3 Short-Time Propagator of the Lévy Noise-Driven Langevin
Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 92
10.4 Joint Probability Distribution and Markovian Properties . . . . .. 92
10.5 Limit Theorems, and Wiener and Lévy Processes . . . . . . . . . .. 93
10.6 Reconstruction of Lévy Noise-Driven Langevin Dynamics
by Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 95
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 97
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 97
xiv Contents

11 Stochastic Processes with Jumps and Non-vanishing

Higher-Order Kramers–Moyal Coefficients . . . . . . . . . . . . . . . .... 99
11.1 Continuous Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . 100
11.2 Non Vanishing Higher Order Kramers–Moyal Coefficients
and the Continuity Condition . . . . . . . . . . . . . . . . . . . . . . . . . . 101
11.3 Stochastic Properties of Poisson Process . . . . . . . . . . . . . . . . . . 102
11.4 Statistical Moments of Poisson Process . . . . . . . . . . . . . . . . . . 105
11.5 The Process in Presence of Jumps, Pure Poisson Process
as an Example, Kramers–Moyal Coefficients . . . . . . . . . . . . . . 106
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
12 Jump-Diffusion Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
12.1 Kramers–Moyal Coefficients of Jump-Diffusion Processes . . . . . 111
12.2 Gaussian Distributed Jump Amplitude . . . . . . . . . . . . . . . . . . . 114
12.3 Mixed Gaussian Jumps—The Variance Gamma Model . . . . . . . 115
12.4 Jump-Drift Process, Stochastic Solution, Example . . . . . . . . . . . 117
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
13 Two-Dimensional (Bivariate) Jump-Diffusion Processes . . . . . . . . . 123
13.1 Bivariate Jump-Diffusion Processes . . . . . . . . . . . . . . . . . . . . . 123
13.2 Kramers–Moyal Coefficients for Jump-Diffusion Processes
in Two Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
14 Numerical Solution of Stochastic Differential Equations:
Diffusion and Jump-Diffusion Processes . . . . . . . . . . . . . . . . . . . . . 129
14.1 Numerical Integration of Diffusion Processes . . . . . . . . . . . . . . 129
14.1.1 Euler–Maruyama Method . . . . . . . . . . . . . . . . . . . . . . 129
14.1.2 Milstein Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
14.1.3 Runge–Kutta-Like Methods . . . . . . . . . . . . . . . . . . . . 132
14.2 Numerical Integration of Jump-Diffusion Processes . . . . . . . . . . 133
14.2.1 Euler–Maruyama Method . . . . . . . . . . . . . . . . . . . . . . 133
14.2.2 Milstein Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
14.3 Stochastic Differential Equation: Packages in R and Python . . . . 135
14.3.1 An R Package (Langevin) for Numerical Solving
and Modeling of Univariate and Bivariate “Diffusion
Processes” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
14.3.2 Simulation of Diffusion Processes, R-Package
“Sim.DiffProc” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
14.3.3 Simulation of Diffusion Processes, R-Package
“DiffusionRimp” . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Contents xv

14.3.4 Simulation of Diffusion Processes, R-Package

“yuima” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
14.3.5 Simulation of Jump-Diffusion Processes,
Python-Solver “JiTCSDE” . . . . . . . . . . . . . . . . . . . . . 136
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
15 The Friedrich–Peinke Approach to Reconstruction of Dynamical
Equation for Time Series: Complexity in View of Stochastic
Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
15.1 Stochastic Processes in (Length or Time) Scale . . . . . . . . . . . . 143
15.1.1 Increments of Stochastic Processes . . . . . . . . . . . . . . . 143
15.1.2 Fractal and Multifractal Time Series: Linear
and Nonlinear Processes . . . . . . . . . . . . . . . . . . . . . . . 144
15.2 Intermittency and Kramers–Moyal Expansion . . . . . . . . . . . . . . 146
15.2.1 Governing Equation for the Statistical Moments
in Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
15.3 Fokker–Planck Equation and (Multifractal) Scaling
Exponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
15.4 Langevin and Jump-Diffusion Modeling in Scale . . . . . . . . . . . 149
15.5 Multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
15.6 Scale Dependent Solution of Fokker–Planck Equation . . . . . . . . 151
15.7 The Castaing Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
15.8 Multiscale Correlation Functions . . . . . . . . . . . . . . . . . . . . . . . 156
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
16 How to Set Up Stochastic Equations for Real World Processes:
Markov–Einstein Time Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
16.1 From Time Series to Stochastic Dynamical Equation . . . . . . . . 165
16.2 Markov–Einstein Time (Length) Scale . . . . . . . . . . . . . . . . . . . 166
16.3 Evaluating Markovian Properties . . . . . . . . . . . . . . . . . . . . . . . 168
16.4 Methods for Estimation of Markov–Einstein Time
or Length Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
16.5 Estimation of Drift and Diffusion Coefficients from
Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
16.5.1 Estimation of the Drift Vector . . . . . . . . . . . . . . . . . . . 172
16.5.2 Estimation of the Diffusion Matrix . . . . . . . . . . . . . . . 173
16.5.3 Higher Order Kramers–Moyal Coefficients . . . . . . . . . . 173
16.5.4 Estimation of Drift and Diffusion Coefficients
from Sparcely-Sampled Time Series . . . . . . . . . . . . . . 174
16.6 Deriving an Effective Stochastic Equation . . . . . . . . . . . . . . . . 175
xvi Contents

16.7 Self-consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
17 The Kramers–Moyal Coefficients of Non-stationary Time Series
and in the Presence of Microstructure (Measurement) Noise . . . . . 181
17.1 The Kramers–Moyal Coefficients for Non-stationary Time
Series: Nadaraya-Watson Estimator . . . . . . . . . . . . . . . . . . . . . 182
17.1.1 Time Dependent Kramers–Moyal Coefficients . . . . . . . 184
17.2 The Kramers–Moyal Coefficients in the Presence
of Microstructure (Measurement) Noise . . . . . . . . . . . . . . . . . . 184
17.2.1 Real-World Data with Microstructure Noise . . . . . . . . . 186
17.2.2 Real-World Data without Microstructure Noise . . . . . . 187
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
18 Influence of Finite Time Step in Estimating of the
Kramers–Moyal Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
18.1 Diffusion Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
18.2 The Kramers–Moyal Conditional Moments for the Langevin
Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
18.3 Conditional Moments of the Jump-Diffusion Equation
for Different Orders of the Time Interval . . . . . . . . . . . . . . . . . 197
18.4 The Kramers–Moyal Coefficients in Vanishing Time
Interval Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
18.4.1 “Apparent” and “True” (in Vanishing Time
Interval Limit) Drift and Diffusion Coefficients
in Diffusion Processes . . . . . . . . . . . . . . . . . . . . . . . . 201
18.4.2 The Optimization Procedure to Extract
Kramers–Moyal Coefficients in Vanishing
Time Interval Limit . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
19 Distinguishing Diffusive and Jumpy Behaviors in Real-World
Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
19.1 Distinguishing Diffusive from Jumpy Stochastic Behavior
in Complex Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
19.2 A Jump Detection Measure Q(x) . . . . . . . . . . . . . . . . . . . . . . . 208
19.3 Application to Real-World Time Series . . . . . . . . . . . . . . . . . . 209
19.3.1 Jump Discontinuity in Non-stationary Time Series . . . . 211
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Contents xvii

20 Reconstruction Procedure for Writing Down the Langevin

and Jump-Diffusion Dynamics from Empirical Uni- and
Bivariate Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
20.1 The Reconstruction Procedure, Diffusion Processes . . . . . . . . . . 215
20.1.1 One Dimensional Time Series . . . . . . . . . . . . . . . . . . . 215
20.1.2 Two Dimensional (Bivariate) Time Series . . . . . . . . . . 217
20.1.3 An R Package for Reconstruction of One- and
Two-Dimensional Stochastic Diffusion Processes:
White Noise-Driven Langevin Dynamics . . . . . . . . . . . 221
20.2 The Reconstruction Procedure for the Lévy Noise-Driven
Langevin Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
20.3 The Reconstruction Procedure and Jump-Diffusion
Stochastic Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
20.3.1 One-Dimensional Time Series . . . . . . . . . . . . . . . . . . . 221
20.3.2 Two-Dimensional (Bivariate) Time Series . . . . . . . . . . 223
20.4 Other Methods for Reconstruction of Time Series . . . . . . . . . . . 224
20.4.1 Multiscale Reconstruction of Time Series . . . . . . . . . . 225
20.4.2 Mapping Stochastic Processes onto Complex
Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
21 Reconstruction of Stochastic Dynamical Equations: Exemplary
Diffusion, Jump-Diffusion Processes and Lévy Noise-Driven
Langevin Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
21.1 One and Two-Dimensional Diffusion Processes . . . . . . . . . . . . 227
21.1.1 Bistable Potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
21.1.2 Reconstruction of Bivariate Data Sets . . . . . . . . . . . . . 228
21.2 Jump-Diffusion Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
21.2.1 Reconstruction of an Ornstein–Uhlenbeck Process
with Jumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
21.2.2 Reconstruction of a Black-Scholes Process
with Jumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
21.3 Lévy Noise-Driven Langevin Dynamics . . . . . . . . . . . . . . . . . . 233
21.4 Phase Dynamics and Synchronization . . . . . . . . . . . . . . . . . . . 235
21.5 Estimation of Kramers–Moyal Coefficients for Time
Series with Finite Markov–Einstein Time Scale . . . . . . . . . . . . 237
21.6 Estimation of Kramers–Moyal Conditional Moments
for Diffusion Processes with Different Precision . . . . . . . . . . . . 238
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
22 Applications and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
22.1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
22.2 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
xviii Contents

22.2.1 Representation of Jump-Diffusion Dynamics

in Terms of Fractional Brownian Motion
of Order k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
22.2.2 Langevin Dynamics Driven by Fractional
Brownian Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
22.2.3 The Integral Fluctuation Theorem for Diffusion
Processes (Cascade Processes) . . . . . . . . . . . . . . . . . . . 250
22.2.4 Estimation of Memory Kernel from Time Series . . . . . 252
22.2.5 Anomalous Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . 254
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
23 Epileptic Brain Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
23.1 Stochastic Qualifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
23.2 Detailing the Stochastic Behavior of Epileptic Brain
Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

Appendix A: Wilcoxon Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

Appendix B: Kernel Density Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Chapter 1
Introduction

1.1 Time Series of Complex Systems

Complex systems are composed of a large number of subsystems that may interact
with each other. The typically nonlinear and multiscale interactions often lead to
large-scale behaviors, which are not easily predicted from the knowledge of only the
behavior of individual subsystems. Such large-scale collective emergent behaviors
may be desired or undesired. Examples of undesired emergent behaviors include
short- and long-term climate changes, hurricanes, cascading failures, and epileptic
seizures. Among the desired ones are evolution, adaptation, learning, and intelli-
gence, to name just a few [1–11].
In complex systems the fluctuations stemming from the microscopic degrees of
freedom play a fundamental role in introducing temporal variations on a fast time
scale that, quite often, can be treated as short-time correlated source of fluctuations.
In such systems, self-organized behaviour arises due to the emergence of collective
variables, or order parameters that, compared to the time or length scales of the micro-
scopic subsystems, vary on slower temporal and larger spatial scales. The interaction
of the order parameters are typically nonlinear, with the microscopic degrees of free-
dom showing up in the fluctuations that participate in the order parameter dynamics
and results in complex time series; see Fig. 1.1. Thus, the analysis of the behaviour of
complex systems must be based on the assessment of the nonlinear mutual interac-
tions, as well as the determination of the characteristics and strength of the fluctuating
forces. This immediately leads to the problem of retrieving a stochastic dynamical
system from data; see Fig. 1.2 for typical stochastic time series.
Analysis of time series has a long history in the field of nonlinear dynamics
[12–16]. The problem of dynamical noise, i.e., fluctuations that interfere with the
dynamical evolution, has not been addressed in much details, although it is of utmost
importance for the analysis of strongly fluctuating time series [17, 18]. In this book,
we provide detailed description and discussion of a non-parametric method, known
as the reconstruction method, which has been developed for analyzing continuous
stochastic processes and stochastic data with jumps in time and/or length scales.
© Springer Nature Switzerland AG 2019 1
M. R. Rahimi Tabar, Analysis and Data-Based Reconstruction of Complex Nonlinear
Dynamical Systems, Understanding Complex Systems,
https://doi.org/10.1007/978-3-030-18472-8_1
2 1 Introduction

Fig. 1.1 Complex systems are composed of a large number of subsystems behaving in a “col-
lective manner”. In such systems, which are far from equilibrium, “collectivity” arises due to
self-organization. It results in the formation of temporal, spatial, spatio-temporal and functional
structures. The state of the subsystems change over time and results in stochastic dynamics (as
shown for subsystem 8). The dynamics of order parameters in complex systems are generally non-
stationary, and the subsystems interact with each other in nonlinear manner. The arrows indicate
causal directions of influence

Fig. 1.2 Segments of

intracranial
electroencephalographic
(iEEG) time series, recorded
during a seizure-free interval
from within the epileptic
focus (red) and from a
distant brain region (black).
Source from [21]

The development of such methods has been stimulated by research on turbulent

flows and neuroscience [19–21], which has demonstrated the necessity of treating
the fluctuations as dynamical variables that interfere with the deterministic dynamics.

1.2 Stochastic Continuous Time Series

Systems under the influence of random forcing, or in the presence of nonlinear

interactions with other systems, can behave in a very complex stochastic manner
[17, 18, 22, 23]. The corresponding time series of such systems generally have
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
1.2 Stochastic Continuous Time Series 3

continuous trajectories, or may possess jump discontinuities. To decipher the problem

of retrieving a stochastic dynamical system from data, the main assumption is that
the measured time series represents a Markov process. A stochastic process with a
degree of stochasticity may have a finite Markov-Einstein (ME) time scale tM —the
minimum time interval over which the data can be considered as a Markov process.
Therefore one should first estimate tM for the measured time series, after which
one can apply the method described in this book to reconstruct the corresponding
dynamical stochastic equation. We will introduce various methods to estimate tM in
Chap. 16.
A process x(t) has continuous trajectories if the following relations for condi-
tional averaging, known as the Kramers–Moyal (KM) conditional moments, hold
for infinitesimal dt:

(x(t + dt) − x(t))1 |x(t)=x  = a(x, t) dt

(x(t + dt) − x(t))2 |x(t)=x  = b2 (x, t) dt

(x(t + dt) − x(t))2+s |x(t)=x  = O(dt 1+χ ) (1.1)

with s > 0 and χ > 0. The KM coefficients are defined as D(j) (x) = limdt→0 j!1 dt1
K (j) (x), where the conditional moments K (j) (x) = [x(t + dt) − x(t)]j |x(t)=x , and
can be determined non-parametrically, i.e., directly form the measured time series
[17]. Throughout this book we will use two definitions for the KM coefficients,
D(j) (x) and M (j) (x), which are related to each other by, D(j) (x) = 1/j! M (j) (x).
The dynamics of continuous stochastic processes is governed by the Langevin
dynamics that has the following expression [24–26],

dx(t) = a(x, t) dt + b(x, t) d W (t), (1.2)

where {W (t), t ≥ 0} is a scalar Wiener (Brownian) motion, and the functions a(x, t)
and b2 (x, t)/2 are known as drift and diffusion coefficients, respectively. In terms of
the conditional probability distribution, a continuous Markov process x(t) satisfies
the following continuity condition, given some δ > 0 [25],

Prob[ |Δx(t)| > δ |x(t)=x ]

C = lim =0 (1.3)
dt→0 dt

where Δx(t) = x(t + dt) − x(t). This means that jumps in the process are unlikely.
This condition is called Lindeberg’s continuity condition.
Generalization of Langevin dynamics for d -dimensional time series is straight-
forward. For d -dimensional continuous time series, a necessary ingredient of the
system under consideration is that its dynamical behavior should be described by a
set of macroscopic order parameters x(t) that are governed by the nonlinear Langevin
equations [17],
4 1 Introduction

d
x = a(x) + b(x) (t) (1.4)
dt
where x(t) denotes the d -dimensional state vector, a(x) the drift vector, and the
matrix b(x) isrelated to the diffusion matrix (second-order KM matrix) according
to, Dij(2) (x) = dk=1 bik (x) bjk (x). The noise terms l (t), lumped together in the vector
(t), are fluctuating forces with Gaussian statistics and rapidly decaying temporal
correlations, such that delta-correlation in time can be assumed, i.e., (t) = 0, and,
l (t)k (t  ) = δlk δ(t − t  ).

1.3 Time Series with Jump Discontinuity

In Eq. (1.2) for smooth functions a(x, t) and b(x, t), the process x(t) is diffusion
process and has continuous trajectory. In general, non-vanishing C in Eq. (1.3) or non-
vanishing higher order KM coefficients indicate that a measured time series does not
belong to the class of continuous diffusion processes; see Chap. 12. Accordingly, an
improvement of the Langevin-type modeling, i.e. Eq. (1.2), is needed. An important
generalization of the Langevin-type modeling is to include jump processes, also
known as the jump-diffusion processes, with properties that can also be determined
from measured time series. The jump-diffusion processes are given by dynamical
stochastic equation:

dx(t) = a(x, t) dt + b(x, t) d W (t) + ξ dJ (t), (1.5)

where {W (t), t ≥ 0} is again a scalar Wiener process, a(x, t) and b(x, t) are the drift
and multiplicative diffusion functions, respectively, and J (t) is a time-homogeneous
Poisson jump process. The jump has state-dependent rate λ(x) and jump size ξ
according to some distribution with variance σξ2 . We assume that ξ has Gaussian dis-
tribution, or follows any symmetric distribution with finite moments. This represents
the minimal modeling of a measured time series that contains jumps. In general, one
may assume any distributions for the jump events. In this book, also we will describe
how one models the jumps characteristics—its rate and variance σξ2 —with mixed
Gaussian jumps, where the variance is distributed according to a Gamma distribution.
We describe in Chap. 12 how all the unknown functions and coefficients in Eq. (1.5)
are determined/computed directly from measured time series. Two typical trajec-
tories corresponding to continuous (Brownian type) process and stochastic process
with jumps are shown in Fig. 1.3. Jumps with amplitudes ξ1 and ξ2 are marked with
arrows.
In Chap. 18, we demonstrate that sampling at discrete times gives rise to nonvan-
ishing higher-order conditional moments K (j) (x) with j > 2, even if the underlying
path is continuous. In Chap. 19 we will derive analytically a criterion that allows one
to check whether for a given, even noisy time series the underlying process has a
continuous (diffusive) or jumpy trajectories.
1.4 Microstructural (Measurement) Noise 5

Fig. 1.3 Exemples of

trajectories of a continuous
process (Brownian type,
black line) and a synthetic
discontinuous stochastic
process (red line). For the
latter, jumps with amplitudes
ξ1 and ξ2 are marked with
arrows

1.4 Microstructural (Measurement) Noise

A measured time series may also contain some other noise, which is not assimilated
by the stochastic process. In this case, the time series is written as y(t) = x(t) + (t),
where x(t) denotes the pure stochastic variable, and (t) is an additional noise that
is assumed to be a short-range correlated noise and statistically independent of x(t).
In general, such noise may have its origin in intrinsic components of the complex
dynamics, or can be caused by an external disturbance, e.g., that added to the time
series by the measurement process.
In the literature, such spoiling noise is called differently, either as observational
or measurement noise, or as microstructural noise; an example is in the financial
sciences. The method described in this book is also able to separate and determine the
stochastic behavior of the pure stochastic variable x(t) and the statistical parameters
of the noise (t), such as its variance and higher order statistical moments, etc.
Therefore, we will be able to estimate precisely the “noise” contributions in a given
time series. The microstructural noise is closely linked to the so-called Epps effect
of financial data. Epps observed that there is a dramatic drop of the absolute value
of correlations among stocks, when the sampling frequency increases. In such short
time scales the noise (t) contributes much to the dynamics of y(t) and, due to its
strong fluctuations over short time scales, the predictability of y(t) is decreased. We
note that generally, an original series x(t) has slower dynamics.

1.5 Intermittency

It turns out that the non-parametric method of analysis can also be successfully
applied to not only fluctuating time series, but also to the analysis of spatio-temporal
disordered systems, such as fluid turbulence [18, 19], or characterization of rough
surfaces [27], and the porosity distributions in large-scale porous media. Such struc-
tures can be analyzed as scale-dependent stochastic processes. Experimental observ-
6 1 Introduction

Fig. 1.4 Probability distribution functions (PDF) of the increment statistics, p(Δr x, r) for wind
power fluctuations. Continuous deformation of the increments’ PDFs for time lags r = 1, 10, 1000
sec in log-linear scale are shown. For better clarity the PDFs have been shifted in the vertical
direction, and Δr x’s are measured in units of their standard deviation σr . Extreme events up to about
20σr=1 are recorded. A Gaussian PDF with unit variance is plotted for comparison. The method
described in Chap. 15 provides an evolution equation for the change of the shape or deformation of
the PDF of p(Δr x, r) with respect to scale r. Source from [28]

ables include the field increments, i.e., the difference in the field between two points
separated by a distance or lag r, Δr x(t) = x(t + r) − x(t). The change of the incre-
ments as a function of the scale r can then be viewed as a stochastic process over a
length scale. The method described in this book provides an evolution equation for
the change of shape or deformation of the probability distribution function (PDF) of
p(Δr x, r) with respect to scale r; see Fig. 1.4.
To study the fractal and multifractal behavior of given time series, the approach
can be viewed as an extension of the multifractal description of stochastic pro-
cesses [17]. The multifractal description focusses on the scaling behavior of the
moments of quantities of interest, such as velocity or temperature increments, as a
function of the scale. The complete information on an increment, Δr x(t), is con-
tained in the probability distribution function (PDF) p(Δr x, r) for a certain scale r.
For a self-similar process, it is assumed that the increments exhibit
 ∞ scaling behavior,
Δr x ≈ r ξ , which means p(Δr x, r) = p̃(Δr x/r ξ )/r ξ , where −∞ p̃(u)du = 1. With
locally-varying scaling exponent ξ the PDF is constructed as a superposition of the
probability distributions

1
p(Δr x, r) = dξ p̃(Δr x/r ξ ) f (ξ, r) , (1.6)
rξ

where the measure f (ξ, r) characterizes the distribution of the regions with different
scaling behavior. Knowledge of the deformation equation for p(Δr x, r) with respect
to the scale r enables one to study the scaling exponents of increments for given time
series; see Chap. 15.
1.5 Intermittency 7

In addition, the approach that we describe, which is based on the characterization

of fluctuating fields by a scale-dependent stochastic process, can describe the joint
statistics of the chosen stochastic variable, i.e. increments, on many different scales.
This is achieved by the knowledge of the joint PDF, p(Δr1 x, r1 ; . . . ; Δrn x, rn ). Using
such joint PDFs, the correlations between the scales are also worked out, demon-
strating how the complexity is linked between the scales. If the statistics of the scale-
dependent measure can be regarded as a Markov process evolving in r, knowledge
of two-scale conditional PDF is sufficient for a complete description of multiscale
joint PDF; see Chap. 15 for more details.
Another consequence of the method that we describe in this book is that it enables
us to understand the cascade nature of scale-dependent processes, as well as the
intermittency of the time series. In the intermittent time series the structures arise in
a time series and exhibit themselves as correlated high peaks at random times. The
intervals between them are characterized by a low intensity and a large size. Rare
peaks are the hallmarks of non-gaussian tails of the PDFs. In Fig. 1.4, as an example,
the PDF of the increment statistics, p(Δr x, r), for wind power fluctuations are shown
[28]. Continuous deformation of the increment PDFs for time lags r = 1, 10, 1000
sec in log-linear scale are shown. Here, Δr x is the wind power increments in lag r
and measured in units of their standard deviation σr . Extreme events up to about 20
σr=1 are recorded that deviates strongly from Gaussian PDF.

References

1. H. Haken, Synergetics, An Introduction (Springer, Berlin, 1983)

2. H. Haken, Advanced Synergetics (Springer, Berlin, 1987)
3. H. Haken, Information and Self-Organization: A Macroscopic Approach to Complex Systems
(Springer, Berlin, 2000)
4. H. Haken, Synergetics: Introduction and Advanced Topics (Springer, Berlin, 2004)
5. G. Nicolis, I. Prigogine, Exploring Complexity (W. H. Freeman & Co., San Francisco, 1989)
6. P. Bak, How Nature Works: The Science of Self-Organized Criticality (Oxford University Press,
Oxford, 1999)
7. L. Schimansky-Geier, T. Poeschel, Stochastic Dynamics (Springer, Berlin, 1997)
8. F. Schweitzer, Self-Organization of Complex Structures: From Individual to Collective Dynam-
ics (Gordon and Breach, London, 1997)
9. R.N. Mantegna, H.E. Stanley, An Introduction to Econophysics: Correlations and Complexity
in Finance (Cambridge University Press, New York, 2000)
10. P.L. Gentili et al., Untangling Complex Systems: A Grand Challenge for Science vol. 36, Issue
8, (Rowman & Littlefield Publishers, 2018)
11. D. Sornette, Critical Phenomena in Natural Sciences, 2nd edn. (Springer, Heidelberg, 2003)
12. P. Grassberger, T. Schreiber, C. Schaffrath, Nonlinear time sequence analysis. Int. J. Bifurc.
Chaos 1, 521 (1991)
13. J.D. Hamilton, Time Series Analysis (Princeton University Press, Princeton, 1994)
14. R. Hegger, H. Kantz, T. Schreiber, Practical implementation of nonlinear time series methods:
the TISEAN package. Chaos 9, 413 (1999)
15. H. Kantz, T. Schreiber, Nonlinear Time Series Analysis (Cambridge University Press, Cam-
bridge, 2003)
Random documents with unrelated
content Scribd suggests to you:
each is some distance above the centre of the mass of the machine.
Each wing is fan-curved from front to rear, with the outermost
segment longer than the innermost. The fan wings are opened or
contracted by a hand-lever arrangement, and besides the hand
levers there is an automatic pendulum mechanism which regulates
their area to the requirements of the wind. Whether or no the
inventor’s exact arrangements may prove on trial to be successful is
a matter on which decisive opinion cannot be given; but the principle
of expanding and diminishing surface is thoroughly sound, and is
worthy of lavish expenditure and experiment. Other ways of attaining
variable speed machines have been suggested, though the method
of a variable surface would seem likely to carry the regulation of
speed to a greater nicety than do the other plans. One of these
projects is to alter the angle of the incidence of the planes while the
machine is in flight; the angle would have to be steep for slow speed,
and gradually flatten for increase of speed.

2. Immediate rising into the air.

It is undoubtedly a disadvantage of the aëroplane that it has to
run on the ground on wheels to get the initial velocity necessary for
flight. In some of the earlier military experiments with aëroplanes the
machines were made to run over ploughed fields, for it was
recognised that machines which could only rise when running on
smooth ground would be useless for military work. But one can
imagine that it may often be expedient in military operations for
machines to rise from land so unequal that with the present method
flight would be impossible.
The perfect military aëroplane should be able to rise in the air at
any time and from any place. The application of horizontal lifting
screws beneath the flying machine would make this a possibility,
though it would be necessary to have two of such screws revolving
in opposite directions. It is indeed curious that so little has been done
in the way of such experiments. It will be said that each added screw
means engine multiplication and complication; but these difficulties
are details of engineering that are not unsolvable.
 In the case of such large aëroplanes as the Russian type that
has been described, it would seem specially feasible to attach the
lifting screws.

3. Hovering in the air.

One great advantage of the lifting screws would be that by their
use the machines could hover in the air. Now, when the vertical
screw is stopped, the aëroplane must fall to earth unless the aviator
makes the “vol-plané.” This necessity brings into strong relief the
present imperfection of the flying machine. When horizontal screws
are attached to a flying machine we really have the essential feature
of sustentation, and the existence of the ordinary supporting surface
becomes superfluous. The flying machine has, in fact, become of the
“Hélicoptère” type, though doubtless for some time the supporting
surface will be retained as a means of additional security; in time it
may vanish altogether, and support as well as progression depend
upon revolving screws.

4. Stability.
It has been stated that the properly constructed airship is stable
when in the air; it has not got to fear the more treacherous side gust
which over and over again has brought the aëroplane to earth, and
coupled its name with tragedy. The vexed problem of the stability
and equilibrium of aëroplanes is the most important that has yet to
be solved; until this is done it is not likely the airship will completely
disappear as an instrument of war. In speaking of the remarkable
exploits of Pégoud, it was said that they were an object-lesson on
the materiality of the air, and we have yet to learn how to use this
materiality to the best advantage, so as to afford us continual
stability. Until the problem is solved, man cannot be said to have
brought himself to the level of the soaring bird; the latter, indeed,
makes good use of the very attributes of the wind which at present
tend to upset the aëroplanist—the vertical component of the wind, its
internal work, i.e., its gustiness; its non-uniformity, i.e., its different
velocities at different levels. Every light, therefore, that can be thrown
experimentally or mathematically on the difficult subject of
equilibrium and stability should be eagerly sought.
Professor G. H. Bryan’s mathematical researches are indeed
epoch-making, and their study by the aëronautical engineer should
be prolific of practical result. He does much to elucidate points of the
problem of stability that before had been imperfectly grasped. For
instance, take the case of his remarks as to distinction between
equilibrium and stability.

We say that the motion of a flying machine is steady when

the resultant velocity is constant in direction and magnitude,
and when the angle of the machine to the horizontal is
constant. If this motion is slightly disturbed the machine may
either return after a time to the original motion, or it may take
up a new and altogether different mode of motion. In the first
case, the steady motion is said to be stable, and in the
second unstable.
It is evidently necessary for steady motion of any kind that
there should be equilibrium—i.e., that there should be no
forces acting on the machine (apart from accidental
disturbances) which tend to vary the motion, and hence it
follows that the number of modes of steady motion of which a
machine is capable is, in general, limited, and that when an
unstable, steady motion is disturbed, the new mode of motion
taken up is entirely different from the old.
It is necessary to distinguish carefully between equilibrium
and stability, as the two are very often confused together.
Equilibrium is necessary to secure the existence of a mode of
steady motion, but is not sufficient to ensure the stability of
the motion. The question of the stability of a rigid body moving
under the action of any forces has been solved by Routh. In
order to apply his results to the stability of flying machines, it
is necessary to know the moment of inertia of the machine
about its centre of gravity, the resistance of the air on the
supporting surfaces as a function of the velocity and angle of
 incidence, and also the point of application of this force—i.e.,
the centre of pressure for different angles of incidence. If
these are known for the surfaces constituting any machine,
then the problem of its stability for small oscillations can be
completely solved. Unfortunately, our knowledge of these
points is very unsatisfactory. Several valuable series of
experiments have been made to determine the resistance on
planes, but there is still some doubt as to the position of the
centre of pressure at small angles of incidence, especially for
oblong planes, and very little indeed is known as to the
movement of the centre of pressure on concave surfaces.
Until experiments are made on this point it will be impossible
to solve the problem of stability for machines supported on
concave surfaces.

The subject of the stability of aëroplanes falls under two

heads:—

1. Automatic stability.
2. Inherent stability.

Attempts have been made to produce the first by the aid of

moving gyroscopes and pendulums without much success, and
Professor Bryan has pointed out, apart from the fact that movable
parts are likely to get out of order, they also increase the degree of
the friction of the machine, thus further adding to the number of
conditions that have to be satisfied for stability.
It would seem, therefore, that the desideratum is inherent
stability. Professor Bryan considers that there is hope of attaining
longitudinal and lateral stability by the use of exhaustive
mathematical researches; these will result in the fixing of
independent auxiliary surfaces in aëroplanes in such happy positions
as will secure stability in all conditions of atmosphere. Or it may well
be that through some unlooked-for observation or simple experiment
the answer will come. In the shape of the aëroplane surfaces alone
may be the solution of the problem. But if the aëroplane be still an
imperfect instrument, it is sufficiently developed to be already one of
the greatest factors of modern warfare.
Printed by Hazell, Watson & Viney, Ld., London and Aylesbury.—1414573
 Transcriber’s Notes
Punctuation, hyphenation, and spelling were made
consistent when a predominant preference was found in the
original book; otherwise they were not changed.
Simple typographical errors were corrected; unbalanced
quotation marks were remedied when the change was
obvious, and otherwise left unbalanced.
Illustrations in this eBook have been positioned between
paragraphs and outside quotations. In versions of this eBook
that support hyperlinks, the page references in the List of
Illustrations lead to the corresponding illustrations.
Page 2: “the battle of Fleurus, 1794” was misprinted as
“the battle of Fleurus, 1784”.
Page 86: “the longest being 22½ feet” was printed that
way.
*** END OF THE PROJECT GUTENBERG EBOOK AIRCRAFT IN
WAR ***

Updated editions will replace the previous one—the old editions

will be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States copyright
in these works, so the Foundation (and you!) can copy and
distribute it in the United States without permission and without
paying copyright royalties. Special rules, set forth in the General
Terms of Use part of this license, apply to copying and
distributing Project Gutenberg™ electronic works to protect the
PROJECT GUTENBERG™ concept and trademark. Project
Gutenberg is a registered trademark, and may not be used if
you charge for an eBook, except by following the terms of the
trademark license, including paying royalties for use of the
Project Gutenberg trademark. If you do not charge anything for
copies of this eBook, complying with the trademark license is
very easy. You may use this eBook for nearly any purpose such
as creation of derivative works, reports, performances and
research. Project Gutenberg eBooks may be modified and
printed and given away—you may do practically ANYTHING in
the United States with eBooks not protected by U.S. copyright
law. Redistribution is subject to the trademark license, especially
commercial redistribution.

START: FULL LICENSE

THE FULL PROJECT GUTENBERG LICENSE
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the

free distribution of electronic works, by using or distributing this
work (or any other work associated in any way with the phrase
“Project Gutenberg”), you agree to comply with all the terms of
the Full Project Gutenberg™ License available with this file or
online at www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand,
agree to and accept all the terms of this license and intellectual
property (trademark/copyright) agreement. If you do not agree to
abide by all the terms of this agreement, you must cease using
and return or destroy all copies of Project Gutenberg™
electronic works in your possession. If you paid a fee for
obtaining a copy of or access to a Project Gutenberg™
electronic work and you do not agree to be bound by the terms
of this agreement, you may obtain a refund from the person or
entity to whom you paid the fee as set forth in paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only

be used on or associated in any way with an electronic work by
people who agree to be bound by the terms of this agreement.
There are a few things that you can do with most Project
Gutenberg™ electronic works even without complying with the
full terms of this agreement. See paragraph 1.C below. There
are a lot of things you can do with Project Gutenberg™
electronic works if you follow the terms of this agreement and
help preserve free future access to Project Gutenberg™
electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright
law in the United States and you are located in the United
States, we do not claim a right to prevent you from copying,
distributing, performing, displaying or creating derivative works
based on the work as long as all references to Project
Gutenberg are removed. Of course, we hope that you will
support the Project Gutenberg™ mission of promoting free
access to electronic works by freely sharing Project
Gutenberg™ works in compliance with the terms of this
agreement for keeping the Project Gutenberg™ name
associated with the work. You can easily comply with the terms
of this agreement by keeping this work in the same format with
its attached full Project Gutenberg™ License when you share it
without charge with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside
the United States, check the laws of your country in addition to
the terms of this agreement before downloading, copying,
displaying, performing, distributing or creating derivative works
based on this work or any other Project Gutenberg™ work. The
Foundation makes no representations concerning the copyright
status of any work in any country other than the United States.

1.E. Unless you have removed all references to Project

Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project
Gutenberg™ work (any work on which the phrase “Project
Gutenberg” appears, or with which the phrase “Project
Gutenberg” is associated) is accessed, displayed, performed,
viewed, copied or distributed:

This eBook is for the use of anyone anywhere in the United

States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it
away or re-use it under the terms of the Project Gutenberg
License included with this eBook or online at
www.gutenberg.org. If you are not located in the United
States, you will have to check the laws of the country where
you are located before using this eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is

derived from texts not protected by U.S. copyright law (does not
contain a notice indicating that it is posted with permission of the
copyright holder), the work can be copied and distributed to
anyone in the United States without paying any fees or charges.
If you are redistributing or providing access to a work with the
phrase “Project Gutenberg” associated with or appearing on the
work, you must comply either with the requirements of
paragraphs 1.E.1 through 1.E.7 or obtain permission for the use
of the work and the Project Gutenberg™ trademark as set forth
in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is

posted with the permission of the copyright holder, your use and
distribution must comply with both paragraphs 1.E.1 through
1.E.7 and any additional terms imposed by the copyright holder.
Additional terms will be linked to the Project Gutenberg™
License for all works posted with the permission of the copyright
holder found at the beginning of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files
containing a part of this work or any other work associated with
Project Gutenberg™.
 1.E.5. Do not copy, display, perform, distribute or redistribute
this electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the
Project Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if
you provide access to or distribute copies of a Project
Gutenberg™ work in a format other than “Plain Vanilla ASCII” or
other format used in the official version posted on the official
Project Gutenberg™ website (www.gutenberg.org), you must, at
no additional cost, fee or expense to the user, provide a copy, a
means of exporting a copy, or a means of obtaining a copy upon
request, of the work in its original “Plain Vanilla ASCII” or other
form. Any alternate format must include the full Project
Gutenberg™ License as specified in paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™
works unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or

providing access to or distributing Project Gutenberg™
electronic works provided that:

• You pay a royalty fee of 20% of the gross profits you derive from
the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
 about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt that
s/he does not agree to the terms of the full Project Gutenberg™
License. You must require such a user to return or destroy all
copies of the works possessed in a physical medium and
discontinue all use of and all access to other copies of Project
Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project

Gutenberg™ electronic work or group of works on different
terms than are set forth in this agreement, you must obtain
permission in writing from the Project Gutenberg Literary
Archive Foundation, the manager of the Project Gutenberg™
trademark. Contact the Foundation as set forth in Section 3
below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on,
transcribe and proofread works not protected by U.S. copyright
law in creating the Project Gutenberg™ collection. Despite
these efforts, Project Gutenberg™ electronic works, and the
medium on which they may be stored, may contain “Defects,”
such as, but not limited to, incomplete, inaccurate or corrupt
data, transcription errors, a copyright or other intellectual
property infringement, a defective or damaged disk or other
medium, a computer virus, or computer codes that damage or
cannot be read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES -

Except for the “Right of Replacement or Refund” described in
paragraph 1.F.3, the Project Gutenberg Literary Archive
Foundation, the owner of the Project Gutenberg™ trademark,
and any other party distributing a Project Gutenberg™ electronic
work under this agreement, disclaim all liability to you for
damages, costs and expenses, including legal fees. YOU
AGREE THAT YOU HAVE NO REMEDIES FOR NEGLIGENCE,
STRICT LIABILITY, BREACH OF WARRANTY OR BREACH
OF CONTRACT EXCEPT THOSE PROVIDED IN PARAGRAPH
1.F.3. YOU AGREE THAT THE FOUNDATION, THE
TRADEMARK OWNER, AND ANY DISTRIBUTOR UNDER
THIS AGREEMENT WILL NOT BE LIABLE TO YOU FOR
ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE
OR INCIDENTAL DAMAGES EVEN IF YOU GIVE NOTICE OF
THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If

you discover a defect in this electronic work within 90 days of
receiving it, you can receive a refund of the money (if any) you
paid for it by sending a written explanation to the person you
received the work from. If you received the work on a physical
medium, you must return the medium with your written
explanation. The person or entity that provided you with the
defective work may elect to provide a replacement copy in lieu
of a refund. If you received the work electronically, the person or
entity providing it to you may choose to give you a second
opportunity to receive the work electronically in lieu of a refund.
If the second copy is also defective, you may demand a refund
in writing without further opportunities to fix the problem.

1.F.4. Except for the limited right of replacement or refund set

forth in paragraph 1.F.3, this work is provided to you ‘AS-IS’,
WITH NO OTHER WARRANTIES OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR
ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of
damages. If any disclaimer or limitation set forth in this
agreement violates the law of the state applicable to this
agreement, the agreement shall be interpreted to make the
maximum disclaimer or limitation permitted by the applicable
state law. The invalidity or unenforceability of any provision of
this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the

Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and
distribution of Project Gutenberg™ electronic works, harmless
from all liability, costs and expenses, including legal fees, that
arise directly or indirectly from any of the following which you do
or cause to occur: (a) distribution of this or any Project
Gutenberg™ work, (b) alteration, modification, or additions or
deletions to any Project Gutenberg™ work, and (c) any Defect
you cause.

Section 2. Information about the Mission of

Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new
computers. It exists because of the efforts of hundreds of
volunteers and donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project
Gutenberg™’s goals and ensuring that the Project Gutenberg™
collection will remain freely available for generations to come. In
2001, the Project Gutenberg Literary Archive Foundation was
created to provide a secure and permanent future for Project
Gutenberg™ and future generations. To learn more about the
Project Gutenberg Literary Archive Foundation and how your
efforts and donations can help, see Sections 3 and 4 and the
Foundation information page at www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-
profit 501(c)(3) educational corporation organized under the
laws of the state of Mississippi and granted tax exempt status by
the Internal Revenue Service. The Foundation’s EIN or federal
tax identification number is 64-6221541. Contributions to the
Project Gutenberg Literary Archive Foundation are tax
deductible to the full extent permitted by U.S. federal laws and
your state’s laws.

The Foundation’s business office is located at 809 North 1500

West, Salt Lake City, UT 84116, (801) 596-1887. Email contact
links and up to date contact information can be found at the
Foundation’s website and official page at
www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission
of increasing the number of public domain and licensed works
that can be freely distributed in machine-readable form
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws

regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or
determine the status of compliance for any particular state visit
www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states

where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot

make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.

Please check the Project Gutenberg web pages for current

donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.

Project Gutenberg™ eBooks are often created from several

printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.