Tracing

Complexity Theory
ESD.83 Research Seminar in
Engineering Systems

P. Ferreira
October 2001

Outline

Views
Definition
Approach
Applications
Early History
People
Institutions
Research
Assessment
References

Views

Study of complicated systems:

A system is complex when it is composed of many parts that interconnect in
intricate ways. (Joel Moses, Complexity and Flexibility). This definition has to
do with the number and nature of the interconnections. Metric for intricateness is
amount of information contained in the system

A system presents dynamic complexity when cause and effect are subtle, over
time. (Peter Senge, The Fifth Discipline). Egs: dramatically different effects in,
the short-run and the long-run; dramatically different effects locally and in other
parts of the system; obvious interventions produce non-obvious consequences

A system is complex when it is composed of a group of related units (subsystems),

for which the degree and nature of the relationships is imperfectly known.
(Joseph Sussman, The New Transportation Faculty). The overall emergent
behavior is difficult to predict, even when subsystem behavior is readily
predictable. Small changes in inputs or parameters may produce large changes in
behavior

Views

Study of complicated systems:

A complex system has a set of different elements so connected or related as to
perform a unique function not performable by the elements alone. (Rechtin and
Maier, The Art of System Architecting). Require different problem-solving
techniques at different levels of abstraction

Scientific complexity relates to the behavior of macroscopic collections of units

endowed with the potential to evolve in time. (Coveney and Highfield, Frontiers
of Complexity). This is different from mathematical complexity (number of
mathematical operations needed to solve a problem, used in computer science)

Complexity theory and chaos theory both attempt to reconcile the unpredictability
of non-linear dynamic systems with a sense of underlying order and structure.
(David Levy, Applications and Limitations of Complexity Theory in
Organizational Theory and Strategy). Implications: pattern of short-term
predictability but long-term planning impossible, dramatic change unexpectedly,
organizations can be tuned to be more innovative and adaptive

Views

Definition

The Newtonian Paradigm is built on Cartesian Reductionism:

Machine Metaphor and Cartesian Dualism (Descartes): Body is a biological machine; mind

as something apart from the body; Intuitive concept of machine: built up from distinct parts
and can be reduced to those parts without losing its machine-like character: Cartesian
Reductionism
The Newtonian Paradigm and the three laws of motion: General Laws of motion, used as
the foundation of the modern scientific method. Dynamics is the center of the framework,
which leads to trajectory

Complexity results from failure of the Newtonian Paradigm to be generic:

Complex and simple systems are disjoint categories that encompass all of nature
But the real world is made up of complex things and the world of simple mechanisms is

fictitious and created by science. Experiments involve reducing the system to its parts and
then studying those parts in a context formulated according to dynamics

How is science done?

Senses (observe the world) + Mental activity (make sense out of that sensory information).

Encode natural system (NS) into formal system (FS); manipulate FS to mimic the causal
change in the NS. From the FS derive an implication that corresponds to the causal event in
the FS; decode the FS and check its success in representing the causal event in the NS

Definition

Definition of Complexity:
The world, from which we single out some smaller part, the NS, is converted
into a FS that our mind can manipulate and we have a model. The world is complex.
The FS we chose to try to capture it can only be partially successful. For years we
were satisfied with the Newtonian Paradigm as the FS, forgot about there even
being and encoding and decoding, and gradually began to change the ontology so
that the Newtonian Paradigm actually replaced or became the real world. As we
began to look more deeply into the world we came up with aspects that the
Newtonian Paradigm failed to capture. Then we needed an explanation. Complexity
was born! This easily can be formalized. It has very profound meaning
Complexity is the property of a real world system that is manifest in the inability of
any one formalism being adequate to capture all its properties. It requires that we find
distinctly different ways of interacting with systems. Distinctly different in the sense that
when we make successful models, the formal systems needed to describe each distinct
aspect are NOT derivable from each other
Bob Rosen and Don Mikulecky, Professors of Physiology
Medical College of Virginia Commonwealth University

Definition
Implications of this definition:
A complex system is non-fragmentable. If it were it would be a machine. Their

reduction to parts destroys important system characteristics irreversibly

A complex system comprises real components that are distinct from its
parts.There are functional components defined by the system which definition
depend on the context of the system. Outside the system they have no meaning. If
removed from the system it looses its original identity
Complex systems have models, analytic or synthetic. But the tools differ. If a
synthetic model can replace an analytic models, the system is fragmentable
No largest model. If there were a largest model, all other models could be derived
from it and fragmentability would result
Causalities in the system are mixed when distributed over the parts. The nature of
causality requires closed loops of excluded in the Newtonian Paradigm

The important attributes of the system are beyond algorithmic definition or

realization: a path to refute Church's thesis (All the models of computation
yet developed, and all those that may be developed in the future, are equivalent
in power. We will not ever find a more powerful model...)

Definition
Ideas related to Complexity:
Size: Egs the size of a genome; the number of species in an ecology. Size is

indication of difficulty in dealing with the system. But for complexity, such parts
need to be inter-related
Ignorance: Egthe brain is too complex for us to understand.Complexity is the
cause of ignorance. Cannot completely associate the two (other significant causes?)
Minimum Description Length: Kolmogorov Complexity is the minimum possible
length of a description in some language (usually that of a Turing machine)
Variety: Eg this species markings are complex due to their great variety. Variety
is necessary for complexity but it is not sufficient for it
(Dis)Order: Complexity is mid-point between order and disorder

Complexity

Disorder

Definition
Complexity is that property of a language expression which makes it difficult to
formulate its overall behavior, even when given almost complete information
about its atomic components and their inter-relations"
Bruce Edmonds, Senior Research Fellow in Logic and Formal Methods
Center for Policy Modeling, Manchester Metropolitan University, UK

Relationship to more specific definitions of complexity:

Computational Complexity: amount of computational resources needed to solve a

class of problems. Lacks the difficulty of providing the program itself

Bennett's Logical Depth: computational resources to calculate the results of a
program of minimal length
Lfgren's Interpretation and Descriptive Complexity: the combined processes of
interpretation and description. Eg: interpretation: decoding of the DNA into the
effective proteins; description: process the result of reproduction and selection on
the information there encoded
Kauffman's number of conflicting constraints: complexity is the number of
conflicting constraints. This represents the difficulty of specifying a successful
evolutionary walk given the constraints

Approach

Abstraction, Modularity and Scales

Eg from Physics: Matter

{-i(22i)/(2me)-i(22j)/(2mn)+e2/(40)i1,i21/|ri1-ri2|+
+z2e2 /(40) j1,j21/|Rj1-Rj2|-ze2 /(40) i,j1/|ri-Rj|}=E

But cannot solve analytically even if i=2 and j=1 (Helium)

What to do? Characterize the behavior of the system at a different scales

Eg: molecules (mass, charge, poles, symmetries,)

Or use Computer Simulation (major tool)

Approach

Approach

But computers have limited expressive power. Computers with 32 bits have
steps of at least 2.328-10. For some systems, a difference of this magnitude in
the input conditions lead to very different outcomes

Eg: M. Feigenbaum studies of population growth models

Populationt = GrowthRate*Populationt-1(1-Populationt-1)

Feigenbaum Constant: 4.6692016

Growth Rate

Approach

But the Feigenbaum constant appears in many other contexts

Eg: the Mandelbrot Set

Equation: Z(n+1)=Z(n)2+C, C and Z imaginary numbers
Mapping: represents the number of iterations need for |Z(n)|>2

The importance of the Feigenbaum constant:

It is an invariant

Approach

Dissipation of the initial conditions:

Eg: The Sierpinski Triangle

Idea of Attractor:
Eg: Lorentz Attractor (dx/dt=-a*x+a*y;dy/dt=b*x-y-z*x;dz/dt=-c*z+x*y; dt =.02, a=5, b=15, c=1)

The importance attractors:

Reduce the space state

Approach

Cellular automata: array of finite state machines (inter-related)

Lattice of sites, each lattice can take one of k values
Levels of lattices implement different scales of the system
Discrete in time, each site updates asynchronously depending on neighbors
Every site updates according to a local pre-defined rule
Fixed point and limiting cycles become common

life.exe

Applications
Complexity Theory appears in many fields:
The more traditional ones: physics, biology, computer
science

Other examples include

Transportation Systems

(Joseph Sussman, Professor Civil and Environmental Engineering, MIT)

Transport systems are complex networks, internally interconnected

at different scales
The system is stochastic by nature and policy-makers introduce
strategies that affect the overall behavior of the system

Dynamic Markets and Firms

(Chris Meyer, E&Y Partner and Director of the Center for Business Innovation)

The market is ever changing, defined by firm interaction

Inside the firm: make boundaries permeable, allow the bottom-up
flow of ideas, give up of the idea of equilibrium

Early History

Complexity is related to the NP-completeness of some problems (combinatorial explosion).

First known problem of this sort is:

Given n points and the distance between every pair of them, find the shortest route which visits each
every point at least once and then returns to the starting point

There was a German book published in 1832 about this problem

The problem entered the mathematical world only one century later by Merrill Flood, who
urged the RAND computer company to offer a prize for its solution. Merrill Flood, together
with Melvin Dresler, were the first to work out formally the Prisoners Dilemma in 1950. They
were involved in researching strategies for nuclear war

Dantzig, Fulkerson and Johnson (Computer Science Department at Stanford University)

published a paper, in 1954, published a paper showing that a solution is optimal by looking at
some inequalities (49-city map of the 48-state United States, needs 25 inequalities)
G. B. Dantzig, R. Fulkerson, and S. M. Johnson, "Solution of a large-scale traveling salesman
problem", Operations Research 2 (1954), 393-410

Researchers understood that problems fall into two-categories: the good and the bad ones. Once
you solve one problem, you actually solve a class of similar problems

People
People related to the field come from primarily from mathematics, physics, computer science
and biology

Among the most prominent people we find:

Stuart Kauffman - Pioneer in complexity theory; MD from University of California (1968), Professor in

Biophysics, Theoretical Biology and Biochemistry (1969-1995), University of Chicago and University
of Pennsylvania; Currently, consultant for Los Alamos National Laboratory and External Professor, Santa
Fe Institute; Publication: At Home In The Universe, Oxford University Press, 1995

Murray Gell-Mann Theoretical physicist; PhD (Physics) 01/51, MIT; Professor Emeritus of

Theoretical Physics,California Institute of Technology; Professor and Co-Chairman of the Science Board
of the Santa Fe Institute; Nobel Prize in 1969, work on the theory of elementary particles (co-discoverer
of Quarks); Currently in the President's Committee of Advisors on Science and Technology; Author of the
book: The Quark and the Jaguar, W. H. Freeman and Company, New York, 1994

John Holland
Anderson
Goedel
Kolgomorov
Wolfram
Selt Lloyd

People
Philip Anderson Condensed matter theorist; PhD Harvard (49);

Professor of Physics at Oxford University and Princeton

University (75-present); Nobel Prize in 1975 for investigations on
the electronic structure of magnetic and disordered systems; Also
at the Bell Labs (49-84) and Santa Fe Institute (70-present)

John Holland first PhD in Computer Science (University of

Michigan); pioneer of evolutionary computation, particularly
genetic algorithms; Professor of Cognition and Perception at the
University of Michigan and Santa Fe Institute

Others: Selt Llyod (Physics), Joseph Sussman (Civil), Christopher

Langton (Computer Science), Brian Arthur (economics), Jack Cowan

(maths), Herbert Simon (economics), John Smith (biology), Per Bak
(physics)

Institutions
Santa Fe Institute
Private, non-profit, multidisciplinary research and

education center, founded in 1984

Largely Supported by the NSF and MacArthur Foundation
Operates as a small visiting institution
Catalyzes new collaborative, multidisciplinary projects
Primarily devoted to Basic Research
Gathers about 100 members, 35 in residence at one time

Research
Areas of research (at SFI) include:

Computation in Physical and Biological Systems

Economic and Social Interactions
Evolutionary Dynamics
Network Dynamics;

Can science achieve a unified theory of complex systems?

From Complexity to Perplexity, by J. Horgan, Scientific American:
Some (at SFI) argue that it might be possible to have a new, unified way of thinking
about nature, human social behavior, life and the universe itself
Some (also at SFI!) argue we dont even know what that means
Some researchers believe that one day computer power will be enough to predict,
control and understand nature
R. Shepard (Stanford University): even if we can capture nature's intricacies on
computers, those models might themselves be so intricate that they elude human
understanding

Assessment
Complexity theory targets at the heart of systems:
Understanding the relationship between emergent behavior and intricateness
of parts (through the non-fragmentable property)
Paradigm to think about systems and scales

Spreads to many areas (but by definition)

Physics, biology, computer science, economics,

Successful: understanding concept of identity of a system

But there is a challenge: complex systems engineering:
Design Purposeful Complex Systems
So far, we have good tools to characterize but not to design
(eg. Attractors and Pattern recognition)
Why bother? Is there another way to account for emergent behavior?

References

Complex Systems:
Founded by Stephen Wolfram in 1987
Contributors from academia, industry, government
General public in 40 countries around the world
Topics: mathematics, physics, computer science, biology

Advances in Complex Systems:

Founded in 1998
Editor-in-Chief: Peter F. Stadler, Dept. of Theoretical

Chemistry and Molecular Structural Biology, U. Vienna

Co-Editor-in-Chief: Eric Bonabeau, Santa Fe Institute
Fields: biology, physics, engineering, economics, cognitive
science and social sciences