LETTER Communicated by Bard Ermentrout

Parameter Space Structure of Continuous-Time Recurrent

Neural Networks

Randall D. Beer
[email protected]
Department of Electrical Engineering and Computer Science and Department of
Biology, Case Western Reserve University, Cleveland, OH 44106, U.S.A.

A fundamental challenge for any general theory of neural circuits is how

to characterize the structure of the space of all possible circuits over a
given model neuron. As a first step in this direction, this letter begins a
systematic study of the global parameter space structure of continuous-
time recurrent neural networks (CTRNNs), a class of neural models that
is simple but dynamically universal. First, we explicitly compute the lo-
cal bifurcation manifolds of CTRNNs. We then visualize the structure of
these manifolds in net input space for small circuits. These visualizations
reveal a set of extremal saddle node bifurcation manifolds that divide
CTRNN parameter space into regions of dynamics with different effec-
tive dimensionality. Next, we completely characterize the combinatorics
and geometry of an asymptotically exact approximation to these regions
for circuits of arbitrary size. Finally, we show how these regions can be
used to calculate estimates of the probability of encountering different
kinds of dynamics in CTRNN parameter space.

1 Introduction

Although tremendous progress has been made on the computational mod-

eling of specific neural circuits, the need for a more general theory of such
circuits is becoming widely recognized in neuroscience. This is especially
true in work on invertebrate pattern generation, where detailed experimen-
tal analyses of several pattern generation circuits have brought questions
of general principles to the fore (Selverston, 1980; Getting, 1989; Marder
& Abbott, 1995; Marder & Calabrese, 1996). For example, recent compu-
tational and experimental studies of neurons and small circuits from the
crustacean stomatogastric ganglion (STG) have become increasingly con-
cerned with the overall parameter space structure of this circuit. In studies
of both model and biological STG neurons, Goldman, Golowasch, Marder,
and Abbott (2001) found that activity patterns were quite robust to some sets

The author is now at the Cognitive Science Program at Indiana University.

Neural Computation 18, 3009–3051 (2006)

C 2006 Massachusetts Institute of Technology
3010 R. Beer

of conductance variations but extremely sensitive to other sets. Golowasch,

Goldman, Abbott, and Marder (2002) found that averaging the conduc-
tance values of different models of an STG cell exhibiting a given firing
characteristic could sometimes fail to produce an average model with
the same characteristic due to nonlinearities in conductance space. Prinz,
Billimoria, and Marder (2003) found that widely disparate parameter sets in
a three-cell pyloric network model could give rise to almost indistinguish-
able network activity. More recently, Prinz, Bucher, and Marder (2004) have
constructed maps of the parameter spaces of model STG neurons through
sampling on a coarse grid of conductance values.
A general theory of neural circuits would benefit such studies in many
ways. It would tell us what kinds of dynamical behavior lie in different re-
gions of a circuit’s parameter space and allow us to quantify the probability
of encountering these different dynamics. It would help us to understand a
circuit’s response to sensory inputs or neuromodulators by distinguishing
those directions of variation in parameter space to which its behavior is
most sensitive from those to which it is robust. It would support the classi-
fication of novel circuits from their parameters alone, without requiring an
exhaustive dynamical analysis of each new circuit. It would provide math-
ematical and computational tools for calculating experimentally testable
predictions regarding novel manipulations. It would provide a methodol-
ogy for the design of neural circuits with desired behavior. Perhaps most
important, it would supply a general context within which to understand
the behavior of particular circuits, allowing us to situate the details of the
actual within the space of the possible.
Characterizing the general parameter space structure of neural circuits is
an extremely difficult problem and is likely to be impossible in full detail for
most models of interest. Nevertheless, even a partial characterization of the
parameter space structure of the simplest nontrivial model neural circuits
would be extremely useful. Studies of simpler models can help us to build
intuition about what such a general theory might look like. Such studies
would also allow us to develop the conceptual framework necessary to
formulate the theory and the mathematical and computational machinery
required to derive its consequences. In addition, they would give us a
better appreciation for the kinds of questions that we can reasonably expect
to answer with such a theory and the kinds of questions that are likely to be
beyond its reach. Only armed with this experience can we hope to extend
our understanding to the parameter space structure of more biophysically
realistic models.
In this letter, we begin the task of characterizing the parameter space
structure of a class of model neural circuits that, though simple, is dy-
namically universal. Section 2 introduces the continuous-time recurrent
neural network (CTRNN) model we will study and reviews some of its key
properties. In section 3, we explicitly compute the local bifurcation man-
ifolds of CTRNNs and visualize their detailed structure for small N. An
CTRNN Parameter Space 3011

approximation to the overall structure of the outer envelope of these mani-

folds for arbitrary N is then derived analytically in section 4. Sections 5 and
6 use the theory to calculate estimates of the probability of encountering dif-
ferent kinds of dynamics in CTRNN parameter space. The letter concludes
with a discussion of the current status of the theory and directions for fu-
ture work. An electronic supplement in the form of a Mathematica (Wolfram,
2003) notebook provides tools for readers to reproduce the main calcula-
tions in the article and to carry out their own explorations (Beer, 2005).

2 Continuous-Time Recurrent Neural Networks

Continuous-time recurrent neural networks are among the simplest pos-

sible nonlinear continuous-time neural models. They are defined by the
vector differential equation

τ ẏ = −y + W σ (y + θ ) + I (2.1)

where τ , ẏ, y, θ , and I are length N vectors, W = {wi j } is an N × N matrix,

and all vector operations (including the application of the output function
σ (x) = 1/(1 + e −x )) are performed element-wise.
The standard neurobiological interpretation of this model is that yi rep-
resents the mean membrane potential of the ith neuron, σ (·) represents
its mean firing rate, τi represents its membrane time constant, θi repre-
sents its threshold/bias, Ii represents an external input, the weights wi j, j=i
represent synaptic connections from neuron j to neuron i, and the self-
interaction wii represents a simple active conductance. This model can also
be interpreted as representing nonspiking neurons (Dunn, Lockery, Pierce-
Shimomura, & Conery, 2004). In this case, σ (·) represents saturating non-
linearities in synaptic input. Note that the distinction between I and θ is
merely semantic; with respect to the output dynamics of equation 2.1, only
the net input I + θ matters, since equation 2.1 can be rewritten in the form
τ ẋ = −x + σ (W x + I + θ) using the substitution y → W x + I. Without loss
of generality, we will often assume that I = 0, so the net input to a CTRNN
is given simply by θ . Thus, an N-neuron CTRNN has N time constants,
N net inputs, and N2 weights, giving CCTRNN (N), the space of all possible
CTRNNs on N neurons, an (N2 + 2N)-dimensional parameter space.
Compared to more biologically realistic neural models, the dynamics
of an individual CTRNN neuron is quite trivial. However, small networks
of CTRNNs can reproduce qualitatively the full range of nerve cell phe-
nomenology, including spiking, plateau potentials, and bursting. More im-
portant, CTRNNs are known to be universal approximators of smooth dy-
namics (Funahashi & Nakamura, 1993; Kimura & Nakano, 1998; Chow &
Li, 2000). Thus, at least in principle, the use of CTRNNs implies no restric-
tion whatsoever on biological realism. CTRNNs can be thought of as a basis
3012 R. Beer

dynamics from which any other dynamics can be reproduced to any desired
degree of accuracy.
CTRNNs are a special case of the general class of additive neural net-
work models τ ẏ = −y + W ξ (y + θ ) + I (Grossberg, 1988). Additive neu-
ral networks have been extensively studied in both their continuous-time
and discrete-time versions (Cowan & Ermentrout, 1978; Cohen & Gross-
berg, 1983; Hopfield, 1984; Hirsch, 1989; Borisyuk & Kirillov, 1992; Blum
& Wang, 1992; Zhaojue, Schieve, & Das, 1993; Beer, 1995; Hoppensteadt &
Izhikevich, 1997; Tiňo, Horne & Giles, 2001; Pasemann, 2002; Haschke &
Steil, 2005). Although our analysis will focus on equation 2.1, the results
we obtain would be qualitatively identical for any additive model with a
(smooth, monotone, bounded) sigmoidal activation function ξ (x).
A particularly convenient class of activation functions can be parame-
terized as
α
σα,β,µ (x) = + β,
1 + e −µx

where α, µ ∈ R+ , β ∈ R (Tiňo et al., 2001). This class contains several com-

mon activation functions, including both the one used here (σ1,0,1 ) and the
hyperbolic tangent function (σ2,−1,2 ). All additive neural network models
using activation functions in this class share an important property: their dy-
namics are topologically conjugate (Haschke, 2004). Specifically, the quan-
titative results we obtain for equation 2.1 can be directly translated to any
other additive model τ  ẏ = −y + W σα,β,µ (y + θ  ) + I via the change of
variables

y = µ−1 y
τ = τ
W = (α µ)−1 W
θ  = µ−1 θ
I = µ−1 I − (α µ)−1 W · β,

where β is the length N vector (β, . . . , β).

The steady-state input-output (SSIO) curve of a single CTRNN neuron
will play an important role in this article (see Figure 1). By transforming
equation 2.1 to the output space defined by o ≡ σ (y + θ ) and setting the time
derivative to 0, we find that the SSIO curve of a neuron with self-weight w is
given by I + θ = σ −1 (o) − w o. A single additive model neuron can exhibit
either unistable or bistable dynamics, depending on the strength of its self-
weight and its net input (Cowan & Ermentrout, 1978). In a single CTRNN
neuron, only unistable dynamics are possible when w < 4 (see Figure 1A).
When w > 4, bistable dynamics occurs when I L (w) ≤ I + θ ≤ I R (w) (see
CTRNN Parameter Space 3013

A B
1 1

0.8 0.8

0.6 0.6
o

o
0.4 0.4

0.2 0.2

0 0
-4 -2 0 2 4 -4 -2 0 2 4
I+ θ I+ θ

Figure 1: Representative steady-state input-output (SSIO) diagrams of a single

CTRNN for (A) w = 2 and (B) w = 8. The solid line shows the output space
location of the neuron’s equilibrium points as a function of the net input I + θ.
Note that the SSIO becomes folded for w > 4, indicating the existence of three
equilibrium points. When the SSIO is folded, the left and right edges of the fold
are given by I L (w) and I R (w), respectively (black points in B). The ranges of
synaptic inputs received from other neurons are indicated by gray rectangles.
The lower (min I) and upper (max I) limits of this range play an important
role in the analysis described in this article. In both plots, two synaptic input
ranges are shown: one for which the neuron is saturated off (left rectangle) and
one for which the neuron is saturated on (right rectangle). The dashed line in
A shows the piecewise linear SSIO approximation used in section 4.2, which
suggests using the intersections of the linear pieces (black points) as the analog
of the fold edges in part B.

Figure 1B), where the left and right edges of the fold are given by (Beer,
1995)

√ √  
w+ w−4 w ± w(w − 4)
I L (w), I R (w) = ±2 ln − ,
2 2

and it is convenient to define the width of the fold as IW (w) ≡ I R (w) −

I L (w). Analogous expressions can be derived for arbitrary sigmoidal acti-
vation functions.
As we study the parameter space of CTRNNs, we will repeatedly en-

counter center-crossing circuits defined by the condition θi∗ = − Nj=1 wi j /2
(Beer, 1995). When this condition is satisfied, the null manifolds of each
neuron intersect at their centers of symmetry, or, equivalently, the SSIO of
each neuron is centered over the range of synaptic inputs that it receives
3014 R. Beer

from the other neurons. Center-crossing circuits are important for a va-
riety of reasons. First, the richest possible dynamics can be found in the
neighborhood of such circuits. By “richest possible dynamics,” I mean dy-
namics that makes maximal use of the available degrees of freedom in the
circuit. Second, the bifurcations of the central equilibrium point of a center-
crossing circuit can often be fully characterized analytically. Finally, for any
given weight matrix, the corresponding center-crossing circuit serves as a
symmetry point in the net input parameter space for that circuit.

3 Visualizing Local Bifurcation Manifolds

Before we attempt to characterize the general structure of CCTRNN (N), it

would be helpful to directly visualize this structure, at least for small N. As
is typical in dynamical systems theory, we will abstract over the details of
individual trajectories and study instead the equivalence classes induced by
topological conjugacy of entire flows (Kuznetsov, 1998). Under this equiv-
alence relation, the parameter space of a dynamical system is divided into
regions of topologically equivalent dynamics by bifurcation manifolds. If
we wish to understand CCTRNN (N), then characterizing the structure of these
manifolds is a good place to begin.
Bifurcations can be local or global. Local bifurcations involve changes in
the neighborhood of a limit set and can be explicitly defined by algebraic
conditions on the vector field and its derivatives in that neighborhood. For
example, the change of stability of an equilibrium point as parameters are
varied is a local bifurcation. In contrast, global bifurcations involve changes
that are not localized to any particular limit set and can usually be studied
only numerically. For example, saddle connections are global bifurcations
in which the unstable manifold of one equilibrium point coincides with
the stable manifold of another. We consider only local bifurcations in this
article.
The two most common local bifurcations are the saddle node and Hopf
bifurcations (Kuznetsov, 1998). In a saddle node bifurcation, the real eigen-
values of an equilibrium point change sign, signaling a change in its stability.
In a Hopf bifurcation, the real parts of a complex conjugate pair of eigenval-
ues of an equilibrium point change sign, signaling a change in its stability
and the production of a limit cycle. These bifurcations are defined by the
conditions

det (J) = 0 (saddle node bifurcation)

det(2J 1) = 0 (Hopf bifurcation)

where J = {∂ f i /∂ y j } is the Jacobian matrix of partial derivatives of the vector

field f(y) ≡ (−y + W · σ (y + θ ))/τ with respect to y evaluated at an equilib-
rium point ȳ, 1 denotes the identity matrix, and is the bialternate matrix
CTRNN Parameter Space 3015

product (Guckenheimer,
   & Sturmfels, 1997). Given two N × N ma-
Myers,
trices A = a i j and B = b i j , A B is the 12 N (N − 1) × 12 N (N − 1) matrix
whose rows are labeled by the multi-index ( p, q ) (where p = 2, 3, . . . , N,
and q = 1, 2, . . . , p − 1), whose columns are labeled by the multi-index
(r, s) (where r = 2, 3, . . . , N, and s = 1, 2, . . . , r − 1), and whose elements
are given by (Kuznetsov, 1998)
   
a a  b b 
1  pr ps   pr ps 
(A B)( p, q )(r, s) =  +  .
2  b qr b q s   a qr a q s 

An implementation of is included in the electronic supplement (Beer,

2005).
The significance of the bialternate matrix product lies in the fact that if an
N × N matrix M has eigenvalues λ1 , . . . , λ N , then 2M 1 has eigenvalues
λi + λ j , with 1 ≤ j < i ≤ N. Thus, the Hopf condition det(2J 1) = 0 is
satisfied whenever a complex conjugate pair of eigenvalues has zero real
parts. Note that an equilibrium point having a pair of real eigenvalues
with equal magnitude but opposite sign will also satisfy this condition
(such a point is called a neutral saddle). Since such points are not actually
Hopf bifurcations, the portions of a solution manifold of det(2J 1) = 0
for which the eigenvalues are not pure imaginary must be removed in a
postprocessing step.
Haschke (2004; Haschke & Steil, 2005) developed a method for comput-
ing local bifurcation manifolds of discrete-time recurrent neural networks
in net input space that can easily be adapted to CTRNNs. If we define
 ≡ σ  (ȳ + θ ) to be the vector of derivatives of σ (·) at an equilibrium point
ȳ, with 0 < ψi ≤ 1/4, then the CTRNN Jacobian at that point can be written
as

J = diag τ −1 · diag() · W − 1 ,

where τ −1 denotes the vector of reciprocals of the time constants τ and

diag(v) denotes the diagonal matrix containing the vector v. This allows
the saddle node and Hopf bifurcation conditions to be expressed directly
in terms of .
For example, for a two-neuron CTRNN, the saddle node bifurcation
manifolds are given by the solutions (ψ1 , ψ2 ) to

w ψ −1 w ψ 
11 1 21 1
 
det(J) = det  w τ1ψ τ1
w22 ψ2 − 1 
12 2
τ2 τ2
= 1 − w11 ψ1 − w22 ψ2 + ψ1 ψ2 det W = 0,
3016 R. Beer

where the time constants have been eliminated by multiplying through by

τ1 τ2 . The Hopf bifurcation manifolds are given by the solutions to
 w ψ −1 w ψ  
11 1 21 1

   10 
det(2J 1) = det 2  w τ1ψ τ1
w22 ψ2 − 1  
12 2 01
τ2 τ2
w11 ψ1 − 1 w22 ψ2 − 1
= + = 0.
τ1 τ2

We can visualize these local bifurcation manifolds by numerically ap-

proximating the solution manifolds defined implicitly by each bifurcation
condition or explicitly solving for one ψi in terms of the others. The results
for two sample two-neuron circuits are shown in Figures 2A and 2B. Note
that these manifolds are functions of W (for saddle node bifurcations) or W
and τ (for Hopf bifurcations).
The bifurcation manifolds in  must then be transformed to net input
space. Using the fact that ȳ = σ  −1 () − θ (where σ  −1 (·) denotes the inverse
of the derivative of σ (·)) and then substituting this for ȳ in the equilibrium
point condition f(ȳ) = 0, we obtain

θi = σ  −1 (ψi ) − W · σ σ  −1 (ψi ) , (3.1)

where

1± 1 − 4ψi − 2ψi
σ  −1 (ψi ) = ln .
2ψi

Note that σ  −1 (·) is two-valued for a sigmoidal function. Since each compo-
nent of θ can come from either branch, each bifurcation manifold in  space
can therefore generate up to 2 N bifurcation manifolds in net input space.
Figures 2C and 2D show the θ-space bifurcation manifolds corresponding
to the -space manifolds shown in Figures 2A and 2B, respectively.
Using the same approach, we can also calculate and display the local
bifurcation manifolds of three-neuron circuits. Figures 3A and 3B provide
an external view of the local bifurcation manifolds for two different three-
neuron circuits, and the slices in Figures 3C and 3D reveal some of the rich
internal structure. In principle, this method can be applied to circuits of
any size. However, the exponential scaling of the number of bifurcation
manifolds in net input space and the difficulty of visualizing manifolds
in dimensions greater than three make it practical for small circuits only.
Mathematica code for the visualization of the local bifurcation manifolds
of CTRNNs in two and three dimensions can be found in the electronic
supplement (Beer, 2005).
CTRNN Parameter Space 3017

A B
0.25 0.25

0.2 0.2

0.15 0.15

ψ2 ψ2
0.1 0.1

0.05 0.05

0 0
0 0.05 0.1 0.15 0.2 0.25 0 0.05 0.1 0.15 0.2 0.25
ψ1 ψ1

C D
-1 -1

-2 -1.5

-3 -2

θ2 θ2

-4 -2.5

-5 -3

-6 -3.5
-6 -5 -4 -3 -2 -1 -4.5 -4 -3.5 -3 -2.5 -2
θ1 θ1

Figure 2: Representative local bifurcation curves of two-neuron CTRNNs. In

all cases, solid black curves represent saddle node bifurcations, dashed curves
represent Hopf bifurcations, and solid gray curves represent neutral saddles.
 space (ψ1 , ψ2 ) of activation function derivatives
(A) Bifurcation curves in the
for a circuit with W = and τ1 = τ2 = 1. (B) Bifurcation curves in (ψ1 , ψ2 )
6 1
  1 6
5.5 −1
for a circuit with W = and τ1 = τ2 = 1. (C) The bifurcation curves from
1 5.5
A in net input space (neutral saddle curves have been removed). Each curve in
(ψ1 , ψ2 ) can produce multiple curves in (θ1 , θ2 ). (D) The bifurcation curves from
B in net input space (neutral saddle curves have been removed).

4 The Global Structure of Local Bifurcation Manifolds

How can we characterize the structure of CTRNN parameter space? As

Figures 2 and 3 demonstrate, the local bifurcation structure of CTRNNs can
be quite complex even in small circuits. Of course, including numerically
computed global bifurcation manifolds would serve only to complicate
these plots (Borisyuk & Kirillov, 1992; Hoppensteadt & Izhikevich, 1997).
3018 R. Beer

Figure 3: Representative local bifurcation

 surfaces
 of three-neuron CTRNNs in
6 1 1
net input space. (A) A circuit with W = 11 16 61 and τi = 1. (B) A circuit with
 6 −1 1 
W = −1 1 6
1
−1
6
and τi = 1. (C) A cutaway view of the interior of the plot in A.
(D) A cutaway view of the interior of the plot in B. The bifurcation manifolds
in the space (ψ1 , ψ2 , ψ3 ) of activation function derivatives are not shown in this
figure, and saddle node and Hopf bifurcation surfaces are not distinguished in
these plots.

If we have any hope of achieving an understanding of CCTRNN (N) that can

be scaled to large N, then we must focus on the overall structure of these
manifolds rather than their fine details.
The plots in Figures 2 and 3 illustrate two key features of the overall
structure of CTRNN parameter space. First, there is always a compact cen-
tral region in net input space (whose location and extent depend on the
CTRNN Parameter Space 3019

connection weights) with the richest dynamics and the highest density of
bifurcations. As it turns out, the center of each of these regions corresponds
to the center-crossing network for that weight matrix. In these central re-
gions, all N neurons are dynamically active: the range of net input each
neuron receives from the other neurons overlaps the most sensitive region
of its activation function σ (·). Dynamically active neurons can respond to
changes in the outputs of other neurons and are therefore capable of par-
ticipating in nontrivial dynamics.
A second key feature of CTRNN parameter space apparent in Figures 2
and 3 is that the local bifurcation manifolds flatten out as we move away
from the central region, forming quasirectangular regions with an apparent
combinatorial structure. This structure is produced by different subsets of
neurons becoming saturated: the range of net input received is such that
σ (·) ≈ 0 or σ (·) ≈ 1. Saturated neurons effectively drop out of the dynamics
and become constant inputs to other neurons because their outputs are in-
sensitive to changes in the outputs of those other neurons (see Figure 4). For
example, in Figure 3A, we see a central “cube” surrounded by six “poles,”
which are in turn interconnected by twelve “slabs.” In the central cube, all
three neurons are dynamically active. In each pole, only two neurons are
dynamically active; one of the neurons is saturated either on or off. Thus, in
these regions, the dynamics of the three-neuron circuit effectively becomes
two-dimensional. Indeed, the structure of the local bifurcation manifolds
visible in the pole cross-sections in Figure 3A is similar to that of the two-
neuron circuit in Figure 2C (Haschke, 2004). In the slabs, only one neuron
is dynamically active, while in the eight remaining corner regions of this
plot, all three neurons are saturated.
We can use this observation to partition the net input space of a CTRNN
into regions of dynamics with different effective dimension depending on
the number of neurons that are dynamically active. A CTRNN with an effec-
tive dimension of M has limit sets whose extent, distribution, and responses
to perturbations span an M-dimensional subspace of the N-dimensional
output space of an N-neuron circuit, with 0 ≤ M ≤ N (see Figure 4). In this
section, we will completely characterize the combinatorics and geometry of
an approximation to these regions for arbitrary N. Due to some differences
in the details, we first consider the case when all wii > 4 and then the case
when some wii < 4.

4.1 All wii > 4. When all wii > 4, the SSIO curves of all neurons are
folded (see Figure 1B), and the edges of these folds play an important
role in structuring CTRNN parameter space (see Figure 4). Specifically,
when the right edge of a neuron’s synaptic input range falls below its
left fold, that neuron will be saturated off regardless of the states of the
other neurons in the circuit (see the left rectangle in Figure 1B). And when
the left edge of a neuron’s synaptic input range falls above its right fold,
that neuron will be saturated on (see the right rectangle in Figure 1B). At
3020 R. Beer

A
1

0.8

0.6
o2
0.4

0.2

0
0 0.2 0.4 0.6 0.8 1
o1

B
1

0.8

0.6
o2
0.4

0.2

0
0 0.2 0.4 0.6 0.8 1
o1

C
1

0.8

0.6
o2
0.4

0.2

0
0 0.2 0.4 0.6 0.8 1
o1
CTRNN Parameter Space 3021

the boundaries of these saturated regions (when a fold is tangent to the

corresponding edge of the synaptic input range), saddle node bifurcations
occur. These saddle node bifurcations are extremal in the sense that as long
as a neuron remains saturated in this way, it cannot participate in any further
bifurcations. Thus, all bifurcations (both local and global) that a given subset
of neurons can undergo must be contained within the extremal bifurcation
manifolds that delineate the dynamically active ranges of those neurons.
When all wii > 4, the extremal bifurcation manifolds are the boundaries that
divide CTRNN parameter space into regions of dynamics with different
effective dimension.
Although the extremal bifurcation manifolds can be calculated analyti-
cally using the method described in the previous section, the expressions
become extremely unwieldy for larger N. Thus, it is far more convenient for
arbitrary N to work with approximations to these manifolds. Our approx-
imation will be based on the fact that as we move away from the center-
crossing point in net input space, these extremal bifurcation manifolds
become asymptotically flat due to the saturation of σ (·). By extrapolating in-
ward toward the center-crossing point, we can approximate the extremal bi-
furcation manifolds by the hyperplanes that they asymptotically approach.
Figures 5A through 5D show the region approximations that correspond
to Figures 2C, 2D, 3A, and 3B, respectively. Note that while these approx-
imations are asymptotically exact and generally quite good, there can be
nontrivial differences when nonlinear effects come into significant play in
the neighborhood of the center-crossing point (see Figures 5B and 5D).
In the remainder of this section, we formally characterize the structure
of the regions bounded by these hyperplanes. We proceed in three steps.
First, we describe the region of M-dimensional dynamics in an N-neuron
circuit (denoted by the symbol R) as a union of disjoint polytopes (denoted
by the symbol Q) corresponding to different subsets of N − M neurons in
saturation. Second, we decompose the structure of each Q into a union of

Figure 4: Phase portraits (left) and SSIO diagrams (right) of two-neuron

CTRNNs with dynamics of different effective dimension. In the phase portrait
plots, stable equilibrium points are shown in black, saddle points are shown in
gray, unstable equilibrium points are shown as circles, and the nullclines are
shown as black curves. In the SSIO plots, the range of synaptic input that each
neuron receives from the other is indicated by a gray rectangle. (A) A phase
portrait with an effective dimension of 2. The folds of both neurons intersect
their range of synaptic inputs. (B) A phase portrait with an effective dimension
of 1. Neuron 1 is saturated on, confining the dynamics to the right edge of out-
put space after transients have passed. (C) A phase portrait with an effective
dimension of 0. Both neurons are saturated on, confining the dynamics to the
upper-right-hand corner of output space after transients have passed.
3022 R. Beer

Figure 5: Asymptotic hyperplane approximations to the extremal saddle node

bifurcation manifolds shown in Figures 2 and 3. (A) Region approximations
corresponding to Figure 2C. The darkest square is R22 , the lighter rectangles are
R21 , and the lightest regions are R20 . (B) Region approximations corresponding to
Figure 2D. (C) Region approximations corresponding to Figure 3A. (D) Region
approximations corresponding to Figure 3B.

overlapping rectangular hypersolids (denoted by the symbol B). Third, we

write the boundaries of each B as a set of N inequalities involving functions
of the connection weights.
Let R M
N
(W) be the asymptotic hyperplane approximation to the region of
M-dimensional dynamics in the net input space of an N-neuron circuit with
weight matrix W, 0 ≤ M ≤ N. Because the saddle node bifurcation condi-
tion is independent of the time constants, R MN
does not depend on τ . Thus,
R M is an N-dimensional volume characterized by N2 weight parameters.
N

For example, Figure 6 shows the structure of R3M for M = 0, . . . , 3 for the
same circuit shown in Figures 3A and 5C.
CTRNN Parameter Space 3023

Figure 6: The individual components of the region approximation shown in

Figure 5C. The front-most polytope 3 Q1 2 3 has been removed from R30 in order
to make the interior visible.

As can be clearly seen in Figure 6, R M N

(W) is composed of disjoint
polytopes corresponding to different subsets of N − M neurons in satu-
i ,...,i N−M
ration. Let N Qik+1
1 ,...,i k
(W) be the polytope with neurons i 1 , . . . , i k satu-
rated off and neurons i k+1 , . . . , i N−M saturated on for any 0 ≤ k ≤ N − M.
Let Z(S) ≡ × {i , i } be the set containing the indices in S in all possible
i∈S
raised and lowered combinations, for example, Z({1, 2}) = {1 , 1 } × {2 , 2 } =
{{1 , 2 }, {1 , 2 }, {1 , 2 }, {1 , 2 }}. Then


RM
N
(W) = N
Q |J (W), (4.1)
N
S∈K N−M
J ∈Z(S)
3024 R. Beer

where K KN denotes the K -subsets of {1, . . . , N} and N Q |J denotes the con-

catenation of the indices J onto N Q. For example, R20 = 2 Q1 2 ∪ 2 Q1 2 ∪
Q1 ∪ 2 Q12 . Note that the index set J is unordered: 2 Q1 2 = 2 Q2 1 . Since there
2 2
N N
are N−M = M different ways of selecting N − M neurons to saturate and
N N−M
each of these may be saturated either off or on, R M N
is composed of M 2
nonoverlapping Q polytopes.
Each N Q|J polytope is in general nonconvex. In order to simplify the
description of R M N
, we will further decompose these N Q|J s into unions of
rectangular hypersolids N B|L(W), where L is an ordered list of raised and
lowered indices: 3 B 1 2 = 3 B 1 2 . Then

N
Q|J(W) = N
B |L (W), (4.2)
L∈P(J )

where P(J ) is the set of permutations of the raised and lowered indices J .
For example, 3 Q1 2 = 3 B 1 2 ∪ 3 B 2 1 (see Figure 7). Here 3 B 1 2 can be interpreted
to mean that neuron 1 is on and neuron 2 is on given that neuron 1 is
on, and 3 B 2 1 means that neuron 2 is on and neuron 1 is on given that
neuron 2 is on. Since L contains N − M elements, each N Q|J is composed
of (N − M)! possibly overlapping N B|Ls. Thus, R M N
is composed of a total
N N−M
of M 2 (N − M)! = M! 2N! N−M
rectangular hypersolids, which obviously
grows very quickly with the codimension N − M.
In order to calculate the bounds of a N B |L rectangular hypersolid, we
will need the synaptic input spectrum IiS (W) of neuron i: the set of possible
synaptic inputs received by neuron i from the subcircuit consisting of the
neurons in S. The min and max elements of this set define the range of
synaptic inputs that neuron i can receive (see Figure 1). I can be defined as

IiS (W) ≡ Pow({wi j | j ∈ S and j = i}), (4.3)

where P ow (W) denotes the power set of the set of weights W and P ow (W)
denotes the set of sums of the elements of the sets in P ow (W). For example,
the synaptic input spectrum of neuron 2 in the {2, 3, 4} subcircuit of a four-
neuron network is

{2,3,4}
I2 = Pow({w23 , w24 })
= {{}, {w23 }, {w24 }, {w23 , w24 }}
= {0, w23 , w24 , w23 + w24 }.

We can now write each N B|L(W) as a set of N inequalities. In order

to obtain the tightest possible bounds, we must treat the saturated and
dynamically active neurons separately. Consider a neuron i in a circuit
with a set of saturated neurons S and a set of dynamically active neurons
CTRNN Parameter Space 3025

Figure 7: An illustration of the structure of Q and B regions. 3 B1 2 (A) and

B (B) differ in the order in which their defining inequalities are constructed.
3 21

(C) The nonconvex polytope 3 Q1 2 is formed by the union of the rectangular

solids 3 B1 2 and 3 B2 1 .

A and let Ii be the net synaptic input that i receives from the neurons in S.
Then neuron i will be saturated off when the left edge of its fold (I L (wii ) −
(θi + Ii )) falls above the right edge of the range of synaptic input it receives
from the active neurons (max IiA ) (see the left rectangle in Figure 1B), it
will be saturated on when the right edge of its fold (I R (wii ) − (θi + Ii ))
falls below the left edge of the range of synaptic input it receives from the
active neurons (min IiA ) (see the right rectangle in Figure 1B), and it will be
dynamically active otherwise.
3026 R. Beer

More formally, each saturated neuron whose index appears in L leads

to an inequality of the form lvi < θvi < uvi . Write the ordered index list L
as the element-wise product of an elevation vector e and an index vector
v : Li = e i vi , where e i = 1 if index vi is raised in L and 0 if it is lowered.
Then the inequality bounds for a saturated neuron can be written as


 −∞ if ei = 0
lvi = N \{v , ..., vi−1 } 
i−1
 I R (wvi vi ) − min Ivi 1 − e j wvi v j if ei = 1
j=1
(4.4)


 I (w ) − max I N \{v1 , ..., vi−1 } − i−1
L vi vi vi e j wvi v j if e i = 0
uvi =
 j=1
∞ if e i = 1

where N = {1, . . . , N} and the set difference S1 \S2 is the set consisting of
all elements of S1 that are not in S2 .
For example, for 4 B 4 1 3 we have L = {4 ,1 , 3 }, with e = (1, 0, 1) and v =
(4, 1, 3). Consider neuron 1 in this circuit, which occurs at an index of i = 2
in L. The notation 4 B 4 1 3 tells us that neuron 1 is saturated off (e 2 = 0),
which requires that the left edge of its fold falls to the right of the maximum
{1, 2, 3, 4}\{4} {1, 2, 3}
synaptic input max I1 = max I1 it receives (see Figure 1B).
The left edge of its fold is given by I L (w11 ) offset by the net input it receives
from the other saturated neurons relative to its own bias: I L (w11 ) − θ1 −
 1
j=1 e j wvi v j = I L (w11 ) − θ1 − e 1 w14 = I L (w11 ) − θ1 − w14 . Thus, the neuron
{1, 2, 3}
1 boundary of 4 B 4 1 3 is I L (w11 ) − θ1 − w14 > max I1 or θ1 < I L (w11 ) −
{1, 2, 3}
max I1 − w14 .
On the other hand, each dynamically active neuron i in A = N \L leads
to an inequality of the form li < θi < ui , with


N−M
li = I L (wii ) − max IiA − e j wi v j
j=1
(4.5)

N−M
ui = I R (wii ) − min IiA − e j wi v j .
j=1

For example, for 4 B 4 1 3 we have A = {1, 2, 3, 4}\{4, 1, 3} = {2}. For neu-

ron 2 to be dynamically active, its fold must overlap the range of synaptic
input it receives from the other dynamically active neurons. The minimum
{2} {2}
and maximum synaptic input are given by min I2 = 0 and max I2 = 0,
respectively. The left (resp. right) edges of the fold are given by I L (w22 )
(resp. I R (w22 )) offset by the net input it receives from the saturated
CTRNN Parameter Space 3027


neurons relative to its own bias: I L (w22 ) − θ2 − 3j=1 e j w2v j = I L (w22 ) −
θ2 − (e 1 w24 + e 2 w21 + e 3 w23 ) = I L (w22 ) − θ2 − w23 − w24 (resp. I R (w22 ) −
θ2 − w23 − w24 ). Thus, the neuron 2 bound is I L (w22 ) − w23 − w24 < θ2 <
I R (w22 ) − w23 − w24 .
Completing this example, the rectangular hypersolid 4 B 4 1 3 would be
defined by the four inequalities

{1,2,3}
θ1 < I L (w11 ) − max I1 − w14
I L (w22 ) − w23 − w24 < θ2 < I R (w22 ) − w23 − w24
{2,3}
I R (w33 ) − min I3 − w34 < θ3
{1,2,3,4}
I R (w44 ) − min I4 < θ4 .

Despite the notational complexity of this section, the basic idea is

straightforward. When all wii > 4, the extremal saddle node bifurcation
manifolds divide the net input parameter space of an N-neuron CTRNN
into regions of dynamics whose effective dimensionality ranges from 0
to N. Furthermore, the combinatorial structure, location, and extent of an
asymptotic approximation to each of these regions can be calculated ex-
plicitly using equations 4.1 to 4.5. Mathematica code for the computation
(and display when N = 2 or 3) of R M N
(W) can be found in the electronic
supplement (Beer, 2005).

4.2 Some wii < 4. In contrast to the wii > 4 case, the SSIO of a neuron
whose self-weight is less than 4 is unfolded (see Figure 1A). Thus, such a
neuron will not undergo the extremal saddle node bifurcations that play
such a crucial role in the parameter space structure characterized above.
Can this analysis be extended to circuits containing such neurons?
To gain some insight into this question, Figure 8 compares the net input
parameter space of a two-neuron CTRNN with w22 = 5 (see Figure 8A) and
w22 = 3 (see Figure 8B). Note that the left and right branches of saddle node
bifurcations disappear when w22 passes below 4, as expected. However,
there are still differences in the effective dimensionality of the dynamics of
this circuit as θ2 is varied. For example, between the two saddle node bi-
furcation curves, the three-equilibrium-point phase portrait changes from
occupying the interior of the state space (and therefore being effectively
two-dimensional in the distribution of its equilibrium points and its re-
sponse to perturbations) at the point C1 to occupying only the bottom edge
(effectively one-dimensional) at C2. Similarly, outside the saddle node bi-
furcation curves, the single equilibrium point phase portrait changes from
occupying the right edge (effectively one-dimensional) at D1 to occupying
the bottom right-hand corner (effectively zero-dimensional) at D2.
3028 R. Beer

Figure 8: An illustration of the extension of the definition of R to CTRNNs

containing self-weights less than 4. (A) Local bifurcation curves and region ap-
proximations for a two-neuron circuit with W = ( 63 35 ). (B) The same circuit as A,
but with w22 = 3. Note that the saddle node bifurcations involving the bistability
of neuron 2 have now disappeared since w22 < 4. However, the extended defini-
tion R̃ can still be used to calculate the regions shown. C1, C2, D1, and D2 show
the phase portraits at the four indicated points in B, with the nullclines drawn as
black curves, stable equilibrium points colored black, and saddle points colored
gray. Although no saddle node bifurcations separate C1 from C2 or D1 from D2
in this circuit, these regions do differ in their effective dimensionality.
CTRNN Parameter Space 3029

Thus, when w22 < 4, regions of dynamics with different effective dimen-
sionality still exist, but there are no sharp boundaries between them because
these regions are no longer delineated by saddle node bifurcations. If we
wish to extend our definitions of the R M N
(W) boundaries from the previous
section to the case when some self-weights are less than four, then we need
to identify some feature of a neuron’s unfolded SSIO curve against which
we can compare the range of synaptic inputs that it receives. Different
choices will lead to somewhat different boundaries.
Perhaps the simplest way to accomplish this is to make use of the piece-
wise linear approximation (the dashed lines in Figure 1A),


0 y < −θ − 2

σ̃ (y + θ ) = y + θ + 1 −θ − 2 ≤ y ≤ −θ + 2


 4 2
1 y > −θ + 2,

and use the points where the linear pieces intersect as markers for boundary
calculations (black points in Figure 1A). By setting the resulting one-neuron
equation to 0 and solving for the net input, we obtain left and right bound-
aries of −2 and 2 − w, respectively, leading to the extended definitions

−2 w<4
Ĩ L (w) =
I L (w) w ≥ 4

2−w w<4
Ĩ R (w) =
I R (w) w ≥ 4.

It is important to reiterate that these extended definitions can no longer

be interpreted as folds when w < 4. Rather, they correspond to boundaries
between saturated and dynamically active behavior.
If we replace I L (w) (resp. I R (w)) by Ĩ L (w) (resp. Ĩ R (w)) everywhere in
our previous analysis, we obtain the extended regions R̃ M N
(W), N Q̃|J (W),
and N B̃|L(W) that are valid for all wii . In our two-neuron example, these
extended definitions give rise to the shaded regions shown in Figure 8B.
Although the original region definitions will be used in the remainder of
this article, the extended definitions could be substituted at any point.

5 Calculating R M
N
Probabilities

In many applications, knowing what can happen in principle is often far

less useful than knowing what typically does happen in practice. Proba-
bilistic calculations can be used to characterize the most likely behavior
under various conditions. In this section, we study the probability that
a random parameter sample will encounter a region of M-dimensional
3030 R. Beer

dynamics in an N-neuron circuit. Such calculations provide estimates of

the dynamical complexity of randomly chosen circuits. In addition, because
the R MN
boundaries become increasingly complex in higher dimensions, a
statistical description can provide a very useful summary of the overall
scaling of the structure of CCTRNN (N) with N. Probabilistic calculations are
also important for the application of stochastic search techniques such as
evolutionary algorithms to CTRNNs (Beer & Gallagher, 1992; Harvey, Hus-
bands, Cliff, Thompson, & Jacobi, 1997; Nolfi & Floreano, 2000), where
search behavior is dominated by the most common dynamics in parameter
space. They can help to determine how best to seed the initial population
of an evolutionary search. They can help select weight and bias parameter
ranges (which is often done in an ad hoc manner) so as to maximize the
chances of encountering interesting dynamics. They can also help to explain
the dynamics of the evolutionary search itself and assess the evolvability of
different types of dynamics.
Assuming that weights are in the range [wmin , wmax ] and biases are in
the range [θmin , θmax ], this probability is given by the fraction of parameter
space volume occupied by the region of interest,

vol R M
N
P RM
N
= , (5.1)
(wmax − wmin ) N2 (θmax − θmin ) N

where vol (R) denotes the volume of the region R.

Since R M
N
is composed of nonoverlapping polytopes N Q|J , we have

vol R M
N
= vol( N Q|J ). (5.2)
N
S∈K N−M
J ∈Z(S)

It is more difficult to compute vol( N Q|J ) because its constituent N B |L

rectangular hypersolids can overlap. In general, the volume of these possi-
bly overlapping sets is given by the sum of the volumes of the individual
sets adjusted by the volumes of their various overlaps,

|P(J )|


   
vol( Q|J ) =
N
vol( B | L) −
N
(−1) i
vol N
B |L ,
L∈P(J ) i=2 P(J )
H∈Ki L∈H

(5.3)

where P(J ) once again denotes the permutations of J , |P(J )| de-

P(J )
notes the cardinality of P(J ) and K K denotes the K -subsets of P(J ).
For example, vol Q 3 12
= vol B3 12
+ vol 3 B 2 1 − vol 3 B 1 2 ∩ 3 B 2 1 (see
Figure 7). Since the N B |L are rectangular hypersolids, each intersection in
CTRNN Parameter Space 3031

equation 5.3 will also be a rectangular hypersolid, the bounds of which can
be found by taking the appropriate maxs and mins of the bounds of the
constituent N B |L s.
Finally, the volume of each rectangular hypersolid N B |L is given by

 
N  
vol( N B|L) = [ui ]θθmax
min
− [l i ]θmax
θ min
,
W i=1

2
where W is the hypercube [wmin , wmax ] N , the expressions ui and li for the
bounds of the ith dimension of N B |L are given in equations 4.4 and 4.5, and
the notation [x]max
min means to clip x to the bounds [min, max]. Since ui and li
depend on only the N weights coming into neuron i, denoted Wi , this N2 -
dimensional integral can be factored into the product of N N-dimensional
integrals as

N 
  
θ
vol N
B |L = [ui ]θmax
min
− [li ]θθmax
min
. (5.4)
i=1 Wi

N
Thus, in order to calculate P(R M ), we must evaluate equations 5.1 to
5.4. Mathematica code supporting the construction and evaluation of such
expressions for sufficiently small N − M is provided in the electronic sup-
plement (Beer, 2005). Note that these expressions are not as efficient as they
could be. By taking into account integral symmetries, it should be possible
to derive equivalent expressions that involve the evaluation of considerably
fewer integrals.
N
As a concrete illustration of the calculation of P(R M ), consider the region
R N of N-dimensional dynamics in an N-neuron circuit. This is not only the
N

simplest case, but also the most important, because all N neurons are dy-
namically active. In this case, R N
N consists of a single rectangular hypersolid
and thus

vol R N
N = vol
N
Q = vol N
B .

Since all N integrals are identical, equation 5.4 can be written as

 wmax  wmax  wmax  θmax
vol R N
N = ··· I R (w) − min I N θmin
4 −wmax −wmax

 θmax N
− I L (w) − max I N θmin dw1 · · · dw N−1 dw , (5.5)

where wi are the incoming weights to an arbitrary neuron, w is that

{1,...,N}
neuron’s self-connection, and I N is an abbreviation for Ii for
3032 R. Beer

arbitrary i. Note that the lower limit of the outermost integral must be
4 because we are using the original region definitions (which are defined
only for w ≥ 4) rather than the extended ones. Note also that we have as-
sumed that wmin = −wmax for simplicity and wmax ≥ 4 so that vol(R N N ) is
nonzero.
Although it is unclear in general how to evaluate these arbitrarily it-
erated piecewise integrals in closed form, it is possible to evaluate them
for fixed w and θ bounds and fixed N (see section A.1). In addition, de-
pending on the range of θmin and θmax relative to the points θmin and
θmax where clipping begins to occur, there are two cases of interest where
evaluation of these integrals for general N is relatively straightforward:
(1) when clipping dominates equation 5.5 and (2) when no clipping
occurs.
The points θmin and θmax can be defined as follows. For 4 ≤ w ≤
 ∈ [I R (wmax ) , −2] and I L (w)
wmax , we have I R (w)  ∈ [I L (wmax ) , −2]. Since
I N has the form 0, ±wmax , . . . , ±wmax (N − 1) , we can conclude that
I R (w) − min I N ∈ [I R (wmax ) , wmax (N − 1) − 2] and that I L (w) − max I N ∈
[I L (wmax ) − wmax (N − 1) , −2], giving

θmin = I L (wmax ) − wmax (N − 1)

θmax = wmax (N − 1) − 2.

The first case in which the iterated integrals can be evaluated in closed
form is when θmin  θmin and θmax  θmax , which will occur when N be-
comes sufficiently large relative to fixed θmin and θmax . In this case, the
integrands are almost everywhere clipped to either θmin or θmax , and the
iterated integrals evaluate to

 N
N−1
N = (θmax − θmin ) (wmax − 4) (2wmax )
vol∞ R N ,

where the ∞ subscript reminds us that this expression is accurate only for
sufficiently large N. The probability of a random parameter sample hitting
RNN therefore scales as


vol∞ R N
N wmax − 4 N
P∞ R N
N = N2
= .
(2wmax ) (θmax − θmin ) N 2wmax

The second case in which the iterated integrals can be calculated in closed
form is when θmin ≤ θmin and θmax ≥ θmax , which will occur when N is small
relative to fixed θmin and θmax . In this case, the θ bounds are sufficiently large
that no clipping occurs and the [·]θθmaxmin
can be dropped from the integrands.
CTRNN Parameter Space 3033

The integrals can then be evaluated (see section A.2) to obtain

vol0 R N
N = (2
N−2
(wmax ) N−1 (wmax (N(wmax − 4) − wmax

+ wmax (wmax − 4) + ln 256 + 4)
 √
− 8(wmax − 1) ln( wmax − 4 + wmax )

+ 2 wmax (wmax − 4) − 8 ln 2)) N ,

where the 0 subscript reminds us that this expression is accurate only for
sufficiently small N. The probability in this case thus scales as

vol0 R N
N
P0 R N
N = .
(2wmax ) N2 (θmax − θmin ) N

Figure 9A shows the scaling of the two approximations P0 and P∞ with

N for the particular case wmax = 16, θmin = −24 and θmax = 24 (here and
throughout the remainder of the article, such specific values are for il-
lustrative purposes only). Superimposed on these curves are data points
taken from 106 random samples of parameter space at each N. Note that
P0 R NN provides the better fit for N < 5, whereas P∞ R N provides the
N

better fit for N > 7. The data actually begin to deviate from P0 by N = 2
(since θmin = −24 < θmin = I L (16) − 16 (N − 1) for N > 1), but the largest
error occurs in the crossover region between these two curves, where the θ
clipping begins to become significant. This becomes even more apparent for
the narrower bounds θmin = −16 and θmax = 16 (see Figure 9B). If higher ac-
curacy is required in this crossover region, then the full iterated integrals for
vol R NN must be evaluated (see section A.1). Such calculations can also be
used to choose appropriate [wmin , wmax ] and [θmin , θmax ] parameter ranges
so as to maximize P(R N N ) for a CTRNN of a given size.

6 Calculating Phase Portrait Probabilities

As classification schemes, the effective dimensionality equivalence rela-

tion described above and the topological conjugacy equivalence relation
normally studied in dynamical systems theory are distinct. A parameter
space region containing dynamics with a given effective dimension may
include many nonequivalent phase portraits. Conversely, a given phase
portrait may appear in multiple regions with different effective dimension-
ality. However, when the defining conditions are simple enough, we can
estimate the densities of individual phase portraits using geometric reason-
ing similar to that employed above. In this section, we give two examples
of such calculations.
3034 R. Beer

A
5

3
%
2

0
2 4 6 8 10
N

B
5

3
%
2

0
2 4 6 8 10
N

N approximations P0 R N (solid curve) and P∞ R N

Figure 9: Plots of the P R N N N
6
(dashed curve) compared to data (black points) from 10 random parameter
space samples at each N, with wi j ∈ [−16, 16] and θi ∈ [−24, 24] (A) or θi ∈
[−16, 16] (B). P0 is most accurate for small N, while P∞ is most accurate for
large N. The maximum error occurs at the crossover point between the two
approximations and can be significant for smaller θ ranges. The crosshairs at
N = 4 in B correspond to the exact value calculated in section A.1.

6.1 Maximal Phase Portraits. The maximum number of equilibrium

points that an N-neuron CTRNN can exhibit is 3 N (Beer, 1995). A suffi-
cient (but not quite necessary) condition for the occurrence of this maximal
phase portrait P3NN is that the range of synaptic input each neuron receives
CTRNN Parameter Space 3035

from the other neurons should fall entirely within its SSIO fold—that is,
I L (wii ) − min IiN ≤ θi ≤ I R (wii ) − max IiN for all neurons i. Thus, the den-
sity of this phase portrait in the parameter space of an N-neuron circuit can
be estimated as

 
P P3NN =

N
w!
max w!
max max "
w! θ  θ #
··· I R (w) − max I N θmax − I L (w) − min I N θmax dw1 · · · dw N−1 dw
min min 0
4 wmin wmin
2
,
(wmax − wmin ) N (θmax − θmin ) N

where the outer clipping to 0 in the integrand enforces the condi-

tion IW (wii ) > range IiN ≡ max I N − min I N , ensuring the existence of this
phase portrait for a given set of weights. Not surprisingly, this arbitrarily
iterated piecewise integral is also difficult to evaluate in general, but can
often be evaluated for particular N and parameter ranges.
As an example, consider the density of phase portraits with nine equi-
librium points P92 in a two-neuron CTRNN whose weights and biases are
in the range [−16, 16]. In this case, the previous integral reduces to

P P92 =
! ! " # 2
16 16
4 −16 [I R (w) − max (0, w1 )]16 16
−16 − [I L (w) − min (0, w1 )]−16 dw1 dw
0
.
324 322
(6.1)

This expression can then be evaluated (see section A.3) to obtain

√ √ √ 2 2
1152 − 576 3 ln 2 + 3 + 240 ln 2 + 3
P P92 = ≈ 0.0060%.
1073741824

For comparison, a random sample of 106 two-neuron circuits exhibited

the nine-equilibrium-point phase portrait with a probability of 0.0059%.
Clearly, the maximal phase portrait is very rare in the parameter space
of two-neuron CTRNNs, and only becomes more so in larger networks.
However, we can use our knowledge of the SSIO geometry underlying this
phase portrait to substantially increase its probability. For example, if we
randomly choose the cross-weights, then randomly choose each self-weight
such that IW (wii ) > range IiN (assuming wii > 4), and then randomly
3036 R. Beer

 
choose each bias from the range I L (wii ) − min IiN , I R (wii ) − max IiN , we
are guaranteed to obtain the maximal phase portrait.

6.2 Central Oscillation Phase Portraits. Some phase portraits do

not have even approximate defining conditions that can be expressed
solely in terms of the geometry of SSIO curves. However, sometimes
we can still calculate a good estimate of occurrence probability in such
cases. Consider, for example, the density of central oscillatory dynam-
ics in a two-neuron CTRNN. By “central,” I mean a phase portrait P1LC 2

in which a single stable limit cycle surrounds a single unstable equi-

librium point in the neighborhood of a center-crossing configuration.
While there are other oscillatory phase portraits that can occur in two-
neuron CTRNNs (Borisyuk & Kirillov, 1992; Beer, 1995; Hoppensteadt &
Izhikevich, 1997; Ermentrout, 1998), their defining conditions are delicate,
and they are therefore likely to make only a small contribution to oscillation
probability.
By linearizing about the center-crossing point (θ1∗ , θ2∗ ) and solving
for the Hopf bifurcation condition (see section A.4), we can derive
two pairs of expressions H1± ( θ2 ; W, τ ) and H2± ( θ1 ; W, τ ) such that
θ1∗ + H1± θ2 − θ2∗ ; W, τ and θ2∗ + H2± θ1 − θ1∗ ; W, τ approximate the Hopf
bifurcation curve in net input space. A comparison of these approxima-
tions (gray curves) to the actual Hopf curves (dashed curves) is shown in
Figure 10. Note that the approximate Hopf curves can extend beyond the
inner edge of the saddle node bifurcation “tongues” even though the cen-
tral oscillations terminate there in saddle node bifurcations on a loop. Note
also that the character of these curves changes from elliptical to hyperbolic
as one of the self-weights passes through 0. Indeed, when one self-weight
becomes sufficiently negative, the oscillatory region splits into two distinct
regions and central oscillations no longer exist.
There are several different ways we can use these curves to esti-
mate P P1LC 2
. Perhaps the simplest
 ∗ approach is to approximate the os-
−
cillatory
 ∗ region by the rectangle θ 1 +
 H1 (0; W, τ ) , θ1∗ + H1+ (0; W, τ ) ×
− ∗ +
θ2 + H2 (0; W, τ ) , θ2 + H2 (0; W, τ ) obtained from the upper and lower
bounds of the Hopf curve approximation at the center-crossing point (see
the gray rectangles in Figure 10). If we clip this rectangle to the θ bounds
and saddle node bifurcation manifolds, clip its width and height to 0, inte-
grate over the weights and time constants, and divide by the total volume
of parameter space, we obtain
2
P P1LC
2
=
(wmax − wmin ) N2 (θmax − θmin ) N (τmax − τmin ) N

 ∗ 
× [θ1 + H1+ (0; W, τ )]ab11 − [θ1∗ − H1− (0; W, τ )]ab11 0
D
 
× [θ2∗ + H2+ (0; W, τ )]ab22 − [θ2∗ − H2− (0; W, τ )]ab22 0
CTRNN Parameter Space 3037

20
A B C
15 15
15
10 10
10
5 5
θ2 5 θ2 θ2
0 0
0
-5 -5
-5
-10 -10
-25 -20 -15 -10 -5 0 -20 -15 -10 -5 0 5 -20 -15 -10 -5 0 5
θ1 θ1 θ1

D E F
15 15 15

10 10 10

5 5 5
θ2 θ2 θ2
0 0 0

-5 -5 -5

-10 -10 -10

-20 -15 -10 -5 0 5 -20 -15 -10 -5 0 5 -20 -15 -10 -5 0 5
θ1 θ1 θ1

Figure 10: An illustration of an approximation to the region of central os-

cillation phase portraits P12LC in two-neuron CTRNNs. The Hopf bifurcation
curve approximations θ ∗ + H ± (θ ; W, τ ) are shown as solid gray curves. The
gray rectangles approximate
 the oscillatory
  regions themselves.
 These rect-
angles are defined by θ1∗ + H1± (0; W, τ ) × θ2∗ + H2± (0; W, τ ) clipped to the
saddle node bifurcation curves. Here τ1 = τ2 = 1. In A, W = ( −16 5 16
6
), and in
w11 16
B to F , W = ( −16 10 ), with w11 = 4 (B), w11 = 1 (C), w11 = −1 (D), w11 = −2
(E) and w11 = −3 (F). Although no central oscillations exist in E and F ,
θ ∗ + H ± (θ; W, τ ) still approximate the Hopf bounds of regions of noncentral
oscillations. A more accurate nonrectangular approximation to the oscillatory
region would be able to take advantage of this fact.

where

max(θmin , I R (wii ) − max(0, wi j,i= j )) wii ≥ 4
ai =
θmin wii < 4

min(θmax , I L (wii ) − min(0, wi j,i= j )) wii ≥ 4
bi =
θmax wii < 4

clip the Hopf curve to the saddle node bifurcation manifolds and θ bounds,
and

D : {(w11 , w12 , w21 , w22 , τ1 , τ2 )|wmin ≤ wii ≤ wmax , 0 ≤ w12 ≤ wmax ,

wmin ≤ w21 ≤ 0, τmin ≤ τi ≤ τmax , κη1 , κη2 ≥ 0}
3038 R. Beer

gives the domain of integration (see section A.4 for an explanation of the
last two conditions). Since it is well known that oscillations can occur in
a two-neuron CTRNN only when the cross-weights are oppositely signed
(Ermentrout, 1995), we have assumed above that w12 > 0 and w21 < 0 and
doubled the integral to account for the opposite possibility.
We will not attempt to evaluate this integral in closed form. Assuming
wi j , θi ∈ [−16, 16] and τi ∈ [0.5, 10], numerical integration using a quasi-
random Monte Carlo method gives P P1LC 2
≈ 0.22%, which accords quite
well with the empirical probability of 0.24% observed in a random sample of
106 two-neuron circuits. The empirical estimate was obtained by randomly
generating 10 initial conditions in the range yi ∈ [−16, 16], integrating each
with the forward Euler method for 2500 integration steps of size 0.1 to skip
transients, and then integrating for an additional 500 integration steps. If
the output of either neuron varied by more than 0.05 during this second
integration for any initial condition, then the circuit was classified as os-
cillatory. Since the empirical value includes both central and noncentral
oscillations, it would be expected to be slightly higher.
How does the probability of oscillation P O N scale with N in CTRNNs?
By “oscillation,” I mean any asymptotic behavior other than an equilibrium
point, so that periodic, quasi-periodic, and chaotic dynamics are included.
Although this question is beyond the theory described in this letter, we
can examine it empirically. A plot of P O N is shown in Figure 11A (black
curve), with the P P1LC 2
value calculated above corresponding to the N = 2
point in this plot. For comparison, the scaling of oscillation probability with
N for random center-crossing circuits is also shown (gray curve). Note that
both curves monotonically increase toward 100%, although oscillations are
clearly much more likely in random center-crossing circuits than they are
in completely random CTRNNs (Beer, 1995; Mathayomchan & Beer, 2002).

Figure 11: Empirical studies of oscillation probability in CTRNNs. (A) The

probability P O N of observing “oscillatory” (nonequilibrium) dynamics in
general (black) and center-crossing (gray) N-neuron circuits with 105 (N ≤ 20)
or 104 (N > 20) random samples from wi j , θi ∈ [−16, 16], τi ∈ [0.5, 10]. Although
oscillations are clearly much more common in center-crossing networks, both
curves increase with N. Samples were obtained by randomly generating 10 ini-
tial conditions in the range yi ∈ [−16, 16], integrating each with the forward
Euler method for 2500 integration steps of size 0.1 to skip transients and then
integrating for an additional 500 integration steps. If the output of any neuron
varied by more than 0.05 during this second integration for any initial condition,
the circuit was classified as oscillatory. (B) The probability P O MN
of observing
M-dimensional oscillatory dynamics in an N-neuron circuit, using 105 parame-
ter space samples for each N and the same sampling protocol as in A. Note that
the distribution shifts to the right, broadens, and rises with increasing N.
CTRNN Parameter Space 3039

A
100

80

60

%
40

20

0
0 20 40 60 80 100
N

3
% 25
2
1 20

0 15
0
N
5 10
10
15 5
M
20
25 0
3040 R. Beer

Interestingly, samples of oscillatory circuits taken from this data set sug-
gest that chaotic dynamics becomes increasingly common for large N,
which is consistent with other work on chaos in additive neural networks
(Sompolinsky & Crisanti, 1988).
In order to gain some insight into the underlying structure of P(O N ),
N
the probability P(O M ) that exactly M neurons are oscillating in an N-
neuron circuit (with the remaining N − M neurons in saturation) is plotted
in Figure 11B. As N increases, note that (1) the most probable oscillating
subcircuit size (denoted M ) increases, (2) the distribution of oscillatory
subcircuits broadens, and (3) the probability of M increases. Several fac-
tors underlie these features. First, the distribution broadens and shifts to
the right because the range of possible subcircuit sizes grows with N. At
least within the range
√ of this plot, the shift in peak location with N is
roughly M ≈ 1.26 N − 0.074. Second, the probability  of M increases be-
 N
cause both the number of possible subcircuits M ∼ 2 N grows expo-
M
nentially and the parameter ranges over which a subcircuit of a given size
can oscillate increases with N − M. As long as P O N = M P O M N
< 1,
the probability of M can continue to increase. However, as P O N ap-
proaches 1, one would expect the probability of M to decrease as a fixed
area is distributed across an increasing range of subcircuit sizes. Finally, the
quantitative details of P O M N
obviously depend on the relative proportion
of different oscillatory regions that fall within the range of allowable bias
values.

7 Discussion

Despite the extreme difficulty involved, there is a growing recognition of

the need for a more general theory of neural circuits, even a partial one. One
path toward such a theory involves the systematic study of the structure
of the space of all possible circuits over a given model neuron. This article
begins such a study for the relatively simple but still dynamically universal
CTRNN model. I have explicitly computed the local bifurcation manifolds
of arbitrary CTRNNs in net input space and presented visualizations of
their structure for small N. I have also shown how the outermost envelope
of saddle node bifurcations formed by the saturation of the sigmoid activa-
tion function divides net input space into regions of dynamics with different
effective dimensionality, and I have derived analytical approximations to
these regions. While these regions by no means exhaust the intricate struc-
ture of CTRNN parameter space, they do provide a coarse map on which
more detailed behavior can be situated. I have also demonstrated how to
calculate estimates of the probabilities of these different regions and of
specific phase portraits.
Although saturation is the most obvious feature of σ (·), it is still remark-
able how this simple property almost completely dominates the structure
CTRNN Parameter Space 3041

of CCTRNN (N). As N increases, the probability of finding circuits with satu-

rated subcircuits exponentially overwhelms the probability of finding cir-
cuits in which all neurons are dynamically active. However, either activity-
dependent regulation (Turrigiano, Abbott, & Marder, 1994; Williams &
Noble, in press) or biased sampling (Beer, 1995; Mathayomchan & Beer,
2002) can counteract this trend. The domination of saturation also suggests
the utility of applying techniques from the logical analysis of switching
networks (Lewis & Glass, 1992; Edwards, 2000; Thomas & Kaufman, 2001)
or modular decomposition (Chiel, Beer, & Gallagher, 1999; Beer, Chiel, &
Gallagher, 1999) to CTRNNs.
While there is no question that the richest dynamics that an N-neuron
CTRNN can exhibit occurs when all N neurons are dynamically active,
it would be a mistake to dismiss circuits in which some neurons are
saturated as somehow less interesting. Even in, say, a 23-neuron circuit,
17-dimensional dynamics can be quite rich. Furthermore, it can be much
easier to evolve 17-dimensional dynamics in a 23-neuron circuit than in a
17-neuron circuit, since P R23 17 can be considerably higher than P R17 .
17

In addition, N-neuron CTRNNs containing saturated subcircuits may be

switchable into different M-dimensional dynamical modes by external in-
puts.
Despite the progress reported in this letter, much work remains to be
done. It would be useful to have an exact closed-form expression for P(R N N)
for general parameter ranges, as well as either approximate or exact closed-
form expressions for P(R M N
) and P P3NN . Although the piecewise function
capabilities of Mathematica (version 5.1 and above) are quite powerful, fur-
ther advances in the evaluation of the arbitrarily iterated piecewise integrals
that arise in these calculations will be necessary. It would also be interesting
2
to derive at least an approximate closed form for the P(P1LC ) calculations
in section 6.2 for general parameter ranges, as well as to improve the accu-
racy of these calculations by using nonrectangular approximations to the
oscillatory region. More generally, it may be possible to calculate probabil-
ity estimates for at least some of the other two-neuron phase portraits, and
perhaps some phase portraits in larger circuits as well. Empirical estimates
of the probabilities of many of the two-neuron phase portraits have already
been determined for a specific set of parameter ranges (Izquierdo-Torres,
2004).
It would also be interesting to characterize the structure of the nonex-
tremal local bifurcation manifolds that appear in Figures 2 and 3. Further
examination of the submanifold of center-crossing circuits may be fruitful
in this regard. Note that while there is only one true center-crossing point
in net input space for any fixed set of weights, each subcircuit region has its
own center-crossing submanifold. For example, at the centers of symme-
try of the poles in Figure 3 lie lines of two-neuron center-crossing circuits
with the third neuron in saturation, while the centers of symmetry of the
slabs likewise contain planes of one-neuron center-crossing circuits. Thus,
3042 R. Beer

the center-crossing (sub)circuits form a kind of “skeleton” for CCTRNN (N)

within the “skin” of the extremal saddle node bifurcations. By studying
small perturbations from these center-crossing submanifolds, it may be
possible to gain some insight into the structure of the nonextremal local
bifurcation manifolds that lie between the skeleton and the skin.
Even in its present form, the theory described in this article has signifi-
cant applications to evolutionary robotics, where CTRNNs are widely used
as controllers (Beer & Gallagher, 1992; Harvey et al., 1997; Nolfi & Floreano,
2000). For example, it would be interesting to explore the utility of seeding
evolutionary searches with circuits drawn uniformly from R N N , which in-
cludes a much greater variety of rich dynamics than the highly symmetrical
center-crossing circuits that have previously been utilized as seeds (Beer,
1995; Mathayomchan & Beer, 2002). The theory can also be used to select
[wmin , wmax ] and [θmin , θmax ] parameter ranges that maximize P(R N N ) for a
CTRNN of a given size. Studies of neutrality (Izquierdo-Torres, 2004) and
evolutionary dynamics in CTRNN parameter spaces (Seys & Beer, 2004)
should also benefit from the theoretical picture of CCTRNN (N) presented
here, and the probability calculations we have demonstrated should be
directly applicable to empirical studies of the implications of saturation
for the evolvability of CTRNNs (Williams & Noble, in press). The theory
could also be used to further work on the impact of network architecture
on CTRNN dynamics (Psujek, Ames, & Beer, 2006), since different circuit
architectures simply correspond to subspaces of CCTRNN (N) with some con-
nection weights fixed to 0. Finally, given the qualitative similarities between
CTRNNs and some models of genetic regulatory networks (de Jong, 2002),
the work described here may even have applications in this area.
Returning to the recent results on the parameter spaces of inverte-
brate pattern generator models with which this article began (Goldman
et al., 2001; Golowasch et al., 2002; Prinz et al., 2003, 2004), what might
our CTRNN results tell us about the maximum conductance parameter
spaces of more biophysically realistic neural models? Although the pa-
rameter space structure of such models is considerably more complex,
some of the insights gained from studies of CTRNNs, and the mathemat-
ical and computational tools developed to support such studies, might
carry over. For example, various kinds of saturation effects, which play
such an important role in structuring CCTRNN (N), also occur throughout
more realistic models. Thus, one would predict that saturation would simi-
larly dominate the parameter spaces of conductance-based models. Fur-
thermore, one would expect that the proper functioning of such mod-
els would impose constraints on the interactions between conductances
analogous to those defining the R N N region in CTRNNs. More gener-
ally, the study of CTRNN parameter space can serve as an exemplar for
parameter space studies of conductance-based models, suggesting fruit-
ful questions and directions and making tentative predictions. Indeed,
the discovery of sensitivity and robustness to different combinations of
CTRNN Parameter Space 3043

conductance variations, failure of averaging, and multiple instantiability in

conductance-based models of invertebrate pattern generators was partly
anticipated by earlier studies with CTRNNs (Chiel et al., 1999; Beer et al.,
1999).
Ultimately, the work described in this article needs to be extended
to these more biophysically realistic neural circuits. Perhaps the best
place to begin such a complexification would be with two-dimensional
model neurons, which can be configured to qualitatively reproduce a wide
range of nerve cell phenomenology (Rowat & Selverston, 1997; Rinzel &
Ermentrout, 1998). Indeed, one interesting possibility would be to study
circuits of two-dimensional model neurons formed from pairs of CTRNN
units, since the theory developed here could be directly applied. From
there, more complex model neurons can be explored, such as the three-
dimensional generic bursting model recently examined by Ghigliazza and
Holmes (2004). Only by studying the simplest possible model circuits and
then incrementally complicating them as our understanding progresses will
we be able to approach a more general theory of neural circuits that can do
justice to the parameter space complexity that studies of biological circuits
are revealing.

Appendix

A.1 Calculating vol(R N

N ) in Closed Form for Fixed N. In this section,
we show how to calculate the value of vol(R N
N ) from equation 5.5 for fixed
N
w and θ ranges and fixed N. We can write vol(R N N ) = (RN − L N ) , where

 wmax  wmax  wmax  θmax

RN ≡ ··· I R (w) − min I N θmin
dw1 · · · dw N−1 dw
4 −wmax −wmax
 wmax  wmax  wmax  θmax
LN ≡ ··· I L (w) − max I N θmin
dw1 · · · dw N−1 dw.
4 −wmax −wmax

By construction, I N has 2 N−1 elements, with N−1

k
k-wise sums. Since the
expression min I N will evaluate to each element of I N precisely when all
terms in that element are negative, RN can be written as

  wmax  wmax  wmax

N−1
RN = ··· [I R (w)]θθmax
min
dw1 · · · dw N−1 dw
0 4 0 0
  wmax  wmax  wmax  0
N−1
+ ··· [I R (w) − w1 ]θθmax
min
dw1 · · · dw N−1 dw
1 4 0 0 −wmax
  wmax  wmax  wmax  0  0
N−1
+ ··· [I R (w) − w1 − w2 ]θθmax
min
dw1 · · · dw N−1 dw
2 4 0 0 −wmax −wmax

+···
3044 R. Beer

    $ %θmax
N−1 wmax 0 0 
N−1
+ ··· I R (w) − wi dw1 · · · dw N−1 dw.
N−1 4 −wmax −wmax i θmin

Evaluating the integrals involving positive weights (which do not appear

in the corresponding integrands), we obtain
  wmax
N−1
RN = (wmax ) N−1 [I R (w)]θθmax
min
dw
0 4
  wmax  0
N−1
+ (wmax ) N−2 [I R (w) − w1 ]θθmax
min
dw1 dw
1 4 −wmax
  wmax  0  0
N−1
+ (wmax ) N−3 [I R (w) − w1 − w2 ]θθmax
min
dw1 dw2 dw
2 4 −wmax −wmax

+···
    $ %θmax
N−1 wmax 0 0 
N−1
+ (wmax )0 ··· I R (w) − wi dw1 · · · dw N−1 dw,
N−1 4 −wmax −wmax i=1 θmin

which sums to

N−1 
 N−1
RN = (wmax ) N−k−1 SRk
k
k=0

 $ %θmax
wmax  0  0 
k
with SRk ≡ ··· I R (w) − wi dw1 · · · dwk dw.
4 −wmax −wmax i=1 θmin

A similar derivation can be applied to L N to obtain

N−1 
 N−1
LN = (wmax ) N−k−1 SLk
k
k=0

 $ %θmax
wmax  wmax  wmax 
k
with SLk ≡ ··· I L (w) − wi dw1 · · · dwk dw.
4 0 0 i=1 θmin

Unfortunately, it is not clear how to evaluate SRk and SLk in closed

form for general k. There are two obstacles to overcome. First, we
! k
must evaluate integrals of the form Dk i=1 xi for general k, with
 k
Dk : {(x1 , . . . , xk )| −wmax ≤ xi ≤ 0, i=1 xi < I R (w) − θmax } for SRk and Dk :
k
{(x1 , . . . , xk )| 0 ≤ xi ≤ wmax , i=1 xi > I L (w) −
! wθmax
min } for SL . Second, we
k

!must evaluate integrals of the form γ R ≡ 4 k

(I R (w)) dw and γ Lk ≡
k
wmax k
4 (I L (w)) dw for general k. Fortunately, using cylindrical decomposi-
tion, Mathematica (Wolfram, 2003) can symbolically tabulate the values of
CTRNN Parameter Space 3045

such integrals for fixed k and fixed w and θ ranges. Thus, for modest N, it
is possible to explicitly calculate vol(R N
N ) in closed form.
For example, consider the point of greatest discrepancy between the
approximate curves and the empirical data in Figure 9B, which occurs at
N = 4. For wmax = 16, θmin = −16, and θmax = 16, we obtain

4
vol(R44 ) = 4096 SR0 − SL0 + 768 SR1 − SL1 + 48 SR2 − SL2 + SR3 − SL3
4
−7798784γ R0 − 557056γ R1 + 10752γ R2 + 128γ R3 − 2γ R4 − 7798784γ L0 + 557056γ L1 + 19968γ L2 + 256γ L3 + γ L4
= ,
331776

which numerically evaluates to a probability of P(R44 ) ≈ 0.375%. This the-

oretical value (the crosshairs in Figure 9B) matches the empirical value of
0.376% quite closely. Details of these calculations, as well as the tabulations
of SRk , SLk , and γ Rk and γ Lk necessary to support them, can be found in the
electronic supplement (Beer, 2005).

A.2 Calculating the Approximation vol0 (R N N ) in Closed Form. In

section 5, a closed-form expression for vol0 (R N
N ) was given. Here, we
show the derivation of this expression. Since the integrand clipping can
be dropped by assumption, the integral 5.5 becomes

 wmax  wmax  wmax

N) =
vol0 (R N ··· (I R (w) − min I N )
4 −wmax −wmax
N
− (I L (w) − max I N ) dw1 · · · dw N−1 dw ,

which is equivalent to

 wmax  wmax  wmax N

··· IW (w) + range I N dw1 · · · dw N−1 dw ,
4 −wmax −wmax

where range I N = max I N − min I N = |w1 | + · · · + |w N−1 |. Thus, we can

rewrite this integral as

 wmax  wmax  wmax

··· IW (w) dw1 · · · dw N−1 dw
4 −wmax −wmax
 wmax  wmax  wmax 
N−1 N
+ ··· |wi | dw1 · · · dw N−1 dw .
4 −wmax −wmax i=1
3046 R. Beer

The first term can be reduced to a one-dimensional integral of IW (w) that

can be evaluated explicitly to obtain

 wmax  wmax  wmax

··· IW (w) dw1 · · · dw N−1 dw
4 −wmax −wmax
 wmax
N−1
= (2wmax ) IW (w) dw
4
 
wmax wmax (wmax − 4) + ln 256
= (2wmax ) N−1
2
√ √
8 (wmax − 1) ln wmax − 4 + wmax
−
2

2 wmax (wmax − 4) − ln 256
+ .
2

In the second term, we can eliminate the absolute value operations by

splitting the integral into 2 N−1 parts, all of which have the same value. Then
we distribute the integrals over the sum and evaluate to obtain

 wmax  wmax  wmax 

N−1
··· |wi | dw1 · · · dw N−1 dw
4 −wmax −wmax i=1

 wmax  wmax  wmax 

N−1
= 2 N−1 ··· wi dw1 · · · dw N−1 dw
4 0 0 i=1

N−1 
 wmax  wmax  wmax
= 2 N−1 ··· wi dw1 · · · dw N−1 dw
i=1 4 0 0

(wmax )2
= 2 N−1 (N − 1) (wmax − 4) (wmax ) N−2 .
2

Raising the sum of the previous two expressions to the Nth power and
simplifying gives

N ) = (2
vol0 (R N (wmax ) N−1 (wmax (N(wmax − 4) − wmax
N−2


+ wmax (wmax − 4) + ln 256 + 4)
 √
− 8(wmax − 1) ln( wmax − 4 + wmax )

+ 2 wmax (wmax − 4) − 8 ln 2)) N .
CTRNN Parameter Space 3047

A.3 Calculating P(P92 ) for Two-Neuron Circuits. In section 6.1, a value

for P P92 was given. Here we show the derivation of this value. Given
4 ≤ w ≤ 16 and −16 ≤ w1 , θ1 , θ2 ≤ 16, the integrand of equation 6.1 can be
simplified to obtain
 
[I R (w) − max(0, w1 )]16
−16 − [I L (w) − min(0, w1 )]−16
16
0
&
I (w) − w1 0 ≤ w1 ≤ IW (w)
= W
IW (w) + w1 −IW (w) ≤ w1 ≤ 0.

We can then split the inner integral of equation 6.1 and evaluate to obtain

! ! ! IW (w)  2
16 0
4 −IW (w) I W (w) + w1 dw1 + 0 IW (w) − w1 dw1 dw
P P92 =
324 322
! 2
16
4 IW (w)2 dw
=
326
√ √ √
(1152 − 576 3 ln(2 + 3) + 240 ln(2 + 3)2 )2
= .
1073741824

A.4 Calculating an Approximation to the Hopf Curve for Two-Neuron

Circuits. In section 6.2, approximations H1± ( θ2 ; W, τ ) and H2± ( θ1 ; W, τ )
to the Hopf bifurcation curve in two-neuron circuits were utilized. Here
we give the derivation of these expressions. Within a small perturbation
θ of a center-crossing circuit, we can replace σ (x) with its linearization
σ̂ (x) = x/4 + 1/2 in order to obtain the linearized dynamics τ ẏ = − y + W ·
σ̂ (y + θ ∗ + θ ). By solving for the simultaneous zeroes of these equations,
we find that the central equilibrium point occurs at ȳ = −θ ∗ + ȳ, where,
for a two-neuron circuit, we have

(4w11 − w11 w22 + w12 w21 ) θ1 + 4w12 θ2

ȳ1 =
w11 w22 − 4w11 − 4w22 − w12 w21 + 16
(4w22 − w11 w22 + w12 w21 ) θ2 + 4w21 θ1
ȳ2 = .
w11 w22 − 4w11 − 4w22 − w12 w21 + 16

If Ĵ is the Jacobian of the linearized system evaluated at ȳ, then the Hopf
bifurcation condition is given by

w11 σ̂  ȳ1 + θ1 + θ1 − 1 w22 σ̂  ȳ2 + θ2 + θ2 − 1

det(2Ĵ 1) = + = 0,
τ1 τ2

where σ̂  (x) = 1/4 − x 2 /16 is the quadratic approximation to σ  (x).

3048 R. Beer

Substituting ȳi = −θi + ȳi , solving for θ1 and θ2 , and simplifying,

we obtain

α θ2 ± β(χ( θ2 )2 + κη1 )
H1± ( θ2 ; W, τ ) = ,
2η1

α θ1 ± β(χ( θ1 )2 + κη2 )
H2± ( θ1 ; W, τ ) = ,
2η2

where

α = 2w22 w21 τ1 (w11 − 4) + 2w11 w12 τ2 (w22 − 4)

β = (w12 w21 + 4(w22 − 4) − w11 (w22 − 4))2
χ = −4w11 w22 τ1 τ2
κ = τ2 (w11 − 4) + τ1 (w22 − 4)
η1 = w11 τ2 (w22 − 4)2 + w22 τ1 (w21 )2
η2 = w22 τ1 (w11 − 4)2 + w11 τ2 (w12 )2 .

Note that these H expressions are real valued only when β(χ( θ2 )2 + κη1 ),
β(χ( θ1 )2 + κη2 ) ≥ 0 and that they have singularities at η1 = 0 and η2 = 0.
The last two conditions in the domain of integration D given in the main
text arise from the requirement that κη1 , κη2 ≥ 0 for the H functions to be
real valued when θ1 = θ2 = 0, since β is strictly positive.

Acknowledgments

I thank Jeff Ames, Michael Branicky, Alan Calvitti, Hillel Chiel, Bard Ermen-
trout, Eldan Goldenberg, Robert Haschke, and Eduardo Izquierdo-Torres
for their feedback on an earlier draft of this letter. This research was sup-
ported in part by NSF grant EIA-0130773.

References

Beer, R. D. (1995). On the dynamics of small continuous-time recurrent neural net-

works. Adaptive Behavior, 3, 469–509.
Beer, R. D. (2005). Electronic supplement to “Parameter space struc-
ture of continuous-time recurrent neural networks.” Available online at
http://mypage.iu.edu/∼rdbeer/CTRNNSupplement.nb.
Beer, R. D., Chiel, H. J., & Gallagher, J. C. (1999). Evolution and analysis of model
CPGs for walking II. General principles and individual variability. J. Computa-
tional Neuroscience, 7, 119–147.
CTRNN Parameter Space 3049

Beer, R. D., & Gallagher, J. C. (1992). Evolving dynamical neural networks for adap-
tive behavior. Adaptive Behavior, 1, 91–122.
Borisyuk, R. M., & Kirillov, A.B. (1992). Bifurcation analysis of a neural network
model. Biological Cybernetics, 66, 319–325.
Blum, E. K., & Wang, X. (1992). Stability of fixed points and periodic orbits and
bifurcations in analog neural networks. Neural Networks, 5, 577–587.
Chiel, H. J., Beer, R. D., & Gallagher, J. C. (1999). Evolution and analysis of
model CPGs for walking I. Dynamical modules. J. Computational Neuroscience, 7,
99–118.
Chow, T. W. S., & Li, X.-D. (2000). Modeling of continuous time dynamical systems
with input by recurrent neural networks. IEEE Trans. on Circuits and Systems—I:
Fundamental Theory and Applications, 47, 575–578.
Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern forma-
tion and parallel memory storage by competitive neural networks. IEEE Trans.
Systems, Man and Cybernetics, 13, 813–825.
Cowan, J. D., & Ermentrout, G. B. (1978). Some aspects of the eigenbehavior of neural
nets. In S. A. Levin (Ed.), Studies in mathematical biology 1: Cellular behavior and the
development of pattern (pp. 67–117). Providence, RI: Mathematical Association of
America.
de Jong, H. (2002). Modeling and simulation of genetic regulatory networks: A
literature review. J. Computational Biology, 9, 67–103.
Dunn, N. A., Lockery, S. R., Pierce-Shimomura, J. T., & Conery, J. S. (2004). A neural
network model of chemotaxis predicts functions of synaptic connections in the
nematode Caenorhabditis elegans. J. Computational Neuroscience, 17, 137–147.
Edwards, R. (2000). Analysis of continuous-time switching networks. Physica D, 146,
165–199.
Ermentrout, B. (1995). Phase-plane analysis of neural activity. In M. A. Arbib (Ed.),
The handbook of brain theory and neural networks (pp. 732–738). Cambridge, MA:
MIT Press.
Ermentrout, B. (1998). Neural networks as spatio-temporal pattern-forming systems.
Reports on Progress in Physics, 61, 353–430.
Funahashi, K. I., & Nakamura, Y. (1993). Approximation of dynamical systems by
continuous time recurrent neural networks. Neural Networks, 6, 801–806.
Getting, P. A. (1989). Emerging principles governing the operation of neural net-
works. Annual Review of Neuroscience, 12, 185–204.
Ghigliazza, R. M., & Holmes, P. (2004). Minimal models of bursting neurons: How
multiple currents, conductances and timescales affect bifurcation diagrams. SIAM
J. Applied Dynamical Systems, 3, 636–670.
Goldman, M. S., Golowasch, J., Marder, E., & Abbott, L. F. (2001). Global structure,
robustness and modulation of neuronal models. J. Neuroscience, 21, 5229–5238.
Golowasch, J., Goldman, M. S., Abbott, L. F., & Marder, E. (2002). Failure of averaging
in the construction of a conductance-based neural model. J. Neurophysiol., 87,
1129–1131.
Grossberg, S. (1988). Nonlinear neural networks: Principles, mechanisms, and archi-
tectures. Neural Networks, 1, 17–61.
Guckenheimer, J., Myers, M., & Sturmfels, B. (1997). Computing Hopf bifurcations
I. SIAM J. Numerical Analysis, 84, 1–21.
3050 R. Beer

Harvey, I., Husbands, P., Cliff, D., Thompson, A., & Jacobi, N. (1997). Evolutionary
robotics: The Sussex approach. Robotics and Autonomous Systems, 20, 205–224.
Haschke, R. (2004). Bifurcations in discrete-time neural networks: Controlling complex
network behavior with inputs. Unpublished doctoral dissertation, University of
Bielefeld.
Haschke, R., & Steil, J. J. (2005). Input space bifurcation manifolds of recurrent neural
networks. Neurocomputing, 64C, 25–38.
Hirsch, M. (1989). Convergent activation dynamics in continuous time networks.
Neural Networks, 2, 331–349.
Hopfield, J. J. (1984). Neurons with graded response properties have collective com-
putational properties like those of two-state neurons. Proc. National Academy of
Sciences, 81, 3088–3092.
Hoppensteadt, F. C., & Izhikevich, E. M. (1997). Weakly connected neural networks.
Berlin: Springer.
Izquierdo-Torres, E. (2004). Evolving dynamical systems: Nearly neutral regions in con-
tinuous fitness landscapes. Unpublished master’s thesis, University of Sussex.
Kimura, M., & Nakano, R. (1998). Learning dynamical systems by recurrent neural
networks from orbits. Neural Networks, 11, 1589–1599.
Kuznetsov, Y. A. (1998). Elements of applied bifurcation theory (2nd ed.). Berlin: Springer.
Lewis, J. E., & Glass, L. (1992). Nonlinear dynamics and symbolic dynamics of neural
networks. Neural Computation, 4, 621–642.
Marder, E., & Abbott, L. F. (1995). Theory in motion. Current Opinion in Neurobiology,
5, 832–840.
Marder, E., & Calabrese, R. L. (1996). Principles of rhythmic motor pattern generation.
Physiological Reviews, 76, 687–717.
Mathayomchan, B., & Beer, R. D. (2002). Center-crossing recurrent neural networks
for the evolution of rhythmic behavior. Neural Computation, 14, 2043–2051.
Nolfi, S., & Floreano, D. (2000). Evolutionary robotics. Cambridge, MA: MIT Press.
Pasemann, F. (2002). Complex dynamics and the structure of small neural networks.
Network: Computation in Neural Systems, 13, 195–216.
Prinz, A. A., Billimoria, C. P., & Marder, E. (2003). Alternative to hand-tuning
conductance-based models: Construction and analysis of databases of model
neurons. J. Neurophysiol., 90, 3998–4015.
Prinz, A. A., Bucher, D., & Marder, E. (2004). Similar network activity from disparate
circuit parameters. Nature Neuroscience, 7, 1345–1352.
Psujek, S., Ames, J., & Beer, R. D. (2006). Connection and coordination: The interplay
between architecture and dynamics in evolved model pattern generators. Neural
Computation, 18, 729–747.
Rinzel, J., & Ermentrout, B. (1998). Analysis of neural excitability and oscillations. In
C. Koch and I. Segev (Eds.), Methods in neuronal modeling (2nd ed., pp. 251–291).
Cambridge, MA: MIT Press.
Rowat, P. F., & Selverston, A. I. (1997). Oscillatory mechanisms in pairs of neurons
with fast inhibitory synapses. J. Computational Neuroscience, 4, 103–127.
Selverston, A. I. (1980). Are central pattern generators understandable? Behavioral
and Brain Sciences, 3, 535–571.
Seys, C. W., & Beer, R. D. (2004). Evolving walking: The anatomy of an evolutionary
search. In S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, & J.-A. Meyer
CTRNN Parameter Space 3051

(Eds.), From animals to animats 8: Proceedings of the Eighth International Conference

on the Simulation of Adaptive Behavior (pp. 357–363). Cambridge, MA: MIT Press.
Sompolinsky, H., & Crisanti, A. (1988). Chaos in random neural networks. Physical
Review Letters, 61(3), 259–262.
Thomas, R., & Kaufman, M. (2001). Multistationarity, the basis of cell differentiation
and memory. II. Logical analysis of regulatory networks in terms of feedback
circuits. Chaos, 11, 180–195.
Tiňo, P., Horne, B. G., & Giles, C. L. (2001). Attractive periodic sets in discrete-time re-
current neural networks (with emphasis on fixed-point stability and bifurcations
in two-neuron networks). Neural Computation, 13, 1379–1414.
Turrigiano, G., Abbott, L. F., & Marder, E. (1994). Activity-dependent changes in the
intrinsic properties of cultured neurons. Science, 264, 974–976.
Williams, H., & Noble, J. (in press). Homeostatic plasticity improves signal propa-
gation in continuous-time recurrent neural networks. Biosystems.
Wolfram, S. (2003). The mathematica book (5th ed.). Champaign, IL: Wolfram Media.
Zhaojue, Z., Schieve, W. C., & Das, P. K. (1993). Two neuron dynamics and adiabatic
elimination. Physica D, 67, 224–236.

Received October 11, 2005; accepted April 25, 2006.