Synergetic Control Theory Approach for Solving Systems of Nonlinear

Equations

Anton BEZUGLOV
Earth Sciences Resources Institute, University of South Carolina
Columbia, SC, 29208

Anatoliy KOLESNIKOV
Synergetics and Controlling Processes Dept, Taganrog State University of RadioEngineering,
Taganrog, Russian Federation

Igor KONDRATIEV
Electrical Engineering Dept, University of South Carolina,
Columbia, SC, 29208

Juan VARGAS
Computer Science & Engineering Dept, University of South Carolina,
Columbia, SC, 29208

ABSTRACT
This paper presents Synergetic Control Theory
(SCT) and discusses how it can be used for solving
systems of nonlinear equations. SCT is a new
methodology for solving systems of nonlinear equations.
The main advantage of SCT is that it maps the original
system of equations to a dynamical system such that (1)
any trajectory in the state space of the system ends in an
attracting point; (2) the attracting point is located at the
solution of the original system and (3) the rate at which
the dynamical system moves towards the attracting point
is controllable. An algorithm based on SCT is discussed.
The algorithm has the following advantages: (1) If
solutions exist, the algorithm finds one at a controllable
rate, independently of initial guesses, (2) If no solutions
exist, the algorithm makes such determination quickly,
using stability analysis.

Keywords: Nonlinear equations, Systems of nonlinear
equations, dynamical systems, stability analysis,
Synergetic Control Theory, attracting regions, attractors.
1. INTRODUCTION
The purpose of this work is to describe a synergetic
method for obtaining solutions to systems of nonlinear
equations. Similar to the traditional Newton-Raphson and
Broydens methods, the synergetic method transforms the
equations to a dynamical system that performs iterations.
The synergetic dynamical system can be considered
superior to other systems, because: (1) It is linear and (2)
It has attractors only where they are needed, i.e. at the
solutions.
This paper is organized as follows. Section 2
discusses the Newton-Raphson and Broydens methods.
In section 3 the basics of SCT are given and the
synergetic algorithm is explained. Section 4 provides
examples of finding roots using the synergetic algorithm
and compares the convergence rate versus the Newton-
Raphson method.
2. BACKGROUND
Iterative methods for finding roots of nonlinear
systems of equations are used, because mathematics does
not have a sufficiently general analytical method. In
contrast to analytical methods, iterative methods consider
the process of solving equations or systems of equations
as a dynamical process, which controls the convergence
from an initial guess to a solution. The dynamical systems
that guide these processes are a by product of the iterative
methods. Therefore, the properties of the methods, such
as convergence rate, robustness to initial guesses,
robustness to local maxima and minima, are mostly
determined by the properties of the dynamical systems.
A typical approach is the expansion of the Taylors
series, which is applied in a family of methods, including
the Newton-Raphson, the Broydens, and their
modifications [1,2]. Let us review approaches used in
these methods. Suppose that a problem is formulated as
follows:

( ) 0 = x F
i
(1)

where M i F
i
1 , is a set of equations that need to
be zeroed, x is a vector of variables. The vector of
functions can be approximated as a part of the
Taylors series as:
F

( ) ( ) x J x F x x F + = + (2)

where is the Jacobian. The iterative formulae can be J
derived by setting ( ) 0 = + x x F and solving (2) for
x :
The Broydens method
The Broydens method [4] does not require an exact
Jacobian, this may save significant computational time as
compared to the Newton-Raphson method. The
Broydens method is a secant method, in which the
Jacobian is approximated by the following formula:

( ) x F J x =
1
(3)


( ) ( ) ( ) ( )
1 1

k k k k k
x F x F x x x J (5)
The Newton-Raphson method
The Newton-Raphson method uses the dynamical
system based on equation (3). The solution is approached
from an initial guess
0
x by the iterative formula:

The Broydens method is computationally more
efficient, and its efficiency can be improved by using
Sherman-Morison matrix inversion formula [5] for
finding ( )
1
J . However, the global convergence
properties of this method are not better than those of the
Newton-Raphson method.

x x x
k k
+ =
+1
(4)

where is an iteration number. The converging process
stops when the distance iterations become less than some
threshold:
k
0 ,
1
> <
+

k k
x x .
Note that in both methods, the dynamical systems
linearize the nonlinear function ( ) x F . This linearization
works when the nonlinear nature of the function is not
dominant or when successful initial guesses are made.
However, the dynamical systems are developed so that
their convergence properties remain unknown. Apart from
converging to the roots, they may converge to local
extrema, diverge, or be unstable, causing numerical
overflow.


The main aim of this work is to suggest a different
approach to the process of finding roots. The synergetic
iterative algorithm, which is described in the following
section, creates a dynamical system with known
properties. Perhaps the single most important feature of
this algorithm is that the generated dynamical system will
be linear according to the solutions of the system of
equations. Therefore, the global convergence of the
dynamical system (and of the method itself) is its unique
property.
Figure 1. Failures of the Newton-Raphson method
The Newton-Raphson method is known to have a
quadratic convergence rate, provided that the initial
guesses happen to be near the roots. If the initial guesses
are not near the roots, the method may fail to converge.
Figure 1 (taken from [3]) shows several types of
unsuccessful choices of initial guesses. In the top left, the
Newton-Raphson will infinitely loop at the local
minimum. The top right and bottom right demonstrate a
numerical overflow if the guess is close to a point where
the first derivative of is close to zero. In the bottom
left graph, the divergence is demonstrated.
( ) F
3. THE SYNERGETIC CONTROL THEORY
APPROACH (SCT)
In general, SCT [6,7] provides methods for
designing optimal controllers for dynamical systems,
where the controllers are coordinated with internal
expectations of the systems. The resulting dynamical
systems with their controllers have areas of attraction that
correspond to the control goals. Introducing such areas of
attraction, or attractors, to dynamical systems, is one of
the main concepts of the SCT. In this work, the attractors
are created at the roots of nonlinear systems of equations.
This allows the Synergetic Algorithm (which is described
later in this section) to converge and find the roots of the
systems.
The insufficient global convergence and instability
at the local extrema are limitations of the Newton-
Raphson method, which can be improved. For instance,
the iteration formula can be modified
x x x
k k
+ =
+

1
, where 1 0 < . The parameter
is chosen so that it minimizes (
1
2
+ k
x F ). This
modification, which is supposed to enhance the
convergence, is a heuristic, and as such, does not
guarantee a global convergence.
An attractor is a region in the state space of a
dynamical system that pulls the trajectories from nearby
areas of the state space. Depending on the dimensionality
of the state space, attractors can be points, contours, tori
or regions of fractal dimensionality. The attractors
represent the internal wishes of the dynamical system.
Whatever the initial conditions are, the system moves
towards one of the attractors and remains there infinitely.
If the requirements that the controllers provide to the
system cannot be fulfilled, the attractors cannot be
created, and the dynamical system becomes unstable.
This situation can be identified by stability analysis.
20 40 60 80 100
Time
-0.05
0.05
0.1
X

If no attractors are present, the system is unstable and
it will not converge. This situation can be diagnosed by
analyzing stability using the Lyapunov Stability Theory
[8, 9] or a potential function of the system [10].
The SCT suggests creating attractors in dynamical
systems at those areas of state spaces that correspond to
the control purposes. The control purposes are formulated
as aggregated macrovariables
i
that need to be zeroed.
The aggregated macrovariables are functions of system
variables x and control signals u : ( ) u x
i i
, .
Hence the attractors need to be introduced where all the
aggregated macrovariables are equal to zero.
Figure 2. Convergence of from different initial
conditions to attractor at 0 =
The parameter T in (8), as it can be seen from the
equation, determines the rate of convergence. By
choosing smaller T , the rate of the transition processes
can be increased.
The SCT gives an equation which can be used for
creating dynamical systems with attractors at 0 =
i
.

( ) 0 = + & T (6)
Now, let us apply the concepts of the SCT for
creating the Synergetic Algorithm for finding roots of the
systems of equations.
where T determines the rate of convergence to the
attractor, & is the derivative of the aggregated
macrovariable by time; and ( ) - some function that
influences the approaching to the attractor. One of
possible expressions for ( ) is: ( ) = . In this
case, the equation is rewritten as:
The Synergetic Algorithm
Suppose that a system of 1 nonlinear
equations is given as a set of:
M j

( ) 0 , , ,
2 1
=
N j
x x x f K (9)

This set of equations represents the purpose of the
control, which is expressed by the aggregated
macrovariables. The aggregated macrovariables in this
case are equal to the equations: ( )
j j
f . This assures
that the attractors will be created at the solutions of the
system of equations.
0 = + & T (7)

The solution of this differential equation gives the
following function for :

( )
T
t
e t

=
0

(8)
The dynamical system is created using equation (7):

( ) ( )
( ) 0 , ,
, , , ,
1
1
1
1
1
= +
|
|
.
|

\
|

N j N
N
N j N j
x x f x
x
x x f
x
x
x x f
T K &
K
&
K

(10)
where stands for time. As it is depicted in Figure 2, t
( ) t is attracted to 0 = from any initial conditions
0
.

In order to model the dynamical system, its
equations need to be solved for time derivatives.
According to time derivatives, these differential equations
are linear, thus the solution can be found using Gauss
Elimination, which has a complexity of ( )
3
n O , or other
methods.
The resulting dynamical system:

( ) T x x x g x
N i i
, , , ,
2 1
K & = (11)

can be iterated using any iterative method, like Runge-
Kutta, etc.
A specific result can be obtained for a one dimensional
case:

20 40 60 80 100
Time
-1
-0.5
0.5
1
1.5
X,Y

Figure 3. Convergence of the system (a= 2, b=1) from
different initial values

( )
( ) x f T
x f
x
x

= &

(12)

An important property of the dynamical system
given by (11), is that its attractors are located where the
system of equations has solutions. Besides, although
are nonlinear, the dynamical system is linear
according to
( )
i
g
j
. This guarantees convergence to the
solutions only, provided that they exist.
If no roots exist, iterating over the dynamical system
is not needed; instead, stability analysis can be used for
checking the systems consistency. An example of such a
system is given in Section 4.
Figure 3 demonstrates the convergence of the dynamical
system for X and Y to the root from several initial
guesses. For cases when there are several roots, several
attractors are created in the dynamical system. The initial
guess determines the root to which the system converges.
The convergence is finished after . The
obtained solution matches the root of the system, found
analytically: for
60 > t
2 = a and , it is 1 = b 38 . 0 = x and
62 . 0 = y .
By using SCT concepts for solving systems of
nonlinear equations, the synergetic algorithm achieves
two goals: (1) It checks if the system has roots prior to
iterating and (2) It converges to the roots globally, i.e.
independently to initial guesses.
Inconsistency in equations
Let us now discuss the case when the equations
have no solution. Without loss of generality and for
illustrative purposes only, a one dimensional case is
considered here.
4. EXPERIMENTS AND DISCUSSION
Examples illustrating the synergetic algorithm are
given in this section. The algorithm is discussed in
relation to: (1) A simple system of equations. (2) A
system that may be inconsistent, (3) A comparison of the
the convergence rate versus the Newton-Raphson
method.
Suppose that the equation is:

0
2
= + a x
(15)

Depending on the value of , the equation can have
one or two roots, or no roots at all. After applying the
procedure to the equation, the dynamical system that
converges to the root(s) is:
a
A simple system of equations
Suppose that we need to find the roots of the
system:

T x
a x
x


=
2
2
&
(16)

=
= +
b y x
a e e
y x
(13)


By introducing a potential function ( ) x V as
The application of the synergetic algorithm in this
case will consist of the following steps. First, the
dynamical system based on the equation (10) is
constructed:
( )
x
x
x V
& =

(17)

( )
( )

= +
= + + +
0
0
b y x y x T
a e e y e x e T
y x y x
& &
& &
(14)
we can represent the energy of the dynamical system
and use it for stability analysis. The set of maxima of the
potential function correspond to unstable states
(analogous to maxima of potential energy), minima to
stable states.
Then, the system is solved for and to obtain
the dynamical system equations. Once that is done, the
dynamical system is iterated over to obtain the solution.
x& y&
The potential function for the obtained dynamical
system is:

( ) ( ) x
T
a
T
x
x V ln
2 4
2

=
(18)

-4 -2 2 4
x
1
2
3
4 V(x)

Figure 6. Potential function V for ( ) x 0 = a
Now consider the three possible situations, based on
parameter . a
Case 1. . 0 > a

-4 -2 2 4
x
-6
-4
-2
2
4
V(x)

Figure 4. Potential function V when a ( ) x 0 >
When 0 = a , the dynamical system is stable, it has
a single attractor and globally converges to the solution as
depicted in Figure 7:

2 4 6 8 10
Time
-0.05
-0.025
0.025
0.05
0.075
0.1
X

Figure 7. Stable system with one attractor when 0 = a
Thus, the potential function has no minima and the
system is unstable (Figure 5):

2 4 6 8 10
Time
-1200
-1000
-800
-600
-400
-200
X

Figure 5. Unstable system when 0 > a
Case 3. 0 < a .
In this case the potential function has two minima as
depicted in

-4 -2 2 4
x
1
2
3
4
5
6
V(x)

Figure 8. Potential function V when ( ) x 0 < a
As it is depicted in Figure 5, the choice of close
initial guesses causes significant deviations of the
trajectories. There is no conversion and the system stops
due to numerical overflow.
Case 2. . 0 = a
In this case, the logarithm is eliminated from the
potential function, and the potential function becomes a
parabola with a focus at as depicted in Figure 6. 0 = x
As in the previous case, the dynamical system is
globally stable (except for ), however now it has
two attractors corresponding to the roots. The choice of
any of the attractors depends on the initial guesses. In this
case, the axes is a potential boundary between the two
attracting regions. A dynamical system cannot intersect
this line by itself. Therefore, the choice of the root is
determined by the initial guesses, as it is illustrated in
Figure 9.
0 = x
y
5. CONCLUSIONS In terms of the SCT, this dynamical system has a
bifurcation point at , where any deviation from
zero causes significant changes in the system.
0 = x
This paper discusses the application of the
Synergetic Control Theory for solving systems of
nonlinear equations. A Synergetic algorithm is presented
and its characteristics are discussed. The algorithm has
the the following advantages vis-a-vis existing iterative
algorithms: (1) The nature of the algorihm is such ahta it
makes possible to directly assess the consistency of the
systems of equations, (2) The global convergence to the
roots of systems is guaranteed and (3) The algorithm
exhibits controllable rate of the convergence, which can
be made faster/slower than the rate of the other
algorithms.

-3
2 4 6 8 10
Time
-2
-1
1
2
3
4 X

Figure 9. Stable system with two attractors when 0 < a
BIBLIOGRAPHY
[1] Robert J. Schilling, Sandra L. Harris, Applied
Numerical Methods for Engineers Using Matlab and
C, Brooks/Cole, 2000, pp. 228-231
[2] Acton, F.S. 1970, Numerical Methods That Work;
1990, corrected edition (Washington: Mathematical
Association of America), Chapter 14, 1970
Convergence rate
The convergence rate of the dynamical system and
hence the algorithm is determined by the value of the
parameter T in (7) and (10). The smaller the T value,
the higher the convergence rate. Since T can be any
positive number, the convergence rate can be chosen so
that the algorithm outperforms the traditional iteration
methods. For instance, the comparison of the
convergence rates between the Synergetic and the
Newton-Raphson methods is illustrated in Figure 10. The
figure demonstrates that the convergence rate can be
controlled.
[3] Stefan Jahn, et al., Qacs: Technical papers,
http://qucs.sourceforge.net/tech/node30.html, 2000
[4] C.G. Broyden, A class of methods for solving
nonlinear simultaneous equations, 1965, Mathematics
of Computation, vol. 19, pp. 577593
[5] G.J. Lastman, N.K. Sinha, Microcomputer-based
numerical methods for science and engineering, 1989,
New York: Saunders
[6] A.A. Kolesnikov, The Basis of the Synergetic Control
Theory, Moscow, Ispo-Servis, 2002 (in Russian)
There is a restriction on the choice of T that comes
from (10). If T is small and iterations for modeling the
dynamical system are large, then large values of and
are possible during the transition process. The
increase of
i
x&
i
x
T helps to keep the transition process under
control.
[7] A.A. Kolesnikov, Synergetic Control Theory,
Moscow, Energoatomizdat, 1994 (in Russian)
[8] W. Hahn, Theory and Application of Liapunov's
Direct Method, Englewood Cliffs, NJ: Prentice-Hall,
1963.
[9] R.E. Kalman, J.E. Bertram, Control System Analysis
and Design Via the 'Second Method' of Liapunov, I.
Continuous-Time Systems, J. Basic Energ. Trans.
ASME 82, 371-393, 1960
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Iterations
X

v
a
l
u
e
Synergetic, T=0.8
Synergetic, T=1.1
Newton-Raphson

[10] A.A. Kolesnikov, et al., The Modern Applied
Control Theory, Part 2, pp. 43-47, Taganrog State
University of Radioengineering, 2000 (in Russian)


Figure 10. Convergence rates of the methods