Internal Agent States: Experiments Using the Swarm Leader Concept

Mohamed H. Mabrouk, Craig W. Murray, Kevin Johnstone and Colin R. McInnes

Abstract — In recent years, an understanding of the the attraction potential strength and range the goal point
operating principles and stability of natural swarms has and the repulsive potential strength and range of the zth
proven to be a useful tool for the design and control of obstacle point, respectively. The equations of motion for N
artificial robotic agents. Many robotic systems, whose agents that contains No obstacle points and a goal point at
design or control principals are inspired by behavioural
aspects of real biological systems such as leader-follower
position G, are then defined for the ith agent as
relationship, have been developed. We introduced an
algorithm which successfully enhances the navigation
performance of a swarm of robots using the swarm v i = ri (1)
leader concept. This paper presents some applications
based on that work using the simulations and
experimental implementation using a swarming
behaviour test-bed at the University of Strathclyde. mi v i = − β i v i − ∇ iV global (ri ) (2)
Experimental and simulation results match closely in a
way that confirms the efficiency of the algorithm as well
as its applicability. (3)
V global (ri ) = Vinteraction (ri ) + V goal (rig ) + Vobstacles (rio )
I. INTRODUCTION
Research activity on autonomous robots has witnessed a N

Vglobal (ri ) = ∑  C r j e i j 
− r −r lr j − ri − r j la j
surge, especially in the field of artificial robotic systems − Ca j e
j ≠i  
that inspire ideas from real biological systems due to their No
important commercial applications [1]. Swarms of self- + ∑ Co z e
− ri − ro z lo z − ri − r g lg
− Cg e (4)
organizing agents that exchange information have a greater z =1
functionality than single robots [2]. In these systems, large
numbers of identical autonomous robots are controlled
using architectures that are inspired from natural systems In earlier work [7], we introduced sets of first order
such as insect swarms, bird flocks and fish schools [3]–[6]. differential equations to describe the free parameters of the
We use a model designed to simulate the motion of a potential field (internal state) to solve the local minimum
swarm of robots, which consists of N agents. The ith agent problem. For artificial potential field based navigation,
is represented with mass mi, position ri and relative there have been several attempts to solve the local
distance rij between the ith and jth agents. The generalized minimum problem. The problem for a swarm of agents
Morse potential, which decays exponentially at large attracted to a goal point at position G is defined such that
distances and represents a comparatively realistic an artificial potential field at G induces motion towards the
description of natural swarming agents, is used to define goal. However, in order to prevent collision with a static
the interactions amongst the swarm agents Vinteraction(ri), the obstacle, an additional repulsive potential field is required.
attraction potential of the goal Vgoal(rig) and the repulsive In general, a local minimum may form due to the
potential of the No obstacles Vobstacles(rio). Unit mass agents superposition of the goal potential and that of the obstacles,
are considered for simplicity. To prevent the agents from resulting in the agent, or swarm of agents, becoming
reaching large speeds, a dissipative frictional force with trapped in a state other than the goal G. Considering this
coefficient βi for the ith agent is added. problem, the entire swarm, or part of the swarm will be
The potential is characterized by attractive and repulsive trapped at the obstacle since the agents trapped inside the
interaction potential fields of strength Ca and Cr with obstacle will experience two virtual opposite forces, as
ranges la and lr respectively, while Cg, lg, Coz and of loz are shown in Fig. 1. We introduced three new concepts; the
swarm leader concept [7], the swarm aggregation concept
Manuscript received May 02, 2008.
M. H. Mabrouk is with the Department of Mechanical Engineering,
[8],[9], and the swarm vortex-like behaviour concept [10]
University of Strathclyde, Glasgow, UK. (phone:+44 (0) 141 548 4851; to enhance the performance of the internal state model for
fax: +44 (0) 141 552 5105; e-mail: mohamed.mabrouk@ strath.ac.uk). agents behaviour that allows them to effectively solve this
C. W. Murray is with the Department of Mechanical Engineering, key problem. In this paper, we introduce applications using
University of Strathclyde, Glasgow, UK (e-mail: craig.murray@
strath.ac.uk).
the internal state model enhanced with the swarm leader
K. Johnstone is with the Department of Mechanical Engineering, concept through simulation and test-bed facility.
University of Strathclyde, Glasgow, UK (e-mail: kevin.johnstone@
strath.ac.uk)
C. R. McInnes is with the Department of Mechanical Engineering,
University of Strathclyde, Glasgow, UK (e-mail: colin.mcinnes@
strath.ac.uk)
 updated position of their leader. In the algorithm used in
this paper [7], the agents have information about the
leaders by sharing global information, which is the agents
potential parameters that are employed to express the
leader-follower relationship. The swarm leader concept
Y
enables the swarm in Fig. 1 to efficiently solve the problem
by making the agents follow the agent that finds a clear
way to the goal, as shown in Fig. 2.

X
Fig.1. a) t=1

Fig.2.a) t=12 X

Fig.1.b) t=26
Fig.1. Classical reactive problem for a swarm of agents

II. AGENT INTERNAL STATE MODEL WITH SWARM Y

LEADER CONCEPT

There is a considerable body of research concerning

building artificial systems inspired by swarm leaders
phenomenon in real biological systems. In [11]–[12], the
authors introduced dynamic models based on the distance
and the angle between leaders and followers, which Fig.2.b) t=24 X

indicates that the agents must know who and where their
leaders are. Different leader roles were discussed in Wang
and [13] and a convergent condition, in which the
followers need to have the leaders’ states by sharing global
information, was constructed by using contraction theory.
In [14], the swarm members know which members are the
leaders in a leader based control strategy. Similarly, the
Y
followers need to know who the leaders are in [15] where
the leader-follower systems are investigated in terms of
controllability and optimal control. In [16], the authors
discuss the importance of updating the follower
information concerning the leader position.
Experimentally, much research work focuses on leader-
follower relationship in multi-agent systems. In [2], the Fig.2.c) t=48
X

author tested a set of communication techniques and a

library of behaviors, among which follow-the-leader
behaviour is programmed, on a swarm of 100 physical
robots.
From this background it can be seen that although local
information can be used to control the relative distances
and angles amongst swarm members, the followers must
know which members are their leaders. In addition, the
followers need to share global information about the
 parameters according to some task; for example a scout
agent finding something of interest for the swarm.

IV. SIMULATION RESULTS

Y

A. Escaping a trap I
To show the swarm leader concept, Eq. (1) – Eq. (4) are
now used to simulate the agents’ motion for Np identical
agents which are trapped behind a barrier that consists of
X
No identical obstacle points, as shown in Fig. 3, where G is
Fig.2.d) t=107 a goal point which has an attraction potential of low
Fig. 2. Behaviour of a swarm using the internal state interaction range lg such that it does not extend to the
model and swarm leader, t =12-107 [7] region inside the trap.
The challenge in this situation is that the agents are stuck
inside a trap whose only exit is located away from the goal
III. RESPONSES TO A TEMPORARY LEADER
position and they are not strongly attracted to the goal. In
For a complete understanding of the role of the leader in this situation, a condition is defined such that if one of the
a free system and how it affects the swarm behaviour using agents finds its way through the exit, it will gain a higher
the artificial potential approach for the agents interactions, attraction potential coefficient Ca and higher attraction
the motion of a free swarm whose agents experience an potential range la. These conditions now make any
attraction to one of them will be considered. For a swarm succeeding agent a temporary leader for the rest of the
of agents, let agent (h) be the temporary leader that has agents and the swarm center therefore accelerates to the
higher Ca and la than the other agents of the swarm. leader position, leading them out of the trap. Then, the
Recalling Eq. (1-4) for obstacles and goal free environment leader and subsequently the swarm are attracted to the goal.
and using Vij instead of Vinteraction for simplicity, the global Figure 3 shows the agents randomly moving inside the trap
potential equation for ith agent is until one agent succeeds in escaping and so becomes a
temporary leader leading the rest of the agents out of the
Np
trap.
V global (ri ) = Vij (ri ) = ∑  C r j e  (5)
− ri −r j lr j − ri −r j la j
− Ca j e
j ≠i   B. Escaping a trap II (a ‘rescue’ mission)
We now consider another application of internal states
Np using communication through interaction between swarm
Defining the swarm center velocity as r = 1
c
Np
∑r
i =1
i members. The application is a ‘rescue’ mission that has
been given to one of the team members, which uses the
and noting that ∇ iVih (rih ) = Vih′ (rih ).rˆih , the equation of internal state model with the swarm leader concept, to
motion of the swarm center will be assist other team members that use fixed internal states and
fail to reach the goal according to the local minimum that
forms behind a C-shape obstacle (which has not been prior
  known).
N Np Np Np  (6)
1  p
rc = − ∑ βvi + ∑ ∑ ∇ iVij + ∑ Vih′ (rih ).rˆih  The key idea is that the rescuing member gains
N p  i =1 i =1 j ≠i i =1
 leadership characteristics as it realizes it has a clear way to
 i≠h 
 j≠h  the goal. This application has the advantage of using the
internal state model with only some agents in the swarm
where Vih is the interaction potential that affects ith agent by whose task will be to act as ‘leaders/scouts’ for the rest of
the leader agent (h). Since rˆij = −rˆ ji , the double summation the swarm, mimicking the behaviour in real biological
in Eq. (6) will cancel to yield systems [17].
The scenario, shown in Fig. 4, demonstrates a swarm
that uses fixed internal states in two groups. The first group
N
β 1 p has a clear path to the goal while the other group is trapped
rc +
Np
rc = − ∑ Vih′ (rih ).rˆih
N p i =1
(7)
in a local minimum formed behind a C-shape obstacle. The
agents that successfully reach the goal clearly did not lead
the individuals of the swarm, which are trapped in the local
which represents a damped oscillator with a forcing term minimum.
generated by the lead agent. It can therefore be concluded
that the agents will be attracted to any agent considered as
a temporary goal if it has a larger attraction interaction
 Y

Fig.4.d) t=35
X

Fig. 3. Swarm leader concept in a trap application, t=0.5, t=9,

t=12, t=15, t=18, t=22.3
Fig.4.d) t=70 X

Fig. 4. Behaviour of a swarm using conventional fixed

internal states, t =2-70

Fig.4.d) t=2 X

Fig.5.d) t=1 X

Fig.4.d) t=30 X

X
Fig.5.d) t=10
 We now compare this scenario with another scenario
through which one of the agents is assigned the rescue task
by using the dynamic internal state model with the swarm
leader concept, as show in Fig. 5. As the leader agent has a
clear way to the goal, it gains leader properties (large Ca)
Y
according to the internal state model with the swarm leader
concept. This will enable the leader agent to manipulate the
potential in the workspace such that the individuals trapped
in the local minimum are attracted to it rather than to the
goal, as shown in Fig. 5, resulting in successful escape
from the trap.
Fig.5.d) t=19 X
V. EXPERIMENTAL RESULTS

The ‘rescue’ mission application is implemented in a

swarming behaviour test-bed; University of Strathclyde.
The test-bed has been built to test the behaviour of agents,
Y which interact via pair-wise interactions of short-range
repulsions and long-range attraction. Every agent, as shown
in Fig. 6, is equipped with an external source of light, a
short-range touch sensor to implement the repulsion type
forces, and long-range light sensors to implement the
attraction type forces. The same procedures in the rescue
mission application are implemented using only 3 agents
Fig.5.d) t=25 X
for simplicity.

Fig.5.d) t=33 X

Fig. 6. One of the agents used in the test

Fig.5.d) t=44 X

Fig. 5. Behaviour of a swarm using the internal state

model and swarm leader, t =1-44
 Transparent C-shape
obstacle

Trapped agent

Fig.7.a) t=2 sec (agent trapped in local minimum) Fig.7.d) t=26 sec (trapped agent follow the leaders)

Internal state agent

Fig.7.b) t= 18 sec (leaders approach) Fig.7.e) t=29 sec (trapped agent escapes)

Fig.7.c) t=23 sec (trapped agent is attracted to the leaders) Fig.7.f) t=36 sec (trapped agent reaches the goal)
Fig. 7. Swarm leader concept implementation, t = 1- 36 sec
 The scenario of the local minimum problem is Proc. American Contr. Conf., pages 1084 -1089. Portland,
implemented using an agent that is attracted to a visible OR,USA, 2005.
[6] Y. L. Chuang, Y. R. Huang, M. R. D’Orsogna, and A. L.
goal (light source located at the goal position) through a Bertozzi, “Multi-vehicle flocking: Scalability of cooperative
transparent barrier. When the agent reaches the barrier, it control algorithms using pairwise potential,” 2007 IEEE
is repelled according to the function of its touch sensor. International Conference on Robotics and Automation,
However, as the agent moves away from the barrier it is 2292-2299, 2007.
[7] M. Mabrouk and C. McInnes, “Swarm robot social potential
attracted again according to the function of its light sensor fields with internal agent dynamics,” in Proceeding of the
and then tries again to reach the goal through the barrier, as 12th International Conference on Aerospace Sciences &
shown in Fig. 7.a-b). In this case, the agent gets stuck in a Aviation Technology (ASAT12), ROB-02:1-14, 2007a.
[8] M. Mabrouk and C. McInnes, “Swarm potential fields with
position away from the goal position defining the local internal agent states and collective behaviour,” in
minimum problem. In the simulation, using the internal Proceeding of Towards Autonomous Robotic Systems,
states model enables the stuck agent to solve the problem TAROS, pp. 90-96, 2007b.
by following another agent ‘the leader’ that has a clear path [9] M. Mabrouk and C. McInnes, “Solving the potential field local
minimum problem using internal agent states,” accepted by
to the goal. This idea is implemented through a the Journal of Robotics and Autonomous systems, 2007c.
modification of the leader agents such that their external [10] M. Mabrouk and C. McInnes, “Wall following to escape local
light source becomes brighter as they have a clear way to minima for swarms of agents using internal states and
the goal (they gain leader properties which is represented emergent behaviour,” accepted by the International
Conference on Computational Intelligence and Intelligent
by higher Ca according to the internal state model with the Systems, ICCIIS 2008.
swarm leader concept). This enables the leader agents to [11] J. Desai, J. Ostrowski, and V. Kumar, “Modelling and control
attract the trapped agent more than the goal such that the of formations of nonholonomic mobile robots,” IEEE
individual trapped in the local minimum is attracted to the transactions on robotics and automation, vol.17, No.6,
leaders rather than to the goal, as shown in Fig. 7.c-f). pp.905-908, 2001.
[12] A. Das, R. Fierro, V. Kumar, J. Ostrowski, J. Spletzer and C.
Taylor, “A vision-based formation control framework,”
VI. CONCLUSIONS IEEE transactions on robotics and automation, vol.18,
No.5, pp.813-825, 2002.
This paper presents some applications based on earlier [13] W. Wang, and J. Slotine, “A theoretical study of different
work which aimed to enhance APF based navigation leader roles in networks,” IEEE transactions on robotics
and automation, vol.51, No.7, pp.1156-1161, 2006.
performance of multi-agent systems using one of the most [14] D. Dimarogonas, M. Egerstedt and K. Kyriakopoulos, “A
common swarming behaviours in natural systems; the leader-based containment control strategy for multiple
swarm leader concept. Two applications are introduced unicycles,” in Proceedings of the IEEE conference on
using simulations. The first application is escaping a trap decision and control, pp. 5968-5973, 2006.
[15] M. Ji, A. Muhammad and M. Egerstedt, “Leader-based multi-
application, through which if one of the agents finds its agent coordination: controllability and optimal control,” in
way through the exit, it will gain a higher attraction Proceedings of the American control conference, 2006.
potential coefficient Ca and higher attraction potential [16] Z. Wang and D. Gu, “Fuzzy control of leader-follower
range la becoming a temporarily leader to the rest of the flocking system,” in proceedings of towards autonomous
robotic systems, TAROS 2007, pp 70 –77, 2007.
individuals inside the trap to help them escape. The second [17] M. Lindauer. Communication among social bees. Cambridge,
application is a ‘rescue’ mission application to use the Massachusetts: Harvard University Press, 1971.
internal state model with only some agents in the swarm
whose task will be to act as ‘leaders’ for the rest of the
swarm, mimicking the behaviour in real biological
systems. The implementation of the second application
using a swarming behaviour test-bed confirms the
applicability of the used model as well as its ability to
enhance the performance for a real swarm of robots.

REFERENCES
[1] B. Webb, “Can robots make good models of biological
behaviour?,” The Behavioural and Brain Sciences 24 (6),
2003.
[2] J. McLurkin, “Stupid Robot Tricks: A behaviour-based
distributed algorithm,” MSc Thesis. MIT, Library for
Programming Swarms of Robots, 2004.
[3] E. Bonabeau, M. Dorigo, and G. Theraulaz. Swarm
intelligence: From natural to artificial systems. Oxford
University Press, New York, 1999.
[4] V. Gazi, and K. Passino, “Stability analysis of swarms,” in
IEEE Trans. Autom. Contr. Vol. 48, pp. 692-697, 2003.
[5] B. Q. Nguyen, Y. L. Chuang, D. Tung, C. Hsieh, Z. Jin, L. Shi,
D. Marthaler, A. L. Bertozzi, and R. M. Murray, “Virtual
attractive repulsive potentials for cooperative control of
second order dynamic vehicles on the caltech MVWT,” in