Ant Colony Optimization

Ant Algorithms
 ACO Concept
• Ants (blind) navigate from nest to food source
• Shortest path is discovered via pheromone trails
– each ant moves at random
– pheromone is deposited on path
– ants detect lead ant’s path, inclined to follow
– more pheromone on path increases probability of
path being followed
 ACO System
• Virtual “trail” accumulated on path segments
• Starting node selected at random
• Path selected at random
– based on amount of “trail” present on possible paths
from starting node
– higher probability for paths with more “trail”
• Ant reaches next node, selects next path
• Continues until reaches starting node
• Finished “tour” is a solution
 ACO System, cont.
• A completed tour is analyzed for optimality
• “Trail” amount adjusted to favor better solutions
– better solutions receive more trail
– worse solutions receive less trail
– higher probability of ant selecting path that is part of a
better-performing tour
• New cycle is performed
• Repeated until most ants select the same tour
on every cycle (convergence to solution)
 ACO System, cont.
• Often applied to TSP (Travelling Salesman
Problem): shortest path between n nodes
• Algorithm in Pseudocode:
– Initialize Trail
– Do While (Stopping Criteria Not Satisfied) – Cycle Loop
• Do Until (Each Ant Completes a Tour) – Tour Loop
• Local Trail Update
• End Do
• Analyze Tours
• Global Trail Update
– End Do
 Background
• Discrete optimization problems difficult to
solve
• “Soft computing techniques” developed in
past ten years:
– Genetic algorithms (GAs)
• based on natural selection and genetics
– Ant Colony Optimization (ACO)
• modeling ant colony behavior
 Background, cont.
• Developed by Marco Dorigo (Milan, Italy),
and others in early 1990s
• Some common applications:
– Quadratic assignment problems
– Scheduling problems
– Dynamic routing problems in networks
• Theoretical analysis difficult
– algorithm is based on a series of random decisions
(by artificial ants)
– probability of decisions changes on each iteration
 Swarm Intelligence

• Collective system capable of accomplishing

difficult tasks in dynamic and varied
environments without any external guidance or
control and with no central coordination

• Achieving a collective performance which could

not normally be achieved by an individual
acting alone

• Constituting a natural model particularly suited

to distributed problem solving
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
 Inherent features

• Inherent parallelism
• Stochastic nature
• Adaptivity
• Use of positive feedback
• Autocatalytic in nature
 Natural behavior of an ant
Foraging modes

• Wander mode
• Search mode
• Return mode
• Attracted mode
• Trace mode
• Carry mode
 Natural behavior of ant

Ant Algorithms – (P.Koumoutsakos – based on notes L. Gamberdella (www.idsia.ch)

 How to implement in a program

•Ants: Simple computer agents

•Move ant: Pick next component in the construction of

solution

•Trace:  k
Pheromone, i , j , a global type of information

•Memory: MK or TabuK

•Next move: Use probability to move ant

 Implementation
• Can be used for both Static and Dynamic
Combinatorial optimization problems
• Convergence is guaranteed, although the
speed is unknown
– Value
– Solution
ACO for the Traveling Salesman Problem

The TSP is a very important problem in the

context of Ant Colony Optimization because it is
the problem to which the original AS was first
applied, and it has later often been used as a
benchmark to test a new idea and algorithmic
variants.

• It is a metaphor problem for the ant colony

• It is one of the most studied NP-hard problems in the combinatorial optimization
• it is very easily to explain. So that the algorithm behavior is not obscured by
too many technicalities.
 Ant Colony Optimization (ACO) for TSP

Graph (N,E): where N = cities(nodes), E = edges

= the tour cost from city i to city j (edge weight)

d ij
Ant move from one city i to the next j with some transition probability.

B
A

C
Ant Colony Optimization (ACO) for TSP

Each edge is associated a static value based on the

edge-cost (r,s) = 1/dr,s.

Each edge of the graph is augmented with a trace

(r,s) deposited by ants. Initially, 0.

Trace is dynamic and it is learned at run-time

Each ant tries to produce a complete tour, using the

probability depending on (r,s) and (r,s) to choose
the next city.
ACO Algorithm for TSP
Initialize

Place each ant in a randomly chosen city

For Each Ant

Choose NextCity(For Each Ant)

yes more cities

to visit

No

Return to the initial cities

Update trace level using the tour cost for each ant

No
Stopping
criteria

yes

Print Best tour

 Algoritma
1. Inisiasi harga parameter-parameter algoritma
2. Inisiasi node pertama setiap semut
3. Mengisi panjang node pertama ke dalam tabu list
4. Menyusun rute kunjungan setiap semut ke setiap node
5. Menghitung panjang ruta setiap semut
6. Mencari rute terpendek
7. Menghitung Perubahan harga intensitas jejak kaki semut antar
node Menghitung harga intensitas jejak kaki semut antar node
untuk siklus berikutnya
8. Reset harga perubahan intensitas jejak kaki semut antar node
9. Mengosongkan tabu list
10. Menglangi langkah 2 jika diperlukan
 A simple TSP example []

[]

A
B
2

[]

C
3

[]

4
D
E []

dAB =100;dBC = 60…;dDE =150

5
 Iteration 1

[A] [B]

2
1
A
B
[C]

3
C

[D] [E]

4
D 5
E
 How to choose next city?

[A]

1
A
[A]
B

 
1
 [  ( t )] [  ]
  [  ( t )] C[  ]  if j  allowed k
[A] ij ij

pij ( t )  
k
ik ik
kallowedk
 [A,D]
1
[A]

0 otherwise
1 1
D
E
 Iteration 2
[E,A] [C,B]

5 3
A
B
[B,C]

2
C

[A,D]
[D,E]

1
D 4
E
 Iteration 3

[D,E,A] [E,A,B]

4 5
A
B
[A,D,C]

1
C

[B,C,D]
[C,B,E]

2
D 3

E
 Iteration 4
[B,C,D,A] [D,E,A,B]

2 4
A
B
[E,A,B,C]

5
C

[C,B,E,D]
[A,DCE]

D 3
1
E
 Iteration 5
[C,B,E,D,A] [A,D,C,E,B]

1
3
A
B
[D,E,A,B,C]

4
C

[E,A,B,C,D]

[B,C,D,A,E]

D 5

E 2
 Path and Trace Update
[A,D,C,E,B]

L1 =300
1
[B,C,D,A,E]

L2 =450
2
Q
[C,B,E,D,A]
 if (i, j )  bestTour
L3 =260
 ik, j   Lk
0
3
 otherwise
[D,E,A,B,C]

L4 =280
4

[E,A,B,C,D]

L5 =420
5
End of First Run

Save Best Tour (Sequence and length)

All ants die

New ants are born

 Ant System (Ant Cycle) Dorigo [1] 1991
t = 0; NC = 0; τij(t)=c for ∆τij=0
Place the m ants onthe n nodes 
 [  ij ( t )] [ ij ]
Initialize


 Update tabu (s)  
if j  allowed k
pijk ( t )    [  ik ( t )] [ ik ]
k
Tabu list management

kallowedk
  [ ( t )] [ ] if j  allowed
ij
 
Choose the city j to move
ij

p (
 
   [  ( t )] [  ]
to. Use probability
 
k
k
ij t )0 ik
otherwise
ik
kallowedk
Move k-th ant to town j.
0 otherwise Insert town j in tabuk(s)

Compute the length Lk of every ant

Update the shortest tour found

 ij (t  1)   (t )  For every =edge (i,j)

Compute  ijij ( t  n )   ij ( t )  ij
 ij
For k:=1 to m do
Q
 if ( i , j )  tour described by tabu k
 k
i, j   Lk
0

Q
otherwise

 if (i, j )  bestTour
 ij :  ij   ijk

 k
  Lk
Yes

iSet
, jt = t + n; NC=NC+1; ∆τ =0
0
ij
NC<NCmax
&& not

 otherwise stagn.
No

End
 Stopping Criteria

• Stagnation
• Max Iterations
ACO : Ant Colony Optimization for TSP
Performance

• Algorithm found best solutions on small problems

(75 city)
• On larger problems converged to good solutions –
but not the best
• On “static” problems like TSP hard to beat specialist
algorithms
• Ants are “dynamic” optimizers – should we even
expect good performance on static problems
• Coupling ant with local optimizers gave world
class results….
 Parameters of ACO

Comparison among three strategies, averages over 10 trials.

Other parameters: Q, constant for trace updates, and

m, the number of ants

Taken from Dorigo [1]

 Pheromone trail and heuristic function:
are they useful?

Comparison between ACS standard, ACS with no heuristic (i.e., we set B=0), and ACS in which ants
neither sense nor deposit pheromone. Problem: Oliver30. Averaged over 30 trials, 10,000/m iterations per trial.
 General ACO

• A stochastic construction procedure

• Probabilistically build a solution
• Iteratively adding solution components to partial
solutions
- Heuristic information
- Trace/Pheromone trail
• Reinforcement Learning reminiscence
• Modify the problem representation at each
iteration
 General ACO

• Ants work concurrently and independently

• Collective interaction via indirect
communication leads to good solutions
 Some inherent advantages

• Positive Feedback accounts for rapid discovery

of good solutions
• Distributed computation avoids premature
convergence
• The greedy heuristic helps find acceptable
solution in the early solution in the early stages
of the search process.
• The collective interaction of a population of
agents.
 Disadvantages in Ant Systems

• Slower convergence than other Heuristics

• Performed poorly for TSP problems larger
than 75 cities.
• No centralized processor to guide the AS
towards good solutions
 Applications

• Traveling Salesman Problem

• Quadratic Assignment Problem
• Network Model Problem
• Vehicle routing
 Conclusion
• ACO is a relatively new metaheuristic approach for
solving hard combinatorial optimization problems.
• Artificial ants implement a randomized construction
heuristic which makes probabilistic decisions.
• The cumulated search experience is taken into
account by the adaptation of the pheromone trail.
• ACO shows great performance with the “ill-
structured” problems like network routing.
• In ACO local search is important to obtain good
results.
 References
• Dorigo M. and G. Di Caro (1999). The Ant Colony Optimization Meta-Heuristic. In D. Corne, M.
Dorigo and F. Glover, editors, New Ideas in Optimization, McGraw-Hill, 11-32.
• M. Dorigo and L. M. Gambardella. Ant colonies for the traveling salesman problem. BioSystems,
43:73–81, 1997.
• M. Dorigo and L. M. Gambardella. Ant Colony System: A cooperative learning approach to the
traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1):53–66, 1997.
• G. Di Caro and M. Dorigo. Mobile agents for adaptive routing. In H. El-Rewini, editor, Proceedings
of the 31st International Conference on System Sciences (HICSS-31), pages 74–83. IEEE
Computer Society Press, Los Alamitos, CA, 1998.
• M. Dorigo, V. Maniezzo, and A. Colorni. The Ant System: An autocatalytic optimizing process.
Technical Report 91-016 Revised, Dipartimento di Elettronica,Politecnico di Milano, Italy, 1991.
• L. M. Gambardella, ` E. D. Taillard, and G. Agazzi. MACS-VRPTW: A multiple ant colony system
for vehicle routing problems with time windows. In D. Corne, M. Dorigo, and F. Glover, editors,
New Ideas in Optimization, pages 63–76. McGraw Hill, London, UK, 1999.
• L. M. Gambardella, ` E. D. Taillard, and M. Dorigo. Ant colonies for the quadratic assignment
problem. Journal of the Operational Research Society,50(2):167–176, 1999.
• V. Maniezzo and A. Colorni. The Ant System applied to the quadratic assignment problem. IEEE
Transactions on Data and Knowledge Engineering, 11(5):769–778, 1999.
• Gambardella L. M., E. Taillard and M. Dorigo (1999). Ant Colonies for the Quadratic
Assignment Problem. Journal of the Operational Research Society, 50:167-176.