482CSM3 - SELECTED TOPICS - Chapter 3 - Ant Colony Optimization Techniques
482CSM3 - SELECTED TOPICS - Chapter 3 - Ant Colony Optimization Techniques
SELECTED TOPICS
Prepared by:
Section 1
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf
Inherent features
• Inherent parallelism
• Stochastic nature
• Adaptivity
• Use of positive feedback
• Autocatalytic in nature
Natural behavior of an ant
Foraging modes
• Wander mode
• Search mode
• Return mode
• Attracted mode
• Trace mode
• Carry mode
Natural behavior of ant
•Pheromone: k
i,j
•Memory: MK or TabuK
[]
A
B
2
[]
C
3
[]
4
D
E []
[A] [B]
1 2
A
B
[C]
3
C
[D] [E]
4
D 5
E
How to build next sub-solution?
[A]
1
A
[A]
B
1
[ ( t )]
[ ]
[ ( t )] C[ ] if j allowed k
[A] ij ij
pij ( t )
k
ik ik
kallowedk
[A,D]
1
[A]
0 otherwise
1
1
D
E
Iteration 2
[E,A] [C,B]
5 3
A
B
[B,C]
2
C
[A,D]
[D,E]
1
D 4
E
Iteration 3
[D,E,A] [E,A,B]
4 5
A
B
[A,D,C]
1
C
[B,C,D]
[C,B,E]
2
D 3
E
Iteration 4
[B,C,D,A] [D,E,A,B]
2 4
A
B
[E,A,B,C]
5
C
[C,B,E,D]
[A,DCE]
D 3
1
E
Iteration 5
[C,B,E,D,A] [A,D,C,E,B]
1
3
A
B
[D,E,A,B,C]
4
C
[E,A,B,C,D]
[B,C,D,A,E]
D 5
E 2
Path and Pheromone Evaluation
[A,D,C,E,B]
Q
if ( i , j ) tour
L1 =300 ik, j Lk
0
1 otherwise
[B,C,D,A,E]
L2 =450
2
[C,B,E,D,A]
total
3 A ,B
1
A ,B
L3 =2602
A ,B 3
A ,B 4
A ,B 5
A ,B
[D,E,A,B,C]
L4 =280
4
[E,A,B,C,D]
L5 =420
5
End of First Run
[ ij (Update t )]tabu [(s)ij ]
allowed k
k
Tabu list management
if j
pijk ( t ) [[ ( tik
)] ([t
)]
] [ ik ]
Choose the city j to move
ij ij
kallowed
if j allowed
to. Use probability
p ( t ) [ k ( t )] [ ]
k
k
ij ik ik
kallowedk
0
Move k-th ant to town j.
0 otherwise otherwise
Insert town j in tabu (s) k
ij ( t n ) ij ( t ) ij
For every =edge (i,j)
Compute ij ( t n ) ij ( t ) ij
For k:=1 to m do
Q
if ( i , j ) tour described by tabuk
k
i,j Lk
0
Q otherwise
k
i, j Lk
Yes
0
Set t = t + n; NC=NC+1; ∆τij=0 NC<NCmax
&& not
otherwise stagn.
No
End
Stopping Criteria
• Stagnation
• Max Iterations
General ACO
• Elitist strategy
e / Lgb
( t ) if arc(i, j) T gb
ijgb ( t )
0 otherwise
• ASrank
w1
ij ( t 1 ) ( 1 ) ij ( t ) ( w r ) ijr ( t ) w ijgb ( t )
r 1
Improvements to AS
• ACS
– Strong elitist strategy
– Pseudo-random proportional rule
j arg max jN k ij ( t ) ij
i
With Probability (1- q0):
[ ij ( t )] [ ij ]
[ ( t )] [ ] if j allowed k
pij ( t )
k
ik ik
kallowedk
0 otherwise
Improvements to AS
ij ( t 1 ) ( 1 ) ij ( t ) ijbest ( t )
• MMAS
min ij max
– High exploration at the beginning
– Only best ant can add pheromone
– Sometimes uses local search to improve its
performance
Dynamic Optimization Problems
• Problem is NP-hard
• Classical combinatorial
optimization problem to
test.
ACO for the Traveling Salesman Problem
Discrete Graph
Ant move from one city i to the next j with some transition probability.
B
A
C
Ant Systems Algorithm for TSP
Initialize
No
Update pheromone level using the tour cost for each ant
No
Stopping
criteria
yes
Comparison between ACS standard, ACS with no heuristic (i.e., we set B=0), and ACS in which ants
neither sense nor deposit pheromone. Problem: Oliver30. Averaged over 30 trials, 10,000/m iterations per trial.
Trail pheromone in AS
K ij (t ) for each edge that it has used. depends on how well the ant
has performed.
4.a candidate list to restrict the choice of the next city to visit.
ACS : Ant Colony System for TSP
ACO State Transition Rule
( A, B) 150 ( A, B) 1 / 10
( A, B) 35 ( A, B) 1 / 7
( A, B) 90 ( A, B) 1 / 15
… similar to evaporation
ACS Global Trail Updating
Here in ACO the global pheromone trail update is applied only to the
best tour since trial began.
ACO : Candidate List
Problem is:
• Assign n activities to n locations (campus and mall
layout).
• F= f
h ,k n ,n , f i , j ,flow from activity h to activity k
• Assignment is permutatio
• Minimize: n
C ( ) d ij f (i ) ( j )
i , j 1
• It’s NP hard
QAP Example
Locations B
Facilities
? C
biggest flow: A - B
B C A B
Example 1 2
4 2 Location
3 4
1 3 Department („Facility“)
Constructive method:
step 1: choose a facility j
Characteristics:
– each ant leaves trace (pheromone) on the chosen couplings (i,j)
– assignment depends on the probability (function of pheromone trail and a
heuristic information)
Ants choose the location according to the heuristic desirability “Potential goodness”
1
ij
sij
AS-QAP Constructing the Solution
[ (t )] [ ]
pijk (t ) if j N ik
ij ij
lNik ij
[ (t )] [ ij ]
where k is the feasible Neighborhood of node i
N i
Q
if facility i is assigned to location j in the solution of ant k
kij Jk
0 otherwise
diversification
Intensification
HAS-QAP algorithms Performance
Comparisons with some of the best heuristics for the QAP have shown that
HAS-QAP is among the best as far as real world, and structured problems are
concerned.
So far, the most interesting applications of ant colony optimization were
limited to travelling salesman problems and quadratic assignment problems..
Section III
Network Routing
Vehicle Routing
Conclusions
3 1
5 2
2
ROUTING IN COMM. NETWORKS
Problem statement
• Dynamic Routing
At any moment the pathway of a message must be
as small as possible. (Traffic conditions and the
structure of the network are constantly changing)
• Load balancing
Distribute the changing load over the system and
minimize lost calls.
ROUTING IN COMM. NETWORKS
Objective:
Traditional way:
“Central Controllers”
Disadvantage:
• Communication overhead.
• Fault tolerance ~ Controller Failure.
• Scalability
• Dynamic ~ Uncertainty
• Authority.
Algorithm I
Ant-based load balancing in telecommunication
networks
(Schoonderwoerd, R. -1996)
• Network has n nodes.
• Each node has its Routing Table (pheromone table) {Ri[n-1][k]}
• Initialize: equilibrium Routing table (all nodes have the same value
or normalized random values)
• Each node lunches {n-1} ants (agents) each to different destination.
• Each ant select its next hop node proportionally to goodness of
each neighbor node
• routing table of the node that just the ant arrived to is updated as
follows:
Reference: Schoonderwoerd, R. (1996) “Ant-based load balancing in telecommunication networks” Adapt. Behav. 5, 169-207
Algorithm I (cont.)
1
Where f
age
Reference: Schoonderwoerd, R. (1996) “Ant-based load balancing in telecommunication networks” Adapt. Behav. 5, 169-207
Algorithm I (cont.)
Example: Pheromone Table @ NODE 6
Next node
1 3 4 7 8
1 0.850 0.100 0.009 0.001 0.090
2 0.045 0.100 0.520 0.325 0.010
7 Destination 3 0.020 0.925 0.045 0.008 0.002
5 Nodes 4 0.004 0.100 0.800 0.090 0.006
5 0.010 0.095 0.470 0.410 0.015
7 0.005 0.003 0.020 0.948 0.024
2 8 0.015 0.005 0.002 0.023 0.955
4
6
Node8 Node1
8
3
Node3
1
Node7
Node4
Reference: Schoonderwoerd, R. (1996) “Ant-based load balancing in telecommunication networks” Adapt. Behav. 5, 169-207
Algorithm I (cont.)
old
Example: 1
Routing table @ node 1
Next node Next node
2 3 2 3
Destination 3 0.50 0.50 Destination 3 0.40 0.60
node 2 0.50 0.50 node 2 0.50 0.50
4 0.50 0.50 4 0.50 0.50
old
2 1
4
1
3
0.25
Reference: Schoonderwoerd, R. (1996) “Ant-based load balancing in telecommunication networks” Adapt. Behav. 5, 169-207
Algorithm I (cont.)
Reference: Schoonderwoerd, R. (1996) “Ant-based load balancing in telecommunication networks” Adapt. Behav. 5, 169-207
Algorithm II
AntNet
(Di Caro & Dorigo - 1997)
r+, r- r+, r-
B-ANT B-ANT B-ANT
1 2 3
F-ANT F-ANT
F-ANT
F-Ants also measure the quality of the trip (# nodes, Node Statistics)
Reference: Schoonderwoerd, R. (1996) “Ant-based load balancing in telecommunication networks” Adapt. Behav. 5, 169-207
Vehicle Routing Problem with Time
Windows (VRPTW)
Subject to:
• Vehicles ( # ,Capacity,time on road,trip length)
• Depots (Numbers)
• Customers (Demands,time windows)
Vehicle Routing Problem with Time
Windows (VRPTW)
[11:00-11:30]
[8:00-9:00]
[10:00-11:45]
[8:30-10:30]
[8:00-12:30]
[8:15-9:30]
Depots
Depot
[10:15-11:45]
VRP “Simple Algorithm”
- Place ants on depots (Depots # = Vehicle #).
- Probabilistic choice
~ (1/distance, di, Q)
~ amount of pheromone
gb MAC-VRPTW Multi-objectives
ACS-VEI ACS-TIME
Single objective
(Min. Vehicles number) (Min. Travel time)
ACS VEI
ACS TIME
Gambardella L.M., Taillard 12. (1999), “Multiple ant colony system for VRPTW”
Parallel implementation
• Populations,Elitism ~ GA
• Probabilistic,RANDOM ~ GRASP
• Constructive ~ GRASP
• Heuristic info, Memory ~ TS
Design Choices
• Number of ants.
• Balance of exploration and exploitation
• Combination with other heuristics techniques
• When are pheromones updated?
• Which ants should update the pheromone.?
• Termination Criteria
Ongoing Projects
• DYVO: ACO for vehicle routing
• MOSCA: Dynamic and time dependent VRP
• Ant@ptima: Research applications
Conclusions
• ACO is a recently proposed metaheuristic approach
for solving hard combinatorial optimization problems.
• Artificial ants implement a randomized construction
heuristic which makes probabilistic decisions.
• The a cumulated search experience is taken into
account by the adaptation of the pheromone trail.
• ACO Shows great performance with the “ill-
structured” problems like network routing.
• In ACO Local search is extremely important to obtain
good results.
References
• Dorigo M. and G. Di Caro (1999). The Ant Colony Optimization Meta-Heuristic. In D. Corne, M.
Dorigo and F. Glover, editors, New Ideas in Optimization, McGraw-Hill, 11-32.
• M. Dorigo and L. M. Gambardella. Ant colonies for the traveling salesman problem. BioSystems,
43:73–81, 1997.
• M. Dorigo and L. M. Gambardella. Ant Colony System: A cooperative learning approach to the
traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1):53–66, 1997.
• G. Di Caro and M. Dorigo. Mobile agents for adaptive routing. In H. El-Rewini, editor, Proceedings
of the 31st International Conference on System Sciences (HICSS-31), pages 74–83. IEEE
Computer Society Press, Los Alamitos, CA, 1998.
• M. Dorigo, V. Maniezzo, and A. Colorni. The Ant System: An autocatalytic optimizing process.
Technical Report 91-016 Revised, Dipartimento di Elettronica,Politecnico di Milano, Italy, 1991.
• L. M. Gambardella, ` E. D. Taillard, and G. Agazzi. MACS-VRPTW: A multiple ant colony system
for vehicle routing problems with time windows. In D. Corne, M. Dorigo, and F. Glover, editors,
New Ideas in Optimization, pages 63–76. McGraw Hill, London, UK, 1999.
• L. M. Gambardella, ` E. D. Taillard, and M. Dorigo. Ant colonies for the quadratic assignment
problem. Journal of the Operational Research Society,50(2):167–176, 1999.
• V. Maniezzo and A. Colorni. The Ant System applied to the quadratic assignment problem. IEEE
Transactions on Data and Knowledge Engineering, 11(5):769–778, 1999.
• Gambardella L. M., E. Taillard and M. Dorigo (1999). Ant Colonies for the Quadratic
Assignment Problem. Journal of the Operational Research Society, 50:167-176.
Thank you