Genetic Algorithms: and Other Approaches For Similar Applications
Genetic Algorithms: and Other Approaches For Similar Applications
Genetic
Algorithms
And other approaches for
similar applications
Opti mi zation
Techniques
• Mathematical Programming
• Network Analysis
• Branch & Bound
• Genetic Algorithm
• Simulated Annealing
• Tabu Search
Ge ne tic A lgo ri th m
• Based on Darwinian Paradigm
Reproduction Competition
Survive Selection
Cross point
• Two point crossover (Multi point crossover)
One-point crossover - Nature
1 2
1 2
2 1 2 1
Two-point crossover
• Randomly two positions in the chromosomes are
chosen
• Avoids that genes at the head and genes at the tail
of a chromosome are always split when recombined
Randomly chosen positions
• Uniform crossover
• Mutation
• Generating new offspring from single parent
• Global Optimal
• Parameter Tuning
• Parallelism
• Random number generators
Example of coding for TSP
Travelling Salesman Problem
• Binary
• Cities are binary coded; chromosome is string of bits
❘ Most chromosomes code for illegal tour
❘ Several chromosomes code for the same tour
• Path
• Cities are numbered; chromosome is string of integers
❘ Most chromosomes code for illegal tour
❘ Several chromosomes code for the same tour
• Ordinal
• Cities are numbered, but code is complex
• All possible chromosomes are legal and only one chromosome
for each tour
• Several others
Roulette wheel
• Sum the fitness of all chromosomes, call it T
• Generate a random number N between 1 and T
• Return chromosome whose fitness added to the
running total is equal to or larger than N
• Chance to be selected is exactly proportional to
fitness
Chromosome : 1 2 3 4 5 6
Fitness: 8 2 17 7 4 11
Running total: 8 10 27 34 38 49
N (1 ≤ N ≤ 49): 23
Selected: 3
Tournament
• Binary tournament
• Two individuals are randomly chosen; the fitter of the two is
selected as a parent
• Probabilistic binary tournament
• Two individuals are randomly chosen; with a chance p,
0.5<p<1, the fitter of the two is selected as a parent
• Larger tournaments
• n individuals are randomly chosen; the fittest one is selected
as a parent
❘ X is indifferent with X ( X
1 2 1 ~ X2), if X1 does not
dominate X2, and X2 does not dominate X1
Multi-Cr iterio n F itness
• Pareto Optimal Set
• If there exists no solution in the search
space which dominates any member in the
set P, then the solutions belonging the the
set P constitute a global Pareto-optimal
set.
• Pareto optimal front
• Dominance Check
Multi-Cr iterio n F itness
• Weighted sum
• F(x) = w1f1(x1) + w2f2(x2) +…+wnfn(xn)
• Problems?
❘ Convex and convex Pareto optimal front
Sensitive to the shape of the Pareto-optimal
front
❘ Selection of weights?
Need some pre-knowledge
Not reliable for problem involving
uncertainties
Multi-Cr iterio n F itness
• Optimizing single objective
• Maximize: fk(X)
Subject to:
fj(X) <= Ki, i <> k
X in F where F is the solution space.
Mul ti -Cr iteri on Fi tnes s
• Weighted sum
• F(x) = w1f1(x1) + w2f2(x2) +…+wnfn(xn)
• Problems?
❘ Convex and convex Pareto optimal front
Sensitive to the shape of the Pareto-optimal
front
❘ Selection of weights?
Need some pre-knowledge
Not reliable for problem involving
uncertainties
Multi-Cr iterio n F itness
• Preference based weighted sum
(ISMAUT Imprecisely Specific Multiple Attribute Utility Theory)
• F(x) = w1f1(x1) + w2f2(x2) +…+wnfn(xn)
• Preference
❘ Given two know individuals X and Y, if we prefer
X than Y, then
F(X) > F(Y),
that is
w1(f1(x1)-f1(y1)) +…+wn(fn(xn)-fn(yn)) > 0
Multi-Cr iterio n F itness
❘ All the preferences constitute a linear space
Wn={w1,w2,…,wn}
w1(f1(x1)-f1(y1)) +…+wn(fn(xn)-fn(yn)) > 0
w1(f1(z1)-f1(p1)) +…+wn(fn(zn)-fn(pn)) > 0, etc
s.t. : Wn
s.t. : Wn
Mul ti -Cr it er ion
Fitn es s
Then,
µ > 0 ⇒ Y' Y' '
40 St_av.
Ge_max
35
Ge_av.
30
25
20
15
10
5
0
0 5 10 15 20
Sim ul ate d
Anne ali ng
• What
• Exploits
an analogy between the
annealing process and the search for
the optimum in a more general
system.
Annealing Process
• Annealing Process
• Raising the temperature up to a very high
level (melting temperature, for example),
the atoms have a higher energy state and
a high possibility to re-arrange the
crystalline structure.
• Cooling down slowly, the atoms have a
lower and lower energy state and a smaller
and smaller possibility to re-arrange the
crystalline structure.
Simu la ted An neali ng
• Analogy
• Metal Problem
• Energy State Cost Function
• Temperature Control Parameter
• A completely ordered crystalline structure
the optimal solution for the problem
• Effective Modeling
• Neighborhood structure
• Objective function (fitness or cost)
❘ Example Graph coloring problem: Find the minimum
number of colors needed such that no two connected
nodes share the same color.
• Aspiration criteria
• The criteria for overruling the tabu constraints and
differentiating the preference of among the
neighbors
Effective Tabu Search
• Effective Computing
• “Move” may be easier to be stored
and computed than a completed
solution
❘ move: the process of constructing of x’
from x
• Computing and storing the fitness
difference may be easier than that of
the fitness function.
Effective Tabu Search
• Effective Memory Use
• Variable tabu list size
❘ For a constant size tabu list
Too long: deteriorate the search results
Too short: cannot effectively prevent from cycling
• Intensification of the search
❘ Decrease the tabu list size
• Diversification of the search
❘ Increase the tabu list size
❘ Penalize the frequent move or unsatisfied constraints
Example
• A hybrid approach for graph coloring
problem
• R. Dorne and J.K. Hao, A New Genetic Local
Search Algorithm for Graph Coloring, 1998
Problem
• Given an undirected graph G=(V,E)
• V={v1,v2,…,vn}
• E={eij}
• Determine a partition of V in a
minimum number of color classes
C1,C2,…,Ck such that for each edge eij, vi
and vj are not in the same color class.
• NP-hard
General Approach
• Transform an optimization problem into
a decision problem
• Genetic Algorithm + Tabu Search
• Meaningful crossover
• Using Tabu search for efficient local search
Encoding
• Individual
• (Ci1, Ci2, …, Cik)
• Cost function
• Number of total conflicting nodes
❘ Conflicting node
having same color with at least one of its adjacent
nodes
• Neighborhood (move) definition
• Changing the color of a conflicting node
• Cost evaluation
• Special data structures and techniques to
improve the efficiency
Implementation
• Parent Selection
• Random
• Reproduction/Survivor
• Crossover Operator
• Unify independent set (UIS) crossover
❘ Independent set
Conflict-free nodes set with the same color
❘ Try to increase the size of the independent set
to improve the performance of the solutions
UIS
Unify independent set
Implementation
• Mutation
• With Probability Pw, randomly pick neighbor
• With Probability 1 – Pw, Tabu search
❘ Tabu search
Tabu list
List of {Vi, cj}
Tabu tenure (the length of the tabu list)
L = a * Nc + Random(g)
Nc: Number of conflicted nodes
a,g: empirical parameters
Summary
• Neighbor Search
• TS prevent being trapped in the local
minimum with tabu list
• TS directs the selection of neighbor
• TS cannot guarantee the optimal result
• Sequential
• Adaptive
Hill climbing
sources
Jaap Hofstede, Beasly, Bull, Martin
Version 2, October 2000