Chapter 2

Genetic Algorithms
What are genetic algorithms?
Why use genetic algorithms?
Fundamentals of genetic algorithms
Applications

What Are Genetic Algorithms?

GAs are stochastic search techniques based on the
mechanism of natural selection and natural genetics to
imitate living beings for solving those \di cult"
problems with high complexity and/or undesirable
structure.

Earliest Predecessors:
1. Fraser, A., \Simulation of Genetic Systems by
Automatic Digital Computers: Part I Introduction", Australian Journal of Biological
Science, Vol. 10, pp. 481-491, 1957.
2. Fraser, A., \Part II - E ects of Linkage on Rates of
Advance under Selection", Australian Journal of
Biological Science, Vol. 10, pp. 492-499, 1957.

Creation of the Field:

1. Holland, J., \Adaptation in Natural and Arti cial
Systems", University of Michigan Press, Ann Arbor,
1975.
2. De Jong, K., \An Analysis of the Behavior of a Class
of Genetic Adaptive Systems", Ph.D. Dissertation,
University of Michigan, Ann Arbor, 1975.

General Structure of Genetic Algorithms

Procedure: Genetic Algorithms
begin
t 0
initialize P (t)
evaluate P (t)
while (not termination condition) do
recombine P (t) to yield C (t)
evaluate C (t)
select P (t + 1) from P (t) and C (t)
t t+1

end
end

Recombination: to yield o spring

1. Genetic operations: crossover and mutation
2. Evolution operations: selection

Genetic

crossover

Algorithms

110010 1010
101110 1110
110010 1110
101110 1010

chromosomes
encoding
solutions

1100101010
1011101110

mutation

0011011001
1100110001

00110 11001
00110 01001
evaluation

selection

offspring

new
population

110010 1110
101110 1010
00110 01001
decoding

roulette wheel

solutions
fitness
computation

Terminology of GAs
Genetic Algorithms
Chromosome (string, individual)
Genes (bits)
Locus
Alleles
Phenotype
Genotype

Explanation
Solution (coding)
Part of solution
Position of gene
Values of gene
Decoded solution
Encoded solution

GAs vs. Conventional Approach

Conventional method

genetic algorithm
initial population

inital single point

improvement
(problem-specific)

terminal ?

improvement
(problem-independent)
No

Yes

terminal ?
Yes
end

end

No

Nature of GAs
1. Genetic algorithms work with a coding of solution
set, not the solutions themselves.
2. Genetic algorithms search from a population of
solutions, not a single solution.
3. Genetic algorithms use payo information ( tness
function), not derivatives or other auxiliary
knowledge.
4. Genetic algorithms use probability transition rules,
not deterministic rules.
5. Genetic algorithms exploit the best solution while
exploring the search space.

Major Advantages of GAs

1. Genetic algorithms do not have much mathematical
requirements about the optimization problems.
2. The ergodicity of evolution operators makes genetic
algorithms very e ective at performing global search
(in probability).
3. Genetic algorithms provide us a great exibility to
hybridize with domain-dependent heuristics to make
an e cient implementation for a speci c problem.

Key Components of GAs

1. Encoding / Decoding
2. Crossover / Mutation
3. Selection

10

Encoding / Decoding
How to encode a solution of the problem into a
chromosome is a key to success.
decoding
coding space

solution space

Evaluation and
Selection

Genetic
Operations
encoding

Issues:
1. The feasibility of a chromosome
2. The legality of a chromosome
3. The uniqueness of mapping

11

Coding Schemes
1. Binary coding
2. Integer coding
3. Real number coding
4. Other specially designed codings

12

Examples
1. Optimization Problems
max f (x1 x2 ) = 21:5 + x1 sin(4 x1 ) + x2 sin(20 x2 )
;3:0

x1 12:1
4:1 x2 5:8

(Binary Coding)

Let the domain of variable xj is aj bj ] and the required

precision is ve places after the decimal point. The
precision requirement implies that the range of domain of
each variable should be divided into at least (bj ; aj ) 105
size ranges. The required bits (denoted with mj ) for a
variable is calculated as follows:
2m ;1 < (bj ; aj ) 105 2m
j

;1

The mapping from a binary string to a real number for

variable xj is straight forward and completed as follows:

bj a
xj = aj + decimal(substringj ) 2m;; j1
where decimal(substringj ) represents the decimal value of
substringj for decision variable xj .
j

13

In this example,
(12:1 ; (;3:0)) 10 000 = 151 000
217 < 151 000 218
m1 = 18
(5:8 ; 4:1) 10 000 = 17 000
214 < 17 000 215
m2 = 15
m = m1 + m2 = 18 + 15 = 33
The total length of a chromosome is 33 bits which can be
represented as follows:
33bits
vj 000001010100101001 101111011111110
18bits
15bits

The corresponding values for variables x1 and x2 are given

below:
Binary Number
Decimal Number
x1 000001010100101001
5417
x2 101111011111110
24318
;
x1 = ;3:0 + 5417 12:1;(;13:0) = ;2:687969
18
2
4:
x2 = 4:1 + 24318 52:8;;11 = 5:361653
15

14

2. Word-Matching Problem
- The word-matching problem tries to evolve an
expression of \to be or not to be" from the
randomly-generated lists of letters.
- Since there are 26 possible letters for each of 13
locations in the list, the probability that we get the
correct phrase in a pure random way is
(1=26)13 = 4:03038 10;19 , which is about two chances
out of a billion.

(Integer Coding)
- We use a list of ASCII integers to encode the string of
letters.
- The lowercase letters in ASCII are represented by
numbers in the range 97, 122] in the decimal number
system.
- Fox example, the string of letters tobeornottobe becomes
116 111 98 101 111 114 110 111 116 116 111 98 101]
15

Genetic Operations: Crossover/Mutation

- The genetic operations mimic the process of heredity of
genes to create new o spring at each generation.
- Crossover operates on two chromosomes at a time and
generates o spring by combining both chromosomes'
features.
- The crossover rate (denoted by pc ) is de ned as the
ratio of the number of o spring produced in each
generation to the population size (usually denoted by
pop size).
- A higher crossover rate allows exploration of more of
the solution space at the cost of computations.
- E ectiveness and feasibility issues (repairing work).

16

- Mutation is a background operator which produces

spontaneous random changes in various chromosomes.
(a) replacing the genes lost from the population during
the selection process so that they can be tried in a
new context.
(b) or providing the genes that were not present in the
initial population.
- The mutation rate (denoted by pm ) is de ned as the
percentage of the total number of genes in the
population.
(a) If it is too low, many genes that would have been
useful are never tried out
(b) If it is too high, there will be much random
perturbation to learn from the history of the search.
- E ectiveness and feasibility issues.

17

Evolution Operation: Selection

The principle behind genetic algorithms is essentially
Darwinian natural selection. It directs a genetic
algorithm search toward promising regions in the search
space.
Basic Issues:
1. Sampling space
2. Sampling mechanism
3. Selection probability

18

1. Sampling Space
(a) Regular Sampling Space:

Parents are replaced by their o spring soon after they

give birth.
Selection based on regular sampling space

new population

population

crossover
mutation

selection

replacement

(b) Enlarged Sampling Space:

Both parents and o spring have the same chance of

competing for survival.

19

selection based on enlarged sampling space

parents

crossover

new population

mutation

selection
offspring

20

2. Sampling Mechanism
- Sampling mechanism concerns the problem of how to
select chromosomes from sampling space.
- Basic approaches:
(a) Stochastic sampling
(b) Deterministic sampling
(c) Mixed sampling

21

(a) Stochastic Sampling

The selection phase determines the actual number of
copies that each chromosome will receive based on its
survival probability.
Roulette Wheel Selection

fk : tness of chromosome k
pk : selection probability
pk = Ppopfk
size
j =1

fj

Then we can make a wheel according to these

probabilities. The selection process is based on spinning
the roulette wheel pop size times. Each time, we select
a single chromosome for the new population.
Prohibition of duplicated chromosomes

22

(b) Deterministic Sampling

- This approach usually selects the best pop size
chromosomes from the sampling space.
- Truncation Selection:
T% best chromosomes are selected and each one
received nearly 100/T copies.
- Block Selection

23

(c) Mixed Sampling

- This approach contains both random and deterministic
features simutaneously.
- Tournament Selection:
This method randomly chooses a set of chromosomes
and picks out the best one from the set for
reproduction.
- Stochastic Tournament Selection:
In this method, selection probabilities are calculated
normally and successive pairs of chromosomes are
drawn using roulette wheel selection. After drawing a
pair, the chromosome with higher tness is inserted in
the new population. The process continues until the
population is full.

24

3. Selection Probability
Scaling and Ranking:
- Scaling method maps raw objective function values
to some positive real values, and the survival
probability for each chromosome is determined
according to these values.
- Ranking method ignores the actual objective
function values and uses a ranking of chromosomes
instead to determine survival probability.
Fitness Scaling
0
fk = gk (fk )
where g( ) transforms the raw tness into scaled tness.

{ To maintain a reasonable di erential between

relative tness ratings of chromosomes.

{ To prevent a too-rapid takeover by some super

chromosomes in order to meet the requirement to

limit competition early on, but to stimulate it later.
25

Examples
1. Linear Scaling

0
fk = a fk + b

2. Sigma Truncation
0
fk = fk ; (f ; c

3. Power Scaling

0
fk = fk

4. Logarithmic Scaling
0
fk = b ; log(fk )

5. Boltzmann Scaling
0
fk = ef =T
k

26

Ranking Selection:

Sort the population from the best to the worst and

assign the selection probability of each chromosome
according to the ranking but not its raw tness.
- Linear Ranking:
pk = selection probability for the k-th ranked chromosome
= q ; (k ; 1)
where q = probability for the best one
= popq;q0 ;1
size
where q0 = probability for the worst one
- Exponential Ranking:

pk = q(1 ; q)k;1
or

pk = qk;1

27

Hybrid Genetic Algorithms

GAs + local optimization heuristics
Lamarckian Evolution:
Environmental changes throughout an organism's life
cause structural changes that are transmitted to
o spring.
hybrid GA based on Darwins & Lamarcks evolution
population
crossover

mutation
hill-climbing
evaluation

28

Procedure: Hybrid Genetic Algorithms

begin
t 0
initialize P (t)
evaluate P (t)
while (not termination condition) do
recombine P (t) to yield C (t)
locally climb C (t)
select P (t + 1) from P (t) and C (t)
t t+1

end
end

Memetic Algorithms

fitness

local optimum
proto-child

search space
29

30

Application of GAs in ATM

1. Topology design / Network con guration
2. Dynamic routing / Multi-casting
3. Bandwidth allocation / Virtual paths
4. QoS allocation
5. Connection admission control
6. Bu er management
7. Cell multiplexing

31

GAs for Multicasting

Minimum Spanning Tree Problem:
G = (V E )
V = fv1 v2

vn g

E = feij j eij = (vi vj )g

W = fwij j wij = w(eij ) > 0g
A spanning tree is a minimal set of edges from E that
connects all the vertices in V and therefore at least one
spanning tree can be found in graph G. The minimum
spanning tree, denoted as T , is the spanning tree whose
total weight of all edges is minimal, i.e.,

T = min
w
T e 2E \T ij
ij

32

Encoding
(1) Edge encoding
e4

e1

e5

e8

e2

e10
e11

e6

e3

e9

e12

e7

e1 e2

e3

e4

e5

e6

e7

e8

e9

e10 e11 e12

Problems:
- Representation space is too large.
- Probability of having a tree is too low.

33

(2) Prufer number encoding:

Cayley's Theorem: There are nn;2 distinct labeled
tree on a complete graph with n vertices.

(Prufer established a one-to-one correspondence

between such trees and the set of all strings of n ; 2
digits.)

34

Examples
4

Tree

6
3

Prufer number

3
3

4
4

Prufer number

3
3

4
4

2
Prufer number

35

Procedure: Encoding
Step 1. Let vertex i be the smallest labeled leaf vertex in a labeled tree T .

Step 2. Let j be the rst digit in the encoding as

the vertex j incident to vertex i is uniquely

determined. Here we build the encoding by
appending digits to the right, and thus the
encoding is built and read from left to right.

Step 3. Remove vertex i and the edge from i to j

thus we have a tree with n ; 1 vertices.

Step 4. Repeat the above steps until one edge is left.

We produce a Prufer number or an encoding

with n ; 2 digits between 1 and n inclusive.

36

Procedure: Decoding
Step 1. Let P be the original Prufer number and let

P be the set of all vertices not included in

P . P designates vertices eligible for consideration in building a tree.

Step 2. Let i be the eligible vertex in P with the

smallest label. Let j be the leftmost digit

of P . Add the edge from i to j into the tree.
Remove i from P and j from P . If j does not
occur anywhere in P, put it into P . Repeat
the process until no digits are left in P .

Step 3. If no digits remain in P , there are exactly two

vertices, r and s, in P and thus still eligible
for consideration. Add edge from r to s into
the tree to form a tree with n ; 1 edges.

37

Crossover
cut-point
parent 1

parent 2

offspring 1

offspring 2

Mutation
select a position at random
parent

replace with a digit at random

offspring

38

Evaluation
Step 1. Convert a chromosome into a tree.
Step 2. Calculate the total weight of the tree.
- Let P be a chromosome, and let P be the set of eligible
vertices.

39

Procedure: Evaluation
begin

T f g
eval(T ) 0
de ne P according to P

repeat

select the leftmost digit from P , say i

select the eligible vertex with the smallest label
from P , say j
T T feij g
eval(T ) eval(T ) + wij
remove i from P
remove j from P
if i does not occur anywhere in remaining P then
put i into P

end
k

k+1
until k n ; 2
T T fers g r s 2 P
eval(T ) eval(T ) + wrs

end

40

Selection
- a mixed strategy with ( + ) selection and roulette
wheel selection is used.

Procedure: Selection
begin

select 0 best di erent chromosomes

if 0 < then
select ; 0 chromosomes by roulette wheel selection

end
end

41

Algorithm
Procedure: Genetic Algorithm for dc-MST
begin
t 0
initialize P (t)
evaluate P (t)
while (not termination condition) do
recombine P (t) to yield C (t)
evaluate C (t)
select P (t + 1) from P (t) and C (t)
t t+1

end
end

42

Parameters:

population size pop size = 50

crossover probability pc = 0:5
mutation probability pm = 0:01
maximum generation max gen = 500

percentage %

100
80
60
40
20
0

optimal value: 2256

66.7

111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000

2256

23.3
111
000
111
000
111
000
111
000
111
000
111
000
111
000
000
111

2292

3.3

111
000
111
000

2332

3.3

2342

objective function value

43

111
000
111
000

3.3

111
000
111
000

2378

References
http://www.ie.ncsu.edu/fangroup/ga.html

44