Ninevah University

College of Electronics Engineering

Computer and Informatics Engineering Department

Intelligent Systems
Lecture 14
Genetic Algorithms (GAs)

Lecturer
Dr. Faris S. Alghareb
PhD in Computer Engineering @ UCF
email: [email protected]
Copyright © 2020 Faris S. Alghareb. All rights reserved.
 Genetic Algorithms – Learning objectives

Objective of this lecture

v The algorithmic structure of a GA
◆ understand the flowchart (flow of operations) of genetic algorithms

v Genetic representation strategies

◆ comprehend the data structure of the genotype

v Genetic operators
◆ be able to compute crossover and mutation

v Applying GA to an optimization problem

◆ be able to answer questions about population changes and trends of the fitness graph

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 1

 Chromosome Representation
v GAs primarily use two types of representations:
◆ Binary Coded
◆ Non-binary Coded

v Each chromosome represents an individual in the population, and this individual a one
solution (algorithm) used to built a software system.

! Binary-Coded GAs encode the chromosome using distinct fields of bit strings
− Each field encodes one orthogonal attribute
− e.g. {0010|0100}, {0101|0000}, {0001|0001}, etc. are three individuals in the population
each of which is encoding two fields with a range of 16 values (24) per field.

◆ Non-binary codings use one or more distinct tuples (sets) of real or integer values
! (1,5) ; (3,2) ; etc.

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 2

 Genetic Operators
v Genetic operators realize new individuals in the population to help increase diversity and
then increase the probability of finding the best solution, or one of the best solutions.

v Many genetic operators possible; two most common

◆ Crossover also known as (aka) Recombination Searches for the most useful traits in the
current population
◆ Mutation introduces diversity in the population to move beyond local optima

v GAs frequently use strings of 1s and 0s to represent candidate solutions

Ø if 100101 is better than 010001 it will have more chance to increase (offspring) and influence
future population

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 3

 Genetic Algorithms
Start
here

Replacement ..
.
offspring
Population of
candidate solutions fitness Evaluate the fitness
Mutation function of individuals

Crossover is the goal Yes

reached?

No
parents Selection of
parents
🛑stop

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 4

 Genetic Algorithms – pseudocode
Procedure Rudimentary_GA{
g = 0;
initialize population(g);
while ( max(evaluate_fitness(population(g)) < threshold ){
while (new_parents_available){
parents = select_two_parents(population(g));
offsprings = crossover(parents);
offsprings = mutation(offsprings);
population(g+1) = union(population(g),offsprings)
}
population(g+1) = select_survivors(population(g+1));
g = g + 1;
}
}

This is because the population size increases over time, thus it is then required to
select survivors to maintain fixed population size across generations
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 5
 Representing Candidate Solutions
v Representation of an individual can be using discrete values (binary, integer, or any other
system with a discrete set of values)

Ø Example of Binary DNA Encoding:

Individual
(Chromosome)

1 0 0 0 0 1 0 0

Gene

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 6

 Genetic Operators

g g+1
selection

reproduction

mutation
Recombination
(crossover)

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 7

 Mutation in Genetic Algorithms
Mutation is a genetic operator used to maintain genetic diversity from one generation of a
population of genetic algorithm chromosomes to the next.

◆ In mutation, the solution may change entirely from the previous solution.
◆ The purpose of mutation in GAs is to introduce diversity into the sampled population
◆ Hence GA can come to find a better solution when applying mutation

Mutation Operators: The conventional way of a mutation operator involves a probability that
an arbitrary bit in a genetic sequence will be flipped (inverted).

v Bit inversion
◆ Used in the binary encoding of chromosomes.
◆ The mutation of bit strings ensue through bit flips at random positions
11001001 à10001001

v Order Changing (Swap Mutation)

◆ This mutation operator replaces the genome with either lower or upper bound randomly.
◆ Can be used for integer and float genes.
(1 2 3 4 5 6 7 8 9 7) à (1 8 3 4 5 6 7 2 9 7)
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 8
 Mutation Operator

Representation Biology Boolean

before a
1 1 1 1 1 1 1 b
AND f
c
d

a
after
b
1 1 1 0 1 1 1 OR f
c
d

mutation gene

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 9

 Crossover Operator

Population:

cut cut

1 1 1 1 1 1 1 0 0 0 0 0 0 0 parents

1 1 1 0 0 0 0 0 0 0 1 1 1 1 offspring

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 10

 Single-Point Crossover
Example:

Ø Parent 1: X X| X XXXX
Ø Parent 2: Y Y| Y YYYY

v Creates by single-point crossover between the 2nd and 3rd bits from the left:
Ø Offspring 1: X XY YYYY
Ø Offspring 2: Y YX XXXX

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 11

 Mutation
v In Binary-Coded GAs, each bit in the chromosome is mutated with probability pm
where (pm= the mutation rate).

Example:
Ø Parent1 = 00100110
Ø Parent2 = 10101010

v Assume crossover occurred between leftmost and next leftmost bit to create 2 offspring:
Ø Offspring1 = 10100110
Ø Offspring2 = 00101010

v Then mutation might randomly modify them to become:

Ø Offspring1 = 10101110
Ø Offspring2 = 00100000

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 12

 Global max/min VS. Local max/min
v A maximum or minimum is said to be local if it is the largest or smallest value of the
function, respectively, within a given range.

v However, a maximum or minimum is said to be global if it is the largest or smallest

value of the function, respectively, on the entire domain of a function

v Thus, sometime the searching algorithm is stuck a local minimum or maximum point to be
found as the global one.

v Applying a mutation operator can help to solve this issue, widening the searching range
through generating diverse genies
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 13
 Genetic Algorithm (GA) ⎼ Example (1/5)
✏ Find the maximum value if the fitness function is f(x) = X2 in the searching range of [0,
31]. Use the Single Point Crossover when generating the offspring of 2nd and 3rd
generations.

ANS:
● Find the number of bits that required to represent the strings as 2N
2N = 32 à 5-bit
● Note that number of strings is arbitrary such as 4 strings (No. of strings = number of bits
– 1)
● Selecting four numbers randomly

● Calculation the new generations (offspring) based on the fitness function given in the
question.
Binary Numbers
Decimal
(Randomly selected)
01101 13
11000 24
01000 8
10011 19
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 14
 Genetic Algorithm (GA) ⎼ Example (2/5)
Find the maximum value if the fitness function is f(x) = X2 in the searching range of [0, 31]. Use the
Single Point Crossover when generating the offspring of 2nd and 3rd generations.

ANS:
First Generation
1st Generation X Fitness Function 𝐗𝟐 𝐗𝟐
No. Round
(binary) (decimal) 𝒇 𝒙 = 𝑿𝟐 ∑ 𝐗𝟐 𝐀𝐕𝐄
1 01101 13 169 0.14 0.58 1
2 11000 24 576 0.49 1.97 2
3 01000 8 64 0.06 0.22 0
4 10011 19 361 0.31 1.23 1

∑ "! ∑ X $ = 1170
Ø Average (AVE) #
= 293

Ø Max Fitness = 576

To generate the next generation, we use the point crossover between parents of 1st
generation.
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 15
 Genetic Algorithm (GA) ⎼ Example (3/5)
Applying the crossover between pair (1,2) and (2,4)
Crossover (1, 2) Crossover (2, 4)
0110 1 11 000
Assume k = 4
Assume k = 2
1100 0 (arbitrary choosing) 10 011

Second Generation
2nd Generation X Fitness Function 𝐗𝟐 𝐗𝟐
No. Round
(binary) (decimal) 𝒇 𝒙 = 𝑿𝟐 ∑ 𝐗𝟐 𝐀𝐕𝐄
1 01100 12 144 0.08 0.32 0
2 11001 25 625 0.35 1.42 1
3 11011 27 729 0.41 1.66 2
4 10000 16 256 0.14 0.58 1

∑ "!
∑ X $ = 1754
Ø Average (AVE) #
= 439

Ø Max Fitness = 729

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 16
 Genetic Algorithm (GA) ⎼ Example (4/5)
Looking at the sum and max, we find they increase (729 > 576) and that gives an indication
sign to continue generating the offspring of the 3rd generation.
Crossover (2, 3) Crossover (3, 4)
110 01 110 11
Assume k = 3
(arbitrary choosing) Assume k = 3
110 11 100 00

Third Generation
3rd Generation X Fitness Function 𝐗𝟐 𝐗𝟐
No. Round
(binary) (decimal) 𝒇 𝒙 = 𝑿𝟐 ∑ 𝐗𝟐 𝐀𝐕𝐄
1 11001 25 625 0.27 1.09 1
2 11011 27 729 0.32 1.27 1
3 11000 24 576 0.25 1.00 1
4 10011 19 361 0.15 0.63 1

Ø Average (AVE)
∑ ""
= 572.75 ∑ X $ = 2291
#

Ø Max Fitness = 729

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 17
 Genetic Algorithm (GA) ⎼ Example (5/5)
Ø Here, we can see that max is the same value, however, sum and average are getting better.
Ø This indicates continue running the GA to find next generation may provide better results, leading to finding the
global maximum and avoiding the issue of sticking at a local maximum.

Fitness training/tuning examples

Fitness Function

Fitness Function
Generations
Fitness Function

Generations

Generations

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 18

 Genetic Algorithm (GA) ⎼ Classwork #1
✏ Given a fitness function { f(x) = X2 + 1 } in the searching range of [0, 53], find the
maximizing value of the fitness function for five iterations/generations. Use the bit
inversion as the mutation operator when generating the offspring.

Note: here you should apply the mutation operator instead of the crossover to
find the offspring for next generations

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 19

 Traveling Salesman Problem (TSP)
v To understand what is the traveling salesman problem (TSP) and why it is so problematic.
◆ Assume there is a salesman given a map like the one illustrated in the figure below.
◆ The map contains a total of 20 locations (cities)
◆ The journey should visit each of these locations to make a sell
◆ Thus, the task is to find a route that can minimize traveling distance and time.

◆ As humans, we can easily find a reasonably good

route without needing to do much more than look
at the map for some minutes.
◆ However, is the determined route the optimal
one?
◆ How can we test to verify if it is truly the optimal
route?

Ans: We cannot, and at least this kind of

problem is not practically for humans’
brains.
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 20
 Traveling Salesman Problem (TSP)
v To understand why it is difficult to our brains, consider 3 locations instead of 20.

v First, let us see what the probability of the possible solutions will be if we need to
choose from a set that has 3 choices
Ø We need to choose one out of three, then one out of two, and the remaining one
à (3 x 2 x 1 = 6), there are different 6 routes to chose from

Ø Thus, the number of possible routes is a factorial of the number of locations to visit
Factorial of 10 è 3,628,800
Factorial of 20 è 2,432,902,008,176,640,000
quintillion, quadrillion, trillion, billion, million, thousand

v Solving this problem using modern computers is impractical, and for bigger problems, it
may be even impossible.
v The nearest neighbor algorithm and swarm optimization are usually used to quickly
find a suitable and good-enough solution to the TSP.

v However, here we employ the GAs as the optimization technique.

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 21
 Employing the GA to Solve the TSP
v A valid solution should represent a route where every location is visited only once.

v If a route contains a single location visited more than once, then it is not a valid route to
be considered

v A missed location is also not valid

v The metrics for this kind of problem will be the computation time total distance.

v To ensure that, the GA requires special types of mutation and crossover methods to
be applied.

v The mutation method should only be capable of shuffling the route and ever add or
remove a location from the route, otherwise, it would risk creating an invalid solution.

v The fitness function is to minimize the total distance while visiting all 20 cities.

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 22

 TSP – Swap Mutation
v One type of mutation method we can use is swap mutation.

v With swap mutation, two locations in the route are selected at random then their
positions are simply swapped (changing the order in which they will be visited).

1 2 3 4 5 6 7 8 9

1 2 8 4 5 6 7 3 9

v Here, the position of location 3 and 8 were switched to create a new list with exact same
values/candidates, just with a different order.

v Swap mutation is only swapping pre-existing values; thus, it will never create a list that
has missing, or duplicate values as compared to the originals

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 23

 TSP – Ordered Crossover
v Ordered crossover method can produce a valid route.
v In this crossover method, a subset from parent 1 is selected, and then add that subset to
the offspring.
v Any missing values are then added to the offspring from parent 2, they should be added in
the same order that they are found.
v Consider the following example:
Parent 1 1 2 3 4 5 6 7 8 9

Parent 2 9 8 7 6 5 4 3 2 1
Note: ignore genes (6, 7, 8) because these are already selected (visited) from parent 1.

Offspring 9 5 4 3 2 6 7 8 1

v Implementing this correctly will end up finding a route that contains all the positions which
the parents have with no missing or duplicated position (city).
EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 24
 Traveling Salesman Problem (TSP)
v Initial distance: 1996
v Final distance: 940 , where the fitness function is to minimize the distance
v An optimal solution was found in 100 generations
v Rank for cities to visit is: {(60, 200), (20, 160), (40, 120), (60, 80), (20, 40), (20, 20), (60, 20),
(100, 40), (160, 20), (200, 40), (180, 60), (120, 80), (140, 140), (180, 100), (200, 160), (180, 200),
(140, 180), (100, 120), (100, 160), (80, 180)}
Start here

Stop
here
an optimal
solution
was found in
100
generations

EECIE20-S4305 Intelligence Systems & Software Engineering: Genetic Algorithms Slide 25

End of Lecture 14