Robust Immunological Algorithms for High-Dimensional Global Optimization

Introduction Optimization Immunological Algorithm Metrics and Dynamic Behavior Results and Comparison Conclusions

Robust Immunological Algorithms for
High-Dimensional Global Optimization

V. Cutello † G. Narzisi ‡ G. Nicosia † M. Pavone †
† Department of Mathematics and Computer Science
University of Catania
Viale A. Doria 6, 95125 Catania, Italy
(cutello, nicosia, mpavone)@dmi.unict.it
‡
Computer Science Department
Courant Institute of Mathematical Sciences
New York University
New York, NY 10012, U.S.A.
narzisi@nyu.edu

Robust Immunological Algorithms for High-Dimensional Global Optimization


Outline
Introduction
Global Optimization
Numerical Minimization Problem
Artiﬁcial Immune System
Optimization Immunological Algorithm
Cloning and Hypermutation Operators.
Aging and Selection Operators.
Metrics and Dynamic Behavior
Inﬂuence of Different Potential Mutations.
Tuning of the ρ parameter.
Convergence and Learning Processes.
Results and Comparison
Conclusions


Global Optimization

◮ Global Optimization (GO): finding the best set of
parameters to optimize a given objective function
◮ GO problems are quite difficult to solve: there exist
solutions that are locally optimal but not globally
◮ GO requires finding a setting x = (x1 , x2 , . . . , xn ) ∈ S,
where S ⊆ Rn is a bounded set on Rn , such that a certain
n-dimensional objective function f : S → R is optimized.
◮ GOAL: findind a point xmin ∈ S such that f (xmin ) is a global
minimum on S, i.e. ∀x ∈ S : f (xmin ) ≤ f (x).
◮ It is difficult to decide when a global (or local) optimum has
been reached
◮ There could be very many local optima where the
algorithm can be trapped
the difficulty increases proportionally with the problem dimension



Numerical Minimization Problem

Let be x = (x1 , x2 , . . . , xn ) the variables vector in Rn ;
Bl = (Bl1 , Bl2 , . . . , Bln ) and Bu = (Bu1 , Bu2 , . . . , Bun ) the lower
and the upper bounds of the variables, such that
xi ∈ Bli , Bui (i = 1, . . . , n).
GOAL: minimizing f (x) the objective function
min(f (x)), Bl ≤ x ≤ Bu
Benchmarks used to evaluate the performances and
convergence ability:
◮ twenty-three functions taken by [Yao et al., IEEE TEC,
1999];
◮ twelve functions taken by [Timmis et al., GECCO 2003]
All these functions belong to three different categories:
unimodal, multimodal with many local optima, and multimodal
with few local optima



Articial Immune Systems - AIS
◮ Immune System (IS) is the main responsible to protect the
organism against the attack from external microorganisms,
that might cause diseases;
◮ The biological IS has to assure recognition of each
potentially dangerous molecule or substance
◮ Artiﬁcial Immune Systems are a new paradigm of the
biologically-inspired computing
◮ Three immunological theory: immune networks, negative
selection, and clonal selection
◮ AIS have been successfully employed in a wide variety of
different applications
[Timmis et al.: J.Ap.Soft.Comp., BioSystems, Curr. Proteomics, 2008]




Clonal Selection Algorithms - CSA
◮ CSA represents an effective mechanism for search and
optimization
◮ [Cutello et al.: IEEE TAC, J. Comb. Optimization, 2007]
◮ Cloning and Hypermutation operators: strength driven of
CSA
◮ Cloning: triggers the growth of a new population of
high-value B cells centered on a higher afﬁnity value
◮ Hypermutation: can be seen as a local search procedure
that leads to a faster maturation during the learning phase.



Pseudo-code of Immunological Algorithm
Immunological Algorithm(d, dup, ρ, τB , Tmax )
FFE ← 0;
Nc ← d · dup;
t ← 0;
P (t) ← Initialize_Population(d);
// xi = Bli + β · (Bui − Bli )
Compute_Fitness(P (t) );
FFE ← FFE + d;
while (FFE < Tmax )do
P (clo) ← Cloning (P (t) , dup);
P (hyp) ← Hypermutation(P (clo) , ρ);
Compute_Fitness(P (hyp) );
FFE ← FFE + Nc ;
(t) (hyp)
(Pa , Pa ) = Aging(P (t) , P (hyp) , τB );
P (t+1) ← (µ + λ)-Selection(P (t) , P (hyp) );
a a
t ← t + 1;
end_while




Cloning Operator
◮ Cloning operator clones each B cell dup times (P (clo) )
◮ Each clone was assigned a random age chosen into the
range [0, τB ]
◮ Using the cloning operator, an immunological algorithm
produces individuals with higher afﬁnities (higher ﬁtness
function values)
◮ Improvement: choosing the age of each clone into the
range [0, 2 τB ]
3




Hypermutation Operator
◮ Tries to mutate each B cell receptor M times
is not based on an explicit usage of a mutation probability.
◮ There exist several different kinds of hypermutation
operator
[Cutello et al.: LNCS 3239 and IEEE Press vol.1, 2004]
◮ We have used Inversely Proportional Hypermutation
as the ﬁtness function value of the current B cell increases, the number of
mutations performed decreases
◮ two different potential mutations are used:
ˆ
e−f (x) ˆ
α= , α = e−ρf (x)
ρ
α represents the mutation rate, and ˆ(x) the normalized
f
ﬁtness function in [0, 1].



How the Hypermutation Operator works
◮ Choose randomly a variable xi (i ∈ {1, . . . , ℓ = n})
◮ replace xi with
(t+1) (t) (t)
xi = (1 − β)xi + βxrandom ,

(t) (t)
xrandom = xi is a randomly chosen variable
◮ To normalize the fitness function value was used the best
current fitness value decreased of a user-defined threshold
θ,
◮ is not known a priori any additional information concerning
the problem [Cutello et al., SAC 2006]




Aging Operator
◮ Eliminates all old B cells, in the populations P (t) , and P (hyp)
◮ Depends on the parameter τB : maximum number of
generations allowed
◮ when a B cell is τB + 1 old it is erased
◮ GOAL: produce an high diversity into the current
population to avoid premature convergence
◮ static aging operator: when a B cell is erased
independently from its ﬁtness value quality
◮ elitist static aging operator: the selection mechanism does
not allow the elimination of the best B cell




(µ + λ)-Selection Operator
◮ The best survivors are selected to generate the new
population P (t+1) ,
◮ If d1 < d are survived then d − d1 are randomly selected
(t) (hyp)
among those “died”, i.e. (P (t) Pa ) ⊔ (P (hyp) Pa )




◮ Two different potential mutations were used to determine the number of
mutations M;
◮ We present their comparisons, and hence their inﬂuence, in order to the
performances
◮ The main goal is to determine best law to use to tackle optimization problems
◮ Next tables show the different “impact” in term of performance and quality of the
solution
◮ We show the mean of the best B cells on all runs, and the standard deviation
◮ We used experimental protocol proposed in [Yao et al., 1999]
◮ Parameters used:
◮ d = 100, dup = 2, τB = 15
◮ 1st mutation rate: ρ ∈ {50, 75, 100, 125, 150, 175, 200}
◮ 2nd mutation rate: ρ ∈ {4, 5, 6, 7, 8, 9, 10, 11}




Unimodal Funtions

ˆ
e−f (x) ˆ
α= ρ
α = e−ρf (x)
f1 4.663 × 10−19 0.0
7.365 × 10−19 0.0
f2 3.220 × 10−17 0.0
1.945 × 10−17 0.0
f3 3.855 0.0
5.755 0.0
f4 8.699 × 10−3 0.0
3.922 × 10−2 0.0
f5 22.32 16.29
11.58 13.96
f6 0.0 0.0
0.0 0.0
f7 1.143 × 10−4 1.995 × 10−5
1.411 × 10−4 2.348 × 10−5




Multimodal Functions

ˆ
−f (x) ˆ ˆ
−f (x) ˆ
α= e ρ α = e−ρf (x) α= e ρ α = e−ρf (x)
f8 −12559.69 −12535.15 f16 −1.017 −1.013
34.59 62.81 2.039 × 10−2 2.212 × 10−2
f9 0.0 0.596 f17 0.425 0.423
0.0 4.178 4.987 × 10−2 3.217 × 10−2
f10 1.017 × 10−10 0.0 f18 6.106 5.837
5.307 × 10−11 0.0 4.748 3.742
f11 2.066 × 10−2 0.0 f19 −3.72 −3.72
5.482 × 10−2 0.0 8.416 × 10−3 7.846 × 10−3
f12 7.094 × 10−21 1.770 × 10−21 f20 −3.293 −3.292
5.621 × 10−21 8.774 × 10−24 3.022 × 10−2 3.097 × 10−2
f13 1.122 × 10−19 1.687 × 10−21 f21 −10.153 −10.153
2.328 × 10−19 5.370 × 10−24 (7.710 × 10−8 ) 1.034 × 10−7
f14 0.999 0.998 f22 −10.402 −10.402
7.680 × 10−3 1.110 × 10−3 (1.842 × 10−6 ) 1.082 × 10−5
f15 3.27 × 10−4 3.2 × 10−4 f23 −10.536 −10.536
3.651 × 10−5 2.672 × 10−5 7.694 × 10−7 1.165 × 10−5




Potential Mutation behavior using different dimension values

Mutation Rate for the Hypermutation Operator
200
dim=30, ρ=3.5
dim=50, ρ=4.0
180 dim=100, ρ=6.0
dim=200, ρ=7.0
160

140
10
120 9
Mutations

8
100 7
6
80 5
4
60 3
2
40 1
0.4 0.5 0.6 0.7 0.8 0.9 1
20

0 0.2 0.4 0.6 0.8 1
Fitness




Potential Mutation behavior on large dimension values

Mutation Rate for the Hypermutation Operator
5000
dim=1000, ρ=9.0
dim=5000, ρ=11.5

4000

3
3000
Mutations

2.5

2
2000

1.5

1000 1
0.7 0.75 0.8 0.85 0.9 0.95 1

0
0 0.2 0.4 0.6 0.8 1
Fitness




Mean performance comparison curves for test function f1 Mean performance comparison curves for test function f6
70000 70000
Legend Legend
Real Real
60000 Binary 60000 Binary

50000 50000
Function value

Function value
40000 40000

30000 30000

20000 20000

10000 10000

0 0
100 200 300 400 500 600 700 5 10 15 20 25 30 35 40 45 50
Generation Generation




-2000 25
Legend Legend
Real Real
-4000 Binary Binary
20

-6000
Function value

Function value
15
-8000
10
-10000

5
-12000

-14000 0
100 200 300 400 500 600 700 800 900 1000 100 200 300 400 500 600 700 800 900 1000




70 0
Legend Legend
Real -1 Real
60 Binary Binary
-2
50 -3
Function value

Function value
-4
40 -5

30 -6
-7
20 -8
-9
10
-10
0 -11
5 10 15 20 25 30 35 40 45 50 5 10 15 20 25 30 35 40 45 50




Learning Process
◮ To analyze the learning process was used the Information Gain.
◮ measures the quantity of information the system discovers during the learning
phase
[Cutello et al.: journal of Combinatorial Optimization, 2007]
◮ B cells distribution function:

(t) Bt Bmt
fm = Ph m =
t d
m=0 Bm

t
with Bm the number of B cells at time step t with ﬁtness function value m
◮ Information Gain:
(t) (t )
X (t)
K (t, t0 ) = fm log(fm /fm 0 )
m

◮ Entropy:
(t) (t)
X
E(t) = fm log fm
m




Maximum Information-Gain Principle
◮ The gain is the amount of information the system has already learnt from the
given problem instance with respect to the randomly generated initial population
P (t=0)
◮ Once the learning process begins, the information gain increases monotonically
until it reaches a ﬁnal steady state
◮ Is consistent with the idea of a maximum information-gain principle [Cutello et al.,
GECCO 2003]:
dK
≥0
dt




Information Gain and Standard Deviation on function 5
*
Clonal Selection Algorithm: opt-IMMALG
25

20

300
15
250
200
10 150
100
50
5
0
16 32 64

0
16 32 64
Generations




Learning Processon functions f5 , f7 , and f10

Information Gain
25
f5
f7
f10

20

15

10

5

0
1 2 4 8 16 32
Generations




Averge ﬁtness versus Best ﬁtness on function 5
*
Clonal Selection Algorithm: opt-IMMALG
4e+09
avg fitness
best fitness
3.5e+09

3e+09
25
2.5e+09 gain
20 entropy
Fitness

2e+09 15

10
1.5e+09
5
1e+09
0
16 32 64
5e+08

0
0 2 4 6 8 10
Generations



◮ IA was extensively compared against several nature
inspired methodologies including differential evolution (DE)
algorithms
DE seems to perform better than many other EAs on the same test bed
◮ IA was compared on 33 optimization algorithms in the
overall, including a deterministic global search algorithm
DIRECT



IA versus DIRECT DIRECT: [Jones et al.,j. opt. theory appl., 1993] and [Finkel, Techn. Report, 2003].

ˆ −ˆ(x)
f
α = e−ρf (x) DIRECT α= e ρ
f5 16.29 27.89 22.32
f7 1.995 × 10−5 8.9 × 10−3 1.143 × 10−4
f8 −12535.15 −4093.0 −12559.69
f12 1.770 × 10−21 0.03 7.094 × 10−21
f13 1.687 × 10−21 0.96 1.122 × 10−19
f14 0.998 1.0 0.999
f15 3.2 × 10−4 1.2 × 10−3 3.27 × 10−4
f16 −1.013 −1.031 −1.017
f17 0.423 0.398 0.425
f18 5.837 3.01 6.106
f19 −3.72 −3.86 −3.72
f20 −3.292 −3.30 −3.293
f21 −10.153 −6.84 −10.153
f22 −10.402 −7.09 −10.402
f23 −10.536 −7.22 −10.536



IA versus CLONALG1 and CLONALG2 - mut01 [De Castro et al., IEEE TEC, 2002] and
[Cutello et al., ICARIS 2005]

IA CLONALG1 CLONALG2
f1 4.663 × 10−19 3.7 × 10−3 5.5 × 10−4
(7.365 × 10−19 ) (2.6 × 10−3 ) (2.4 × 10−4 )
f2 3.220 × 10−17 2.9 × 10−3 2.7 × 10−3
(1.945 × 10−17 ) (6.6 × 10−4 ) (7.1 × 10−4 )
f3 3.855 1.5 × 10+4 5.9 × 10+3
(5.755) (1.8 × 10+3 ) (1.8 × 10+3 )
f4 8.699 × 10−3 4.91 8.7 × 10−3
(3.922 × 10−2 ) (1.11) (2.1 × 10−3 )
f5 22.32 27.6 2.35 × 10+2
(11.58) (1.034) (4.4 × 10+2 )
f6 0.0 2.0 × 10−2 0.0
(0.0) (1.4 × 10−1 ) (0.0)
f7 1.143 × 10−4 7.8 × 10−2 5.3 × 10−3
(1.411 × 10−4 ) (1.9 × 10−2 ) (1.4 × 10−3 )



IA versus CLONALG1 and CLONALG2 - mut01

f8 −12559.69 −11044.69 −12533.86
(34.59) (186.73) (43.08)
f9 0.0 37.56 22.41
(0.0) (4.88) (6.70)
f10 1.017 × 10−10 1.57 1.2 × 10−1
(5.307 × 10−11 ) (3.9 × 10−1 ) (4.1 × 10−1 )
f11 2.066 × 10−2 1.7 × 10−2 4.6 × 10−2
(5.482 × 10−2 ) (1.9 × 10−2 ) (7.0 × 10−2 )
f12 7.094 × 10−21 0.336 0.573
(5.621 × 10−21 ) (9.4 × 10−2 ) (2.6 × 10−1 )
f13 1.122 × 10−19 1.39 1.69
(2.328 × 10−19 ) (1.8 × 10−1 ) (2.4 × 10−1 )




f14 0.999 1.0021 2.42
(7.680 × 10−3 ) (2.8 × 10−2 ) (2.60)
f15 3.270 × 10−4 1.5 × 10−3 7.2 × 10−3
(3.651 × 10−5 ) (7.8 × 10−4 ) (8.1 × 10−3 )
f16 −1.017 −1.0314 −1.0210
(2.039 × 10−2 ) (5.7 × 10−4 ) (1.9 × 10−2 )
f17 0.425 0.399 0.422
(4.987 × 10−2 ) (2.0 × 10−3 ) (2.7 × 10−2 )
f18 6.106 3.0 3.46
(4.748) (1.3 × 10−5 ) (3.28)
f19 −3.72 −3.71 −3.68
(8.416 × 10−3 ) (1.5 × 10−2 ) (6.9 × 10−2 )
f20 −3.293 −3.23 −3.18
(3.022 × 10−2 ) (5.9 × 10−2 ) (1.2 × 10−1 )
f21 −10.153 −5.92 −3.98
(7.710 × 10−8 ) (1.77) (2.73)
f22 −10.402 −5.90 −4.66
(1.842 × 10−6 ) (2.09) (2.55)
f23 −10.536 −5.98 −4.38
(7.694 × 10−7 ) (1.98) (2.66)




f1 0.0 9.6 × 10−4 3.2 × 10−6
(0.0) (1.6 × 10−3 ) (1.5 × 10−6 )
f2 0.0 7.7 × 10−5 1.2 × 10−4
(0.0) (2.5 × 10−5 ) (2.1 × 10−5 )
f3 0.0 2.2 × 104 2.4 × 10+4
(0.0) (1.3 × 10−4 ) (5.7 × 10+3 )
f4 0.0 9.44 5.9 × 10−4
(0.0) (1.98) (3.5 × 10−4 )
f5 16.29 31.07 4.67 × 10+2
(13.96) (13.48) (6.3 × 10+2 )
f6 0.0 0.52 0.0
(0.0) (0.49) (0.0)
f7 1.995 × 10−5 1.3 × 10−1 4.6 × 10−3
(2.348 × 10−5 ) (3.5 × 10−2 ) (1.6 × 10−3 )




f8 −12535.15 −11099.56 −1228.39
(62.81) (112.05) (41.08)
f9 0.596 42.93 21.75
(4.178) (3.05) (5.03)
f10 0.0 18.96 19.30
(0.0) (2.2 × 10−1 ) (1.9 × 10−1 )
f11 0.0 3.6 × 10−2 9.4 × 10−2
(0.0) (3.5 × 10−2 ) (1.4 × 10−1 )
f12 1.770 × 10−21 0.632 0.738
(8, 774 × 10−24 ) (2.2 × 10−1 ) (5.3 × 10−1 )
f13 1.687 × 10−21 1.83 1.84
(5.370 × 10−24 ) (2.7 × 10−1 ) (2.7 × 10−1 )




f14 0.998 1.0062 1.45
(1.110 × 10−3 ) (4.0 × 10−2 ) (0.95)
f15 3.2 × 10−4 1.4 × 10−3 8.3 × 10−3
(2.672 × 10−5 ) (5.4 × 10−4 ) (8.5 × 10−3 )
f16 −1.013 −1.0315 −1.0202
(2.212 × 10−2 ) (1.8 × 10−4 ) (1.8 × 10−2 )
f17 0.423 0.401 0.462
(3.217 × 10−2 ) 8.8 × 10−3 ) (2.0 × 10−1 )
f18 5.837 3.0 3.54
(3.742) (1.3 × 10−7 ) (3.78)
f19 −3.72 −3.71 −3.67
(7.846 × 10−3 ) (1.1 × 10−2 ) (6.6 × 10−2 )
f20 −3.292 −3.30 −3.21
(3.097 × 10−2 ) (1.0 × 10−2 ) (8.6 × 10−2 )
f21 −10.153 −7.59 −5.21
(1.034 × 10−7 ) (1.89) (1.78)
f22 −10.402 −8.41 −7.31
(1.082 × 10−5 ) (1.4) (2.67)
f23 −10.536 −8.48 −7.12
(1.165 × 10−5 ) (1.51) (2.48)



IA versus BCA and HGA [Timmis et al., GECCO 2003]
ˆ
−f (x) ˆ
α= e ρ α = e−ρf (x) BCA HGA
g1 −1.12 ± 1.17 × 10−3 −1.12 ± 1.62 × 10−3 −1.08 −1.12
g2 −1.03 ± 8.82 × 10−4 −1.03 ± 7.129 × 10−4 −1.03 −0.99
g3 −12.03 ± 8.196 × 10−4 −12.03 ± 9.28 × 10−4 −12.03 −12.03
g4 0.3984 ± 6.73 × 10−4 0.3985 ± 8.859 × 10−4 0.40 0.40
g5 −178.51 ± 11.49 −178.88 ± 9.83 −186.73 −186.73
g6 −179.27 ± 11.498 −179.12 ± 10.02 −186.73 −186.73
g7 −2.529 ± 0.2026 −2.571±0.253 0.92 0.92
g8 1.314 × 10−12 ± 4.668 × 10−12 1.314 × 10−12 ± 4.668 × 10−12 1.0 1.0
g9 −3.51 ± 1.464 × 10−3 −0.351 ± 1.62 × 10−3 −0.91 −0.99
g10 −186.67 ± 8.17 × 10−2 −186.65 ± 0.1158 −186.73 −186
g11 3.81 × 10−5 ± 5.58 × 10−15 3.81 × 10−5 ± 6.98 × 10−14 0.04 0.04
g12 0.0 ± 0.0 0.0 ± 0.0 1 1



IA versus PSO, arPSO and SEA - n = 30 [Thomsen et al., CEC 2004]

IA PSO arPSO SEA
ˆ
e−f (x) ˆ
α= ρ
α= e−ρf (x)
f1 0.0 0.0 0.0 6.8 × 10−13 1.79 × 10−3
0.0 0.0 0.0 5.3 × 10−13 2.77 × 10−4
f2 0.0 0.0 0.0 2.09 × 10−2 1.72 × 10−2
0.0 0.0 0.0 1.48 × 10−1 1.7 × 10−3
f3 0.0 0.0 0.0 0.0 1.59 × 10−2
0.0 0.0 0.0 2.13 × 10−25 4.25 × 10−3
f4 5.6 × 10−4 0.0 2.11 × 10−16 1.42 × 10−5 1.98 × 10−2
2.18 × 10−3 0.0 8.01 × 10−16 8.27 × 10−6 2.07 × 10−3
f5 21.16 12 4.026 3.55 × 10+2 31.32
11.395 13.22 4.99 2.15 × 10+3 17.4
f6 0.0 0.0 4 × 10−2 18.98 0.0
0.0 0.0 1.98 × 10−1 63 0.0
f7 3.7 × 10−5 1.52 × 10−5 1.91 × 10−3 3.89 × 10−4 7.11 × 10−4
5.62 × 10−5 2.05 × 10−5 1.14 × 10−3 4.78 × 10−4 3.27 × 10−4




IA PSO arPSO SEA
ˆ
−f (x) ˆ
α= e ρ α = e−ρf (x)
f8 −1.257 × 10+4 −1.256 × 10+4 −7.187 × 10+3 −8.598 × 10+3 −1.167 × 10+4
8.369 25.912 6.72 × 10+2 2.07 × 10+3 2.34 × 10+2
f9 0.0 0.0 49.17 2.15 7.18 × 10−1
0.0 0.0 16.2 4.91 9.22 × 10−1
f10 4.74 × 10−16 0.0 1.4 1.84 × 10−7 1.05 × 10−2
1.21 × 10−15 0.0 7.91 × 10−1 7.15 × 10−8 9.08 × 10−4
f11 0.0 0.0 2.35 × 10−2 9.23 × 10−2 4.64 × 10−3
0.0 0.0 3.54 × 10−2 3.41 × 10−1 3.96 × 10−3
f12 1.787 × 10−21 1.77 × 10−21 3.819 × 10−1 8.559 × 10−3 4.56 × 10−6
5.06 × 10−23 7.21 × 10−24 8.4 × 10−1 4.79 × 10−2 8.11 × 10−7
f13 1.702 × 10−21 1.686 × 10−21 −5.969 × 10−1 −9.626 × 10−1 −1.143
4.0628 × 10−23 1.149 × 10−24 5.17 × 10−1 5.14 × 10−1 1.34 × 10−5




IA PSO arPSO SEA
ˆ
−f (x) ˆ
f14 9.98 × 10−1 9.98 × 10−1 1.157 9.98 × 10−1 9.98 × 10−1
5.328 × 10−4 2.719 × 10−4 3.68 × 10−1 2.13 × 10−8 4.33 × 10−8
f15 3.26 × 10−4 3.215 × 10−4 1.338 × 10−3 1.248 × 10−3 3.704 × 10−4
3.64 × 10−5 2.56 × 10−5 3.94 × 10−3 3.96 × 10−3 8.78 × 10−5
f16 −1.023 −1.017 −1.032 −1.032 −1.032
1.52 × 10−2 3.625 × 10−2 3.84 × 10−8 3.84 × 10−8 3.16 × 10−8
f17 4.19 × 10−1 4.2 × 10−1 3.98 × 10−1 3.98 × 10−1 3.98 × 10−1
2.9 × 10−2 3.5158 × 10−2 5.01 × 10−9 5.01 × 10−9 2.20 × 10−8
f18 4.973 5.371 3.0 3.516 3.0
2.9366 3.0449 0.0 3.65 0.0
f21 −10.15 −10.15 −5.4 −8.18 −8.41
1.81 × 10−6 1.018 × 10−7 3.40 2.60 3.16
f22 −10.4 −10.4 −6.946 −8.435 −8.9125
1.19 × 10−6 9.3 × 10−6 3.70 2.83 2.86
f23 −10.54 −10.54 −6.71 −8.616 −9.8
6.788 × 10−7 7.29 × 10−6 3.77 2.88 2.24




IA PSO arPSO SEA
ˆ
e−f (x) ˆ
α= ρ
α= e−ρf (x)
f1 0.0 0.0 0.0 7.4869 × 10+2 5.229 × 10−4
0.0 0.0 0.0 2.31 × 10+3 5.18 × 10−5
f2 0.0 0.0 1.804 × 10+1 3.9637 × 10+1 1.737 × 10−2
0.0 0.0 6.52 × 10+1 2.45 × 10+1 9.43 × 10−4
f3 0.0 0.0 3.666 × 10+3 1.817 × 10+1 3.68 × 10−2
0.0 0.0 6.94 × 10+3 2.50 × 10+1 6.06 × 10−3
f4 7.32 × 10−4 6.447 × 10−7 5.312 2.4367 7.6708 × 10−3
2.109 × 10−3 3.338 × 10−6 8.63 × 10−1 3.80 × 10−1 5.71 × 10−4
f5 97.02 74.99 2.02 × 10+2 2.36 × 10+2 9.249 × 10+1
54.73 38.99 7.66 × 10+2 1.25 × 10+2 1.29 × 10+1
f6 0.0 0.0 2.1 4.118 × 10+2 0.0
0.0 0.0 3.52 4.21 × 10+2 0.0
f7 1.763 × 10−5 1.59 × 10−5 2.784 × 10−2 3.23 × 10−3 7.05 × 10−4
2.108 × 10−5 3.61 × 10−5 7.31 × 10−2 7.87 × 10−4 9.70 × 10−5




IA PSO arPSO SEA
ˆ
−f (x) ˆ
f8 −4.176 × 10+4 −4.16 × 10+4 −2.1579 × 10+4 −2.1209 × 10+4 −3.943 × 10+4
2.08 × 10+2 2.06 × 10+2 1.73 × 10+3 2.98 × 10+3 5.36 × 10+2
f9 0.0 0.0 2.4359 × 10+2 4.809 × 10+1 9.9767 × 10−2
0.0 0.0 4.03 × 10+1 9.54 3.04 × 10−1
f10 1.18 × 10−16 0.0 4.49 5.628 × 10−2 2.93 × 10−3
6.377 × 10−16 0.0 1.73 3.08 × 10−1 1.47 × 10−4
f11 0.0 0.0 4.17 × 10−1 8.53 × 10−2 1.89 × 10−3
0.0 0.0 6.45 × 10−1 2.56 × 10−1 4.42 × 10−3
f12 5.34 × 10−22 5.3169 × 10−22 1.77 × 10−1 9.219 × 10−2 2.978 × 10−7
9.81 × 10−24 5.0655 × 10−24 1.75 × 10−1 4.61 × 10−1 2.76 × 10−8
f13 1.712 × 10−21 1.689 × 10−21 −3.86 × 10−1 3.301 × 10+2 −1.142810
9.379 × 10−23 9.877 × 10−24 9.47 × 10−1 1.72 × 10+3 2.41 × 10−8



IA versus IA∗ - unimodal class

n = 30 n = 100
IA IA∗ IA IA∗
f1 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f2 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f3 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f4 0.0 0.0 6.447 × 10−7 0.0
0.0 0.0 3.338 × 10−6 0.0
f5 12 0.0 74.99 22.116
13.22 0.0 38.99 39.799
f6 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f7 1.521 × 10−5 7.4785 × 10−6 1.59 × 10−5 1.2 × 10−6
2.05 × 10−5 6.463 × 10−6 3.61 × 10−5 1.53 × 10−6



IA versus IA∗ - multimodal class

n = 30 n = 100
IA IA∗ IA IA∗
f8 −1.256041 × 10+4 −9.05 × 10+3 −4.16 × 10+4 −2.727 × 10+4
25.912 1.91 × 10+4 2.06 × 10+2 7.627 × 10−4
f9 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f10 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f11 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f12 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
f13 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0



IA and IA∗ versus several DE variants - n = 30 [Coello et al., GECCO 2006]

Unimodal Functions
f1 f2 f3 f4 f6 f7
∗
IA 0.0 0.0 0.0 0.0 0.0 2.79 × 10−5
IA 0.0 0.0 0.0 0.0 0.0 4.89 × 10−5
DE rand/1/bin 0.0 0.0 0.02 1.9521 0.0 0.0
DE rand/1/exp 0.0 0.0 0.0 3.7584 0.84 0.0
DE best/1/bin 0.0 0.0 0.0 0.0017 0.0 0.0
DE best/1/exp 407.972 3.291 10.6078 1.701872 2737.8458 0.070545
DE current-to-best/1 0.54148 4.842 0.471730 4.2337 1.394 0.0
DE current-to-rand/1 0.69966 3.503 0.903563 3.298563 1.767 0.0
DE current-to-rand/1/bin 0.0 0.0 0.000232 0.149514 0.0 0.0
DE rand/2/dir 0.0 0.0 30.112881 0.044199 0.0 0.0



IA and IA∗ versus several DE variants - n = 30 [Coello et al., GECCO 2006]

Multimodal Functions
f5 f9 f10 f11 f12 f13
∗
IA 16.2 0.0 0.0 0.0 0.0 0.0
IA 11.69 0.0 0.0 0.0 0.0 0.0
DE rand/1/bin 19.578 0.0 0.0 0.001117 0.0 0.0
DE rand/1/exp 6.696 97.753938 0.080037 0.000075 0.0 0.0
DE best/1/bin 30.39087 0.0 0.0 0.000722 0.0 0.000226
DE best/1/exp 132621.5 40.003971 9.3961 5.9278 1293.0262 2584.85
DE current-to-best/1 30.984666 98.205432 0.270788 0.219391 0.891301 0.038622
DE current-to-rand/1 31.702063 92.263070 0.164786 0.184920 0.464829 5.169196
DE current-to-rand/1/bin 24.260535 0.0 0.0 0.0 0.001007 0.000114
DE rand/2/dir 30.654916 0.0 0.0 0.0 0.0 0.0



IA and IA∗ versus DE rand/1/bin variants [Thomsen et al., CEC 2004]

30 dimension 100 dimension
IA IA∗ DE rand/1/bin IA IA∗ DE rand/1/bin
f1 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
f2 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
f3 0.0 0.0 2.02 × 10−9 0.0 0.0 5.87 × 10−10
0.0 0.0 8.26 × 10−10 0.0 0.0 1.83 × 10−10
f4 0.0 0.0 3.85 × 10−8 6.447 × 10−7 0.0 1.128 × 10−9
0.0 0.0 9.17 × 10−9 3.338 × 10−6 0.0 1.42 × 10−10
f5 12 0.0 0.0 74.99 22.116 0.0
13.22 0.0 0.0 38.99 39.799 0.0
f6 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
f7 1.521 × 10−5 7.48 × 10−6 4.939 × 10−3 1.59 × 10−5 1.2 × 10−6 7.664 × 10−3
2.05 × 10−5 6.46 × 10−6 1.13 × 10−3 3.61 × 10−5 1.53 × 10−6 6.58 × 10−4



IA and IA∗ versus DE rand/1/bin variants [Thomsen et al., CEC 2004]

30 dimension 100 dimension
IA IA∗ DE rand/1/bin IA IA∗ DE rand/1/bin
f8 −1.256 × 10+4 −9.05 × 10+3 −1.2569 × 10+4 −4.16 × 10+4 −2.727 × 10+4 −4.189 × 10+4
25.912 1.91 × 104 2.3 × 10−4 2.06 × 10+2 7.63 × 10−4 1.06 × 10−3
f9 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
f10 0.0 0.0 −1.19 × 10−15 0.0 0.0 8.023 × 10−15
0.0 0.0 7.03 × 10−16 0.0 0.0 1.74 × 10−15
f11 0.0 0.0 0.0 0.0 0.0 5.42 × 10−20
0.0 0.0 0.0 0.0 0.0 5.42 × 10−20
f12 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
f13 0.0 0.0 −1.142824 0.0 0.0 −1.142824
0.0 0.0 4.45 × 10−8 0.0 0.0 2.74 × 10−8



IA and IA∗ versus rand/1/exp and best/1/exp variants [Iba et al., GECCO, 2005]

IA∗ IA DE rand/1/exp DE best/1/exp
n = 50 dimensional search space
0±0 309.74 ± 481.05
f1 0±0 0±0 0±0 0±0
0.0535 ± 0.0520 0.0027 ± 0.0013
79.8921 ± 102.611 3.69 × 10+5 ± 5.011 × 10+5
f5 1.64 ± 8.7 30 ± 21.7 52.4066 ± 19.9109 54.5985 ± 25.6652
90.0213 ± 33.8734 58.1931 ± 9.4289
0±0 0.61256 ± 1.1988
f9 0±0 0±0 0±0 0±0
0±0 0±0
0±0 0.2621 ± 0.5524
−6
f10 0±0 0±0 9.36 × 10 ± 3.67 × 10−6 6.85 × 10−6
± 6.06 × 10−6
0.0104 ± 0.0015 0.0067 ± 0.0015
0±0 0.1651 ± 0.2133
f11 0±0 0±0 9.95 × 10−7 ± 4.3 × 10−7 0±0
0.0053 ± 0.010 0.0012 ± 0.0028




1.58 × 10−6 ± 3.75 × 10−6 0.0046 ± 0.0247
f1 0±0 0±0 59.926 ± 16.574 30.242 ± 5.93
2496.82 ± 246.55 1729.40 ± 172.28
120.917 ± 41.8753 178.465 ± 60.938
f5 26.7 ± 43 85.6 ± 31.758 12312.16 ± 3981.44 7463.633 ± 2631.92
3.165 × 10+6 ± 6.052 × 10+5 1.798 × 10+6 ± 3.304 × 10+5
0±0 0±0
f9 0±0 0±0 2.6384 ± 0.7977 0.7585 ± 0.2524
234.588 ± 13.662 198.079 ± 18.947
1.02 × 10−6 ± 1.6 × 10−7 9.5 × 10−7 ± 1.1 × 10−7
f10 0±0 0±0 1.6761 ± 0.0819 1.2202 ± 0.0965
7.7335 ± 0.1584 6.7251 ± 0.1373
0±0 0±0
f11 0±0 0±0 1.1316 ± 0.0124 1.0530 ± 0.0100
20.037 ± 0.9614 13.068 ± 0.8876




50.005 ± 16.376 26.581 ± 7.4714
f1 0±0 0±0 5.45 × 10+4 ± 2605.73 4.84 × 10+4 ± 1891.24
1.82 × 10+5 ± 6785.18 1.74 × 10+5 ± 6119.01
9370.17 ± 3671.11 6725.48 ± 1915.38
f5 88.65 ± 91.85 165.1 ± 71.2 4.22 × 10+8 ± 3.04 × 10+7 3.54 × 10+8 ± 3.54 × 10+7
3.29 × 10+9 ± 2.12 × 10+8 3.12 × 10+9 ± 1.65 × 10+8
0.4245 ± 0.2905 0.2255 ± 0.1051
f9 0±0 0±0 1878.61 ± 60.298 1761.55 ± 43.3824
5471.35 ± 239.67 5094.97 ± 182.77
0.5208 ± 0.0870 0.4322 ± 0.0427
f10 0±0 0±0 15.917 ± 0.1209 15.46 ± 0.1205
19.253 ± 0.0698 19.138 ± 0.0772
0.7687 ± 0.0768 0.5707 ± 0.0651
f11 0±0 0±0 490.29 ± 21.225 441.97 ± 15.877
1657.93 ± 47.142 1572.51 ± 53.611



IA and IA∗ versus memetic versions of rand/1/exp and best/1/exp
DE variants [Iba et al., GECCO, 2005]

IA∗ IA DEﬁrDE DEﬁrSPX
0±0 0±0
f1 0±0 0±0 0±0 0±0
−4
0.0026 ± 0.0023 1 · 10 ± 4.75 · 10−5
72.0242 ± 47.1958 65.8951 ± 37.8933
f5 1.64 ± 8.7 30 ± 21.7 53.1894 ± 26.1913 45.8367 ± 10.2518
66.9674 ± 23.7196 52.0033 ± 13.6881
0±0 0±0
f9 0±0 0±0 0±0 0±0
0±0 0±0
0±0 0±0
f10 0±0 0±0 2.28 × 10−5 ± 1.45 × 10−5 3.0 × 10−6 ± 1.07 × 10−6
0.0060 ± 0.0015 0.0019 ± 4.32 × 10−4
0±0 0±0
f11 0±0 0±0 0±0 0±0
−4
4.96 · 10 ± 6.68 · 10−4 5.27 · 10−4 ± 0.0013




0±0 0±0
f1 0±0 0±0 11.731 ± 5.0574 1.2614 ± 0.4581
358.57 ± 108.12 104.986 ± 22.549
107.5604 ± 28.2529 99.1086 ± 18.5735
f5 26.7 ± 43 85.6 ± 31.758 2923.108 ± 1521.085 732.85 ± 142.22
2.822 × 10+5 ± 3.012 × 10+5 16621.32 ± 6400.43
0±0 0±0
f9 0±0 0±0 0.1534 ± 0.1240 0.0094 ± 0.0068
17.133 ± 7.958 27.0537 ± 20.889
1.2 × 10−6 ± 6.07 × 10−7 0±0
f10 0±0 0±0 0.5340 ± 0.1101 0.3695 ± 0.0734
3.7515 ± 0.2773 3.4528 ± 0.1797
0±0 0±0
f11 0±0 0±0 0.7725 ± 0.1008 0.5433 ± 0.1331
3.7439 ± 0.7651 2.2186 ± 0.3010




17.678 ± 9.483 0.8568 ± 0.2563
f1 0±0 0±0 9056.0 ± 1840.45 2782.32 ± 335.69
44090.5 ± 6122.35 9850.45 ± 1729.9
5302.79 ± 2363.74 996.69 ± 128.483
f5 88.65 ± 91.85 165.1 ± 71.2 2.39 × 10+7 ± 6.379 × 10+6 1.19 × 10+6 ± 4.10 × 10+5
3.48 × 10+8 ± 1.75 × 10+8 1.21 × 10+7 ± 4.73 × 10+6
0.1453 ± 0.2771 0.0024 ± 0.0011
f9 0±0 0±0 352.93 ± 46.11 369.88 ± 136.87
1193.83 ± 145.477 859.03 ± 99.76
0.3123 ± 0.0426 0.1589 ± 0.0207
f10 0±0 0±0 9.2373 ± 0.4785 6.6861 ± 0.3286
14.309 ± 0.3706 9.4114 ± 0.4581
0.5984 ± 0.1419 0.1631 ± 0.0314
f11 0±0 0±0 78.692 ± 11.766 28.245 ± 4.605
368.90 ± 41.116 85.176 ± 12.824



IA on Large Dimensional Search Space

f1 f5 f9 f10 f11
Tmax = 10000
−1 +3 −2 −3
n = 1000 1.93 × 10 1.01 × 10 2.29 × 10 1.21 × 10 1.27 × 10−2
2.44 × 10−2 2.94 × 10+2 5.09 × 10−3 7.76 × 10−5 1.7 × 10−3
n = 5000 1.6 × 10+1 9.11 × 10+3 1.83 2.76 × 10−3 3.26 × 10−1
2.86 × 10+1 3.56 × 10+3 8.13 2.31 × 10−3 5.61 × 10−1
Tmax = 50000
n = 1000 1.97 × 10−2 8.93 × 10+2 9.48 × 10−12 8.53 × 10−16 3.28 × 10−8
9.81 × 10−2 2.76 × 10+2 4.70 × 20−11 1.52 × 10−15 1.58 × 10−7
n = 5000 4.44 × 10+1 5.95 × 10+3 1.71 1.13 × 10−3 2.79 × 10−1
9.6 × 10+1 2.76 × 10+3 2.73 3.3 × 10−3 8.18 × 10−1
Tmax = 100000
n = 1000 3.35 × 10−3 9.54 × 10+2 7.06 × 10−4 3.76 × 10−8 6.66 × 10−12
2.22 × 10−2 1.54 × 10+2 4.72 × 10−3 2.63 × 10−7 4.56 × 10−11
n = 5000 3.52 5.95 × 10+3 3.64 × 10−1 8.14 × 10−4 8.99 × 10−2
5.14 1.98 × 10+3 6, 34 × 10−1 1.59 × 10−3 3.33 × 10−1



Conclusion 1/3.

◮ We have presented an extensive comparative study
illustrating the performance
◮ IA was compared with 33 state-of-the-art optimization
algorithms (deterministic and nature inspired
methodologies):
FEP; IFEP; three versions of CEP; two versions of PSO and AR PSO;
EO; SEA; HGA; immunological inspired algorithms, as BCA and two
versions of CLONALG; CHC algorithm; Generalized Generation Gap
(G3 − 1); hybrid steady-state RCMA (SW-100), Family Competition (FC);
CMA with crossover Hill Climbing (RCMA-XHC); eleven variants of DE
and two its memetic versions
◮ Two variants of IA were presented



Conclusion 2/3.

◮ Main features of the designed immune algorithm:
1. cloning operator, which explores the neighborhood of a
given solution
2. inversely proportional hypermutation operator, which
perturbs each candidate solution as a function of its ﬁtness
function value
3. aging operator, that eliminates the oldest candidate
solutions from the current population in order to introduce
diversity and thus avoiding local minima during the search
process
◮ A large set of experiments was used divided in two different
categories of functions [Yao et al., IEEE TEC, 1999] and [Timmis et al., GECCO 2003]



Conclusion 3/3.

◮ The dimensionality of the problems was varied from small
to high dimensions (5000 variabiles).
◮ The results suggest that the proposed immune algorithm is
an effective numerical optimization algorithm (in terms of
solution quality)
◮ All experimental comparisons show that IA and IA∗ are
comparable, and often outperform, all nature inspired
methodologies used, and one well-known deterministic
optimization algorithm (DIRECT).


Robust Immunological Algorithms for High-Dimensional Global Optimization

Recommended

More Related Content

What's hot (20)

Similar to Robust Immunological Algorithms for High-Dimensional Global Optimization (20)

More from Mario Pavone (15)

Robust Immunological Algorithms for High-Dimensional Global Optimization