2013 IEEE 7th International Power Engineering and Optimization Conference (PEOCO2013), Langkawi, Malaysia.

3-4 June
2013

Assessment of Evolutionary Programming Models

for Single-Objective Optimization
Nur Izzati Abdul Aziz1, Shahril Irwan Sulaiman1, Ismail Musirin1, Sulaiman Shaari2
1
Faculty of Electrical Engineering
2
Faculty of Applied Sciences
Universiti Teknologi MARA, 40450 Shah Alam, Selangor, Malaysia
[email protected]

Abstract- This paper presents an assessment of different These intelligent approaches provide various features such
evolutionary programming (EP) techniques for solving single- as the ability to learn from example, capability to handle
objective optimization problem. Evolutionary programming has noisy, incomplete data and non-linear problem as well as
been widely used and applied with success in solving many kinds ability to perform prediction and generalization at high speed
of optimization problem. However there is no benchmark to test
[2]. AI is one of intelligent techniques which fall under the
which techniques of EP models will give a better result in solving
single objective optimization. Three distinct EP models used are computational intelligence (CI) hierarchy and consists of three
classical evolutionary programming (CEP), fast evolutionary main branches which are artificial neural network (ANN),
programming (FEP) and improved fast evolutionary fuzzy logic (FL) and evolutionary computation (EC) [2-3].
programming (IFEP). These EP techniques considered here Evolutionary computation (EC) is a generic terms that has
differ in terms of search operator- Gaussian, Cauchy and mixed been used in many kinds of population-based metaheuristic
Gaussian-Cauchy during mutation process. Therefore, selected optimization and mechanism contains in this EC includes
test functions are used as a benchmark to test which models generation, mutation, selection and reproduction [4].
perform better for single-objective optimization. The three EP Generally, EC can be categorized into four types which are
models showed that FEP is very good in having lowest
genetic algorithm (GA), genetic programming (GP),
computation time and significantly better than CEP and IFEP in
terms of fitness solution. evolutionary strategy (ES), and evolutionary programming
Keywords-evolutionary programming (EP); gaussian mutation; (EP) [5]. Evolutionary Programming (EP) was originally
cauchy mutation; test function proposed 50 years ago by Lawrence J. Fogel in the US in 1960
when he studied the artificial intelligence. EP is a stochastic
I. INTRODUCTION optimization technique based on search algorithm and quite
similar to Genetic algorithm (GA) in term of principles of
Optimization is one of the mathematical procedures for natural evolution [6] in which this method capable to solve
determining optimal allocation of scarce resources, in the both constrained and unconstrained optimization problems. EP
sense that optimization is the mathematical discipline which is one of such discipline that has been employed to improve
concerned with finding the maxima and minima value of the search for optimal solutions in complex problems. In the
functions. Nowadays, continuous development in computer past decade, EP techniques have been used and applied in
system especially in software offers the researcher to deal with several applications and solving many difficult optimization
an optimization problem by utilizing computational resources. problems [7-10].In the field of evolutionary computation, it is
Apart from that, there are huge numbers of optimization common to compare different algorithms using a large test set,
methods have been applied in solving complicated problems especially when the test involves function optimization.
such as economic load dispatch, reactive power dispatch, This paper presents an assessment of the different EP
sizing PV system and so on. One of the optimization models for single objective optimization (SOO) by using
techniques is artificial intelligence (AI). AI offers several selected test functions such as De Jong’s function,
methods in solving problems. An intelligent-based method Rosenbrock’s function, axis parallel hyper–ellipsoid function
such as AI has been widely used as a tool to predict or and rotated hyper-ellipsoid function [11]. Selected test
forecasting, making a decision as well as an optimizer to function has been tested by using three different EP models
select the best solution among the population. which are classical EP (CEP), fast EP (FEP) and improved
AI techniques are becoming useful as an alternative method fast EP (IFEP). Each EP model considered in this paper differs
to replace the existing conventional techniques in solving in the kind of mutation operator that is used.
complicated problems in various areas. Generally, the term
artificial intelligence was first used in July 1956 at Darmouth II. METHODOLOGY
College conference [1]. AI has been employed in solving all
kinds of numerical optimization and complicated practical A. Single Objective Optimization (SOO)
problems and widely used in many different fields. In evolutionary programming, stopping criterion is crucial
because it might affect the duration of an optimization process.
In addition, optimization process could be terminated earlier
This work was supported in part by the Excellence Fund, before the populations reach convergence criteria due to having
UniversitiTeknologi MARA, Malaysia (Ref: 600-RMI/ST/DANA two different stopping criteria.
5/3/Dst (283/2009).

Authorized licensed use limited to: UNIVERSITI TEKNOLOGI MARA. Downloaded on July 15,2023 at 05:40:10 UTC from IEEE Xplore. Restrictions apply.
978-1-4673-5074-7/13/$31.00 ©2013 IEEE 304
 2013 IEEE 7th International Power Engineering and Optimization Conference (PEOCO2013), Langkawi, Malaysia. 3-4 June
2013

Therefore two distinct stopping criteria will be implemented function. Fitness function can be a single mathematical
in a single objective optimization. Single objective equation or a set of sub-program or subroutine. A set of
optimization consists of one objective function which is parents are generated before fitness for each parent is
either to minimize or maximize and a set of constraints evaluated.
[12]. General equation for single objective optimization Secondly, process of adding random number from a certain
can be expressed using (1) where is a vector of n decision distribution to a parent known as mutation. Standard deviation
variables and m is an integer numbers. or strategy parameter will control the degree of variation of
the Gaussian mutation [8]. In EP there are several models that
minimize , =(x1,x2,x3…..,xn) (1) considered distinct in the kinds of mutation process. The
classical evolutionary programming (CEP) employing the
0 ( j=1,…..,m) Gaussian distribution operator during mutation [7]. Cauchy
mutation-based EP known as fast EP (FEP) is introduced in
Apart from that, the main goal of single objective [13] as it converges faster than CEP. This approach used
optimization is to find the best solutions or configurations Cauchy distribution instead of Gaussian distribution in
which correspond to the minimum or maximum value of a gathering random number during mutation process.
single objective function. Meanwhile the same author proposed an improved fast-EP
(IFEP) that uses both Gaussian and Cauchy mutations to
B. Evolutionary Programming (EP) create offspring from the same parent and the better ones are
Evolutionary programming (EP) is one of the branches of chosen for the next generation.
evolutionary algorithm (EA) which falls under the artificial In Classical EP (Gaussian mutation), a single offspring (y’i)
intelligence (AI) hierarchy. EP is a computational model that is made from each parent (yi) by adding a random number
mimics natural evolution and genetics. The basic of EP with zero mean and standard deviation to each vector of
techniques consist of four main parts which are initialization, parent. General mathematical expression for Gaussian operator
mutation, combination and selection. Fig. 1 shows the is:
flowchart of general procedure for EP for single objective y’i =yi + N(0,σ²i),for i = 1,2.........n (2)
optimization.
where N(0,σ²i) represents a Gaussian random variable with
Start mean 0 and standard deviation σi.
Fast EP using Cauchy operator, an offspring is created by
the expression below. Where Ci(0,1) is a Cauchy random
Generation of initial population variable with scale parameter k=1 centered at 0 that is
generated anew for each value of i.
y’i= yi+ σi . Ci(0,1), for i = 1,2.........n (3)
Fitness value evaluation (parents)
Apart from that, by using the method of choosing the better
one from two offsprings that have been generated by each
Mutation parent (using combination of Gaussian-Cauchy mutation)
known as Improved-Fast EP, an offspring (y1’i ,y2’i) is
generated from the parent (yi).
Fitness value evaluation (offspring)
y1’i = yi + σi.Ni(0,1) (4)
y2’i = yi + σi.Ci(0,1) (5)
Combination (Parents + offspring)
From the methods that have been mentioned, the standard
deviation, σi is given by the expression
Selection σi = β . ƒi/ƒmax (yimax – yimin) (6)

where scaling factor is represented by β which has to be tuned

No
Convergence test during the search process for the optimum around the initial
points. If β is set to a high value, the convergence would be
Yes time consuming and vice versa [6]. ƒi indicates the fitness
End value of ith individual. Meanwhile ƒmax represents the
maximum fitness among the parents. After mutation process,
Fig. 1. Evolutionary programming flowcharts the fitness of each offspring is then evaluated before
combination process.
Firstly, an initial N sets of random number will be Thirdly, combination process involved parent and offspring
generated which represent variable to control the objective are combined together such that the number of the population
function. The fitness is calculated based on its objective becomes 2N.

Authorized licensed use limited to: UNIVERSITI TEKNOLOGI MARA. Downloaded on July 15,2023 at 05:40:10 UTC from IEEE Xplore. Restrictions apply.
305
 2013 IEEE 7th International Power Engineering and Optimization Conference (PEOCO2013), Langkawi, Malaysia. 3-4 June
2013

Lastly, the selection is a genetic operator where the new CEP FEP IFEP

population can be determined from the parent and offspring CT_CEP CT_FEP CT_IFEP

population and this process involved selection of the best

0.0852
0.09 16
candidates from parents and offspring. Generally, it is
performed through the stochastic tournament method based on 0.08

0.0685

0.0685
14
the fitness value of the individuals.
To show the relative performance of the three difference 0.07
12
evolutionary programming techniques, four test functions 0.06

Fitness value, Unitless

have been considered in this case which are De Jong’s 10

t, sec
function, Rosenbrock’s function, axis parallel hyper–ellipsoid 0.05

function and rotated hyper ellipsoid function. Each test 8

0.04
function consists of several features such as continuous,
6
convex and unimodal [14]. The simplest test function is De

0.0206

0.0206
0.03
Jong’s. The function can be expressed using 4
n 0.02

0.0081

0.0081
f DeJong's = ∑x 2
(7)

0.0034

0.005

0.0034
i

0.0027
0.0026

0.0026
0.0017
2

0.0002
0.01
i =1
0 0

where global minimum 0 is obtainable for 20 40 60 80 100

Population
=0, i=1,…..,n
The axis parallel hyper-ellipsoid function is also known as a Fig. 2. Comparison between CEP, FEP and IFEP using De Jong’s
function
weighted sphere model and has the following general
definition Fig. 2 shows the performance comparison between
n
different EP models in terms of computation time and
f axisparallel −ellipsoid = ∑ (i.xi ) (8) optimization accuracy of each EP model. De Jong’s function
i =1
has been implemented as a benchmark function to evaluate
the performance of CEP, FEP and IFEP at different number
Apart from that, Schwefel’s function known as rotated of population. From the result obtained, 20 numbers of
hyper-ellipsoid function is actually an extension of axis populations record the lowest fitness which is 0.0002. Apart
parallel hyper-ellipsoid function. The function has the from that, the computation time during 20 population is not
following general expression that much different between three EP models. However, the
n i
computation time was increased as the amount of population
fSchwefel' s = ∑∑ xj 2 (9) increased. As a result FEP performed the lowest computation
i =1 j =1 time in each population time as compared to CEP and IFEP.

The banana function is also known as Rosenbrock’s valley

function. The Rosenbrock’s function has the following
definition
n −1
fRosenbrock' s = ∑[100( xi + 1 − xi 2 ) 2 + (1 − xi )2 (10)
i =1

All EP models will be tested with the selected test function

in order to analyze the performances of each EP models in
terms of fitness solution and computation time.

III. RESULTS AND DICUSSIONS

The development of evolutionary programming (EP) for

the selected test function was aimed to compare the
performance of different EP models in terms of optimization
accuracy and computation time between CEP, FEP and IFEP.
Each EP model was tested using different numbers of
population between 20 until 100 of random numbers. Fig
2,3,4,5 shows the performance of each EP model and each EP
model has been tested by using the selected test function.

Authorized licensed use limited to: UNIVERSITI TEKNOLOGI MARA. Downloaded on July 15,2023 at 05:40:10 UTC from IEEE Xplore. Restrictions apply.
306
 2013 IEEE 7th International Power Engineering and Optimization Conference (PEOCO2013), Langkawi, Malaysia. 3-4 June
2013

CEP FEP IFEP CEP FEP IFEP

CT_CEP CT_FEP CT_IFEP CT_CEP CT_FEP CT_IFEP

0.1517
0.45 14
0.16 16

0.3827
0.4
0.14 14 12

0.3043
0.35
0.12 12 10

Fitness valua, Unitless

0.3
0.1 10
Fitness value, Unitless

0.0769
0.0767

t, sec
0.25 8

t, sec
0.08 8
0.2 6

0.149

0.149
0.06 6

0.0381

0.0381
0.15

0.0839

0.0839

0.0815
4
0.04 4 0.1

0.0441
0.0118
0.0117

0.0091
0.0079

0.0158
0.0152
0.0069
0.0067

2
0.0057

0.0102
0.0051

0.0051

0.0099

0.0099

0.0101
0.0038

0.0018
0.02 2 0.05

0 0 0 0

20 40 60 80 100 20 40 60 80 100
Population
Population
Fig. 3. Comparison between CEP, FEP and IFEP using Axis parallel Fig. 5. Comparison between CEP, FEP and IFEP using Rosenbrock’s
hyper-ellipsoid function function

Axis parallel hyper ellipsoid scores the lowest fitness value The performance of CEP, FEP, and IFEP with different
is at 40 numbers of populations as the optimal population. In population in optimizing Rosenbrock’s function is shown in
addition, an increasing number of the population will affect fig. 5. The optimal population for all EP models is 100
the duration of the optimization process. As a result, initial populations as the number of population produced the lowest
population produced the lowest computation as compared to fitness value in each EP models. Apart from that, the
100 populations which time consuming. computation time is increasing with the increase of
population.
CEP FEP IFEP The results show that FEP performs much better than other
CT_CEP CT_FEP CT_IFEP EP models in terms of solution time. FEP had outerperfomed
0.1277

0.14 16
0.1238

IFEP since IFEP involved more algorithms during mutation

14
process. This will result IFEP more computation time. Apart
0.12
from that, IFEP consists of two offspring that are produced by
12
Gaussian and Cauchy distribution. These offspring will be
evaluated and selected based on the better solution and
Fitness value, Unitless

0.1
0.0796

10 competition. In addition the performance of FEP is better than

0.08 CEP and IFEP in terms of optimization accuracy and offer
t, sec

8 better fitness solution.

0.06
6 IV. CONCLUSION
0.04
This paper was proposed to analyze EP techniques by
0.0231

0.0231

4
using selected test function. Three different evolutionary
0.0126
0.0125

0.0088
0.0087
0.0075

programming (EP) models were developed and tested by using

0.0046

0.02
0.0026

0.0026
0.0016

2
0.0009

selected test function. This kind of problems is quite simple

0 0 and takes small time to reach or obtained final solution. FEP
20 40 60 80 100 offers a better approach in terms of computation times and
Population
scores better optimization accuracy among others EP
Fig. 4. Comparison between CEP, FEP and IFEP using rotated hyper- techniques.
ellipsoid function

The results obtained from each EP models are shown in fig. ACKNOWLEDGMENT
4 by using rotated hyper-ellipsoid function. This result
indicates that FEP scores the lowest computation whereas the The authors would like to thank Green Energy Research
optimal population was occurred at 40 numbers of Center (GERC), Universiti Teknologi MARA (UiTM) Shah
populations. Alam for providing the necessary facilities to conduct the
research.

Authorized licensed use limited to: UNIVERSITI TEKNOLOGI MARA. Downloaded on July 15,2023 at 05:40:10 UTC from IEEE Xplore. Restrictions apply.
307
 2013 IEEE 7th International Power Engineering and Optimization Conference (PEOCO2013), Langkawi, Malaysia. 3-4 June
2013

REFERENCES dispatch problems," Electric Power Systems Research, vol. 73, no. 2, pp.
169-176, 2005.
[8] Q. H. Wu and J. T. Ma, "Power system optimal reactive power dispatch
[1] E. S. Brunette, R. C. Flemmer and C. L. Flemmer, "A review of
using evolutionary programming," Power Systems, IEEE Transactions
artificial intelligence," in Autonomous Robots and Agents, 2009.
on, vol. 10, no. 3, pp. 1243-1249, 1995.
ICARA 2009. 4th International Conference on, 2009, pp. 385-392.
[9] U. Bhanja, S. Mahapatra and R. Roy, "An evolutionary programming
[2] A. Mellit, S. A. Kalogirou, L. Hontoria and S. Shaari, "Artificial
algorithm for survivable routing and wavelength assignment in
intelligence techniques for sizing photovoltaic systems: A review,"
transparent optical networks," Information Sciences, vol. 222, no. 0, pp.
Renewable and Sustainable Energy Reviews, vol. 13, no. 2, pp. 406-419,
634-647, 2013.
2009.
[10] S. Das, R. Mallipeddi and D. Maity, "Adaptive evolutionary
[3] H. Katagiri, K. Hirasama and J. Hu, "Genetic network programming -
programming with p-best mutation strategy," Swarm and Evolutionary
application to intelligent agents," in Systems, Man, and Cybernetics,
Computation, vol. no. pp. 2012.
2000 IEEE International Conference on, 2000, pp. 3829-3834 vol.5.
[11] D. Kothari, "Power system optimization," in Computational Intelligence
[4] J. Verboomen, D. V. H., P.H. Schavemaker, W.L. Kling and R.Belmans.
and Signal Processing (CISP), 2012 2nd National Conference on, 2012,
(2006). Coordinated Phase Shifter Control using Meta-Evoluitionary
pp. 18-21..
Programming and Evoltuion Strategies. Paper presented at the 3rd IEEE
[12] X. Hu and R. Eberhart, "Solving constrained nonlinear optimization
Benelux Young Reserchers Symposium in Electrical Engineering,
problems with particle swarm optimization," in Proceedings of the sixth
Ghent, Belgium
world multiconference on systemics, cybernetics and informatics, 2002,
[5] N. Sinha, R. Chakrabarti and P. K. Chattopadhyay, "Evolutionary
pp. 203-206.
programming techniques for economic load dispatch," Evolutionary
[13] N. Sinha, R. Chakrabarti and P. K. Chattopadhyay, "Fast evolutionary
Computation, IEEE Transactions on, vol. 7, no. 1, pp. 83-94, 2003.
programming techniques for short-term hydrothermal scheduling,"
[6] S. I. Sulaiman, T. K. A. Rahman, I. Musirin, S. Shaari and K. Sopian,
Electric Power Systems Research, vol. 66, no. 2, pp. 97-103, 2003.
"An intelligent method for sizing optimization in grid-connected
[14] M. Molga and C. Smutnicki, "Test functions for optimization needs,"
photovoltaic system," Solar Energy, vol. 86, no. 7, pp. 2067-2082, 2012.
Test functions for optimization needs, vol. no. pp. 2005.
[7] T. Jayabarathi, K. Jayaprakash, D. N. Jeyakumar and T. Raghunathan,
"Evolutionary programming techniques for different kinds of economic

Authorized licensed use limited to: UNIVERSITI TEKNOLOGI MARA. Downloaded on July 15,2023 at 05:40:10 UTC from IEEE Xplore. Restrictions apply.
308