See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/265016158

A proposed iteration optimization approach integrating backpropagation

neural network with genetic algorithm

Article in Expert Systems with Applications · January 2015

DOI: 10.1016/j.eswa.2014.07.039

CITATIONS READS

96 3,457

3 authors, including:

Han-Xiong Huang Cheng-Long Xiao

South China University of Technology University of South China
221 PUBLICATIONS 4,032 CITATIONS 12 PUBLICATIONS 231 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Hunan Provincial Natural Science Founding View project

All content following this page was uploaded by Cheng-Long Xiao on 13 May 2019.

The user has requested enhancement of the downloaded file.

 Expert Systems with Applications 42 (2015) 146–155

Contents lists available at ScienceDirect

Expert Systems with Applications

journal homepage: www.elsevier.com/locate/eswa

A proposed iteration optimization approach integrating backpropagation

neural network with genetic algorithm
Han-Xiong Huang ⇑, Jiong-Cheng Li, Cheng-Long Xiao
Lab for Micro Molding and Polymer Rheology, The Key Laboratory of Polymer Processing Engineering of the Ministry of Education, South China University of Technology,
Guangzhou 510640, PR China

a r t i c l e i n f o a b s t r a c t

Article history: An iteration optimization approach integrating backpropagation neural network (BPNN) with genetic
Available online 4 August 2014 algorithm (GA) is proposed. The main idea of the approach is that a BPNN model is first developed and
trained using fewer learning samples, then the trained BPNN model is solved using GA in the feasible
Keywords: region to search the model optimum. The result of verification conducted based on this optimum is added
Iteration optimization as a new sample into the training pattern set to retrain the BPNN model. Four strategies are proposed in
Backpropagation neural network the approach to deal with the possible deficiency of prediction accuracy due to fewer training patterns
Genetic algorithm
used. Specifically, in training the BPNN model, the Bayesian regularization and modified Levenberg–Mar-
Blow molding
quardt algorithms are applied to improve its generalization ability and convergence, respectively; elitist
strategy is adopted and simulated annealing algorithm is embedded into the GA to improve its local
searching ability. The proposed approach is then applied to optimize the thickness of blow molded poly-
propylene bellows used in cars. The results show that the optimal die gap profile can be obtained after
three iterations. The thicknesses at nine teeth peaks of the bellow molded using the optimal gap profile
fall into the desired range (0.7 ± 0.05 mm) and the usage of materials is reduced by 22%. More impor-
tantly, this optimal gap profile is obtained via only 23 times of experiments, which is far fewer than that
needed in practical molding process. So the effectiveness of the proposed approach is demonstrated.
Ó 2014 Elsevier Ltd. All rights reserved.

1. Introduction deduced from some physical laws. For this reason, some approxi-
mation-based process modeling methods, including response sur-
Many engineering problems, such as system design, process face methodology, radial basis function, Kriging model, and
control and prediction, and part manufacturing, are related to opti- neural network (NN), are usually used to approximate the explicit
mization. The purposes of the engineering optimizations can be functions in many engineering applications (Elsayed & Lacor, 2012;
generally summarized as follows: enhancing the system perfor- Gao & Wang, 2008; Huang, Li, Li, & Huang, 2011; Huang & Lu, 2005;
mance (Li & Yang, 2008; Wang, Zhao, Li, & Guan, 2011), increasing Mirmohseni & Zavareh, 2011). Among these methods, NN is a syn-
the process control and prediction precisions (Chang & Shih, 2010; ergistic representation of mathematical methods helpful in the
Wang, Dong, & Sun, 2010), improving the product quality (Liu & modeling of nonlinear multivariate systems. The feature of the
Yang, 2008; Raja & Baskar, 2012), saving the cost (Lee & Lin, NN is its ability to capture complex nonlinear relationships
2009; Wang, Wang, & Wang, 2013), etc. In engineering optimiza- between output and input patterns through appropriate learning.
tions, a mathematical model is first developed for representing Among NN approaches, backpropagation neural network (BPNN)
the quantitative relationship between the outputs and inputs of is the most classically and generally used training algorithm, and
the investigated system or process, and then is solved in feasible can provide effective solutions to industrial applications. BPNN is
region using an optimization algorithm to obtain the optimal multilayer feed-forward neural network that is trained by the error
process parameters. BP algorithms. Although the BPNN is successful, it has some disad-
However, precise explicit functions mapping the outputs and vantages. The algorithm is not guaranteed to find a global optimum
inputs of a system in engineering problems are often complex and the convergence rate tends to be extremely low. In addition,
and nonlinear, and so quite difficult or even impossible to be the selection of the learning factor and inertial factor, which is usu-
ally determined by experience, affects its convergence. Genetic
⇑ Corresponding author. Tel./fax: +86 20 22236799. algorithm (GA) is a heuristic and stochastic optimization algorithm
E-mail address: [email protected] (H.-X. Huang). based on evolution theory and genetic principles. It is an aggressive

http://dx.doi.org/10.1016/j.eswa.2014.07.039
0957-4174/Ó 2014 Elsevier Ltd. All rights reserved.
 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155 147

search approach that quickly converges to find the optimal solu- BP algorithm. Ho and Chang (2011) used a GA in the BPNN to find the
tion in a large solution domain by the genetic manipulations. optimal parameters to investigate the promoted effectiveness of
Various investigations have demonstrated that combining BPNN predicting platelet transfusion requirements for acute myeloblastic
and GA is a helpful methodology to obtain desirable solutions to leukemia patients. Irani and Nasimi (2011) presented a GA evolved
optimization problems (Ahmad, Jeenanunta, Chanvarasuth, & BPNN, which can improve the reliability and predictability of BPNN.
Komolavanij, 2014; Chatterjee & Bandopadhyay, 2012; Chen, Lai, Each initial weight of the gradient descent-based BPNN was selected
Wang, & Hung, 2011; Chen et al., 2014; Cho, Moon, Kim, & Yun, by a standard GA and the fitness of the GA was determined by the
2012; Cook, Ragsdale, & Major, 2000; Dehghani, Sefti, Ameri, & BPNN. The genetic operators and parameters were carefully
Kaveh, 2008; Esmaeili & Dashtbayazi, 2014; Gossard, Lartigue, & designed and set, avoiding premature convergence and permutation
Thellier, 2013; Ho & Chang, 2011; Huang & Huang, 2007; Irani & problems. The methodology combines the local searching ability of
Nasimi, 2011; Kim & Han, 2003; Ko et al., 2009; Krishna, the gradient decent method with the global searching ability of the
Rangajanardhaa, Hanumantha, & Sreenivasa, 2009; Mirarab, GA. Su et al. (2011) found that better initial weight/bias for the NN
Sharifi, Ghayyem, & Mirarab, 2014; Nasseri, Asghari, & Abedini, can be calculated by the GA. Chatterjee and Bandopadhyay (2012)
2008; Singh, Cooper, Blundell, Pratihar, & Gibbons, 2014; Sinha, used a GA for the selection of BPNN parameters to forecast the reli-
Sikdar (Dey), Chattopadhyay, & Datta, 2013; Solenimani, ability of a load–haul–dump machine. In the work of Esmaeili and
Shoushtari, Mirza, & Salahi, 2013; Su, Yang, & Huang, 2011; Dashtbayazi (2014), a BPNN model was used for predicting the char-
Versace, Bhatt, Hinds, & Shiffer, 2004; Wang et al., 2010; Yuen, acteristics of the prepared Al/SiC nanocomposite, and then GA was
Wong, Qian, Chan, & Fung, 2009). For example, Kim and Han applied to optimize the process parameters. The results showed that
(2003) proposed a hybrid model composed of BPNN and GA, in the combination of the BPNN and GA would make good on appropri-
which the GA globally searches and seeks an optimal or near-opti- ate use of data for predicting and optimizing preferred parameters in
mal BPNN topology. Huang and Huang (2007) proposed a hybrid materials processing technology. Singh et al. (2014) applied a GA to
method consisting of finite element method, BPNN and GA to opti- enhance the prediction accuracy of BPNN by altering its topology.
mize the parison thickness distribution for an extrusion blow Mirarab et al. (2014) performed an optimization procedure based
molded plastics part with required thickness distribution. The on GA to select the best BPNN architecture and determine the opti-
results showed that the proposed method can be used to effectively mum neuron numbers in the hidden layer of the BPNN.
obtain the optimal parison thickness distribution. Dehghani et al. As a combined prediction-optimization approach, the hybrid
(2008) used GA to optimize the connection weights, network archi- BPNN–GA model processes excellent performance since it com-
tecture and learning rules of BPNN model. Ko et al. (2009) investi- bines the inherent merits of BPNN (i.e. accurate nonlinear data fit-
gated the process modeling for the growth rate in pulsed laser ting or regression capabilities) and GA (i.e. efficient and parallel
deposition-grown ZnO thin films using BPNN and GA. The results global searching ability). From the foregoing, the optimization
showed that this modeling methodology can explain the character- strategy coupling BPNN with GA is a very effective approach to
istics of the thin film growth mechanism varying with process solve engineering optimization problems. Moreover, following
conditions. Wang et al. (2010) coupled a BPNN with a GA to predict two main aspects are covered in applying GA into BPNN. One is
the saturates of sour vacuum gas oil. The study showed that the GA to optimize the topological structure of the network, and the other
can find the optimal architecture of the NN and the parameters of the is to optimize some parameters of the network. However, there are

Fig. 1. Flow chart of proposed iteration optimization approach integrating BPNN with GA in this work.
148 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155

Table 1 2. Iteration optimization approach integrating backpropagation

Experimental design with L18 (37) orthogonal array and objective function values y.
neural network with genetic algorithm
Exp. No. O1 O2 O3 O4 O5 O6 O7 y
1 65 65 65 65 65 65 65 0.0144 Fig. 1 shows the flow chart of the proposed iteration optimiza-
2 65 80 80 80 80 80 80 0.0120 tion approach integrating BPNN with GA. The main idea of the pro-
3 65 95 95 95 95 95 95 0.1455 posed optimization approach is described as follows. A BPNN
4 80 65 65 80 80 95 65 0.0355
model is first established and trained using fewer learning sam-
5 80 80 80 95 95 65 65 0.0988
6 80 95 95 65 65 80 80 0.0135 ples, which can be acquired by experiments or numerical simula-
7 95 65 80 65 95 80 95 0.0322 tions according to the design of experiment (DOE), such as
8 95 80 95 80 65 95 65 0.0307 orthogonal design method, Latin hypercube sampling technique,
9 95 95 65 95 80 65 80 0.1631 and Box–Behnken or center composite experimental designs.
10 65 65 95 95 80 80 65 0.2451
Thereafter, using the trained BPNN model as the fitness function
11 65 80 65 65 95 95 80 0.0787
12 65 95 80 80 65 65 95 0.0397 of the GA, the model is solved in the feasible region to search the
13 80 65 80 95 65 95 80 0.0082 model optimum. The verification experiment or simulation based
14 80 80 95 65 80 65 95 0.0556 on the derived optimum is conducted. Then the optimum and cor-
15 80 95 65 80 95 80 65 0.0851
responding experimental or simulated results are added into the
16 95 65 95 80 95 65 80 0.0992
17 95 80 65 95 65 80 95 0.0152
training pattern set as a new learning sample to retrain the BPNN
18 95 95 80 65 80 95 65 0.0191 model. Therefore, only one additional experiment or simulation is
required in each iteration. The training and searching processes
repeat until the optimization process is converged.
To deal with the possible deficiency of prediction accuracy due
to fewer training patterns used in the proposed approach, the fol-
still some drawbacks of this combined approach needed to be dealt
lowing four strategies are adopted in this work. (i) The Bayesian
with. For example, a great number of learning samples acquired by
regularization algorithm (MacKay, 1992, 1995) is used during the
costly experiments or time-consuming numerical simulations
training of the BPNN model to improve its generalization ability.
should be provided as the training patterns to train the BPNN
(ii) Standard Levenberg–Marquardt training algorithm is modified
model with accurate generality. When trained using the standard
to accelerate the BPNN convergence. (iii) Simulated annealing algo-
training algorithm such as standard Levenberg–Marquardt algo-
rithm (SAA) is embedded into the GA to enhance its local searching
rithm, the convergence of the BPNN is relatively slow, especially
ability, which facilitates to effectively improve the quality of solu-
for the networks with large scale. Moreover, the standard GA tends
tions. (iv) For each iteration, the currently obtained best solution is
to be trapped into a local optimum rather than a global one in the
always selected as an individual in the initial population of the GA;
search process. To address these problems encountered in engi-
moreover, an elitist strategy is applied, in which the elite individ-
neering optimizations, a novel iteration optimization approach
ual (namely the current best individual during evolution) is
combining BPNN with GA is proposed in this work, in which the
directly copied to the population of the next generation without
standard Levenberg–Marquardt algorithm and GA are modified
performing any genetic operations. By adopting the four strategies,
and improved. By using the proposed approach, the optimization
the solution obtained using the proposed iteration optimization
results can be easily and quickly obtained without carrying out
approach is better and better with the iteration. In addition, the
too many experiments or numerical simulations. The effectiveness
gradual increase of the learning samples number in the training
of the approach is confirmed by its application in the wall thick-
pattern set can also improve the generalization ability and predic-
ness optimization for a complex-shaped industrial plastics part.
tion accuracy of the BPNN model and thus facilitate to improve the

Fig. 2. Schematics of extrusion blow molding process.

 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155 149

Fig. 3. Schematics of parison thickness adjustment using die gap programming in extrusion blow molding.

quality of solutions. It is noteworthy that, to the authors’ best et al., 2011). The update rule for network weight values using the
knowledge, no literature reports such hybrid iteration optimiza- Levenberg–Marquardt algorithm is defined as:
tion approach. In the following, aforementioned strategies (ii)
and (iii) are described in detail. Dwk ¼ g½Hðwk1 Þ þ ki I1 rEðwk1 Þ ð1Þ

where Dwk is the weight value increment of the neural network in

2.1. Accelerate BPNN convergence using modified Levenberg– the kth update step, g is the network learning rate, H is the Hessian
Marquardt algorithm matrix, ki is a positive constant, and rE is the gradient of the net-
work error function. The calculation of the Hessian matrix inversion
Conventional backpropagation algorithms (e.g., the steepest at each update step is required in Eq. (1), which is a quite time-
descent method) are the first-order training algorithm for they consuming process and significantly lowers the network conver-
require only the first derivatives of the error function. The sec- gence rate. In light of this, a modified Levenberg–Marquardt
ond-order algorithms (e.g., Levenberg–Marquardt algorithm), in algorithm is proposed to avoid computing the inverse matrix and
which the second derivatives of the error function need to be esti- thus to increase the network training efficiency in this work. Start
mated, are demonstrated to be more effective for training NN with the initial weight given in the first stage and denote the term
model with a relatively small number of weight and bias (Su of Hðwk1 Þ þ ki I as matrix G, Eq. (1) can be rewritten as:

Dwk ¼ gG1 ðwk1 ÞrEðwk1 Þ ð2Þ

Left-multiplication by matrix G on both sides of Eq. (2) results in:

Gðwk1 ÞDwk ¼ grEðwk1 Þ ð3Þ

The G is a nonsingular matrix, so it can be decomposed into the
multiplication of a lower triangular matrix L and a upper triangular
matrix U, namely G = LU. In this way, Eq. (3) can be solved by
readily performing the forward substitution and back substitution
to find the Dwk. This is a computationally efficient process in com-
parison to the direct calculation of G1. The latter takes about three
times as many arithmetic operations as the former. Moreover, the
results obtained using the LU decomposition method may be more
accurate.

2.2. Embed simulated annealing algorithm into genetic algorithm

SAA is a heuristic, stochastic, and global optimization algorithm

for nonlinear programming problems proposed by simulating the
Fig. 4. Schematics of car bellow investigated in this work. slow cooling process of molten metal (Kirapatrick, Gelatt, &
150 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155

Fig. 5. Flow chart of optimization process based on proposed approach for case study.

Vecchi, 1983). Its basic idea is to generate a random point to avoid bine the merits of both algorithms. That is, it possesses not only
getting trapped at a local optimum. The random search not only excellent global searching ability of the GA, but also powerful local
accepts the changes that decrease the fitness function f (make it searching ability of the SAA. The detailed procedure of the inte-
better), but also the changes that increase the f (make it worse) grated algorithm is described as follows.
with a probability p = exp(D/T), where D is the increment of
the f and T is a control parameter known as system absolute tem- Step 1: Initialization. Initialize the population size N of the GA,
perature. The detailed implementation of the SAA can be found in the initial annealing temperature T0, the final annealing tem-
Abbasi and Mahlooji (2012). perature Ts, and the number of repetitions L allowed at each
Considering that the standard GA tends to be trapped in a local temperature level of the SAA.
optimum rather than global one in the searching process, the SAA Step 2: Genetic operations.
is embedded into the standard GA to improve its local searching Step 2.1: Set iteration number t = 1. Randomly generate N
ability in this work. This integrated algorithm can effectively com- individuals in feasible region to form the first population P1.
 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155 151

molding (EBM) is one of main polymer manufacturing technologies

and has a wide product range from packaging containers to indus-
trial complex parts such as those supplied to the automobile, office
automation equipment, and pharmaceutical sectors. Recently,
more and more automobile parts are gradually turning to be
produced using the EBM technology with the rapid development
of automobile industry. This is attributed to its advantages of
design flexibility, lower production cost, double-wall hollow part
construction, and high part strength-to-weight ratio. Unsuitable
process parameter settings in EBM may cause production
problems, reduce price competitive advantage, and decrease a
company’s profitability. So in this section, the wall thickness
optimization of a polypropylene bellow molded using the EBM
technology is taken as an example to illustrate the engineering
application of the proposed iteration optimization approach.
In the EBM process as schematically shown in Fig. 2, the poly-
mer is melted and mixed in an extruder, and then the molten poly-
mer is extruded through a parison die to form an annular parison,
which is located between two mold halves. When the extruded
parison reaches the predetermined length, the mold is closed and
then the clamped parison is inflated with compressed air to take
the shape of the mold cavity, followed by the part cooling and ejec-
tion. Strictly controlling the wall thickness distribution of EBM
parts is usually required to achieve required mechanical properties
of parts and to minimize the usage of materials. The part wall
thickness distribution depends mainly on the thickness distribu-
Fig. 6. Architecture of BPNN model used in this work.
tion along the parison just prior to inflation, which can be con-
trolled by manipulating the die gap as a function of time during
Step 2.2. Crossover. All the individuals in population Pt are parison extrusion, known as parison programming. Fig. 3 schemat-
randomly chosen as parents for crossover to create the off- ically illustrates the parison thickness adjustment using the die gap
springs, which are then added into the Pt. programming.
Step 2.3. Mutation. Mutate each individual x in the Pt with a
predefined mutation rate. 3.1. Objection function establishment
Step 2.4. Selection. Select N fitter individuals from the Pt
based on their fitness and copy them into population Qt. Fig. 4 schematically shows the EBM bellow investigated in this
Step 3: Simulated annealing operations. work. This bellow used in cars has nine triangular teeth, and the
Step 3.1: Each individual x in the Qt is separately selected as distance between adjacent two peaks of five teeth near the small
the initial solution S0 of the SAA for update. Set T = T0, the and big ends is 10 and 15 mm, respectively. The durability of the
current solution S = S0, the best solution S⁄ = S0 and the rep- bellow in applications is mainly dependent on the thicknesses at
etition counter n=1; then calculate the function f(S0). the teeth peaks, which should not be less than a certain value.
Step 3.2: Generate the neighboring solution Sn in the neigh- Considering the material usage, the peaks should not be too thick.
borhood of the S; calculate D = f(Sn)  f(S). If D 6 0, then set In this work, the thicknesses at the nine teeth peaks are controlled
S = Sn; If D > 0, generate a random number 0 < r < 1, if in the range of 0.7 ± 0.05 mm. This is not an easy task and is
r 6 p = exp(D/T), set S = Sn and n = n + 1. If f(S) < f(S⁄), set
S⁄ = Sn.
Step 3.3: If n < L, go to step 3.2; else, go to step 3.4.
Step 3.4: Add N updated individuals into the Qt, then select N
fitter individuals from the Qt based on their fitness and copy
them into population Pt+1.
Step 3.5: Reduce the temperature T according to Ti = CTi1, in
which C is a constant less than 1. Typically, 0.75 6 C 6 0.95
(Abbasi & Mahlooji, 2012).
Step 4: If the T is reduced to the Ts, terminate the search and
output the current best individual x⁄ in the Pt+1 as the optimal
solution; else, set t = t + 1, then go to step 2.2.

3. Application of proposed approach in wall thickness

optimization of extrusion blow molded part

Determination of optimal process parameter settings, which

affects productivity, quality, production cost and delivery time, is
crucial work and regarded as the most challenging part in manu-
facturing industries. To ascertain the optimal process parameter
settings, numerous process trials are generally required to evaluate Fig. 7. Measured thicknesses at nine peaks of bellow molded using initial die gap
the process variables and their interactions. Extrusion blow profile (‘‘gap 0’’).
152 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155

Fig. 8. Measured thicknesses at nine peaks of bellow molded using ‘‘gap 1’’.

difficult to be achieved using the conventional strategy based on

the experience and intuition of operators. To achieve this goal, it
is necessary to optimize the thickness distribution along the pari-
son, and so seven programming points are set to manipulate the
die opening in the parison extrusion stage. The die openings at
the seven discrete points, denoted as O1, O2, . . ., O7, are identified
as the design variables.
The peaks of the nine teeth (denoted as 1#, 2#, . . ., 9# shown in
Fig. 4) of the bellow are selected as the measured positions of wall
thicknesses, and the following objective function is developed to
evaluate the wall thickness distribution:
Pm 2
j¼1 ½Thj ðnÞ  Thobj 
min y ¼ gðnÞ ¼ ð4Þ
m
where n = f(O1, O2, . . ., O7) is the vector of design variables, Thj is the
peak thickness at the jth measured position, Thobj denotes the objec-
tive wall thickness for the nine teeth peaks, and m is the total num-
ber of the measured positions. Obviously, the smaller the value of
the objective function y, the smaller the deviation between the
actual and objective wall thicknesses at the nine teeth peaks of
the molded bellows is.

3.2. Optimization process

The purpose of the optimization for this case study is to employ

the proposed approach to model and acquire the optimal die gaps
corresponding to the seven parison sections for obtaining the
desired wall thicknesses at the nine teeth peaks in the molded
bellows. Fig. 5 illustrates the flow chart of the optimization process
in detail. The process includes two steps, that is, the design and
analysis of experiment and the iteration optimization for die gap
profile.
In the design and analysis of experiment, the orthogonal design
method is used to collect the representative samples for training
the BPNN model more effectively. The levels of the design variables
are determined based on the initial die gap profile with a fixed
opening of 80% (denoted as ‘‘gap 0’’). Three levels (65%, 80% and
95% die openings) for each design variable are set, so eighteen
experimental runs depended on L18 (37) orthogonal array as shown
in Table 1 are performed.
The orthogonal experiment results are analyzed using the
sensitivity analysis method, in which the sensitivities of the design
variables at each level are calculated. Then, the optimal level of Fig. 9. Evolution of measured thicknesses at nine peaks of molded bellows with
each design variable is obtained according to the analyzed results, iteration in iteration optimization process.
 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155 153

Table 2 3.3. Results and discussion

Objective function values y and weights for bellows molded in optimization process.

No. 1 2 3 4 5 The molded bellows are cut open along their axial plane of sym-
Die gap profile Gap 0 Gap 1 Gap 2 Gap 3 Gap 4 metry. The wall thicknesses at the nine peaks (as shown in Fig. 4)
y 0.0066 0.0029 0.0373 0.0052 0.0006 are measured using a digimatic caliper with a resolution of
Weight (g) 61.39 55.02 55.42 51.37 47.98 0.001 mm. Fig. 7 gives the measured thicknesses of the nine peaks
of the bellow molded using the initial die gap profile (gap 0). As can
be seen, only four peaks exhibit the thicknesses falling into the
desired range (0.7 ± 0.05 mm), and the difference between maxi-
and the corresponding die gap profile is denoted as ‘‘gap 1’’. The mum and minimum thicknesses among the nine peaks is about
verification experiment with ‘‘gap 1’’ is conducted and the thick- 0.23 mm. For each set of design variables in the orthogonal array
nesses at the nine measured positions on the molded bellow are (shown in Table 1), a corresponding EBM experiment is conducted.
detected. If the bellow molded using ‘‘gap 1’’ meets the desired The measured thicknesses are used to estimate the objective func-
thickness requirement, output ‘‘gap 1’’ as the optimal die gap pro- tion values y shown in Eq. (4), which are listed in the correspond-
file for blow molding process; else, totally 20 samples obtained so ing column in Table 1. The sensitivity analyses on the orthogonal
far by conducting the EBM experiments according to ‘‘gap 0’’, L18 experiment results show that the optimum set of design variable
orthogonal array and ‘‘gap 1’’ are used as the training patterns of levels is a combination of O12, O22, O32, O41, O52, O61, and O71,
the BPNN model in the following iteration optimization step. which is denoted as gap 1. Gap 1 corresponds to 80% die opening
In the iteration optimization for die gap profile, all the currently for 1#, 2#, 3#, and 5# programming points, and 65% opening for
obtained samples are used as the training patterns to train the 4#, 6#, and 7# points. Fig. 8 shows the measured thicknesses at
BPNN model. A three-layer (namely, an input layer, a hidden layer the nine peaks of the bellow molded using gap 1. It can be clearly
and an output layer) BPNN model is developed to build the quan- observed that the uniformity of the thickness distribution is greatly
titative relationship between the objective function (Eq. (4)) and improved, the difference between maximum and minimum thick-
die openings. Sigmoid and linear transfer functions are used for nesses decreases from initial 0.23 to 0.12 mm. The y value obtained
the hidden and output layers, respectively. The seven die openings using gap 1 is 0.0029, which is much less than the minimum y
in all the samples are selected as the input parameters of the BPNN value obtained in the orthogonal experiment (0.0082, Table 1).
model, their corresponding objective function values are the out- However, still four peaks exhibit the thicknesses out of the desired
put parameters. Because there is no definite rule to determine range. So the thickness optimization procedure goes into the iter-
the appropriate number of neurons in the hidden layer, it is deter- ation optimization step.
mined by trail-and-error method in this work. The results showed Fig. 9 shows the evolution of the measured wall thicknesses at
that the BPNN model with 10 neurons in the hidden layer can give the nine peaks with iteration in the iteration optimization process.
better generalization and effectively avoid the overfitting of neural It can be clearly seen that the deviation between the actual and
network. So a BPNN model with 7-10-1 architecture is constructed, desired thickness distributions is gradually decreased with itera-
as shown in Fig. 6. In training the BPNN process, the aforemen- tion. After three iterations, the thicknesses at all measured posi-
tioned Bayesian regularization algorithm and modified Leven- tions fall into the desired range and the average thickness is
berg–Marquardt algorithm are applied. Moreover, the tolerance 0.701 mm, then the iteration optimization process is terminated.
of the mean square error (MSE) is set at 0.0001. Once the MSE of Table 2 summarizes the objective function values y and weights
the network is reduced within the given tolerance, the training for the bellows molded using the aforementioned gap 0 and gap
process is terminated and the trained BPNN model is obtained. 1 and the die gap profiles obtained in the three iterations (denoted
The trained BPNN model can be used to evaluate the objective as gap 2, gap 3, gap 4). As can be seen, the y value is very small
function value for different sets of design variables within the (0.0006) in the final iteration. Significantly, the bellow weights
feasible region. So the trained BPNN model is directly used as the are reduced from the initial 61.39 g to the optimized 47.98 g, sav-
fitness function of the GA for solving the model to search the opti- ing about 22% materials. This indicates that the proposed iteration
mal die gap profile. For each iteration optimization, ‘‘gap n  1’’ optimization approach in this work is feasible and very effective to
(namely the current best die gap profile) is selected as an individ- obtain the optimal die gap profile for achieving the desired thick-
ual in the initial population of the GA. Meanwhile, the elitist strat- ness distribution in final parts. More importantly, this optimal
egy is adopted and the SAA is embedded into the GA to improve its gap profile is obtained via only 23 experiment times, which is far
local searching ability in the searching process of the GA. The GA fewer than that needed in practical molding process. The corre-
implements an investigation procedure in four stages: initializa- sponding die gap profile (gap 4) is taken as the optimal one input
tion, selection, crossover, and mutation. In this work, the initial into the controller of the EBM machine for the continuous blow
population size is set at 40, arithmetic crossover is employed and molding process.
the crossover rate is 0.6, the nonuniform mutation operator is used
and the mutation rate is 0.01. Moreover, the initial and final
annealing temperatures for the SAA are set at 1000 and 10 °C, 4. Conclusions
respectively; and the number of repetitions allowed at each tem-
perature level is set at 200. The contributions of the current work include both methodol-
The derived optimum solution from the GA is used as a newly ogy and application aspects. In the aspect of methodology, a hybrid
updated die gap profile (denoted as ‘‘gap n’’). The corresponding optimization approach integrating both BPNN and GA is proposed.
EBM experiment using ‘‘gap n’’ is conducted. If the wall thicknesses Four strategies are adopted to overcome the possible deficiency of
at the nine teeth peaks of the molded bellow meets the desired prediction accuracy due to fewer training patterns used in the
requirement, then ‘‘gap n’’ is input into the controller of the EBM approach. Specifically, in training the BPNN model, the Bayesian
machine for blow molding process; or else, ‘‘gap n’’ and corre- regularization and modified Levenberg–Marquardt algorithms are
sponding objective function value are added as a new sample into applied to improve its generalization ability and convergence,
the training pattern set to retrain the BPNN model. The training respectively; the elitist strategy is adopted and the SAA is embed-
and searching processes repeat until the iteration optimization ded into the GA to improve its local searching ability. The proposed
process is converged. approach can avoid getting trapped at a local optimum. More
154 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155

importantly, it can avoid using a great number of learning samples, Elsayed, K., & Lacor, C. (2012). Modeling and Pareto optimization of gas cyclone
separator performance using RBF type artificial neural network and genetic
which are acquired by expensive experiments or numerical simu-
algorithm. Powder Technology, 217, 84–99.
lations, for training the BPNN model. Esmaeili, R., & Dashtbayazi, M. R. (2014). Modeling and optimization for
The hybrid optimization approach proposed in this work shows microstructural properties of Al/SiC nanocomposite by artificial neural
great potential in complicated industrial applications. The wall network and genetic algorithm. Expert Systems with Applications, 41(13),
5817–5831.
thickness optimization of the polypropylene bellows molded using Gao, Y. H., & Wang, X. C. (2008). An effective warpage optimization method in
the blow molding technology is taken as an example to illustrate injection molding based on the Kriging model. The International Journal of
the application of the proposed approach. The trained BPNN model Advanced Manufacturing Technology, 37(9–10), 953–960.
Gossard, D., Lartigue, B., & Thellier, F. (2013). Multi-objective optimization of a
is used as the fitness function of the GA for solving the model to building envelope for thermal performance using genetic algorithms and
search the optimal die gap profile. The results show that the pro- artificial neural network. Energy and Buildings, 67, 253–260.
posed approach is feasible and very effective to find the optimal Ho, W. H., & Chang, C. S. (2011). Genetic-algorithm-based artificial neural network
modeling for platelet transfusion requirements on acute myeloblastic leukemia
die gap profile for achieving the desired thickness distribution in patients. Expert Systems with Applications, 38(5), 6319–6323.
final bellow without doing too many experiments. After three iter- Huang, G. Q., & Huang, H. X. (2007). Optimizing parison thickness for extrusion
ation optimizations, the thicknesses at all the nine peaks fall into blow molding by hybrid method. Journal of Materials Processing Technology,
182(1–3), 512–518.
the desired range (0.7 ± 0.05 mm) and the average thickness is Huang, H. X., Li, J. C., Li, D., & Huang, G. Q. (2011). New strategies for predicting
0.701 mm. Moreover, the usage of materials is reduced by 22%. parison dimensions in extrusion blow molding. Polymer–Plastics Technology and
That is, satisfactory result is obtained via fewer experiment num- Engineering, 50(13), 1329–1337.
Huang, H. X., & Lu, S. (2005). Modeling parison formation in extrusion blow molding
ber, which is only 23 and far fewer than that needed in practical
by neural network. Journal of Applied Polymer Science, 96(6), 2230–2239.
molding process for such complex part. The proposed approach Irani, R., & Nasimi, R. (2011). Evolving neural network using real coded genetic
can effectively help to determine optimal process parameter set- algorithm for permeability estimation of the reservoir. Expert Systems with
tings and achieve competitive advantages on product quality and Applications, 38(8), 9862–9866.
Kim, K. J., & Han, I. (2003). Application of a hybrid genetic algorithm and neural
costs in manufacturing industries. network approach in activity-based costing. Expert Systems with Applications,
Work is still needed to be performed in three major aspects. (1) 24(1), 73–77.
This work focuses on the optimal die gap profile in blow molding. Kirapatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated
annealing. Science, 220, 671–680.
For future work the main concern may be towards integration of Ko, Y. D., Moon, P., Kim, C. E., Ham, M. H., Myoung, J. M., & Yun, I. (2009). Modeling
more design variables. The proposed approach may be extended and optimization of the growth rate for ZnO thin films using neural networks
to other complicated engineering optimization problems for and genetic algorithms. Expert Systems with Applications, 36(2), 4061–4066.
Krishna, M. R. G., Rangajanardhaa, G., Hanumantha, R. D., & Sreenivasa, R. M. (2009).
obtaining optimal solutions via less cost and time spent on exper- Development of hybrid model and optimization of surface roughness in electric
iments and analyses. (2) The trial and error method is used for discharge machining using artificial neural networks and genetic algorithm.
determining the parameters in the GA, including the population Journal of Materials Processing Technology, 209(3), 1512–1520.
Lee, W. S., & Lin, L. C. (2009). Optimal chiller loading by particle swarm algorithm
size, crossover rate, and mutation rate. In the future, GA parame- for reducing energy consumption. Applied Thermal Engineering, 29(8–9),
ters may be considered to improve the predictive performance of 1730–1734.
the model. (3) Other heuristic optimization algorithms, such as Li, S. Y., & Yang, M. (2008). Particle swarm optimization combined with finite
element method for design of ultrasonic motors. Sensor and Actuators A, 148(1),
particle swarm optimization, may be used to deal with similar
285–289.
optimization problems, and their performances are compared to Liu, W., & Yang, Y. Y. (2008). Multi-objective optimization of sheet metal forming
that of the GA. process using Pareto-based genetic algorithm. Journal of Materials Processing
Technologies, 208(1–3), 499–506.
MacKay, D. J. C. (1992). Bayesian interpolation. Neural Computation, 4(3), 415–447.
Acknowledgments MacKay, D. J. C. (1995). Bayesian neural networks and density networks. Nuclear
Instruments and Methods in Physics Research A, 354(1), 73–80.
Financial support provided by National Natural Science Founda- Mirarab, M., Sharifi, M., Ghayyem, M. I., & Mirarab, F. (2014). Prediction of solubility
of CO2 in ethanol–[EMIM][Tf2N] ionic liquid mixtures using artificial neural
tion of China and National High-Tech Research and Development networks based on genetic algorithm. Fluid Phase Equilibria, 371, 6–14.
Program is gratefully acknowledged. Mirmohseni, A., & Zavareh, S. (2011). Modeling and optimization of a new impact-
toughened epoxy nanocomposite using response surface methodology. Journal
of Polymer Research, 18(4), 509–517.
References Nasseri, M., Asghari, K., & Abedini, M. J. (2008). Optimized scenario for rainfall
forecasting using genetic algorithm coupled with artificial neural network.
Abbasi, B., & Mahlooji, H. (2012). Improving response surface methodology by using Expert Systems with Applications, 35(3), 1415–1421.
artificial neural network and simulated annealing. Expert Systems with Raja, S. B., & Baskar, N. (2012). Application of particle swarm optimization
Applications, 39(3), 3461–3468. technique for achieving desired milled surface roughness in minimum
Ahmad, I., Jeenanunta, C., Chanvarasuth, P., & Komolavanij, S. (2014). Prediction of machining time. Expert Systems with Applications, 39(5), 5982–5989.
physical quality parameters of frozen shrimp (litopenaeus vannamei): an Singh, A., Cooper, D. E., Blundell, N. J., Pratihar, D. K., & Gibbons, G. J. (2014).
artificial neural networks and genetic algorithm approach. Food and Bioprocess Modelling of weld-bead geometry and hardness profile in laser welding of plain
Technology, 7(5), 1433–1444. carbon steel using neural networks and genetic algorithms. International Journal
Chang, W. D., & Shih, S. P. (2010). PID controller design of nonlinear systems using of Computer Integrated Manufacturing, 27(7), 656–674.
an improved particle swarm optimization approach. Communications in Sinha, A., Sikdar (Dey), S., Chattopadhyay, P. P., & Datta, S. (2013). Optimization of
Nonlinear Science and Numerical Simulation, 15(11), 3632–3639. mechanical property and shape recovery behavior of Ti-(49 at.%) Ni alloy
Chatterjee, S., & Bandopadhyay, S. (2012). Reliability estimation using a genetic using artificial neural network and genetic algorithm. Materials & Design, 46,
algorithm-based artificial neural network: an application to a load–haul–dump 227–234.
machine. Expert Systems with Applications, 39(12), 10943–10951. Solenimani, R., Shoushtari, N. A., Mirza, B., & Salahi, A. (2013). Experimental
Chen, G. Y., Fu, K. Y., Liang, Z. W., Sema, T., Li, C., Tontiwachwuthikul, P., & Idem, R. investigation, modeling and optimization of membrane separation using
(2014). The genetic algorithm based back propagation neural network for MMP artificial neural network and multi-objective optimization using genetic
prediction in CO2-EOR process. Fuel, 126, 202–212. algorithm. Chemical Engineering Research and Design, 91(5), 883–903.
Chen, W. C., Lai, T. T., Wang, M. W., & Hung, H. W. (2011). An optimization system Su, C. L., Yang, S. M., & Huang, W. L. (2011). A two-stage algorithm integrating
for LED lens design. Expert Systems with Applications, 38(9), 11976–11983. genetic algorithm and modified Newton method for neural network training in
Cho, E. N., Moon, P., Kim, C. E., & Yun, I. (2012). Modeling and optimization of ITO/Al/ engineering systems. Expert Systems with Applications, 38(10), 12189–12194.
ITO multilayer films characteristics using neural network and genetic Versace, M., Bhatt, R., Hinds, O., & Shiffer, M. (2004). Predicting the exchange traded
algorithm. Expert Systems with Applications, 39(10), 8885–8889. fund DIA with a combination of genetic algorithms and neural networks. Expert
Cook, D. F., Ragsdale, C. T., & Major, R. L. (2000). Combining a neural network with a Systems with Applications, 27(3), 417–425.
genetic algorithm for process parameter optimization. Engineering Applications Wang, G. L., Zhao, G. Q., Li, H. P., & Guan, Y. J. (2011). Research on optimization
of Artificial Intelligence, 13(4), 391–396. design of the heating/cooling channels for rapid heat cycle molding based on
Dehghani, S. A. M., Sefti, M. V., Ameri, A., & Kaveh, N. S. (2008). Minimum miscibility response surface methodology and constrained particle swarm optimization.
pressure prediction based on a hybrid neural genetic algorithm. Chemical Expert Systems with Applications, 38(6), 6705–6791.
Engineering Research & Design, 86(A2), 173–185.
 H.-X. Huang et al. / Expert Systems with Applications 42 (2015) 146–155 155

Wang, H. S., Wang, Y. N., & Wang, Y. C. (2013). Cost estimation of plastic injection Yuen, C. W. M., Wong, W. K., Qian, S. Q., Chan, L. K., & Fung, E. H. K. (2009). A hybrid
molding parts through integration of PSO and BP neural network. Expert Systems model using genetic algorithm and neural network for classifying garment
with Applications, 40(2), 418–428. defects. Expert Systems with Applications, 36(2), 2037–2047.
Wang, S. C., Dong, X. C., & Sun, R. J. (2010). Predicting saturates of sour vacuum gas
oil using artificial neural networks and genetic algorithm. Expert Systems with
Applications, 37(7), 4768–4771.

View publication stats