Xiao et al.

/ J Zhejiang Univ SCI 2004 5(12):1597-1603 1597

Journal of Zhejiang University SCIENCE

ISSN 1009-3095
http://www.zju.edu.cn/jzus
E-mail: [email protected]

Ant colony system algorithm for the optimization of

beer fermentation control

XIAO Jie (N ~ ) t , ZHOU Ze-kui ( N ] ~ d : ) , ZHANG Guang-xin (~t~;~,~)

(National Key Lab of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China)
rE-mail: [email protected]
Received Sept. 23, 2003; revision accepted Oct. 12, 2003

Abstract: Beer fermentation is a dynamic process that must be guided along a temperature profile to obtain the desired
results. Ant colony system algorithm was applied to optimize the kinetic model of this process. During a fixed period of
fermentation time, a series o f different temperature profiles of the mixture were constructed. An optimal one was chosen at
last. Optimal temperature profile maximized the final ethanol production and minimized the byproducts concentration and
spoilage risk. The satisfactory results obtained did not require much computation effort.

Keywords: Beer fermentation, Kinetic model, ACS algorithm, Optimization, Optimal temperature profile
doi: 10.1631/jzus.2004.1597 Document code: A CLC number: TP20

INTRODUCTION behavior of real ant colonies, in particular, by their

foraging behavior. It is a population-based ap-
Long well known industrial beer fermentation proach using positive feedback as well as greedy
is usually carried out in open loop conditions, using search. The effectiveness of its different versions as
a predefined and empirically established tempera- powerful tools for solving complicated optimiza-
ture profile. The off-line measurement of the wort tion problems was demonstrated in a number of
density, which is well correlated to the fermentable applications (Dorigo et al., 2002), which gave us
sugar concentration, is widely used on industrial inspiration to apply it in dynamic process optimi-
scale for monitoring the fermentation. However, for zation problems.
optimization of operational conditions and process In order to implement ant algorithm in
control purposes, we need a mathematical model optimization appropriately, two aspects should be
for accurately simulating the industrial process taken into consideration:
under different operational conditions. There is no 1) Selecting a suitable version of ant algorithm
steady state in the process, and model equations are (Ant Colony System-ACS algorithm was finally
generally non-linear. adopted in this paper);
Further work based on the model is finding an 2) Adapting algorithm to the specific
optimal or pseudo-optimal operation regime, which optimization problem with some revisions to the
calls for an efficient algorithm to fulfill this purpose. original one. Enough attention should be paid to
Ant algorithm is a new general-purpose heuristic the stochastic greedy rule (the state transition rule)
algorithm, which was first applied in TSP (Trav- and the pheromone updating rule, which are the
eling Salesman Problem) problems, inspired by the essences of the algorithm. The optimization of the
1598 Xiao et al. / J Zhejiang Univ SCI 2004 5(12):1597-1603

process (in forms of some mathematical equations) perature increase cannot be controlled in plants in
has been accomplished with the help o f MATLAB, the later parts of the fermentation process.
which is a high-performance language for tech-
nical computing.
OPTIMIZATION PROBLEM STATEMENT

PROCESS DESCRIPTION AND MATHEMATIC- The optimization of fermentation process is a

AL MODEL multi-objective optimization problem. An objective
function used to evaluate how good the fermenta-
Beer fermentation exhibits complex charac- tion is, should take time and efficiency criteria,
teristics. During the process, sugars are converted together with some constrains, into consideration.
to ethanol. Without mechanical stirring, tempera- The original one (Andres-Toro et al., 1997a) is a
ture is the most important manipulated intervention group of five functions; each of them has a
in the process. Low temperature makes the process sub-objective. To make values of functions com-
time-consuming, and high temperature accelerates parable, some modifications to the original
the process greatly. The later one (above 15 °C) will sub-objective parameters were done:
cause the loss of some ethanol efficiency, and Ethanol production must be as high as possi-
multiply the risk of spoilage. Some byproducts ble:
(especially ethyl acetate and diacetyl), in addition Function I:
to ethanol, are produced depending on temperature. ./1 = +lOC.(t) (1)
They must be constrained within certain limits, in
order to eliminate bad effects on flavor and aroma. in which Ce(t) can be calculated from the following
The brewery uses a specific temperature profile functions:
(which is often confidential) to drive the process.
This work is aimed at finding an optimized dC~(t)
= f,lleXac t (t) (2)
temperature profile characterized by good ethanol dt
efficiency, no spoilage risks, and being not over the
f ~ l - - ~Co(t) and f l o - ~¢°C~(t) (3)
concentration limits of ethyl acetate and diacetyl. 0.5C~o ks + C~(t)
For this purpose, a useful kinetic model dX ~(t)
(Andres-Toro et al., 1998) was adopted. - - - ,UxXact (t) - ,udtXa¢t (t) + fllagXlag (t) (4)
dt
It should be noted that there is a heating circuit
/.txoC~(t)
in the experimental set-up, because the model pa- - (5)
0.5C o + co(t)
rameter values were calculated from the experi-
mental data obtained under isothermal runs (al- dXsd(t)
- - --/./dtSact (t) --/~sdX~ (t) (6)
though the same model can be applied at the dt
non-isothermal pilot plant scale). To keep a con- dC~(t) = _/./sXact(t) (7)
stant temperature, it is essential to have both dt
cooling and heating circuits. However, in many P~oC~(t)
breweries, fermentation tanks only have cooling
(8)
ks + C s(t)
circuits, so the increase of temperature relies on the
heat released by the biochemical process itself. A Diacetyl and ethyl acetate concentration
conclusion can be drawn from this fact that if the should be limited:
optimized temperature profile could be used in real Functions II and III
plants, it should obey the rule that once the tem-
perature has been decreased, it cannot go up again J3 = -5.73 ×exp(l 1×Cdy(t)-11.51) (9)
(Increase Once Rule, in short). Because the tem- 4= - 1 . 1 6 × e x p ( 6 . 7 × C e a ( t ) - 6 6 . 7 7 ) (10)
 Xiao et al. / J Zhejiang Univ SCI 2004 5(12):1597-1603 1599

in which Cdy(t) and Cea(t) can be calculated from the successful research direction in ant algorithm. Ant
next two functions: colony system (ACS) algorithm, one version of
ACO, was introduced by Dorigo and Gambardella
dC~a (t) dC (t) (1997) to improve the performance of Ant System
= r as (-
dt Ot (11) (Dorigo et al.,1996), that was able to find good so-
dC ,(t) lutions for only small TSP problems within a rea-
- - -/ ayC (t)&c, (t) (t)co (t) (12) sonable time. The ACS differs from AS in three
dt
main aspects: 1) the state transition rule provides a
direct way to achieve balance between exploration
Spoilage risk along the process:
of new edges and exploitation of a prior and accu-
Function IV:
mulated knowledge on the problem; 2) the global
updating rule is applied only to edges which belong
Jz = - I £ 9-91x 10-~7 x exp(2.3 lx T)dt (13) to the best tour; 3) while ants construct a solution, a
local pheromone updating rule (local updating rule,
Brisk changes in temperature should be pe- for short) is applied to shuffle the tours. In this way
nalized: ants will make better use of pheromone information.
Function V: These three modifications enable ACS to solve large
problems effectively. With about 160 nodes for ants
J, = - t ~ IlT~ - T~-l[ (14)
to visit, ACS algorithm has been chosen to deal with
i=l At
this optimization problem.
To apply ACS algorithm, a piecewise
Grouping all the five functions:
approximation of the temperature profile by a series
of breakpoints is made. The breakpoints in a tem-
Jmax=Jl +J2+J3 +J4 +J5
(15) perature profile are the visited nodes in an ants' tour,
and are regularly spaced, every one hour. Between
The value obtained depends on the tempera-
each time interval, the temperature is constant.
ture profile applied by the brewery along the fer-
Integer numbers between 0 °C and 16 °C are used to
mentation process. That means a dynamic optimi-
represent temperature and fixed fermentation time
zation problem: the objective function should be
is 160 hours. Until now, the construction graph is
maximized by the application of a certain tem-
clear. There are 161 nodes for each ant to build a
perature profile.
complete tour (a temperature profile). Each node
For the application of ant colony system al-
will be the integer temperature value at the end of
gorithm, the format of this objective function has to
each time interval. At the beginning of each time
be changed:
interval, an ant will decide the temperature in the
next time interval. After this kind of discretization,
Jmin= CONS-Jmax (16)
Jmin and other parameter values can be easily cal-

where CONS (CONS>Jmax)is a positive integer, and culated at each end of the time intervals according
Jmin should be minimized. In addition, the tem- to 16 equations listed above.
perature profile obtained should obey the Increase Fig.1 is an industrial temperature profile
Once Rule. (Andres-Toro et al., 1998). If the process follows
this trajectory, and CONS is set as 1000 (this value
is decided according to some literature and it should
OPTIMIZATION WITH ANT COLONY SYSTEM be bigger than the suggested maximal value Of Jmax),
ALGORITHM values Of Jmi, along the process are shown in Fig.2
showing that the value of Jrnin decreases as time
Ant colony optimization (ACO) is a particular goes on. So it is feasible to minimize Jmin by ACS
1600 Xiao et al. /JZhejiang Univ SC1 2004 5(12):1597-1603

approach. Here Jmin is like tour length in TSP the trail, rgr, u) is a heuristic function given by the
problems, and the only difference is that Jmin in- next Eq.(18):
creases negative values along the tour construction.
Informally, ants search the optimal tempera-
i
ture profile as follows: m ants are initially posi-
tioned on (t(0), T(0)): it represents coordinate in
construction graph, in which X axis represents time
and Y axis represents temperature. Here, t(0) is the
beginning of the first time interval, and next time
q(r, u) =

f--
1
- minj<~...,16] (J + 1)

where J= JmJ. (i + 1, j ) - Jmi. [i, T(i)].

if minj~[0,t ...,16]( J r )

otherwise
~-~ 0;

(18)
interval begins at t(1). That means there are 161 X
fl is a parameter which weighs the relative
axis coordinates: t(i), i=0,1 ..... 160. Using 10 °C
importance of pheromone versus heuristic value, q
(T(0) set as this value) as the beginning of the so-
is a value chosen randomly with uniform probabil-
lution is recommended by the industrial tempera-
ity in [0,1], qo (0-<q0<-l) is a parameter, and S is a
ture profile shown in Fig. 1. When the program goes
random variable selected according to the distri-
into the inner loop, each ant begins to build a tour (a
bution given by Eq.(19), which gives the probabil-
feasible temperature profile). Take an ant k
ity with which an ant in node r choose the node s to
(k=-l,2 .... ,m) for example, if it is now in node r:
(t(i),T(i)), i=0,1 .... ,160, it should choose the next move to.
node s: (t(i+l), T(i+I)) to move to. There are 17
choices available: (t(i+l), 0), (t(i+l), 1), (t(i+l), [r(r,s)][rl(r,s)] p (19)
2) ..... (t(i+l), 16). N(r) is the set consisting of these p(r,s)- ~, [r(r,u)][rl(r,u)]p
uEN(r)
choices. Jmin(i+l, j ) , j=0,1 ..... 16, can be easily
calculated. Each of them corresponds to a choice of
node (t(i+l), j). Then the state transition rule is While building a solution, ants visit nodes and
applied: change their amount of pheromone trail by applying
the following local updating rule:

s:~argmaxu~N(r){r(r,u)[rl(r,u)]P} if q<qo (17) z(r, s)+--(l-p)~(r, s)+pAz(r, s) (20)

Ls otherwise
where p (O<p<l) is the pheromone decay parameter.
where z(r, u) is the amount of pheromone trail on A~(r, s)--r0 is the initial pheromone level.
edge. There are 160 time intervals, and in each of it, After all ants had completed their tours, global
a matrix (17× 17) is used to record the intensity of updating is performed. The pheromone amount is

16 lO00

G" 12
L,
14

10
/ i
--~
900

8O0

700
a) 8
Y6 6OO

4 5OO
2 I I I
50 100 150 200 4000 210 4'0 60 80 100 120 1'~0 160
Time (h) Time (h)
Fig.1 Industrial temperature profile Fig.2 Objective function value along the process
 Xiao et al. / J Zhejiang Univ SCI 2004 5(12).'1597-1603 1601

updated by applying the follow global updating rule. ter point of every time range (40 hours). From 0 to
To simplify the problem, global updating rule only 20 hours (expressed as [0,20]), the temperature (T)
apply to those tours that once belonged to the is the first mean value (expressed as meanl);
global-best-tour: [20,40], T changes from mean 1 to mean2 (straight
line links this two values); [40,60], T=mean2;
r(r, s)+--(1-a)z(r, s)-t-og(Jmin gb)-1 [60,80], T changes from mean2 to mean2+(mean3-
if (r, s) ~ global-best-tour (21) mean2)~2; [80,100], T changes from mean2+
(mean3-mean2)/2 to mean3; [100,120], T=mean3;
0<ct<l is the pheromone decay parameter, and [120,140], T changes from mean3 to mean4;
Jmin gb is the Jminvalue of the global best temperature [140,160], T=mean4. Fig.4 shows the final tem-
profile from the beginning of the trail. perature profile, which is suitable for implementa-
A version of ant colony system algorithm for tion, obtained after smoothing process.
this case is rapidly developed by MATLAB lan- Using this smoothed temperature profile, the
guage. behavior of the kinetic variables of the fermentation
Initial tests were done with recommended process model can be shown in Figs.5, 6, 7 and 8.
parameter settings (Dorigo and Gambardella, 1997): Fig.5 illustrates time courses of the total suspended
m=10; fl=2; q0=0.9; a=p=0.1; CONS=IO00; biomass and its components: active, latent, and
r0=( 161 × (1000-541.5504))-1; iteration number=5. dead cells. Fig.6 is the sugar concentration curve,
After several runs, the parameter fl changes to 4; q0 Fig.7 is the ethanol concentration curve and Fig.8
changes to 0; iteration number changes to 10. With illustrates diacetyl concentration.
these parameters, a promising temperature profile Several runs have been made changing the
has been achieved. initial parameters related to the ACS algorithm. It is
not recommended to use a large number of itera-
tions to get "better results", because of the fol-
RESULTS AND DISCUSSION lowing reasons: (1) After 20 iterations, Jmin s e e m s
to be constant in the experiment; (2) More iterations
The temperature profile obtained using ACS means much more time to get satisfactory result; (3)
algorithm for the selected parameters in this paper Most importantly, the trend of the industrial tem-
is shown in Fig.3. This jagged profile calls for a perature profile together with Increase Once Rule
smoothing process. This process can be done by will keep some results out of feasible solutions. So
means of average calculation for every 40 hours of it is better to have more runs with moderate itera-
the obtained temperature profile. Four new tem- tions in each run than the contrary approach. Ac-
perature values are obtained and placed in the cen- tually, in these r u n s Jmin are always satisfactory, but

16 145

15 14

14 13.5
? 6"
13
13
12
o
v 12.5
11
1o Y. 12

9 11.5

8 11
0 160 1;o 2'o 4'0 6'0 8; 1;0 140 160
Time (h) Time (h)

Fig.3 Optimal temperature profile obtained by ACS Fig.4 Smoothed temperature profile
1602 Xiao et al. / J Zhejiang Univ SC1 2004 5(12):1597-1603

sometimes results are infeasible solutions. After guage, ACS algorithm is first applied to the beer
reviewing all the results, the best temperature pro- fermentation process optimization problem. The
file with minimum Jmin was selected and smoothed promising results o f this work show that the ACS
for feasible implementation. This new temperature approach can compete with the original GA ap-
profile is obtained not only by minimizing the value proach (Andres-Toro e t a l . , 1997a; 1997b;
o f ffmin, which ensures getting the required ethanol Carrillo-Ureta e t al., 2001) in terms o f solution
level without sacrificing the quality o f beer, but quality and computation speed in this problem and
also by taking application factors into considera- indicate the potential o f ACS as a reliable and
tion. useful tool for solving dynamic process optimiza-
Table 1 shows the results and comparison with tion problems.
previous results found in the literature. Further work to be done includes: using ACS
algorithm in continuous spaces without discretiza-
tion approximation; getting a smooth profile
CONCLUSION without an additional smoothing process; and con-
sidering other industrial implementation aspects:
In this paper, with the help o f M A T L A B lan- economic factors, energy management, etc.

9 t i i i i i i 140 i i i i i i i

8 ,_., 120

t-
100
6
~
O

80
o
5
4 ~ 60
3 8

m 20
1
' --i:" ...... l .... I I I I ~ ~1-'-- - 0 I

20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160

Time(h) Time (h)
Fig.5 Suspended biomass behavior Fig.6 Time course of sugar concentration

60 i i i i i i i 1.6 i i i ~ i i i

1.4
50
¢-
1.2
.2 40 1
x

~ 30 0.8
8 .~ 0.6
- 20o ¢2
e-
0.4
N 10 0.2
I I 0
20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160
Time(h) Time (h)
Fig.7 Time course of ethanol concentration Fig.8 Time course of diacetyl concentration

Table 1 Comparison table

Temperature profile Temperature profile
Industrial temperature profile obtained by GA obtained by ACS
Jmax 546.3 585.5214 585.5865
Computation effort (s) NA 30.87
 Xiao et al. /JZhejiang Univ SCI 2004 5(12):1597-1603 1603

References
Andres-Toro, B., Giron-Sierra, J.M., Lopez-Orozco, J.A., Carrillo-Ureta, G.E., Roberts, P.D., Becerra, V.M., 2001.
Femandez-Conde, C., 1997a. Application of Genetic Genetic Algorithms for Optimal Control of Beer Fer-
Algorithms and Simulations for the Optimization of mentation. Proceedings of the 2001 IEEE International
Batch Fermentation Control. Proc. IEEE International Symposium on Intelligent Control, Mexico City,
Conference on System, Man, and Cybernetics, 'Compu- Mexico, p.391-396.
tational Cybernetics and Simulation', p.392-397. Dorigo, M., Gambardella, L.M., 1997. Ant colony system: a
Andres-Toro, B., Giron-Sierra, J.M., Lopez-Orozco, J.A., cooperative learning approach to the traveling sales-
Fernandez-Conde, C., 1997b. Using Genetic Algo- man problem. 1EEE Transactions on Evolutionary
rithms for Dynamic Optimization: An Industrial Fer- Computation, 1(1):53-66.
mentation Case. Proceedings of the 36th Conference Dorigo, M., Maniezzo, V., Colorni, A., 1996. Ant system:
on Decision & Control. San Diego, California, optimization by a colony of cooperating agents. IEEE
USA,p.828-829. Transactions on Systems, Man, and Cybernetics-Part
Andres-Toro, B., Giron-Sierra, J.M., Lopez-Orozco, J.A., B: Cybernetics, 26(1):29-41.
Fernandez-Conde, C., Peinado, J.M., Garcia-Ochoa, F., Dorigo, M., Gambardella, L.M., Middendorf, M., Stutzle,
1998. A kinetic model for beer production under in- T., 2002. Guest editorial special section on ant colony
dustrial operational conditions. Mathematics and optimization. IEEE Transactions on Evolutionary
Computers in Simulation, 48:65-74. Computation, 6(4):317-319.

Append~ A

Table 2 Nomenclature used

Parameter Description Unit Parameter Description Unit
Xact Suspended active biomass g/L ,L/lag Specific rate of activation h -~
Xhg Suspended lag biomass g/L Specific yeast growth rate h -l
Xsd Suspended dead biomass g/L /-J-dt Inhibition rate of yeast growth h -~
Xinc(0) Inoculum value: 4 g/L /A Substrate consumption rate h -~
Co(t) Ethanol concentration g/L ks Sugar inhibition parameter g/L
Cs(t) Sugar concentration g/L f Fermentation inhibitor factor
Cca(t) Ethyl acetate concentration 10-6 /A Ethanol production rate h -1
Cdy(t) Diacetyl concentration 10-6 Yoas Ethyl acetate coefficient rate
Cso(t) Initial sugar concentration value: 130 g/L ,//-dy Diacetyl appearance rate
t Time H /Ab Diacetyl reduction rate
T Temperature °C /Ad Specific yeast settling down rate h -~

Appendix B

ACS algorithm:
Loop /* at this level each loop is called an iteration*/
Each ant is positioned on a starting node
Loop /* at this level each loop is called a step*/
Each ant applies a state transition rule to incrementally build a solution
and a local pheromone updating rule
Until all ants have built a complete solution
A global pheromone updating rule is applied
Until End condition