Expert Systems With Applications 122 (2019) 118–136

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Comparison of new metaheuristics, for the solution of an integrated

jobs-maintenance scheduling problem
Massimo Bertolini, Davide Mezzogori∗, Francesco Zammori
Department of Engineering and Architecture, University of Parma, Parco Aree delle Scienze, 181/A, 43124, Parma, Italy

a r t i c l e i n f o a b s t r a c t

Article history: This paper presents and compares new metaheuristics to solve an integrated jobs-maintenance schedul-
Received 17 July 2018 ing problem, on a single machine subjected to aging and failures. The problem, introduced by Zammori
Revised 8 November 2018
et al. (2014), was originally solved using the Modified Harmony Search (MHS) metaheuristic. However, an
Accepted 18 December 2018
extensive numerical analysis brought to light some structural limits of the MHS, as the analysis revealed
Available online 20 December 2018
that the MHS is outperformed by the simpler Simulated Annealing by Ishibuchi et al. (1995). Aiming to
Keywords: solve the problem in a more effective way, we integrated the MHS with local minima escaping proce-
Cuckoo search dures and we also developed a new Cuckoo Search metaheuristic, based on an innovative Lévy Flight. A
Harmony search thorough comparison confirmed the superiority of the newly developed Cuckoo Search, which is capable
Metaheuristics to find better solutions in a smaller amount of time. This an important result, both for academics and
Scheduling practitioners, since the integrated job-maintenance scheduling problem has a high operational relevance,
Single machine
but it is known to be extremely hard to be solved, especially in a reasonable amount of time. Also, the
Planned maintenance
developed Cuckoo Search has been designed in an extremely flexible way and it can be easily readapted
and applied to a wide range of combinatorial problems.
© 2018 Elsevier Ltd. All rights reserved.

1. Introduction of fact, scheduling models proposed in technical literature are

becoming more and more complex, since current research’s trends
Scheduling is an important tool for manufacturers and produc- are toward integrating new and practical constraints in traditional
tion engineers, as it deals with the way in which human and ma- scheduling models (El-Khoukhi, Boukachour, & Alaoui, 2017). In
chinery resources should be allocated to meet customers’ demand. this regard, an issue of recent interest concerns the possibility to
Specifically, given a set of production orders (or jobs), scheduling increase machines’ availability and to reduce the overall production
allows one to decide when and in which order jobs should be costs, by including Planned Maintenance tasks (PMs) into the jobs’
processed, with which resources and on which equipment. The schedule (Salmasnia and Mirabadi-Dastjerd (2017)). Traditionally,
objective is to control and to optimize workloads, to minimize in fact, machines and equipment have been considered as fault
production time and costs, respecting all production constraints free and the focus was limited to the way in which jobs should be
(such as jobs’ routing and jobs’ releasing time) and, possibly, deliv- sequenced and allotted to the available resources. Yet, this assump-
ery due dates. Therefore, an optimized schedule may have a major tion is rather heroic because machines can fail and, due to failures
impact on the performance of a manufacturing system and can and reactive maintenance tasks (needed to restore the operating
be source of competitive advantage, as it allows reducing work- state of the fault machines) processing and waiting times gets
in-process, queuing times and industrial costs. It is worth noting longer, routings could be changed accordingly and, consequently,
that scheduling problems have always been of great interest also the original schedule may become non-optimal and should be
for the academia because, being very hard to be solved, they offer re-evaluated. Also, and perhaps more important, since breakdowns
a challenging test bench to assess performance of sophisticated reduce productivity and increase costs, PMs should also be sched-
algorithms and/or metaheuristics (Allahverdi, 2015). As a matter uled, as additional tasks, as an effective way to improve machines’
availability (Safari & Sadjadi, 2011). The opportunity to define the
scheduling sequence, taking into account possible failures and
∗
Corresponding author. the need to perform PMs is becoming even more engaging, now
E-mail addresses: [email protected] (M. Bertolini), that we are entering in the so-called forth industrial revolution
[email protected] (D. Mezzogori), (Macchi, Roda, & Fumagalli, 2017). Indeed, due to the recent
[email protected] (F. Zammori).

https://doi.org/10.1016/j.eswa.2018.12.034
0957-4174/© 2018 Elsevier Ltd. All rights reserved.
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 119

advances in manufacturing industry, which is leading toward astonishing as the SA outperformed the MHS, in any operating
the systematical deployment of Cyber-Physical Systems, it is condition.
now possible not only to remotely monitoring the system’s Due to the above-mentioned issues, the speculative and oper-
state and to take actions based on real time data, but also ational goal of this paper is to ascertain if and how this specific
to synchronize the physical factory floor and the cyber com- instance of the IJMS problem could be optimally solved also by a
putational space (Lee, Bagheri, and Kao (2015)), where opti- population-based metaheuristic. To this aim: (i) we modified the
mization algorithms can be effectively used to dynamically MHS, by introducing new local minima escaping techniques and
solve the scheduling problem, in an acceptable amount of (ii) we developed a new population-based metaheuristic that im-
time. plements a searching strategy like the one of the Cuckoo Search
Among the papers dealing with the Integrated Job Mainte- (CS) by Yang and Deb (2009). Specifically, the CS has been cho-
nance Scheduling problem (IJMS), this work focuses on the one sen because, with respect to other population-based heuristics, it
by Zammori, Braglia, and Castellano (2014), who faced the prob- is less prone to early convergence, as cross-influence among saved
lem of scheduling n jobs on a single machine subjected to ag- solutions plays a minor role in the evolutionary process.
ing and failures and proposed a Modified-Harmony-Search (MHS) The rest of the paper is organized as follows. Literature review
metaheuristic to generate a quasi-optimal solution, in terms of on the IJMS problem is given in Section 2, and the specific sin-
earliness-tardiness penalties. gle machine IJMS problem considered in this paper is detailed in
The rational aspect of this choice is twofold: Section 3. New searching strategies and metaheuristics are intro-
duced in Section 4 and 5 and their parameters are fine-tuned in
• The underlying hypotheses of the model are very pertinent
Section 6. Furthermore, in Section 6 a comprehensive experimen-
with the industrial reality, as the single machine is character-
tal campaign is made to assess and compare performance of the
ized by a time dependent (and increasing) failure rate and both
proposed metaheuristics. Lastly, conclusions and directions for fu-
reactive and planned maintenance tasks are executed to restore
ture researches are drawn in Section 7.
the functionality of the system.
• The model has a high practical relevance: although the single
2. Literature review on the IJMS problem
machine is the simplest form of shop floor configuration, it is
also the one with the largest number of practical applications
Scheduling problems have been generally treated under the as-
(Quan & Xu, 2013).
sumption of full machines’ availability. Only in the last decade this
Concerning the last point, we note that the single machine simplifying hypothesis has been questioned by several researchers,
can be used to adequately represent traditional continuous process who postulated that, due to failures, the overall productivity of the
plants (such as refining), but it can be easily readapted to model manufacturing system may benefit from the inclusion of PMs be-
batch processes such as those of bakeries, biotechnology, chemical tween jobs. However, and quite surprisingly, almost the entirety
and pharmaceutical firms. Not only these production systems are of papers that start from this hypothesis do not fully consider
very common, but it is exactly in these production environments the effect that PMs may have on machines’ availability. Indeed,
that scheduling decisions are extremely important (to boost pro- most of the researchers implicitly assume that, if properly per-
ductivity) and where failures may have a dramatic impact, as they formed, PMs will ensure sufficient machine availability to com-
could interrupt the whole manufacturing process. Additionally, the plete productions’ orders in the allotted time. Basically, the prob-
single machine can also be used to schedule production in a pure lem is simplified as it follows: jobs’ characteristics (in terms of
flow shop or even in a job shop system, provided that the single processing times, set up times, release dates, routings, etc.) and
machine is used to represent the real bottleneck of the manufac- the optimal number (or alternatively the optimal frequency) of the
turing system. required PMs are considered as known inputs, and the schedul-
We also note that our interest (in this model) is mostly op- ing problem reduces to the insertion of jobs between two con-
erational and computational and the aim is to introduce alterna- secutive PMs, in a way that optimizes a certain objective (such
tive solution approaches, more performing than the original MHS. as the minimization of makespan or of the maximal tardiness),
The need to find better solutions was urged by the results of the without violating (at least within a certain tolerance limit) the ex-
analysis made by Zammori et al. (2014), who comparted the Mod- pected PMs frequency. Yet, the possibility that faults may still oc-
ified Harmony Search (MHS) to the Hybrid Kangaroo Simulated cur is neglected, as well as the fact that failures’ probability is
Annealing by Soltani, Jolai, and Zandieh (2010), a metaheuristic strictly related to the time in which planned maintenance was
known to be particularly effective with NP-hard scheduling prob- performed.
lems. Although MHS resulted as the best approach, the gap was Among the most recent papers belonging to this category, the
not as wide as one could expect, when comparing a sophisti- following ones can be mentioned. Mosheiov and Sarig (2009) ex-
cated algorithm as the MHS with a simpler one. To shed light on tended the well-known scheduling problem of the “single machine
this unexpected fact, we repeated the experimental campaign on with due-windows assignment”, by including in the schedule one
a wider problem landscape and we observed that the solutions’ additional maintenance activity. This maintenance activity is op-
space is rather flat (i.e., the average gradient is low), but it is dis- tional and, to be performed, it requires a fixed amount of time,
seminated of many local minima, with values close to the global during which production must be halted. Although this mainte-
one. As noted by Yang (2010), in these peculiar conditions per- nance task implies a waste of processing time, it may be bene-
formances of the Harmony Search (HS) get worse and the prob- ficial because, once it has been performed, the single machine is
ability to get stuck, almost immediately, in a local minimum be- supposed to become more efficient, and the successive workings
comes high. Conversely, when the solution space is flat, only a will require a lower processing time. The authors also presented
search strategy assuring a high degree of randomness and little a polynomial time algorithm that generates the optimal schedule
to none influence to the initial starting point, may find the global that minimizes costs related to jobs’ earliness and tardiness and to
optimum. To verify this assumption, we solved all the original the size and starting time of the due-windows.
problems’ set using the very famous and general-purpose Simu- Low, Hsu, and Su (2010) considered the problem of scheduling
lated Annealing (SA) by Ishibuchi, Misaki, and Tanaka (1995) that, jobs on a single machine subject to periodic maintenance activi-
due to a relatively simple iterated greedy strategy, fully satisfies ties. In this case, both the planned start time and the duration of
the above-mentioned requirements. Obtained results were quite each maintenance activity are known in advance, and the problems
120 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

is to insert jobs between consecutive PMs, trying to minimize the (that utilizes several dominance rules) are proposed for large and
makespan. To this aim the authors proposed an innovative version medium sized problems, respectively.
of the Particle Swarm algorithm that makes use of: (i) a “job-to- Yazdani, Khalili, Babagolzadeh, and Jolai (2017) considered a
position” representation for the particles and (ii) the largest posi- single machine scheduling problem characterized by multiple un-
tional value rule to generate a particle sequence. availability periods, occurring at known and constant intervals,
Zhu, Li, and Zhou (2015) also considered the single machine during which overhauls are performed. Aiming to minimize the
scheduling problem, but the peculiarity is that they introduced sum of earliness-tardiness penalties, the authors proposed an al-
a deteriorating and resource-dependent “rate-modifying” planned gorithm characterized by a variable neighbourhood search that,
maintenance. As usual, the hypothesis made is that planned main- in addition to local search, also implements a “shake” proce-
tenance alters (i.e., increases) the productivity of the machine. dure, which diversifies solutions, by performing a leap toward an-
However, on the other one hand, planned maintenance is con- other local neighbourhood. The algorithm is also integrated with
sidered “deteriorating”, as its duration depends both on the re- a “knowledge module” that keeps track of the structure of good
sources allocated to it and on its starting time, i.e., the lateer it solutions and that takes advantage of it during the search.
is performed, the worse the conditions of the machine become, El-Khoukhi et al. (2017) investigated the flexible job shop
and the longer the maintenance duration becomes. The authors scheduling problem, when a predefined number of PMs, of known
also demonstrated that this problem can be optimally solved in and deterministic length, are needed. To minimize makespan, they
a polynomial time with respect to many objective functions (i.e., solved the problem using an innovative Dual-Ants Colony Optimi-
makespan, flowtime, maximum tardiness and combination of ear- sation. The metaheuristic is also integrated with a dynamic mem-
liness, tardiness and due-date). ory, which takes advantage of local search strategies and several
A similar problem was faced by Luo, Cheng, and Ji (2015), who dispatching rules.
assumed that PM must start before a given deadline and that its Woo and Kim (2018) considered the parallel machines schedul-
duration increases with its starting time. The authors also provided ing problem characterized by “rate-modifying” PMs activities.
polynomial-time algorithms to solve the problem with respect to Specifically, each machine is characterized by a starting processing
the same objective functions used by Zhu, Li, and Zhou (2015). time that increases through time, accordingly to a certain deterio-
Similarly, Mor and Mosheiov (2015) studied the single ma- ration rate. However, if PMs is performed on a machine, the pro-
chine due-windows assignment problem, in case of a deteri- cessing time is restored to its original value. Given a certain num-
orating maintenance task. Specifically, two different types of ber of time slots where PMs can be performed and considering as
deterioration were considered: time-dependent (where the main- known and deterministic the PMs’ time, the main decisions to be
tenance time increases as a function of its starting time), and taken concern: (i) the number and position of PMs and (ii) the or-
position-dependent (where it is a function of its position in the dered assignment of jobs on parallel machines. Specifically, to min-
sequence). In both scenarios a polynomial-time O(n4 ) optimi- imize the makespan, the authors proposed a specific heuristic that
sation algorithm is proposed to minimize costs related to jobs’ takes advantage of Simulated Annealing and of Genetic Algorithm
earliness and tardiness and to the size and starting time of the to decompose the problems into sub-problems than can be analyt-
due-windows. ically and optimally solved.
Yu and Seif (2016) considered a flow shop scheduling problem, As we have already said, all these papers consider PMs as a
with the peculiarity that the operating conditions of each machine prerequisite needed to assure a minimum level of availability, as
is expressed by a set of different Maintenance Levels (MLs). When required by the manufacturing system. Therefore, the number and
a job is processed on a machine, each MLs is decreased and, as the duration of the PMs are assumed to be known and the posi-
soon as a ML drops to zero, PM is performed. With the objective tive effect that PMs may have on the hazard rate of the machines
of minimizing tardiness and maintenance costs, the problem be- is disregarded. Very few papers reduced these drawbacks, by con-
comes NP-hard and it is solved with a lower-bound based genetic sidering a time-dependent hazard rate λ(t) and by including in
algorithm. the analysis also the additional losses that may be generated by
Lee and Wang (2017) considered a single machine that pro- faults and by reactive maintenance tasks. As far as the knowledge
cesses two types of jobs, and the objective is to minimize the of the authors, in addition to the work by Zammori et al. (2014),
makespan of Type #1 jobs, provided that the maximum tardiness which will be discussed in depth in Section 3, this approach
of Type #2 jobs does not violate a predefined threshold limit. Ad- was followed only by eight recent papers that share the fol-
ditionally, PMs must be completed within specific time windows. lowing common hypotheses: (i) Machines’ deterioration increases
Also, in this case, the number of PMs and their (deterministic) through time and it is described by a time dependent hazard
duration are known in advance and the problem is solved using rate, (ii) when performed, planned maintenance restores the orig-
branch and bound (based on a set of dominance rules) and via ge- inal operating conditions of the machine (i.e., perfect repair) or,
netic algorithms. at least, it reduces the current degradation state, (iii) although
Similarly, also Touat, Bouzidi-Hassini, Benbouzid-Sitayeb, and PMs are performed, failures may still occur. In this case reactive
Benhamou (2017) considered the problem of scheduling jobs on maintenance tasks are needed to restore the machines, nonethe-
a single machine that requires PMs. In this case the specificity is less their degradation remains unchanged (i.e., minimal repair
that PMs must be performed inside predefined time windows, un- model).
der human resources competences and availability constraints. To Safari and Sadjadi (2011) are, probably, the first authors that
minimize the total tardiness, the authors proposed two fuzzy ge- fully considered the effects of PMs on the hazard rate. Specifi-
netic algorithms, differing in the way in which jobs, PMs and hu- cally, the authors deal with a flow-shop configuration under the
man resources are scheduled. assumption of condition-based maintenance to minimize expected
Cui and Lu (2017) considered the single machine scheduling makespan. More precisely, every T time units, each machine is in-
problem with jobs’ release dates. The single machine requires flex- spected and, depending on its current degradation (modelled with
ible PMs that are activated following a “run-based” policy, i.e., a a Poisson distribution) a reparation may be performed. Also, fail-
maintenance task must be performed any time the machine work- ures can occur in between two consecutive inspections, according
ing age exceeds a predefined threshold value. To solve the prob- to a non-homogeneous Poisson process of parameter λ(t). A hy-
lem minimizing the makespan, a performing heuristic (based on brid heuristic based on genetic algorithm and simulated annealing
some properties of the optimal solution) and a branch and bound is also proposed to minimize makespan.
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 121

Cui, Lu, and Pan (2014) tackles the problem of scheduling jobs machines’ assignments that minimize the makespan; next the op-
and PMs on a single machine subject to unexpected failures. The timal time between PMs is defined via simulation.
objective function is evaluated both in terms of solution robust- Cui, Lu, Li, and Han (2018) dealt with the integration of pro-
ness and of solution quality. Specifically, the first metric is com- duction scheduling and maintenance planning to optimize robust-
puted as the total deviation between the planned start time and ness and solution quality (evaluated as in Cui et al. (2014)), for a
the actual start time of the jobs, whereas the second metric is flow shop with failure uncertainty. As in Lu et al. (2015), buffer
measured as the maximum tardiness of the jobs. Also, to gener- times are proactively added into the initial schedule and an initial
ate optimal solutions, a three-phased heuristic is proposed. At first solution is obtained following a constructive mathematical model.
the jobs’ sequence and the relative position of PMs is determined; Next, the solution is improved using a two loops algorithm that
next some extra idle time (or “time buffer” ) is included in the takes advantage of numerical simulation and Monte Carlo sampling
schedule to protect due dates from unpredictable failures. Lastly, methods. More precisely, the outer loop performs a local search
in the third and final phase, a pair-wise-swap-based local search to optimize jobs’ sequence and PMs’ positions; the inner loop im-
is performed on the jobs’ sequence, aiming to improve the initial plements a genetic algorithm that tries to optimize the idle times
solution. when jobs’ sequence and PMs’ positions are fixed. Results of the
Lu, Cui, and Han (2015) tackled the problem of finding a robust inner loop are returned as an evaluation criterion to guide the
and stable schedule for a single machine that suffers of unexpected search direction of the outer loop.
breakdowns. According to the authors, the best way to maximise For the sake of clarity, results of the literature review on IJMS
robustness is to: (i) add time buffers to protect the initial sched- problems published starting from 2014 are summarized in Table 1.
ule and (ii) execute PMs anytime machine’s reliability drops below Although the considered models are case specific and cannot be
a threshold limit. Owing to these issues, the authors solved the directly compared, we tried to summarize, in the table, pros and
problem with a genetic algorithm based on a double chromosome: cons of each one of them.
the first chromosome codifies (with natural number) the jobs’ se-
quence, the second one codifies (in binary code) the desired relia-
bility limit.
Mokhtari and Dadgar (2015) considered a flexible flow-shop 3. The Single Machine Integrated Job Maintenance Problem
scheduling problem, in which machines’ failure rates are time-
varying and proposed a heuristic, based on Simulated Annealing In this paper, we focus on the single machine IJMS problem in-
and on Monte Carlo simulation, to minimize the number of tardy troduced by Zammori et al. (2014), whose features are listed be-
jobs. The job shop is considered flexible because any operation can low:
be performed by more than one machine and failure rate is consid-
ered time-varying as it depends on environmental situations like - The single machine must process a certain number of jobs, all
temperature, light, humidity, etc. Quite interestingly, this model available at the starting time t0 ;
is based on the hypothesis that the time needed to complete a - For each job, its processing time, due date and sequence depen-
batch of orders is much smaller than the average time between dent setup time are known;
two planned maintenance tasks. Consequently, the use of PMs is - The machine is subjected to failures and production may be
not contemplated and, indeed, only reactive maintenance tasks are halted both to perform reactive and planned maintenance
executed anytime a failure occurs. tasks;
Nouiri et al. (2017) focused on the flexible job shop scheduling - A time dependent hazard rate λ(t) is considered and mini-
problems under machines breakdowns. Makespan minimization is mal and perfect repair models are used to describe machine’s
considered as the main objective, but, as secondary objective, the restoration after the execution of a reactive and of a planned
obtained solution must guarantee a stable schedule even in case of maintenance task, respectively;
breakdowns. To this aim a two-stages particle swarm optimization - The goal is to find the integrated sequence of jobs and planned
was proposed: at first breakdowns are disregarded and an initial maintenance tasks that minimizes the sum of Earliness-
sequence is found, next breakdowns are randomly generated (sup- Tardiness Penalties (ETP).
posing that machines with the heavier workload are more likely to
fail) and the original solution is modified to assure robustness and In short, using the Kendall’s notation, the problem can be classi-
stability. As in the work by Mokhtari and Dadgar (2015), also in fied as: (1| Due Dates, Sequence Dependent Set-Up Times, Planned
this case PMs are not considered, and only reactive maintenance is Maintenance| Earliness-Tardiness Penalties).
used to counteract machines’ failures.
Salmasnia and Mirabadi-Dastjerd (2017) considered the
scheduling problem of a degraded single machine subjected
to failures. Specifically, they considered different levels of PMs: 3.1. Computing the expected completion time of the jobs
although planned maintenance always leads to machine’s reju-
venation, only if the top and most expensive kind of planned Let N and M be, respectively, the number of jobs and of PMs.
maintenance is performed the machine is restored to its original Then, any possible solution can be expressed as a combination of
state (i.e., perfect repair). The objective is to jointly minimize the two vector π and μ. The vector π = (π 1 , π 2 ,…, π i , …, π N ) rep-
total maintenance cost and the total jobs completion time. Since resents one of the N! possible permutations of N jobs and each
there is a clear trade-off between these objectives, the Pareto Set one of its element π i ∈ {1, 2, …, N} codifies the job that will
of the non-dominated solutions was generated using a genetic be processed in position i. Similarly, μ = (μ1 , μ2 ,…, μj , …, μM )
algorithm. N−1
represents one of the ( ) possible ways in which PMs can
Rahmati, Ahmadi, and Govindan (2017) extended the flexible M
job shop scheduling problem presented by Yu and Seif (2016), by be inserted between the first and the last job; in this case, each
considering machines characterized by an increasing hazard rate. element of μj codifies the index of the job (within π ) that pre-
Briefly, to schedule jobs and PMs, a simulation-based optimization cedes the j-th PM. For instance, with N = 6 and M = 2, the schedule
similar to the one by Nouiri et al. (2017) was proposed. At first, {j1 ,j3 ,PM, j5 ,j4 ,PM, j2 ,j6 } would be denoted as: π = (1, 3, 5, 4, 2, 6),
Harmony Search is used to find both the jobs’ sequence and the μ = (2, 4).
 122
Table 1
Main papers dealing with the IJMS problem from 2014 to now.

Paper System Features of PMs Objective Function Solution Approach Pros Cons

Cui et al. (2014) Single Machine PMs executed at variable time Schedule Robustness Three-phased heuristic Hazard rate λ(t) is considered. A single level of PM (i.e., perfect
intervals to increase machine’s and Quality Robustness is obtained using time repair) is considered.
availability. buffers and the number of PMs is
analytically optimized.
Single Machine. PMs executed at variable time Earliness and Tardiness Modified Harmony Hazard rate λ(t) is considered. A single level of PM (i.e., perfect

M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136
Zammori et al. (2014) intervals to increase machine’s Penalties. Search. Failures and reactive maintenance repair) is considered.
availability. are considered. The number of PMs Solution approach is not very
is analytically optimized. performing.
Luo et al. (2015) Single Machine PM is used to improve Makespan; Other Optimal Polynomial A deteriorating rate, affecting PM’s Hazard rate λ(t) is not considered;
machine’s productivity; PM is classic objectives. Time Algorithm. time is considered; Faults are not considered;
“deteriorating”, as is duration Optimal solution can be obtained A single maintenance task must be
increases over time. in polynomial time included in the schedule.
Lu et al. (2015) Single Machine PM is executed any time λ(t) Robust Schedule Genetic Algorithm with Hazard rate λ(t) is considered; A single level of PM (i.e., perfect
drops below a threshold limit, Stable Schedule double chromosome Failures are considered; repair) is considered.
to restore its initial state. The optimal threshold limit for λ(t)
is optimized.
Mokhtari and Flexible Flow Machines are characterized by Number of tardy jobs Simulated Annealing Time varying hazard rate Only faults and reactive
Dadgar (2015) Shop a time-varying failure rate. and Monte Carlo depending on environmental maintenance tasks are considered.
When faults occur, simulation conditions. PMs are not considered as a way to
maintenance is needed to improve availability.
reactivate the system.
Mor and Single Machine PM is used to improve Costs minimization. Optimal Polynomial Two kinds of deteriorating rates Hazard rate λ(t) is not considered;
Mosheiov (2015) machine’s productivity; Time Algorithm. are considered, and several costs Faults are not considered;
PM is “deteriorating”, i.e. are considered. At most one maintenance task can
duration is time and position Optimal solution can be obtained be included in the schedule.
dependent. in polynomial time.
Zhu, Li, and Zhou Single Machine PM is used to improve Makespan; Optimal Polynomial A deteriorating rate, affecting PM’s Hazard rate λ(t) is not considered;
(2015) machine’s productivity; Other classic objectives. Time Algorithm. time, is considered; Faults are not considered;
PM is “deteriorating”, as is Optimal solution can be obtained At most one maintenance task can
duration increases over time. in polynomial time. be included in the schedule.
Yu and Seif (2016) Flow Shop Machines degrade over time Tardiness and Genetic Algorithm Different Maintenance Levels can Hazard rate λ(t) is not considered
and their productivity decrease. Maintenance costs be used. The number and the kind (i.e., it is substituted by a
PMs are needed as soon as the of PMs to be executed is an output degradation rate);
Maintenance Level drop to of the genetic algorithm. Faults are not considered;
zero.
Cui and Lu (2017) Single Machine. The machine degrades and its Makespan. Specific Heuristic; Introduces a “degradation” concept. Hazard rate λ(t) is not considered
productivity decreases. PMs are Branch and Bound PMs executed accordingly to a (i.e., it is substituted by a
executed, anytime the procedure. run-based policy to reduce degradation rate);
degradation exceeds a degradation. Faults are not considered;
threshold.
El- Flexible Flow PMs are needed to assure the Makespan Dual Ant Colony with The model is standard, the Hazard rate λ(t) is not considered;
Khoukhi et al. (2017) Shop. required availability and do not dynamic memory innovative part lies in the solution Faults are not considered;
alter machine productivity. approach that is very performing. Number and duration of PMs are
known in advance.
(continued on next page)
Table 1 (continued)

Paper System Features of PMs Objective Function Solution Approach Pros Cons

Lee and Single Machine. PMs are needed to assure the Makespan; Genetic Algorithm. Number and duration of PMs are Hazard rate λ(t) is not considered;
Wang (2017) required availability and must Tardiness. known, but their starting time can Faults are not considered;
be performed within vary. Jobs are classified into Number and duration of PMs are

M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136
predefined time windows. families with different importance. known in advance.

Nouiri et al. (2017) Flexible Job Fault probability depends on Makespan Optimization via Failure rate is time varying and it Only faults and reactive
Shop. the work load; anytime a fault simulation also depends on work load. maintenance tasks are considered.
occurs, reactive maintenance is PMs are not considered as a way to
needed. improve availability.
The simulation model is case
specific
Flexible Job PMs executed at variable time Makespan. Harmony Search and Hazard rate λ(t) is considered. A single level of PM (i.e., perfect
Rahmati et al. (2017) Shop. intervals to increase machine’s Simulation Failures and reactive maintenance repair) is considered.
availability. are considered. The number of PMs Optimization is obtained via
is analytically optimized. simulation and it is case specific.
Touat et al. (2017) Single Machine. PMs are needed to assure the Total Tardiness. Fuzzy Genetic PMs must be placed within Hazard rate λ(t) is not considered;
required availability and must Algorithms. predefined time windows, that also Faults are not considered;
be performed within depends on human resources Number and duration of PMs are
predefined time windows. competences and availability known in advance.
constraints.
Salmasnia and Single Machine PMs executed at variable time Total Cost Genetic Algorithm with Hazard rate λ(t), failures and Due to tradeoff among the
Mirabadi (2017) intervals to increase machine’s Total Compl. Time double chromosome reactive maintenance are objectives, a global optimum
availability. considered. Different levels of PM cannot be found; only a Pareto
are considered, and their number Optimal frontier can be generated.
is analytically optimized.
Single Machine Overhauls are performed Earliness and Variable Neighborhood The model is standard, the Hazard rate λ(t) is not considered;
Yazdani et al. (2017) (stopping production) at Tardiness Penalties Search innovative part lies in the solution Faults are not considered;
predefine time intervals. approach that is very performing. Number and duration of PMs are
known in advance.

Cui et al. (2018) Flow Shop PMs executed at variable time Schedule Robustness Dispatching Rules, Hazard rate λ(t) is considered. A single level of PM (i.e., perfect
intervals to increase machine’s and Quality Simulations and Monte Failures and reactive maintenance repair) is considered.
availability. Carlo Sampling are considered. The number of PMs Optimization is obtained via
is analytically optimized. simulation and it is case specific.
Time buffers are used to increase
robustness
Woo and Parallel The machine degrades and its Makespan Simulated Annealing A machines’ deteriorating rate is Hazard rate λ(t) is not considered
Kim (2018) Machines productivity decreases. PMs and Genetic Algorithm considered. (i.e., it is substituted by a
must be executed, within time The optimal number of time degradation rate);
windows to reduce windows (and their duration) is Faults are not considered;
degradation. optimized.

123
124 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

Using this notation, the expected completion time for each job 3.2. Solution approach
E[Ci(π ) ] can be expressed as in Eq. (1):
⎧i

Zammori et al. (2014) also proposed a Modified Harmony

⎪ 
⎪
⎪ sπk−1 πk + pπk i f i ≤ μ1 Search (MHS) to solve the (1| Due Dates, Sequence Dependent Set-
⎪
⎪k=1   Up Times, Planned Maintenance| ETP) scheduling problem. Basi-
⎪
⎪ +TR ·  i pπ
⎪
⎪ cally, the MHS is an advancement of a standard Harmony Search
⎪
⎪  k=1i
k
⎪
⎪ 
(HS), a metaheuristic inspired by the creativity process of jazz mu-
⎪ π)
⎪E Cμ( l−1 + sπk−1 πk + pπk if μl−1 < i ≤ μl ,
⎪
⎪
sicians that, during jam sessions, expand their repertoire using
 π ⎨
k=1+μl−1
l = 2, . . . , M
pure inspiration and/or reworking on old harmonies (Geem, Kim,
E Ci( ) =   (1) & Loganathan, 2001).
⎪ +TP + TR · 
⎪ i
⎪
⎪
p πk Formally, the HS has its essence in the Harmony Memory (HM),
⎪
⎪ k=1+μl−1
a matrix that represents a musician’s repertoire. This matrix con-
⎪
⎪  

⎪E Cμ(πM−1
i
)
⎪
⎪ + sπk−1 πk + pπk i f i > μM tains K row vectors t k , where the superscript k ∈ [1, K] indicates
⎪
⎪ k=1+μM−1
  the position in the HM, each one coding a feasible solution and its
⎪
⎪
⎪
⎪  i
Objective Function (OF). For the present problem, vectors π and μ
⎩ +TP + TR ·  p πk
k=1+μM−1 are joined to form an (N + M + 1)-dimensional vector t k = (t1 , t2 ,…,
t(N + M) , t(N + M + 1) ), whose generic element tik (with i ≤ N + M) con-
where: tains the index of the i-th task of the integrated job-maintenance
- J = {1, 2, …, N} is the set of jobs; schedule and the element t(kN+M+1) contains the ETP of the sched-
- P = (p1 , p2 ,…, pi , …, pN ) is the vector of the jobs’ processing ule. Since tasks may be either jobs or maintenance, natural num-
times; bers in the interval [1; N] and in the interval [(N + 1); (N + M)] de-
- S = (sij ) is the N × N matrix of the sequence dependent setup note jobs and PMs, respectively.
times; Randomly initialized, solutions of the HM follow an evolution-
- d = (d1 , d2 ,…, di , …, dN ) is the vector of the due dates; ary process based, mainly, on the Harmony Memory Consideration
- TR is the average time of a reactive maintenance task; (HMC). Specifically, new solutions t∗ are generated by recombining,
- TP is the average time of a planned maintenance task; one element at a time, the elements tik taken from some “promis-
- (t) is the expected number of failures in the interval [0, t]. ing” solutions stored in the HM. To avoid rapid convergence, selec-
tion is made using a roulette wheel selection rule so that, although
According to Eq. (1) the expected completion time of a job can rarely, even the elements of a solution tik with a poor objective
be computed using a recursive formula that takes into considera- function may be selected. For the same aim, two additional strate-
tion the position of the job within the scheduling sequence, rel- gies, namely Random Selection (RS) and Pitch Adjustment (PA), are
atively to the nearest PM. More precisely, the whole schedule is used to convey a certain degree of randomness to the generative
seen as an aggregate of sub sequences of jobs divided by consec- process. According to the first strategy, to create a new solution,
utive PMs. Also, due to the perfect repair hypothesis, all these se- some elements ti∗ may be randomly generated from the values of
quences can be considered independent because any time PM is the domain [1; (N + M)] to which they belong to. Similarly, PA is
performed, the hazard rate λ(t) is reset to its original value and, used to slightly perturb one or few elements of the new solution.
consequently, the interval [0, t], on which the expected number For instance, in the MHS, PA is implemented as a pairwise inter-
of failures (t) is computed, shrinks accordingly. Owing to these change generation scheme, that is two positions i and j of the new
issues, the completion time of a job belonging to a generic sub- solution t∗ are randomly selected and the corresponding elements
sequence s equals the sum of: (i) the completion time of the last ti∗ and t ∗j are swapped. Lastly, a new solution is accepted if its ob-
subsequence (s − 1), (ii) the processing and setup times of all the jective function is better than that of the worst solution stored in
preceding jobs in the same subsequence s and (iii) the expected the HM; in this case, the worst solution is replaced by the new
down time, due to both failures and maintenance, from the start one.
of subsequence s up to the considered job. The peculiarity of the MHS is the implementation of an addi-
Also, under the reasonable and common assumption of the time tional local minima escaping strategy, called Large Portion Recov-
to failure being distributed as a Weibull, with characteristic life η ery (LPR). Anytime LPR is triggered, the HM is completely regener-
shape parameter β , the hazard rate λ(t) becomes: ated with new solutions created by recombining, through 2-points
λ(t ) = η−β β t β −1 (2) cross-over, entire sub-arrays of the precedent solutions contained
in the HM. LPR is activated with a rate (LPRR) that depends on the
Hence, (t) can be easily obtained as: Saturation’s Level of the HM, a measure of similarity of the solu-
 t β tions of the HM, computed as in Eq. (7):
(t ) = (3)
η SL =
C
(7)
K −1
Based on the expected completion times, ETP can be finally
evaluated as in (4), where aπi and bπi are the earliness and tar- Where C and K are, respectively, the number of copies and the
diness penalties (per unit of time) relative to job π i . total number of solutions of the HM.

N
π

 π 3.3. Problem peculiarities

ET P = aπi · Ei( ) + bπi · Ti( ) (4)
i=1
As anticipated in Section 1, when the ETP is used as objective
where: function, regardless of others problem settings, the solution space
π  π  becomes rather flat, but it is scattered by many points of local min-
Ti( ) = max 0, Ci( ) − dπi (5)
imum. For the sake of brevity, we will not report the detailed re-
  sults of this preliminary analysis, but we will show a simple exam-
π   π  ple, which illustrates this concept well. Specifically, Fig. 1a shows
Ei( ) = min 0, Ci( ) − dπi  (6)
the ETP of all the 3600 solutions of three randomly generated
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 125

Fig. 1. (a) Neighbour solutions of three instances of a 6-Jobs 1-Planned Maintenance problem. (b) Sorted solutions of three instances of a 6-Jobs, 1-Maintenance problem.

instances of a 6-Jobs, 1-PM scheduling problem. In each problem, for five times and PM was added after job in position 1 for the first
the average job’s processing time equals 100-time units, planned group of 720 permutations, in position 2 for the next 720 permu-
and reactive maintenance equals 40 and 20-time units, respec- tations, and so on, as shown in the generation scheme of Table 2
tively. What differentiates each case is the value of the shape pa- As it can be seen from Fig. 1a, the general pattern does not sig-
rameter η of the Weibull probability distribution, which equals nificantly change: neighbor solutions tend to scatter, almost ran-
90%, 60% and 30% of the average jobs’ processing time. domly, around an average value that does not display a significant
All contiguous points on the graph (i.e., points labelled as Si , trend. The fact that the solution space is rather flat is even better
Si + 1 ) are neighbour solutions generated as follows. At first, start- highlighted by Fig. 1b, which shows the ETP curves obtained after
ing from the basic sequence {1, 2, 3, 4, 5, 6}, its 720 permutations sorting (in ascending order) all the 3600 solutions of the problem.
were generated using the shifting procedure i.e., two locations m Lastly, we note that also the number of local minima is high, as
and k are randomly selected and job π m is moved before π k if shown by Table 3, which reports the main features of the above-
m > k and vice-versa. Next, all these permutations were replicated mentioned example.
126 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

Table 2
The neighbors’ generation scheme.

S1 S2 … S720 S721 … S3600

π = (1, 2, 3, 4, 5, 6), π = (1, 2, 3, 4, 6, 5), π = (6, 5, 4, 3, 2, 1), π = (1, 2, 3, 4, 5, 6), π = (6, 5, 4, 3, 2, 1),
μ = (1) μ = (1) … μ = (1) μ = (2) … μ = (5)

Table 3 verges from the real optimum.

Basic features of the problems.
∗
∗

1  fi, j,• − fi,•,•

N
η = 90 η = 60 η = 30 AP E j = (8)
N ∗
fi,•,•
Minimum Cost 10,094 13,455 19,838 i=1
Maximum Cost 13,067 17,861 26,608 ∗

1   fi, j,k − fi,•,•

K N
Solutions within 10% from the optimum 404 703 698
Solutions within 5% from the optimum 82 165 170 AP D j = (9)
N·K fi,∗•,•
k=1 i=1

Where:

3.4. Computational drawbacks - fi,j, k is the k-th solution generated by the j-th algorithm for
the i-th problem; in this case i ∈ [1, 10], j ∈ [SA, MHS] and
As noted by Yang (2010), when the solution space is so flat, nei- k ∈ [1, 8];
ther Random Selection nor Pitch Adjustment can convey enough - fi,∗ j,• is the best solution for problem i found by algorithm j i.e.,
randomness to effectively escape an early convergence process: all fi,∗ j,• = minK { fi, j,k };
the solutions of the Harmony Memory become similar, if not even
- fi,∗•,• is a priori known global optimum for problem i.
identical, and the probability to get immediately stuck in a local
minimum becomes high. Although the MHS implements a third lo- In case of SA, both APE and APD were equal to zero; conversely,
cal minimum escaping strategy (i.e., the Large Portion Recovery), for MHS an APE of 0.5% and an APD of 1.8% were found. These val-
it is licit to ask if this approach is sufficient to obviate the early ues demonstrated that, whereas SA consistently found the global
convergence problem. To answer this question, we compared MHS minimum, MHS always converged to some local minimum rather
with the very famous and general-purpose Simulated Annealing distant from the global one.
(SA) by Ishibuchi et al. (1995), whose main features are detailed
in Appendix A. The rational of this choice can be motivated as fol- 4. Enhancements of the MHS
lows:
Due to the preliminary results discussed in Section 3, we tried
to improve the MHS through the addition of four additional local
- The metaheuristic implements a relatively simple iterated
minima escaping strategies that will be detailed in the following
greedy strategy that assures a high degree of randomness dur-
Sub-Sections.
ing the exploration of the solution space;
- The metaheuristic is flexible, as its performance does not de-
4.1. Block shifting: A new Pitch Adjustment procedure
pend, significantly, neither on the specific nature of the com-
binatorial problem under analysis, nor on the initial condition
As explained in Section 3, in the MHS PA has been implemented
used as the starting point;
as a simple pairwise interchange operator. Consequently, due to
- The number of operating parameters to be fine-tuned is very
the small number of PMs compared to the number of jobs, it is
low.
quite rare for the PA to swap a job with a PM. In general, this is
not an issue and the evolving process works fine, as PMs repo-
We also note that we opted for a general-purpose algorithm sitioning is assured by the Harmony Memory Consideration. Yet,
rather than for one specifically designed to tackle IJMS prob- when the Harmony Memory begins to saturate, Harmony Memory
lems, as it was not possible to find an algorithm suitable for the Consideration loses effectiveness and the exploration of the neigh-
specific IJMS under consideration. Indeed, as clearly discussed in borhood proceeds mainly thanks to PA. To exploit this search, it
Section 2 and showed in Table 1, most of the approaches proposed is thus fundamental to add to the original pairwise interchange,
in the literature have the limit not to consider the effect of PMs on which modifies jobs’ sequencing leaving almost unaltered the posi-
the hazard rate of the machines, which is, instead, one of the pe- tion of PMs, a dual PA operator that, conversely, acts on PMs leav-
culiar feature of the model herein considered. Even the few works ing almost unaltered jobs’ sequencing. Owing to these considera-
that considered the hazard rate, cannot be used as benchmark, tions, we defined the Block Shifting (BS) operator, which works as
since they are very specific and cannot be effectively readapted to follows:
the framework of the problem under consideration.
- A whole subsequence of jobs (i.e., jobs placed between two
Obtained results were astonishing, as SA outperformed MHS,
PMs) is randomly selected;
even in case of 10 randomly generated problems of small size
- Let Ji be the selected subsequence, then a new neighbor solu-
(with parameters defined as indicated in Section 6), character-
tion is generated by moving the block {PM(i − 1) , Ji ,PMi } either
ized by 14-Jobs and 2-PMs and for which the optimum was found
after the first job of the subsequence Ji + 1 or before the last jobs
through an exhaustive search. More precisely, each problem in-
of the subsequence Ji − 1 .
stance was solved 8 times and obtained results where quantified
in terms of Average Percentage Error (APE) and Average Percent- For instance, in case of sequence S1 = {j1 ,j2 ,PM1 ,j4 ,j5 ,j6 ,PM2 ,j3 ,
age Deviation (APD), computed as in Eqs. (8) and (9), respectively. j7 ,j8 ,j9 ,PM3 ,j10 ,j11 ,j12 }, if subsequence J2 = {j4 ,j5 ,j6 } was selected
Note that APEj quantifies the average difference between the best and block {PM1 ,j4 ,j5 ,j6 ,PM2 } was moved forward, the sequence
solution found by the j-th algorithm and the real optimal one. Sim- S2 = {j1 ,j2 ,j3 ,PM1 ,j4 ,j5 ,j6 ,PM2 ,j7 ,j8 ,j9 ,PM3 ,j10 ,j11 ,j12 } would be finally
ilarly, APDj measures how much, on average, the j-th algorithm di- obtained. Please note that S2 could have been generate also by
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 127

means of pairwise interchange; yet this would be extremely im- Lastly, aiming to select the best crossover operator we: (i) de-
probable because, considering that sequences with 2 PM in adja- veloped three standard genetic algorithms, differentiated only by
cent positions are not allowed, it would require a minimum of five the type of crossover and (ii) we tested them on the same prob-
consecutive moves. lems’ set described in Section 3.4. At the end of the analysis the
choice fell on the ordered crossover, as it was the only one able to
4.2. A second crossover operator find the best solution for every instance of the problems set.

Aiming to improve the regeneration of the HM through Large 4.3. Maximizing the harmony memory degree of differentiation
Portion Recovery, in addition to the 2-points crossover, we also
considered and compared two alternative types of crossover: (i) In the original MHS, anytime the Large Portion Recovery is in-
the order crossover and (ii) the double 2-1-points crossover, voked, crossover is activated to regenerate the HM. Ideally, at the
readapted from the recent one proposed by Salmasnia and end of this process, the HM should be completely new, yet, as
Mirabadi-Dastjerd (2017). clearly shown by Fig. 2, the true picture is rather different.
The ordered crossover was chosen to foster the exploration of As it can be seen, anytime the Saturation Level (SL) reaches a
large portions of the solution space, as it is known to have the threshold limit (in this case set to 0.3), the Large Portion Recovery
ability to generate offspring solutions sensibly different from the is triggered and, immediately after, the SL drops down; yet, unlike
parent ones (Gen et Cheng, 20 0 0). Briefly, we recall that the order one could expect, it is always greater than zero. This fact implies
crossover is based on the following steps: that, due to an excessive degree of similarity among parents, du-
plicates are created in the offspring. Conversely, to maximize the
- Select a random swath of consecutive genes from Parent 1; degree of differentiations of the HM, without losing good solutions,
- Drop the swath down to Child 1 and mark out these genes in we decided to limit the regenerative process only to the duplicated
Parent 2; solutions, which are substituted with newly generated ones. More
- Starting on the right side of the swath, take genes from Parent precisely, letting K, C and O = (K − C) be, respectively, the number
2 and insert them in Child 1, at the right edge of the swath; of solutions, of copies and of original (non-duplicated) solutions
- Generate Child 2 by flipping Parent 1 and Parent 2 and repeat- contained in the HM, the regenerative process works as follows:
ing all the other steps.
- Identify all different solutions contained within HM;
To clarify this process, a simple example is given next, where - Use a roulette wheel selection (without replacement if O ≥
the swath of genes selected from Parent 1 is highlighted in Italics: C, with replacement otherwise) to select C parents among the
original solutions and apply order-crossover, to generate C off-
- Parent 1 = {j1 , j3 ,PM1 , j5 , j4 ,PM2 ,j2 ,j6 }, spring solutions
- Parent 2 = {j6 ,j3 ,j4 ,PM1 ,j2 ,j5 ,PM2 ,j1 }, - Selectively replace all the copies contained in the HM using the
- Child 1 = {j6 , j3 ,PM1 , j5 , j4 ,j2 ,PM2 ,j1 }, C generated offspring solutions.
- Child 2 = {j1 , j3 , j4 ,PM1 , j2 , j5 ,PM2 ,j6 }.
It is worth noting that, the regeneration of the HM may intro-
The integrated 2-1-points crossover is the only alternative duce solutions that are much worse than the current best, as in-
crossover operator that was found in the literature, among the sub- dicated by the spikes in the “worst value curve” (in red) shown in
set of papers that deal with the IJMS problem and that fully con- Fig. 3. Anyhow, as indicated by the “current optimum curve” (in
sider the effect of PM on machines’ hazard rate (see Table 2 in green), the regenerative process strongly improves the exploration
Section 2, for further details). In its original version, 2-1-points of the solution space and, indeed, improvements often occur im-
crossover operates on two distinct chromosomes that codify jobs mediately after a regenerative process has taken place.
and different levels of PM, respectively. However, in the present
problem there is a single kind of PM, and so we used the second 4.4. A dynamic large portion recovery rate
gene to codify the position of PMs within the jobs’ sequence.
In virtue of this slight modification, the 2-1-points crossover As a last modification, instead of using a fixed LPRR, we decided
operates as it follows: to dynamically update this threshold limit, using the cubic function
of Eq. (10). In this way, LPRR remains low for small values of SL, but
- Jobs and PMs are separated and coded with the π - μ notation
it rapidly rises as SL increases. This behavior is rationale because
of Section 3.1;
when SL is high almost all the solutions of the HM are identical
- 2-points crossover and 1-point crossover are made on the π
and only crossover can refresh the HM. For this reason, also LPRR
vector (i.e., jobs part) and on the μ vector (i.e., maintenance
must approach one, to activate almost certainly the Large Portion
part), respectively;
Recovery.
- The two constructed parts are recombined together to generate

a new solution. LP RR = LP RRLow i f SL <
 SLLow

LP RR = α + β · SL3 i f SL ∈ SLLow ; SLHigh (10)

To clarify this process, the integrated 2-1-points crossover is ap-
LP RR = 1 i f SL = SHigh
plied to the parents used in the example above; note that both
vectors have been recoded using the π - μ notation and that the Where:
symbol ˈ|ˈ has been used to highlight the cutting points.
- LPRRLow ≥ 0 is the minimum value of LPRR;
- Parent 1 = {j1 ,j3 ,PM1 ,j5 ,j4 ,PM2 ,j2 ,j6 } → π 1 (1, 3, |, 5, 4, 2, |, 6), - SLLow ≥ 0 is the value of SL below which LPRR takes its mini-
μ1 (2, |, 4), mum value;
- Parent 2 = {j6 ,j3 ,j4 ,PM1 ,j2 ,j5 ,PM2 ,j1 } → π 2 (6, 3, |, 4, 2, 5, |, 1), - SLHigh > SLLow is the value of SL above which LPRR becomes 1;
μ2 (3, |, 5), - α = 1 − β · SL3High
- Generated Vectors: π 1,2 (1, 3, 4, 2, 5, 6), π 2,1 (6, 3, 5, 4, 2, - β = (1 − LP RRLow ) / (SL3High − SL3Low ).
1), μ1,2 (2, 5), μ2,1 (3, 4),
- Child 1 = {j1 ,j3 ,PM1 ,j4 ,j2 ,j5 ,PM2 ,j6 }, We conclude this section noting that all the above-mentioned
- Child 2 = {j6 ,j3 ,j5 ,PM1 ,j4 ,PM2 ,j2 ,j1 }. improvements take advantage of the specific structure of the IJMS
128 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

Fig. 2. Typical trend of the Saturation Index during a computational run.

Fig. 3. Typical trend of the objective function during a computational run.

problem herein considered. Consequently, from here on, we will equal to d, a parameter that represents the distance travelled by
refer to this metaheuristic as the Harmony Search for Integrated the cuckoo. Also, this travel is modelled as a Lévy Flight that is,
Job Maintenance Scheduling, indicated as HS/4/IJMS. For further essentially, a random walk in which the length of the random step
details the interested readers are referred to Appendix B, where is drawn from a probability distribution that is heavy-tailed i.e., for
the full pseudo-code can be found. which Pr(x > X ) = O (x−α ), with 1 < α < 3, so that the variance is
infinite. For α < 2, the Pareto distribution (see Eq. (11)) is one of
5. Cuckoo search via Lévy flight and possible enhancements the distributions satisfying these requirements.
 x α
min
We also decided to investigate the performances of the Cuckoo 1− if x ≥ xmin
P r (x > X ) = x (11)
Search via Lévy Flight (CS-LF), a popular population-based meta- 1 if x < xmin
heuristic introduced by Yang and Deb in 2009. CS-LF was originally
designed to solve continuous problems, but its preformances are So, using a Pareto, in virtue of the inverse transformation tech-
excellent also in case of discrete combinatorial problems (see for niques, the step length d can be generated as in Eq. (12):
example Komaki, Teymourian, Kayvanfar, & Booyavi, 2017; Ouaarab,
Ahiod, & Yang, 2014). Also, and perhaps more important, since its dmin
d= (12)
search strategy assures little or none cross influence among saved ( − u )α
1
solutions, CS-LF is a very good candidate to tackle the IJMS prob-
where u is a random number uniformly distributed in the interval
lem.
[0; 1].
Also, to model the possibility that some of the eggs are detected
5.1. Main features of the standard CS-LF and smashed, a P% of the worst solutions are substituted with ran-
domly generated ones. For the full algorithm, the interested read-
CS-LF is a nature inspired metaheuristic, which mimics the ers are referred to Yang and Deb (2009); an extended and modified
brood parasitism of some species of cuckoo, a bird which does not version is detailed in Sections 5.2 and 5.3.
brood its eggs but lays them in other nests. Each cuckoo explores
its surroundings, by means of a Lévy Flight (LF), in search of a good
nest to lay its egg. Evaluating the options, the cuckoo will select 5.2. Cuckoo search with migration and nests location memory
the best one and it will lay its egg. Once an egg has been laid, it (CS-MNM)
can be detected and purged by the owner of the nest, or it can suc-
cessfully hatch. If so, the new cuckoo will continue the exploration We also altered the searching strategy implemented by the CS-
of the surrounding area, looking for new nests to colonize. LF to explore the solution space in an even better way. The key
Briefly, the key element of the CS-LF is that, at each step, a pair- of such re-design is based on a modification of the standard Lévy
wise interchange scheme is iteratively applied to generate a neigh- Flight and on the introduction of three different features, namely:
bor x1 from an initial solution x0 , then a neighbor x2 from x1 and (i) Nests Location Memory, (ii) Risk Aversion and (iii) Clan Migra-
so on (i.e., x0 → x1 → x2 → … → xd ) for, at most, a number of times tion.
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 129

This is shown in Eq. (13).

  
f
NRP = min 1, exp − (13)
Ri, j
Where f is the change in the objective function and Ri,j , is the
parameter that links the risk aversion of cuckoo i with the number
of performed iterations j. To this aim Ri,j is progressively and lin-
early reduced from an initial value R0 , until it reaches an ending
value RJ at the last iteration J, as shown in Eq. (14):
 
Ri, j − RJ
Fig. 4. Cuckoo’s flight modes. Ri, j = R0 − j · (14)
J
Concerning R0 , as suggested by Kirkpatrick, Gelatt, and Vec-
5.2.1. Double shaped Lévy flight chi (1983), this value can be chosen to assure a high initial Non-
As shown in Fig. 4, two flight modes are possible: Return Probability NRP0 as in Eq. (15):
- Mode #1 – A flight of progressive departure from the starting f
point l0 ; R0 = (15)
ln(NRP0 )
- Mode #2 – A peripheral flight centered on an intermediate
point lj . where  f is the average change of the objective function over a
sample of neighbor solutions. Also note that Eq. (15) can be used to
Note that, starting from l0 (highligthed in black in Fig. 4), the compute Rj too, by substituting NRP0 with a low final Non-Return
overall flight (from 0 to 7) can be decomposed into a series of Probability NRPJ .
intermediate points lj (each one representing a feasible solution)
where the cuckoo pauses to assess the goodness of the surranding 5.2.3. Clan migration (CM)
area. As long as the quality improve, with respect to the last vis- We also introduced a migration strategy, to mimic the ability
ited point, the cuckoo continues to move away (as for points {l1 , of birds to share part of their past experience. Specifically, since
l2 , l3 } and {l6 , l7 }). Otherwise (as for points {l4 , l5 }), it explores the members of a clan tend to follow the behavior of the most suc-
neighborhood of the last point of improvement (as for point l3 ). cessful individuals, we hypothesized that, after a certain number
The latter case can be seen as the cuckoo coming back and then of iterations (i.e., an epoch), some of the cuckoos that ended up in
restarting its search or, simply, as the cuckoo performing a sort of areas that offer low opportunities for improvement, migrate in the
circular flight centered on the last point of improvement. Anyhow, surrounding of the best and most promising areas so far discov-
the maximum number of visited points d is defined by the Lévy ered by them and or by other cuckoos.
parameter of Eq. (12) and, as in the standard CS-LF, the search is Operatively, this behavior has been implemented through the
immediately interrupted if a point lj improves the overall best so- following steps:
lution found so far. In this case the cuckoo stops its search and lays
an egg. - Solutions are sorted in descending orders, in terms of the con-
Operatively, starting from l0 , all the visited points lj are gener- secutive number of flights that did not produce any further im-
ated applying the pairwise interchange operator to the last solu- provement (i.e., solutions corresponding to cuckoos which are
tion that produced an improvement. More precisely the first point more likely to be stuck in a local minimum are placed at the
l1 is generated as a neighbor of l0 ; next, if l1 is better than l0 , the top of the list);
second point l2 is generated as a neighbor of l1 , otherwise l2 is - The first P% cuckoos are preselected and, among these one, all
generated as a neighbor of l0 , provided that l2 = l1 . This process is those who exceeded a predefined threshold limit (expressed as
iteratively repeated until at most d point have been explored. For number of consecutive failed flights) are finally selected for mi-
instance, in the case of Fig. 4, the generating scheme is the follow- gration;
ing one: {l0 → l1 ; l1 → l2 ; l2 → l3 ; l3 → l4 ; l3 → l5 ; l3 → l6 ; l6 → l7 }. - The starting points of migrating cuckoos are substituted with
new points, generated as neighbors of the P% best solutions
found so far.
5.2.2. Nest location memory and risk aversion (RA)
Most of the times, the cuckoo starts a new flight from the last
5.3. CS-MNM full implementation
point of improvement lj (with j ≤ k) found during its last flight (i.e.,
l7 in the example of Fig. 4). This always happens in case of a suc-
The implementation is based on two sets E and N storing, for
cessful mission when a new host nest has been identified and the
each cuckoo i, its current starting point ei and its nest ni i.e., the
cuckoo has laid its egg. This also happens in case of an unsuccess-
best solution found so far by cuckoo i. At each iteration, both sets
ful mission, provided that Mode #1 has been used at least once,
are updated accordingly to the outcome of the flight. Let us con-
during the Double Shaped Lévy Flight. Conversely, if only Mode #2
sider a generic cuckoo i placed at point ei . Also, let L ≡ {l1 ,l2 ,…,
has been used, the cuckoo may try to exploit all the information
lj ,..lk }, with k ≤ d, be the points visited by the cuckoo during its
obtained during its peripheral flight. If so, it might decide not to
flight and let l∗ be the best among them. In accordance with the
come back to the initial starting point l0 , but rather to restart from generation scheme of the set L (i.e., Double Shaped Lévy Flight),
the best intermediate point l∗ found during the flight, although
anytime lj proves to be a point of improvement, it replaces ei as
this point was judged inadequate to lay an egg and it is proba-
the new starting point, from which the following neighbor lj + 1 is
bly a bad move in the short-term. Clearly this choice carries a risk obtained. Also note that, as clarified in Sub Section 5.2.1, the flight
that is accepted by the cuckoo with a certain probability (namely
is immediately interrupted as soon as a lj improves the current
the Non-Return Probability NRP). Also, the NRP is not fixed, but, as
best ni . That is, whenever k < d, the last point lk ∈ L coincides
in Simulated Annealing, it is exponentially decreased over time, to
with l∗ and it is also better than ni . In this case the mission has
better represent the risk aversion profile of the cuckoo, which is
been successful, and thus both vectors ni ∈ N and ei ∈ E are up-
supposed to become more and more circumspect as it gets older.
dated so that ni ≡ ei ≡ lk ≡ l∗ . Conversely, if k = d, the mission
130 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

Table 4
CS-MNM pseudocode.

has been unsuccessful, ni is not updated and, typically, the cuckoo to be tardy, and RD = 0.4 is the relative Range of Due Dates
will go back to l∗ . However, if ei has not changed during the flight (Tan, Narasimhan, Rubin, & Ragatz, 20 0 0);
(Mode #2 was employed exclusively), even if l∗ is worse than ei , - The earliness and tardiness penalties have been set, respec-
the cuckoo might choose to go in l∗ and resume its next explo- tively, at a = 1 and b = 8;
ration from that point, with a certain probability (as explained in - Time-to-Failure follows a Weibull distribution with scale pa-
Sub-Section 5.2.2). In this case ei is updated so that ei ≡ l∗ . rameter β = 1.2 and shape parameter η = 90;
For the sake of clarity, the full pseudocode of the CS-MNM is - The average time to execute Planned and Reactive Maintenance
shown in Table 4. Briefly, the Double Shaped Lévy Flight is im- are set, respectively, to TP = 40 and TR = 20.
plemented from Line #14 to #24 and the part of code between
Line #20 to #21 is responsible to switch between the two flight
modes, as described in Section 5.2.1. Cuckoo’s Risk Aversion is im- 6.1. Parameters optimization
plemented by Line #25 to #28 and, as it can be seen, it is enabled
anytime the Double Shaped Lévy Flight proved unsuccessful. Lastly, To assure a fair comparison of the metaheuristics introduced in
Clan Migration is detailed by Line #29 to #32. Sections 3 to 5, their operating parameters were fine-tuned using
a test set of 50 small size problems, characterized by 14-Jobs and
6. Performed tests
2-PMs. Problems of this size were chosen because: (i) their com-
plexity is high enough to prevent the metaheuristics from find-
All the tests discussed in this Section were performed on a test
ing, almost always, the optimal solution and, (ii) they can be opti-
set of randomly generated problems characterized by the following
mally solved in a reasonable amount of time (i.e., around one and
features (Zammori et al., 2014):
a half hour on a computer with a 3.2 GHz processor and 16 GB of
- Processing Times are Normally distributed with mean μP and RAM memory), through an exhaustive search. Also, parameters that
standard deviation σ P of 100 and 25 time-units, respectively; can be reasonably considered independent were fine-tuned one by
- Setup Times are Uniformly distributed in the interval [0; 19]; one, using a sequential optimization strategy. Conversely, parame-
- Due Dates are Uniformly distributed in the interval [NμP · ters for which interaction effects cannot be excluded a priori, were
(1 − TF ) ± 0.5μP RD ], where N is the number of jobs, TF = 0.5 pre-analyzed and optimized using Design-Of-Experiment (DOE).
is the Tardiness Factor, representing the quota of jobs likely More precisely:
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 131

Table 5
Sequential optimization results.

Algorithm Parameter Range Step Size Optimal Value

MHS Random Selection Rate RSR – – 0.05

Pitch Adjustment Rate PAR – – 0.1
Harmony Memory Size HMS – – 0.5NJ
SA Start Probability IAP [0.05; 0.50] 0.05 0.1
End Probability EAP [0.001; 0.03] 0.005 0.02
CS-LF Minimum Flight dmin [1; 10] 1 7
Pareto Shape parameter α [1; 2] 0.3 1.9
Purge percentage P% [0.01; 0.25] 0.05 0.1
Population Size [0.5NJ ; 2NJ ] 0.25 NJ
Where NJ is the total number of Jobs

- MHS was not considered in the fine-tuned process, as we More precisely, the 10 hardest problem instances (among the 50
used the parameters (reported in Table 5) suggested by previously generated ones) were selected and each one was solved
Zammori et al. (2014), which had been already optimized on 8 times, both with HS/4/IJMS and with CS-MNM. Testing 23 combi-
the same problem set; nations of parameters levels, a total of 640 solutions (i.e., 10·8·23 )
- SA and CS-LF (i.e., the original metaheuristics) were optimized were generated with each metaheuristic. It is worth noting that,
first. Parameters were assumed independent and fine-tuned fol- to assess interaction effects through 3-ways ANOVA, there is the
lowing the sequential approach; need to collect more than a single value of the response variable
- HS/4/IJMS and CS-MNM were optimized next. In this case, pa- for each one of the 10·23 combinations of problems and param-
rameters shared with the original metaheuristics were directly eters setting. Thus, we replaced APE with the Relative Percentage
inherited from the previous optimization step; conversely, Error defined in Eq. (16):
specific parameters were optimized using DOE, to better inves- fi, j,k − fi∗
tigate possible interactions among them. RP Ei, j,k = (16)
fi∗
6.1.1. Sequential optimization Where, fi∗ is the (a priori known) optimal solution of problem i
Let us consider a generic algorithm A, and let P = {p1 ,p2 ,…, pm } ∈ [1; 10] andfi,j, k is the solution found at attempt k ∈ [1; 8] by the
be the set of its parameters, ordered from the most to the least used metaheuristic, with parameters configuration j ∈ [1; 23 ].
important one. Sequential optimization proceeds as follows: Operating in this way, two remarkable results were found, as
DOE and the related ANOVA tests revealed that:
- Initialize each parameter pi using a reasonable starting value.
To this aim we used values suggested in the literature; - Concerning possible interaction effects, all p-values were higher
- Given these initial settings, all problems of the test set are than 0.25 (definitely greater than the usual threshold of 0.05),
solved and the accuracy of A is computed. To this aim, we mea- and so parameters can be considered independent;
sured accuracy in terms of APE, as in Eq. (8), using 8 repetitions - Concerning the effect of the operating parameters on the algo-
of each problem instance; rithm performance, all p-values were higher than 0.1, and so
- The parameter p1 , having the higher influence on the algorithm performances are stable and little influenced by specific values
performance, is selected and its value is slightly modified. All of the operating parameters.
the other parameters {p2 ,…, pm } are kept unaltered and all the Since both results confirmed the robustness of the proposed
problems of the test set are solved again. This process is itera- metaheuristics, as well as the absence of any interactions among
tively repeated, proceeding at small step, until the accuracy of parameters, we completed the optimization process following the
A continues to grow; same sequential approach detailed above. Obtained results are
- When accuracy does not increase anymore, the value of p1 is shown in Table 6.
frozen and the second parameter p2 is selected for possible im-
provements and it is fine-tuned using the same approach de- 6.2. Numerical Assessment
scribed above;
- The procedure is repeated for all the other parameters of the Lastly, SA, CS-LF, HS/4/IJMS and CS-MNM were compared in rel-
set P. ative terms. Please note that the original MHS was not included in
this analysis, because, as detailed in Section 3.6, it is outperformed
Obtained results are shown in Table 5, where optimized param-
by the SA, even with problems of relatively small dimensions. We
eters are listed in order of importance. The optimal value is shown
also note that, to get a preliminary idea of the quality of the mod-
in the last column, whereas the range and the step size used in
ifications made on the MHS, we tested its improved version (i.e.,
the optimization are reported in the third and fourth column, re-
the HS/4/IJMS) against the same test set of 10 problems with 14-
spectively.
Jobs and 2-PMs. In this case, results were encouraging as HS/4/IJMS
behaved almost like the SA and it consistently found the global op-
6.1.2. Design of experiment
timum (i.e., APE = 0 and APD = 0.01).
Given the higher complexity of HS/4/IJMS and CS-MNM, a 2-
Comparisons were made relatively to two test sets with a dif-
levels full factorial design was made (using the levels shows in
ferent level of complexity: (i) 10 hard problems with 25-Jobs
Table 6), limited to the following parameters:
and 3-PMs, and (ii) 10 very hard problems with 50-Jobs and 8-
- Concerning HS/4/IJMS, we considered LPRRLow , SLLow and SLHigh , PMs. In both cases, the number of PMs coincides with the op-
because these parameters conjointly govern the rate at which timal value, calculated accordingly to the procedure proposed by
the Large Portion Recovery is triggered; Zammori et al. (2014), to which the interested reader is referred
- Concerning CS-MNM we considered R0 , RJ , and the Epoch’s for further insights.
Length, because these parameters conjointly influence the evo- Each problem instance was solved 8 times, by each meta-
lution of the solutions contained within sets N and E. heuristic, starting from a different initial population. Also, we let
132 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

Table 6
DOE results.

Algorithm Parameter Levels Optimal Value

HS/4/IJMS Low saturation threshold SLow [0.1; 0.3] 0.3

High saturation threshold SHigh [0.5; 0.7] 0.7
Large Portion Recovery Rate LPRRLow [0.05; 0.1] 0.05
CS-MNM Starting Risk Aversion SRA [0.1; 0.5] 0.1
Ending Risk Aversion ERA [0.01; 0.05] 0.01
Epoch E [50; 100] 75

Table 7
Final comparison.

25-Jobs 3-PMs 50-Jobs 8-PMs

Algorithm APE APD Max Error # Wins APE APD Max Error # Wins

HS/4/IJMS 0.007 0.023 0.0491 0% 0.0580 0.2141 0.3562 0%

SA 0.001 0.011 0.0400 20% 0.0030 0.0213 0.0647 40%
CS-LF 0.087 0.104 0.1416 0% [NC] [NC] [NC] [NC]
CS-MNM 0 0.004 0.0109 40% 0.0 0 08 0.0074 0.0187 60%

each metaheuristic run until convergence, so that computation was As a final note, we want to highlight the following fact. Since
halted when the best solution did not improve for a sufficiently convergence was used as stop criterion, one could argue that com-
high number of consecutive iterations. Lastly, performances were parison between CS-MNM and SA was not fair. Indeed, being a
assessed using APE and APD, as defined in Eqs. (8) and (9), respec- population-based metaheuristic, CS-MNM might have found bet-
tively. Please note that, in this case, the real optimum were not ter solutions only because it ran for a total number of iterations
obtained through an exhaustive search; thus, in both equation the higher than that of the SA. More precisely, given a fixed number
parameter fi,∗•,• is substituted by the best solution found for prob- of iterations I, SA evolves the same solution exactly I times; con-
lem i, i.e. fi,∗•,• = min j,k { fi, j,k }. versely, considering that at each iteration CS-MNM selects a so-
Obtained results are reported in Table 7. lution among the N of the set E, at iteration I each solution will
As expected, the SA works very well and, even with 25-Jobs have been evolved I/N times, on average. It is thus plausible to ex-
and 3-PMs it outperforms both CS-LF and HS/4/IJMS. Specifically, pect that, to find the best solution, CS-MNM needs to run for a
whereas performance of HS/4/IJMS are still acceptable, in its stan- number of iterations N times bigger than that of the SA. To ascer-
dard form the Cuckoo Search is definitely the worst metaheuristics, tain this fact, we compared the evolution path of SA with that of
both in terms of solutions’ quality and stability. For these reasons CS-MNM, for each generated solution. Quite surprisingly, the test
CS-LF was not even considered in the second test set (as denoted showed that 75% of times, when SA reached convergence, CS-MNM
by the [NC] values in columns 6 to 9 of Table 7). Frankly speaking, had already found a better solution and its evolution process was
the very poor performance of the CS-LF was not expected. Prob- still on progress; a fact that further certifies the superiority of the
ably, although solutions evolve independently and cross influence CS-MNM, which is also faster than the SA in finding the optimal
among them is minimized, the solution space is explored neither solution. This result is graphically displayed by Fig. 5a and Fig. 5b
efficiently nor effectively. Alone, the Lévy Flight does not seem that show an example of the evolution path followed by the two
to be enough to assure the right random behavior required for a metaheuristics.
thorough exploration of the solution space. Also, purging the worst
solutions (at every iteration) may limit the neighborhood search,
thus further decreasing the overall effectiveness of the approach. 6.3. Operational Implications
Conversely, in its improved version, the CS-MNM turned out to be
the best approach. CS-MNM, indeed, shows the minimum values Once the superiority of the CSM-MNM algorithm was verified,
of APE, APD and Max Error; particularly, a value of APE equal to a last check was made to quantify the operational advantage that
zero ascertains that, for each problem instance, at least one of the could be achieved using the CSM-MNM algorithm to schedule jobs
eight solutions generated by the CS-MNM was the overall best one and PMs in an integrated way. To this aim we compared the cost
(relatively to the 32 generated by all the tested metaheuristics). of the best schedule found by the CSM-MNM algorithm with the
CS-MNM also proved to be very precise in founding the best solu- cost of the presumably best schedule that could be generated by a
tions as certified by a very low value of both APD and Max Error production manager equipped with a standard scheduling system.
and by the highest value of percentage wins. Relatively to this The latter one was generated using the following naïve algorithm:
last metric, it is important to note that summing the percentage
wins of column 5 a total of 60% is obtained. This means that 40% • At first only jobs are considered (i.e., the single machine is con-
of the times SA and CS-MNM converged to the same, presumably sidered as fault free) and the best schedule S is generated. This
optimal, solution. schedule, that using Kendall’s notation can be classified as (1| |
The superiority of the CS-MNM is even better outlined by the Earliness-Tardiness Penalties), corresponds to the one that could
outcomes of the very hard problems’ set. Not only CS-MNM is the be generated by the standard scheduling system and, in the
metaheuristic that located the best solution with the highest fre- present case is obtained evaluating all the (n!) permutations of
quency but, as certified by the lowest APD value, it proved to be the jobs’ sequence.
extremely robust because, even when it missed the best solution, • Next, the optimal number M of PMs is inserted in S, without

it still found a solution very close to it. All these outcomes indi- altering the ordering of the jobs. This is made in the following
cate that the introduced modifications (i.e., nests location mem- reasonable way:
ory, cuckoo’s risk aversion, clan migration and double shaped Lévy - S is split into (M + 1) parts of almost equal duration;
Flight), made it possible to overcome the drawbacks of the stan- - PMs are inserted at these subdivision points, to assure the same
dard approach, assuring an efficient and effective exploration of level of availability in each sub-section of the schedule.
the solution space.
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 133

Fig. 5. (a) Typical evolution in the initial phases. (b) Typical evolution before convergence is reached.

The comparison has been made testing the exhaustive solver ures and aging with the objective to minimize earliness-tardiness
and the CS-MNM against 5 different instances of the IJMS problem penalties. These features make the problem particularly challeng-
characterized by 50-Jobs and 8-PMs. The CS-MNM always outper- ing, because the solution space becomes very flat, but dissemi-
formed the naïve exhaustive, and solutions generated by the CS- nated of many local minima.
MNM were on average (23 ± 1.1) % more efficient, in terms of ex- Considering the practical relevance of the problem and its
pected operating cost, than that obtained by the naïve exhaustive. inherent complexity, we developed and tested different meta-
These results stress even more the relevancy and applicability of heuristics, aiming to find robust and quasi-optimal schedules
the proposed approach. in a reasonable amount of time. Specifically, we benchmarked
the famous SA by Ishibuchi et al. (1995), with two population-
7. Conclusions and future works based metaheuristics, namely the Modified Harmony Search by
Zammori et al. (2014) and the Cuckoo Search by Lévy Flight by
In this paper, we focused on an integrated jobs-maintenance Yang and Deb (2009). MHS was chosen because it was specifically
scheduling problem, relevant to many industrial realities, where designed to tackle this problem, CS-LF was chosen as it imple-
failures and/or reduced availability may jeopardize productivity. ments a very effective local minimum escaping strategy. In addi-
Specifically, we considered a single machine subjected to fail- tion, we also developed two alternative approaches: (i) HS/4/IJMS,
134 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

a hybrid Harmony Search and Genetic Algorithm, specifically tied implemented the rest of the algorithms used as benchmark.
to this problem and (ii) CS-MNM, an innovating modification of the D.Mezzogori executed the Design of Experiment section and
Cuckoo Search via Lévy Flight. F.Zammori performed final numerical comparison. M.Bertolini fi-
After a fine-tuning procedure, a comprehensive comparison was nally supervised and checked the findings of this work. All authors
made. Outcomes showed that, in these peculiar conditions, sim- discussed the results and contributed to the final manuscript.
pler heuristics characterized by a high degree of randomness may
perform better than sophisticated population-based metaheuris- Appendix A. Simulated Annealing (SA)
tics. Indeed SA (i.e., the simpler algorithm) outperformed MHS, CS-
LF and HS/4/IJMS, especially in case of very hard problems, char- Introduced by Metropolis, Rosenbluth, Rosenbluth, Teller, and
acterized by 50-Jobs and 8 PMs. Only the newly developed CS- Teller (1953) and made popular by Kirkpatrick et al. (1983), SA is
MNM subverted this situation, as it was the best approach both in a probabilistic heuristic, whose inspiration come from annealing in
term of precision and accuracy of the obtained solutions. We also metallurgy, a slow cooling process used to bring melted metals to
showed that CS-MNS offers the following operating advantages: a uniform, low-energy crystallization state. Briefly, at each one of
N iterations, SA generates a new solution y from the neighborhood
• It has relatively few parameters to optimize;
of the current solution x and accepts it as the new starting point,
• Parameters are independent (there are no interaction effects
with a probability that, as iterations go on, decreases following the
among them) and can be optimized independently;
negative exponential law show in Eq. (A.1), valid for a minimiza-
• It performs an excellent exploration of the solution space but,
tion problem.
at the same time, it assures a fast convergence and a low com-   
putation time, also for large problems. f
P r = min 1, exp − (A.1)
Ti
All these features make the CS-MNS algorithm a good candidate
for a possible inclusion in the optimizer module of a Cyber Physi- Where f = (f(y) − f(x)) is the change in the objective function
cal System, fully compliant with the principles of the Industry 4.0 and Ti , the so-called temperature at iteration i, is a parameter that
paradigm. accounts for the number of completed iterations. To this aim, the
Frankly speaking we must admit that, at present, our model re- temperature is progressively and linearly reduced from an initial
quires the number of PMs that must be inserted in the schedule value T0 , as in Eq. (A.2):
as input; although this value can be analytically estimated, a bet- T − T 
i N
ter approach would be to use a two-level optimization to generate, Ti = T0 − i · (A.2)
not only the optimal job-maintenance sequence, but also the op- N
timal number of PMs. Another limit of the model is that a single For the sake of clarity, the full SA Pseudo-Code is reported in
level of planned maintenance is considered because, anytime PMs Table A.1
are executed, the hazard rate of the machine is brought back to its Also, neighbor solutions are generated using the pairwise inter-
original level (i.e., perfect repair model). However, a more realistic change operator i.e., two locations i and j are randomly selected
scenario could include different levels of maintenance tasks, each and the corresponding elements xi and xj are swapped: xi ← xj
one characterized by different costs and times and capable to re- and xj ← xi
duce (without necessarily reset) the aging and/or the degradation Concerning the initial and ending temperatures, their
of the machine. Lastly, we note that, although the single machine values are set accordingly to the classical approach by
scheduling problem is a rather flexible model, as it can be applied Kirkpatrick et al. (1983). Specifically, T0 is chosen to assure a
to a variety of common production systems, it could also be ex- desired (high) Initial Acceptance Probability (IAP) computed as
tended to other common layouts, such as parallel machines, flow in Eq. (A.3), where  f is the average variation in the objective
shops or even job shops. All these issues could be topics for future function when a neighbor is generated.
researches.
f
T0 = (A.3)
ln(IAP )
Authors contributions section
Please note that Eq. (A.3) can be used to compute TN too, by
F.Zammori and D.Mezzogori conceived the presented idea and simply substituting IAP with the desired (low) Ending Acceptance
curated the development of the CS-MNM algorithm. M.Bertolini Probability (EAP).

Table A.1
SA by Ishibuchi et al. (1995).

1 Set N, T0 , TN # Number of Iterations, Initial and Ending Temp.

2 Set K # Number of Solutions explored for each level of T
3 Randomly generate an initial solution y
4 x, x∗ ← y # Current and Best Solution
5 x, x∗ ← f(y) # Best and Current Objective Function
6 i←0
7 Do While T0 < TN
8 Generate K new neighbors of x {y1 ,…, yK }
9 y ← min f (yk ) # Best OF, among the newly generated solutions
k
10 y ← yk : f(yk ) = y # Best of the K generated solutions
11 Accept y accordingly to Eq. (a.1)
12 If Accepted (y) Then: x ← y, x ← y
13 If y < x∗ Then: x∗ ← y, x∗ ← y
14 i ← i+K
15 Update Ti as in Eq. (a.2) # Temperature is linearly reduced
16 Loop
17 Return x∗ , x∗
 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136 135

Appendix B. HS/4/IJMS Pseudo-Code

All modifications (made to the original MHS algorithm) are Cui, W., & Lu, Z. (2017). Minimizing the makespan on a single machine with flexi-
ble maintenances and jobs’ release dates. Computers & Operations Research, 80,
clearly defined in the HS/4/IJMS pseudocode.
11–22.
Specifically: Cui, W., Lu, Z., Li, C., & Han, X. (2018). A proactive approach to solve integrated
production scheduling and maintenance planning problem in flow shops. Com-
- Line #9 is used to dynamically compute the value of LPRR, puters & Industrial Engineering, 115, 342–353.
El-Khoukhi, F., Boukachour, J., & Alaoui, A. L. H. (2017). The ‘‘Dual-Ants
based on the saturation of the Harmony Memory, which will
Colony”: A novel hybrid approach for the flexible job shop scheduling prob-
trigger either the original Harmony Memory Consideration or lem with preventive maintenance. Computers & Industrial Engineering, 106, 236–
the Large Portion Recovery; 255.
- Line #18 introduces the Block Shifting operator; Geem, Z., Kim, J., & Loganathan, G. (2001). A new heuristic optimization algorithm:
Harmony search. Simulation, 76(2), 60–68.
- Lines from #21 to #32 specify how the Large Portion Recovery Gen, M., & Cheng, R. (20 0 0). Genetic algorithms and engineering optimization. New
works. Specifically, lines #29 to #32aim to minimize the satu- York, USA: John Wiley & Sons.
ration of the Harmony Memory, by replacing all repeated solu- Ishibuchi, H., Misaki, S., & Tanaka, H. (1995). Modified simulated annealing algo-
rithms for the flow shop sequencing problem. European Journal of Operational
tions. Research, 81(2), 388–398.
Kirkpatrick, S., Gelatt, C., & Vecchi, M. (1983). Optimization by simulated annealing.
Note that standard HS corresponds to lines #10 and line #20, Science, 220(4598), 671–680.
except for line #18. Komaki, G., Teymourian, E., Kayvanfar, V., & Booyavi, Z. (2017). Improved discrete
cuckoo optimization algorithm for the three-stage assembly flowshop schedul-
ing problem. Computers & Industrial Engineering, 105, 158–173.
References Lee, J., Bagheri, B., & Kao, H. A. (2015). A cyber-physical systems architecture
for industry 4.0-based manufacturing systems. Manufacturing Letters, 3, 18–
23.
Allahverdi, A. (2015). The third comprehensive survey on scheduling problems with Lee, W. C., & Wang, J. Y. (2017). A three-agent scheduling problem for minimiz-
setup times/costs. European Journal of Operational Research, 377(2), 345–378. ing the makespan on a single machine. Computers & Industrial Engineering, 106,
Cui, W., Lu, Z., & Pan, E. (2014). Integrated production scheduling and maintenance 147–160.
policy for robustness in a single machine. Computers & Operations Research, 47,
81–91.
136 M. Bertolini, D. Mezzogori and F. Zammori / Expert Systems With Applications 122 (2019) 118–136

Low, C., Hsu, C., & Su, C. (2010). A modified particle swarm optimization algorithm Salmasnia, A., & Mirabadi-Dastjerd, D. (2017). Joint production and preventive main-
for a single-machine scheduling problem with periodic maintenance. Expert Sys- tenance scheduling for a single degraded machine by considering machine fail-
tem with Applications, 37, 6429–6434. ures. TOP, 25, 544–578.
Lu, Z., Cui, W., & Han, X. (2015). Integrated production and preventive maintenance Soltani, R., Jolai, F., & Zandieh, M. (2010). Two robust meta-heuristics for scheduling
scheduling for a single machine with failure uncertainty. Computers & Industrial multiple job classes on a single machine with multiple criteria. Expert Systems
Engineering, 80, 236–244. with Applications, 37(8), 5951–5959.
Luo, W., Cheng, T. C. E., & Ji, M. (2015). Single-machine scheduling with a variable Tan, K. C. T., Narasimhan, R., Rubin, R. A., & Ragatz, G. L. (20 0 0). A comparison
maintenance activity. Computers & Industrial Engineering, 79, 168–174. of four methods for minimizing total tardiness on a single processor with se-
Macchi, M., Roda, I., & Fumagalli, L. (2017). On the advancement of maintenance man- quence dependent setup times. Omega, 28, 313–326.
agement towards smart maintenance in manufacturing (pp. 383–390). IFIP Ad- Touat, M., Bouzidi-Hassini, S., Benbouzid-Sitayeb, F., & Benhamou, B. (2017). A
vances in Information and Communication Technology. hybridization of genetic algorithms and fuzzy logic for the single-machine
Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., & Teller, E. (1953). Equa- scheduling with flexible maintenance problem under human resource con-
tion of State Calculations by Fast Computing Machines. The Journal of Chemical straints. Applied Soft Computing, 59, 556–573.
Physics, 21(6), 1087–1092. Woo, Y., & Kim, B. (2018). Methaeuristic approaches for parallel machine scheduling
Mokhtari, H., & Dadgar, M. (2015). Scheduling optimization of a stochastic flexible problem with time-dependent deterioration and multiple rate-modifying activ-
job-shop system with time-varying machine failure rate. Computers & Opera- ities. Computers & Operations Research, 95, 97–112.
tions Research, 61, 31–45. Yang, X.-S. (2010). Nature-inspired metaheuristic algorithms (2nd Ed.). Frome, UK: Lu-
Mor, B., & Mosheiov, G. (2015). Scheduling a deteriorating maintenance activity and niver Press.
due-window assignment. Computers & Operations Research, 57, 33–40. Yang, X.-S., & Deb, S. (2009). Cuckoo search via Lévy flights. In Proceeding of
Mosheiov, G., & Sarig, A. (2009). Scheduling a maintenance activity and due-win- World Congress on Nature & Biologically Inspired Computing, December 2009
dow assignment on a single machine. Computers & Operations Research, 36, (pp. 210–214). IEEE Publications.
2541–2545. Yazdani, M., Khalili, S. M., Babagolzadeh, M., & Jolai, F. (2017). A single machine
Nouiri, M., Bekrar, A., Jemai, A., Trentesaux, D., Ammari, C. A., & Niar, S. (2017). scheduling problem with multiple unavailability constraints: A mathematical
Two stage particle swarm optimization to solve the flexible job shop predic- model and an enhanced variable neighborhood search approach. Journal of Com-
tive scheduling problem considering possible machine breakdowns. Computers putational Design and Engineering, 4, 46–59.
& Industrial Engineering, 112, 595–606. Yu, A. J., & Seif, J. (2016). Minimizing tardiness and maintenance costs in flow shop
Ouaarab, A., Ahiod, B., & Yang, X. Y. (2014). Discrete cuckoo search algorithm for the scheduling by a lower-bound-based GA. Computers & Industrial Engineering, 97,
travelling salesman problem. Neural Computing & Application, 24, 1659–1669. 26–40.
Quan, O. Y., & Xu, H. Y. (2013). The review of the single machine scheduling problem Zammori, F., Braglia, M., & Castellano, D. (2014). Harmony search algorithm for sin-
and its solving methods. Applied Mechanics and Materials, 411, 2081–2084. gle-machine scheduling problem with planned maintenance. Computers & In-
Rahmati, S. H. A., Ahmadi, A., & Govindan, K. (2017). A novel integrated condi- dustrial Engineering, 76, 333–346.
tion-based maintenance and stochastic flexible job shop scheduling problem: Zhu, H., Li, M., & Zhou, J. (2015). Machine scheduling with deteriorating and re-
Simulation-based optimization approach. Annals of Operations Research, 1–39. source-dependent maintenance activity. Computers & Industrial Engineering, 88,
Safari, E., & Sadjadi, S. (2011). A hybrid method for flowshops scheduling with con- 479–486.
dition-based maintenance constraint and machines breakdown. Expert System
with Applications, 38, 2020–2029.