International Journal of Computers

Rodrigo E. Morales-Navarro et al. http://www.iaras.org/iaras/journals/ijc

A Parallel Variable Neighborhood Search approach for solving the

Traveling Salesman Problem
RODRIGO E. MORALES-NAVARRO1 MARCO ANTONIO CRUZ-CHAVEZ
RAFAEL RIVERA-LOPEZ2 ALINA MARTINEZ-OROPEZA
ABELARDO RODRIGUEZ-LEON2 Universidad Autónoma del Estado de Morelos
Instituto Tecnológico de Veracruz CIICAp
Departamento de Sistemas y Computación Av. Universidad 1001, Col. Chamilpa
M.A. Quevedo 2779, Col. Formando Hogar Cuernavaca, MOR
Veracruz, VER MÉXICO
MÉXICO {mcruz,alinam}@uaem.mx
1 [email protected]
2 {rrivera,arleon}@itver.edu.mx

Abstract: This paper presents a parallel Variable Neighborhood Search (pVNS) algorithm for solving instances
of the Traveling Salesman Problem (TSP). pVSN uses two parallelization levels in order to reach near-optimal
solutions for TSP instances. This parallel approach is evaluated by means of an experimental multi-clusters of
computers in which the nodes in each cluster uses several threads for calculating the path cost of a candidate
solution and a message passing based communication scheme between two clusters is implemented for sharing
their solutions. The results obtained indicate that the use of this parallelization scheme leads to an execution time
reduction of the VNS method and one near optimal solution is reached due to a best exploitation of the solution
space.

Key–Words: Variable neighborhood search, parallel algorithm, traveling salesman problem

1 Introduction eling salesman and spans over several areas of knowl-

edge including mathematics, computer science, oper-
An objective of combinatorial optimization is to im- ations research, genetics, engineering and electronics
plement better algorithms for searching an optimal so- [6]. TSP is among the NP-hard problems and many
lution to a problem inside a finite collection of can- different approaches are used to solve this problem
didate solutions [1]. This area brings together many in an acceptable time especially when the number of
computationally complex problems in which usually cities is high [7]. Within the process of finding near-
the number of solutions is very high, making it im- optimal TSP solutions, several techniques have been
practical to evaluate all of them in order to find the op- proposed that use different neighborhood structures
timal solution. The use of parallelization for solving and local search procedures ([8, 9, 10, 11, 12]). A hy-
combinatorial optimization problems is an approach brid approach that combines different neighborhood
that has gained importance in recent years [2]. Par- structures to find better solutions within an iterated lo-
allel combinatorial optimization algorithms are rais- cal search algorithm is presented in [13].
ing significant interest in science and technology since
they are able to solve complex problems in different High-performance computing (HPC) is an ap-
domains (communications, genomics, logistics and proach that has been used in recent years for solving
transportation, environment, engineering design, etc.) complex problems in science, engineering and busi-
[3]. For example, in [4] it is described an efficient load ness. In [14] it is established that HPC can be achieved
balancing strategy for grid-based branch and bound using a cluster of computers or a Grid computing or a
algorithm and in [5] a parallel genetic annealing algo- workstation personal computer with a multi-core sys-
rithm for solving the Job Shop Scheduling Problem is tem. A cluster is one single system comprised of in-
presented. terconnected computers that communicate with one
The traveling salesman problem (TSP) is per- another either via message passing or by direct inter-
haps the most well known combinatorial optimiza- node memory access using a single address space
tion problem. Applications of TSP and its variants [15]. In México the Instituto Tecnológico de Veracruz
go way beyond the route planning problem of a trav- (ITVer) and the Universidad Autónoma del Estado de

ISSN: 2367-8895 278 Volume 1, 2016

 International Journal of Computers
Rodrigo E. Morales-Navarro et al. http://www.iaras.org/iaras/journals/ijc

Morelos (UAEM) share the resources of their clusters cj,i for i, j : 1, . . . , N , otherwise the problem is re-
in an experimental multi-clusters architecture named ferred as a asymmetric TSP. Euclidean TSP is a spe-
Tarantula minigrid [16]. cial case of symmetric TSP in which the vertices are
This paper presents a parallel approach of a Vari- points in the Euclidean space and the weight on each
able Neighborhood Search (VNS) method described edge is the Euclidean distance between its endpoints.
in [17]. This parallel approach, called pVNS, per-
forms a concurrent finding of new candidate solutions
for TSP instances in a multi-clusters of computers. 3 Neighborhood structures
pVNS uses two levels of parallelization: first, a set of Any combinatorial optimization problem requires the
threads in each node of a cluster is used for calculat- use of techniques that allows a better exploitation of
ing the path cost of a TSP solution and then a message the solution space in order to reach near-optimal so-
passing mechanism for the communication between lutions. Local search techniques utilize a neighbor-
clusters is implemented. The experimental results ob- hood structure for defining a set N of candidate solu-
tained show that a reduction in the execution time of tions. Therefore, any solution s0 is directly reachable
pVNS method is produced and a substantial improve- from s through a movement ρ, thus s0 ∈ N (s). This
ment in the solution is reached. The remainder of this neighborhood structure is implemented in an iterative
paper is organized as follows: The definition of TSP, technique with the purpose of improving the quality
the neighborhood structures and the VNS method is of solutions, according to the objective function of the
presented in sections 2, 3 and 4, respectively. Section problem. A critical aspect of designing some opti-
5 describes the parallel approach for VNS proposed mization algorithms is choosing an appropriate neigh-
in this work and one analysis of the experimental re- borhood structure.
sults is provided in section 6. Finally, in section 7 the In [17] four different neighborhood structures
conclusions and future works are detailed. were applied for searching a near-optimal solution of
a TSP instance:
2 Traveling salesman problem • One adjacent pair [21, 22, 23]: One element in
position i of a solution s is randomly selected and
TSP is a classical reference in combinatorial optimiza- it is permuted with the element in position i + 1
tion and it has become a standard case for testing the of the same solution.
effectiveness of several optimization methods [18]. In • One random pair [22, 24, 25]: This structure
[19] it is established that TSP is probably the most im- performs a single permutation between two non-
portant among the combinatorial optimization prob- adjacent elements of s placed on random posi-
lems that it has served as a testbed for almost every tions i and j.
new algorithmic idea, and was one of the first opti-
mization problems conjectured to be hard in a specific • Two adjacent pairs [26]: Two elements of s in
technical sense. random positions i and j are selected and they are
interchanged with their adjacent elements: Ele-
In its mathematical formulation, cities are the ver-
ment in position i permutes with element in po-
tices of a graph K, edges of this graph represent the
sition i + 1, and element in position j permutes
opportunity of to travel between two cities, and a tour
with element in position j + 1.
is a hamiltonian cycle in K. The task in TSP is to
arrange a tour of several cities such that each city is • Two random pairs [25, 27, 28]: This structure
visited only once and the length of the tour (or some performs two permutations between four non-
other cost function) is minimized [20]. adjacent elements of s placed on random posi-
In [9] it is defined that given N cities, TSP re- tions ia , ib , ja and jb . Element in position ia per-
quires a search for a permutation π : {1, . . . , N } → mutes with element in position ib and element in
{1, . . . , N }, using a N × N cost matrix C, where ci,j position ja permutes with element in position jb .
denotes the cost of travel from city i to j, that mini-
mizes the path length in equation 1.
4 Variable Neighborhood Search
N −1
VNS is a framework for building heuristics based
X
cπ(i),π(i+1) + cπ(N ),π(1) (1)
i=1 upon systematic changes of neighborhoods both in
a descent phase, to find a local minimum, and in a
where π(i) denotes the city at i-th location in the tour. perturbation phase to escape from the corresponding
Different classes of TSP can be identified by the valley [29]. VNS is able to handle variable neigh-
properties of the cost matrix. In symmetric TSP ci,j = borhood sizes, due to the random interaction of the

ISSN: 2367-8895 279 Volume 1, 2016

 International Journal of Computers
Rodrigo E. Morales-Navarro et al. http://www.iaras.org/iaras/journals/ijc

structures during the execution time, which improves cess between clusters is implemented. For construct-
the exploitation of the solution space. In [17] the ing this parallelization scheme one node in each clus-
four different neighborhood structures previously de- ter is asigned as the master node in which the VNS
scribed have been implemented in order to search procedure is executed. Master node sends a candidate
near-optimal solutions of TSP instances using a VNS solution to each one of the other nodes in the cluster
method. Figure 1 shows the structure of this VNS (the slave nodes) in order to calculate the path costs
method. of this group of candidate solutions (figure 2). Im-
plementation details of these two parallelization levels
are described in the next paragraphs.

Figure 2: Master node sends candidate solutions and

receives the path cost of each one.

Parallelization Level 1: It is a shared memory paral-

lelization scheme in which a slave node i in the cluster
Figure 1: Flowchart of VNS method proposed in [17]. receives a candidate solution si and evaluates it. This
solution is divided into n segments and each one is
proccesed by a independent thread Tj in order to de-
termine its partial path cost f (si,Tj ) of each segment.
5 Parallel Variable Neighborhood When all threads have finished their part of the paral-
lel computation, slave node i determines the total path
Search cost f (si ) and sends it to the master node. This paral-
lelization level is depicted in figure 3.
An analysis of the execution time profile of the VNS
sequential version proposed in [17] indicates that the
determination of the path cost of each candidate solu-
tion constructed using the neighborhood structures is
the most expensive procedure since it consumes 99%
of the total execution time, then one parallel VSN
(pVNS) is implemented in order to reduce this execu-
tion time. For improving the exploitation of the search
space, pVNS uses a group of solutions instead only
one candidate solution in its iterative process.
pVNS performs a concurrent finding of new can-
didate solutions of a TSP instance in a multi-clusters
of computers in order to reach a near-optimal solu-
tion. pVNS uses two levels of parallelization: first, a
set of threads on each cluster node for calculating the
path cost of a TSP solution is used and then a mes- Figure 3: A set of threads calculates the path cost of
sage passing mechanism for the communication pro- solution si .

ISSN: 2367-8895 280 Volume 1, 2016

 International Journal of Computers
Rodrigo E. Morales-Navarro et al. http://www.iaras.org/iaras/journals/ijc

Parallelization Level 2: In order to exploit the advan- libraries for implementing parallel tasks are utilized:
tages of a parallel processing scheme, pVNS utilizes OpenMP for the parallelization level 1 and MPI for
a group of candidate solutions instead of only one as the parallelization level 2.
in the sequential version of VNS proposed in [17]. One TSP instance randomly generated for 4000
First, a master node delivers candidate solutions and cities was utilized for evaluating the pVNS perfor-
receives their path costs calculated by the slave nodes mance and the average results of 30 executions of
in the cluster (figure 2) and then updates the group the method using a group of 60 candidate solutions
of solutions comparing the path costs received. This were obtained. Due to the heterogeneity of the nodes
send-receive process with the slave nodes is repeated used in the multi-clusters architecture, tests were per-
until a stop condition is reached. Additionally, be- formed using different node configurations:
fore of delivering new candidate solutions, each mas-
ter node of the multi-clusters architecture shares its Test 1: Sequential VNS (1 quad-core node).
better solutions with the others using a message pass-
ing mechanism as is shown in figure 4. With this inter- Test 2: Sequential VNS (1 dual-core node).
change procedure only the best solutions of this dis- Test 3: pVNS, 10 processes (2 quad-core nodes, 1
tributed searching scheme is preserved as a the new dual-core node).
set of candidate solutions.
Test 4: pVNS, 10 processes (3 quad-core nodes).
Test 5: pVNS, 20 processes (5 dual-core nodes).
Test 6: pVNS, 20 processes (5 quad-core nodes).
Test 7: pVNS, 20 processes (9 dual-core nodes, 1
quad-core node).
Test 8: pVNS, 30 processes (9 dual-core nodes, 3
quad core nodes).
Figure 4: Solutions interchange between two clusters.
6.1 Execution time and total path cost
Figures 5 and 6 show the average total execution time
6 Experimental results and the average total path cost obtained applying the
eigth tests described previously. Figure 5 shows a sub-
Experimental tests of pVNS were realized in the stantial reduction in the execution time of pVNS in
Tarantula minigrid that is a multi-clusters architecture comparison with the better execution time of the se-
configured as a Virtual Private Network (VPN) using quential version of VNS. The generation of 60 candi-
OpenVPN in order to avoid the need for grid pack- date solutions using 30 processes (test number eight)
ages such that Globus or gLite 32. Basic components yields a total cost much smaller than the sequential
of Tarantula miniGrid are shown in table 1. one: the execution time was reduced approximately
Table 1: Tarantula miniGrid components. 23 times compared to the execution time of sequential
version using one quad-core node (test number two).
Software
Red Hat Enterprise Linux 4, Compiler gcc version
3.4.3, OpenMPI 1.2.8, MPICH2-1.0.8, Ganglia 3.0.6, NIS
ypserv-2.13-5, NFS nfs-utils-1.0.6-46,OpenVPN, Torque +
Maui.
Hardware
ITVer cluster: A master node with a dual-core system of
3.2 Ghz and 14 slave nodes (dual and quad core systems)
of 2.33 GHz.
UAEM cluster: A master node with a dual dual-core sys-
tem of 2.8 GHz and 18 slave nodes with a dual-core system
of 2.0 Ghz.

pVNS method is codified in C language and two Figure 5: Average total execution time of pVNS.

ISSN: 2367-8895 281 Volume 1, 2016

 International Journal of Computers
Rodrigo E. Morales-Navarro et al. http://www.iaras.org/iaras/journals/ijc

A comparative graph of several average total path

costs produced by the different node configurations is
shown in figure 6. The better average total path cost
produced by the tests with the sequential algorithm
was 187,339 (test number two), while the better av-
erage total path cost was 186,025 (test number three)
for the tests with the parallel version. Parallel versions
of VNS always produce better candidate solutions due
to the fact that they implement a better exploitation of
the search space by using a group of candidate solu-
tions instead of to use only one candidate solution in
each iteration of a sequential version of VNS.

Figure 7: Speed-up of pVNS.

allows concurrently evaluating several candidate so-

lutions. It can be seen that the use of parallelization
reduces approximately 23 times the execution time of
the algorithm.
The hybridization of OpenMP and MPI is an in-
teresting approach that merits further study. The next
steps in this work will be oriented in two directions:
First the experimental study will be improved using
several test cases obtained from the literature and then
a study about the effect of latency when the nodes of
Figure 6: Average total path costs of pVNS. different clusters are shared across the multi-clusters
architecture using MPI will be developed. Another fu-
ture work is to implement different heuristics as evo-
lutionary algorithms and swarm intelligence methods
6.2 Speedup in order to solve TSP instances in a parallel environ-
Speedup is the measure for the gain of the parallel pro- ment.
gram over the sequential version. In figure 7 is de-
picted the speedup obtained using different node con-
figurations for the pVNS method. It can be seen that References:
the speedup varies depending on the combination of [1] A. Schrijver, Combinatorial optimization: polyhedra
the nodes of the Tarantula minigrid. The speedup in- and efficiency, Springer, 2002.
crease curve turns out to be as expected, obtaining an [2] K. Fujisawa, M. Kojima, A. Takeda and M. Ya-
efficiency of 66% when the node configuration use 30 mashita, Solving Large Scale Optimization Prob-
processes (9 dual-core nodes and 3-quad core nodes). lems via Grid and Cluster Computing, Journal of the
Operations Research Society of Japan, 47(4), 2004,
pp. 265–274.
7 Conclusions and future work [3] E.G. Talbi, Parallel combinatorial optimization, Wi-
ley, 2006.
The implementation of this proposal of two paral-
[4] M. Mezmaz, N. Melab and E.G. Talbi, An efficient
lelization levels using OpenMP and MPI generates
load balancing strategy for grid-based branch and
a substantial reduction in the execution time for the
bound algorithm, Parallel computing, 33(4), 2007,
VNS method, obtaining results in less time than the
pp. 302–313.
sequential version of VNS. Parallelization with MPI
allows to expand the search in the space of solutions, [5] M.A. Cruz-Chávez, A. Rodrı́guez-León, E.Y. Ávila-
since having several candidate solutions improves the Melgar, F. Juárez-Pérez, M.H. Cruz-Rosales and
R. Rivera-Lopez, Genetic-Annealing Algorithm in
explotaition of the solution space and it is more prob-
Grid Environment for Scheduling Problems, Com-
able that a near-optimal solution can be reached.
munications in Computer and Information Science:
In addition, with the use of a multi-clusters ar- Security-Enriched Urban Computing and Smart
chitecture, a greater computing power is obtained that Grid, Springer, 2010, pp. 1–9.

ISSN: 2367-8895 282 Volume 1, 2016

 International Journal of Computers
Rodrigo E. Morales-Navarro et al. http://www.iaras.org/iaras/journals/ijc

[6] G. Gutin and A.P. Punnen, The traveling salesman [20] D.B. Fogel, An Evolutionary Approach to the Travel-
problem and its variations, Springer, 2002. ing Salesman Problem, Biological Cybernetics, 60,
[7] V. Zharfi and A. Mirzazadeh, A Novel Metaheuristic Springer, pp. 139–144, 1988.
for Travelling Salesman Problem, Journal of Indus- [21] S. Lin and W. Kernighan, An Effective Heuristic for
trial Engineering, 2013, 5 pages, 2013. the Traveling Salesman Problem, Operations Re-
[8] M. Hahsler and K. Hornik, TSP – Infrastructure for search 21(2), pp. 498–516, 1973.
the Traveling Salesperson Problem, Journal of Sta- [22] R. González-Velázquez and M.A. Bandala-Garcés,
tistical Software, (23)2, pp. 1–21, 2007. Hybrid Algorithm: Scaling Hill and Simulated An-
nealing to Solve the Quadratic Allowance Problem,
[9] D.S. Johnson and L.A. McGeoch, The Traveling
Proc. of 3th. Latin-Iberoamerican Workshop of Op-
Salesman Problem: A Case Study in Local Optimiza-
eration Research, 2009.
tion, Local Search in Combinatorial Optimization,
Wiley, pp. 215–310, 1997. [23] J. Pacheco and C. Delgado, Different Experiences
Results with Local Search Applied to Path Problem,
[10] S.B. Liu, K.M. Ng and H.L. Ong, A New Heuris- Electronic Journal of Electronics of Comunications
tic Algorithm for the Classical Symmetric Traveling and Works, 2(1), pp. 54–81, 2000.
Salesman Problem, World Academy of Science, En-
[24] C.H. Papadimitriou and K. Steiglitz, Combinatorial
gineering and Technology, 27(48), pp. 34–348 2007.
Optimization, Algorithms and Complexity, Dover,
[11] M.K. Rafsanjani, S. Eskandari and A.B. Saeid, A 1998.
similarity-based mechanism to control genetic algo- [25] D.B. Kenneth, Cost Versus Distance in the Traveling
rithm and local search hybridization to solve travel- Salesman Problem, Tech Report, UCLA Computer
ing salesman problem, Neural Computing and Ap- Science Dept, 1995.
plications, 26(1), pp. 213–222, 2015.
[26] H.R. Lourenço and O.C. Martin, Iterated Local
[12] Y. Lin, Z. Bian and X. Liu, Developing a dynamic Search, Handbook of Metaheustics, 57, Springer,
neighborhood structure for an adaptive hybrid sim- pp. 320–353, 2003.
ulated annealing–tabu search algorithm to solve the
[27] O. Martin, S.W. Otto and E.W. Felten, Large Step
symmetrical traveling salesman problem. Applied
Markov Chains for the Traveling Salesman Problem,
Soft Computing, 49, pp. 937–952, 2016.
Complex Systems, 5(3), pp. 299–326, 1991.
[13] M.A. Cruz-Chávez, A. Martı́nez-Oropeza and [28] O. Martin, S.W. Otto and E.W. Felten, Large Step
S.A. Serna-Barquera, Neighborhood Hybrid Struc- Markov Chains for the TSP Incorporating Local
ture for Discrete Optimization Problems, Proc. of Search Heuristics, Operations Reasearch Letters,
IEEE CERMA 2010, pp. 108–113, 2010. 11(4), pp. 219–224, 1992.
[14] E. Mahdi, A Survey of R Software for Parallel Com- [29] P. Hansen, N. Mladenović, R. Todosijević and
puting, American Journal of Applied Mathematics S. Hanafi, Variable neighborhood search: basics and
and Statistics, 2(4), pp. 224–230, 2014. variants, EURO Journal on Computational Opti-
[15] G. Bell and J. Gray, What’s next in high-performance mization, pp. 1–32, 2016.
computing?, Communications of the ACM, 45(2),
pp. 91–95, 2002.
[16] R. Rivera-Lopez, A. Rodrı́guez-León, M.A. Cruz-
Chávez and I.Y. Hernández-Baez, Tarántula: Una
grid de clusters de cómputo para el desarrollo de apli-
caciones paralelas y en grid en México, Memorias
del Coloquio de Investigación Multidisciplinaria,
Orizaba, MEX, 2011.
[17] M.A. Cruz-Chávez, A. Martı́nez-Oropeza, J. del Car-
men Peralta-Abarca, M.H. Cruz-Rosales and
M. Martı́nez-Rangel, Variable Neighborhood Search
for Non-deterministic Problems, ICAISC 2014 LNAI
8868, Springer, pp. 468–478, 2014.
[18] D.L. Applegate, R.E. Bixby, V. Chvátal, and
W.J. Cook, The Traveling Salesman Problem:
A Computational Study, Princeton University
Press,2006.
[19] D.S. Johnson and C.H. Papadimitriou, Computa-
tional Complexity, The Traveling Salesman Problem,
Wiley, pp. 37–85, 1985.

ISSN: 2367-8895 283 Volume 1, 2016