1

Formulations for the Traveling Salesman Problem

Drive 1 car via n cities, minimising travel. History: Dantzig, Fulkerson, Johnson paper: B&B, cutting planes. Cook, Karp, Levin: complexity theory. Some formulations for the TSP: Assignment. Assignment + tight subtours (DFJ). Assignment + loose subtours (MTZ). Symmetric. Flow (Svestka). Steps (Dantzig). Next time: solution methods. Assignment + cuts. Assignment + B&B. Min spanning tree.

The Travelling Salesman Contest

Here are the distances between 9 cities. Find the shortest tour.
1 1 2 3 4 5 6 7 8 2 41.0 3 53.0 25.8 4 5 62.9 106.3 17.1 69.6 31.9 67.3 53.9 6 86.2 60.8 72.3 44.2 37.0 7 78.2 48.2 57.9 31.2 32.5 14.5 8 74.5 46.6 58.6 29.9 37.6 14.3 5.1 9 47.7 36.4 59.3 28.8 67.0 39.5 35.0 30.2

This is all the data you need to solve the problem exactly. The problem is easy to describe, but very hard to solve exactly. A graphical display is on the next page.

90

The TSP, graphically. Best: 287.1.

5 3

60 North
2 4 8 7 6

30
9

30

East

60

90

1930s-50s: Publicised in the math & OR community by Merrill Flood.

Early history of the TSP

1948: Popularised at RAND Corp., a U.S. military thinktank, which helped developed OR.
1954: Solution of a large-scale traveling-salesman problem, Dantzig, Fulkerson & Johnson, J. of Ops Research of America. At RAND, they solved a 49-city TSP to optimality. DF&J thought a nearly optimal tour could be improved, and then optimality could be guaranteed, by adding just a few cuts. DF&J were the first to use cuts and B&B for integer programming.

1971 Cook, 1972 Karp, 1973 Levin: computational complexity. How hard is a problem to solve, as a function of the input data? If Problem 1 converts easily to Problem 2 & vice versa, then Problem 1 is as easy or hard as Problem 2.
Some problems are easy: shortest path, min spanning tree, assignment. impossible: given a computer program & input, will it stop? Stephen Cook hard: TSP because no good algorithms have been found. Many hard problems, such as job shop scheduling, can be converted algebraically to & from the TSP. A good TSP algorithm will be good for other hard problems. The TSP is easy to state, takes no math background to understand, and no great talent to find feasible solutions. Its fun, and invites recreational problem solvers.

Why is the TSP important?

2 general strategies for the TSP

Given that the TSP is hard, what can we do? 1) Approximate. Use heuristics! Nearest neighbour, nearest merger, Love & Norback (angles), Lin-Kernighan (3-opt), min spanning tree + shortcuts (Christofides), and many others. 2) Spend a lot of time at it. Enumeration! Cutting planes, branch & bound, dynamic programming.

Well do (2) here. Next: formulations for the TSP.

Assignment problem - easy, naturally integer.

Indices: Parameters: Variables: Model AP: i = teacher, j = course. cij = value if teacher i is assigned to course j. xij = 1 if teacher i is assigned to course j, else 0. 1) Max i j cijxij subject to 2) j xij = 1, for all i, 3) i xij = 1, for all j, 4) xij {0,1}, for all i,j.

Explanation: 1) Maximise value of assignments. 2) Assign each teacher i to one course. 3) Assign each course j to one teacher.

Almost the TSP. Is AP a possible formulation for the TSP? Indices: i, j = city. Parameter: cij = cost to go from city i to city j. Variables: xij = 1 if we drive from city i to city j, else 0.

The AP formulation wont work.

WhatsBest formulation. Whats wrong? At least we have a lower bound of 0.

So just leave out variables xii.

I reset the variables on the diagonal so they are fixed to 0.
Looks good. Or does it?

A better lower bound of 279.4.

We have subtours.
3 1

10

2
4 5

8
7

Oops. How do we get rid of these? 1. Can you write AMPL code to print the tour from an optimal solution? 2. Can you write AMPL code to print a subtour of an infeasible solution? 3. Do we need an integer solution to find subtours?

3 The Dantzig, Fulkerson & Johnson (DFJ) model. Indices, parameters, & decision variables as before. 2 Minimise total cost: min ij cij xij , Enter each city once: i xij = 1 for all j. 4 Leave each city once: j xij = 1 for all i. Subtour breaking constraints: i,jS xij |S| 1, for every subset S. Binary integrality: xij {0, 1} for all i, j.

Ways to break subtours:

2n

subtour constraints.

11

For the subtour shown, add: x3,2 + x2,4 + x4,3 2. What are the others? After solving again with the new constraints, more subtours appear. For a large TSP, we may need many subtour breaking constraints. In the worst case, we may need 2n subtour breaking constraints. Next week, we will see a way to generate these constraints. The solution becomes fractional, so we also need to do B&B. However, every solution gives a lower bound on the optimum.

Ways to break subtours: MTZ model

Indices & parameters as before. Variables: xij = 1 if we drive from city i to city j, else 0. ui = number of cities visited at city i.

12

3 2

Minimise total cost: min ij cij xij , 4 Enter each city once: i xij = 1 for all j. Leave each city once: j xij = 1 for all i. Subtour breaking: ui + 1 uj + n(1 xij), for i = 2, , n, i j, j = 2,,n, xij {0, 1} for all i, j; ui 0 for all i. Fewer constraints, but harder to solve! The LP relaxation is not as tight. Okay for small problems, but is bad for large ones. Related variations are a bit tighter.
Ref: C. E. Miller, A. W. Tucker, and R. A. Zemlin, Integer programming formulations and traveling salesman problems, J. ACM, 7 (1960), pp. 326329.

13

Well use this data for our AMPL models.

param N := 12;
param C 1 1 . 2 26 3 40 4 34 5 35 6 48 7 30 8 23 9 48 10 17 11 20 12 29 := [*,*] : 2 3 4 26 40 34 . 64 61 64 . 3 61 3 . 59 5 3 78 8 10 54 10 8 47 15 13 50 24 26 29 30 28 28 33 30 33 39 35 5 35 59 5 3 . 13 5 10 29 25 28 33 6 48 78 8 10 13 . 18 23 16 38 39 33 7 30 54 10 8 5 18 . 5 34 20 23 28 8 23 47 15 13 10 23 5 . 38 15 18 23 9 48 50 24 26 29 16 34 38 . 23 21 16 10 17 29 30 28 25 38 20 15 23 . 3 8 11 20 28 33 30 28 39 23 18 21 3 . 5 12:= 29 33 39 35 33 33 28 23 16 8 5 .;

MTZ formulation in AMPL, ~6 secs.

param N integer > 2; # Number of nodes set Nodes ordered := {1..N}; set Arcs := {i in Nodes, j in Nodes: i <> j}; param C{(i,j) in Arcs}; var x {(i,j) in Arcs} binary; var u {Nodes} >= 0; minimize Tourlength: sum {(i,j) in Arcs} C[i,j]*x[i,j]; subject to Degree1 {i in Nodes}: sum{(i,j) in Arcs} x[i,j]=1; subject to Degree2 {i in Nodes}: sum{(j,i) in Arcs} x[j,i]=1;

14

subject to NoSubtour1 {(i,j) in Arcs: i<>j and i>=2 and j>=2}: u[i] - u[j] + N*x[i,j] <= N - 1; subject to NoSubtour2 {i in Nodes: i >= 2}: u[i] <= N - 1 - (N - 2)*x[1,i]; subject to NoSubtour3 {i in Nodes: i >= 2}: u[i] >= 1 + (N - 2)*x[i,1];

AMPL solution
u[1] u[4] u[12] u[6] u[2] u[11] u[9] u[3] u[7] u[10] u[5] u[8] 0 1 2 3 4 5 6 7 8 9 10 11 1,4 4,12 12,6 6,2 2,11 11,9 9,3 3,7 7,10 10,5 5,8 8,1

min 500 x1,1 + 70.9 x1,2 + 41.6 x1,3 + 29.1 x1,4 + 17.1 x1,5 + 56.1 x1,6 + 55.6 x1,7 + 7.3 x1,8 ... + 104 x12,10 + 39.4 x12,11 + 500 x12,12 subject to 3] x2,1 + x3,1 + x4,1 + x5,1 + x6,1 + x7,1 + x8,1 + x9,1 + x10,1 + x11,1 + x12,1 = 1 4] x1,2 + x1,3 + x1,4 + x1,5 + x1,6 + x1,7 + x1,8 + x1,9 + x1,10 + x1,11 + x1,12 = 1 ... 25] x1,12 + x2,12 + x3,12 + x4,12 + x5,12 + x6,12 + x7,12 + x8,12 + x9,12 + x10,12 + x11,12 = 1 26] x12,1 + x12,2 + x12,3 + x12,4 + x12,5 + x12,6 + x12,7 + x12,8 + x12,9 + x12,10 + x12,11 = 1 27] 12 x2,2 <= 11 28] 12 x2,3 + u2 - u3 <= 11 29] 12 x2,4 + u2 - u4 <= 11 30] 12 x2,5 + u2 - u5 <= 11 If we go from city 2 to ... city 5, then u2 + 1= u5. 143] 12 x12,8 - u8 + u12 <= 11 144] 12 x12,9 - u9 + u12 <= 11 145] 12 x12,10 - u10 + u12 <= 11 146] 12 x12,11 - u11 + u12 <= 11 147] 12 x12,12 <= 11 end inte x1,1 inte x1,2 ...

15

Parts of the MTZ model in Lindo

The symmetric TSP

Symmetric TSP: cij = cji.

16

1
$3

$3

Indices: i, j = city. 9 Parameter: cij = cost to go from city i to city j. Variables: xij = 1 if we drive from city i to city j, else 0, defined only for i<j. Half as many variables as the asymmetric! Minimise total cost: Enter each city once: Subtour breaking: Binary integrality: min ij>i cij xij , j<i xji + j>i xij = 2 for all i. i,jS xij |S| 1, for each subset S. xij {0, 1} for all i, j. 1 $8 $3 9

The homework is a symmetric TSP.

The asymmetric TSP, cij cji, is more realistic. Why?

How does the row 2 summation work?

Model: 1. Min ni=1 nj=i+1 ci,jxi,j, 2. i=1j1xi,j + ni=j+1xj,i = 2, for all j. 3. i,jS xi,j |S| 1, for every subset S, 4. xi,j{0,1} for all i,and j: j>i.

17

The variables into city 5 are: x15, x25, x35, x45, x65, x75, x85, x95. The variables out of city 5 are: x51, x52, x53, x54, x56, x57, x58, x59. Since costs are symmetric, cij = cji, let's drop half the variables. For xij, require i < j. Allow only the variables going out. We need only variables x15, x25, x35, x45, x56, x57, x58, x59. The meaning is not Go in or come out, but use this arc. The summation makes sure that we cover only the variables we need. x15 + x25 + x35 + x45 + x56 + x57 + x58 + x59 = 2.

A flow with gain model of the TSP

Parameter: Decision variables: f = gain in flow from city i to city j. xij = 1 if we drive from city i to city j, else 0. yij = flow from city i to city j.

18

1. Min total cost: min ij cij xij , 2. Arrive in each city: ji yji 1, for i = 2,,n, 3. Flow with gain of f: ji yij ji yji = f, for i = 1,,n, 4. Only n positive vars: ij xij n, cardinality constraint, 5. Force the yij vars: yij (1 + nf)xij for all i, j, 6. Nonnegative, binary: yij 0 for all i, j, xij {0, 1} for all i, j. This formulation flows f material from city to city. As f approaches 0, the solution approaches the optimum. Hard to solve because of (4), the cardinality constraint. Other rows can be added to tighten the model.
Ref: JA Svestka, A continuous variable representation of the TSP, Math Prog, v15, 1978, pp211-213.

# TSP, Svestka formulation. Ref: JA Svestka, A continuous variable # representation of the TSP, Math Prog, v15, 1978, pp211 -213. param N integer > 2; # Number of nodes set Nodes ordered := {1..N}; set Arcs := {i in Nodes, j in Nodes: i <> j};

19

param C{(i,j) in Arcs}; param F := 0.01; var x {(i,j) in Arcs} binary;

var y {(i,j) in Arcs} >= 0;

TSP gain model in AMPL. About 7 secs.

in Arcs} y[j,i] >= 1;
y[j,i] = F; N; + N*F)*x[i,j];

minimize Tourlength: sum {(i,j) in Arcs} C[i,j] * x[i,j];

subject to subject to sum {(i,j) subject to subject to

Demandy {i in Nodes: i>=2}: sum{(i,j) Flowy {i in Nodes: i>=2}: in Arcs} y[i,j] - sum {(i,j) in Arcs} Totalx: sum {(i,j) in Arcs} x[i,j] <= Arcgain {(i,j) in Arcs}: y[i,j] <= (1

# These rows tighten the model. subject to Demandx {i in Nodes: i>=2}: sum{(i,j) in Arcs} x[j,i] = 1; subject to Flowx {i in Nodes: i >= 2}: sum {(i,j) in Arcs} x[i,j] = sum {(i,j) in Arcs} x[j,i]; subject to Startx: sum {i in Nodes: i >= 2} x[1,i] = 1; subject to Starty: sum {i in Nodes: i >= 2} y[1,i] = 1;

y1,8 y8,7 y7,5 y5,4 y4,3 1.00 1.01 1.02 1.03 1.04 x1,8 x8,7 x7,5 x5,4 x4,3

y3,6 1.05 x3,6

y6,9 y9,12 y12,11 y11,10 y10,2 1.06 1.07 1.08 1.09 1.1 x6,9 x9,12 x12,11 x11,10 x10,2

AMPL solution

y2,1 1.11 x2,1

A formulation of steps for the TSP, Dantzig

Indices: i, j, k = city, t = step. Variables: xijt = 1 if we drive from city i to city j at step t, else 0.

20

Minimise total cost: min ijt cij xijt , Flow in = flow out: i xi,j,t k xj,k,t+1 = 0 for all j, and t=1,,n. Go to each city once: jt xijt = 1, for i=1,,n. Binary integrality: xijt {0, 1} for all i, j, t.
This is tighter than the earlier formulations. Unfortunately this requires n3 variables. A 50-city problem would require 125,000 variables! Next: solution methods. Assignment + cuts. Assignment + B&B. Min spanning tree.

21

# TSP steps. Ref Linear Progg & Extensions, Dantzig, 1963; param N integer > 2; # Number of nodes set Nodes ordered := {1..N}; set Arcs := {i in Nodes, j in Nodes, k in Nodes: i <> j};
param C{i in Nodes, j in Nodes: i <> j}; var x {(i,j,k) in Arcs} binary;

TSP step model in AMPL. About 7 secs.

minimize Tourlength: sum{(i,j,k) in Arcs: i<>j} C[i,j]*x[i,j,k]; subject to Demand {i in Nodes}: sum{(i,j,k) in Arcs} x[i,j,k] = 1; subject to End {i in Nodes}: sum {j in Nodes: j <> i} x[j,i,N] = sum {k in Nodes: k <> i} x[i,k,1]; subject to Step {t in Nodes, j in Nodes: t <= N - 1}: sum {i in Nodes: i <> j} x[i,j,t] = sum {k in Nodes: k <> j} x[j,k,t+1];

8,1,1 1,2,2 2,10,3 10,11,4 11,12,5

AMPL solution
12,9,6

9,6,7 6,3,8 3,4,9 4,5,10 5,7,11 7,8,12