User:LIU CS MUN/draft of the page for Held-Karp algorithm

The Held-Karp algorithm, also called Bellman-Held-Karp algorithm, is a dynamic programming algorithm proposed in 1962 independently by Bellman and by Held and Karp to solve the Traveling Salesman Problem (TSP). TSP is an extension of the [Hamiltonian circuit problem]. The problem can be described as: find a tour of N cities in a country (assuming all cities to be visited are reachable), the tour should (a) visit every city just once, (b) return to the starting point and (c) be of minimum distance. Broadly, the TSP is classified as symmetric travelling salesman problem (sTSP), asymmetric travelling salesman problem (aTSP), and multi-travelling salesman problem (mTSP).The mTSP is generally treated as a relaxed vehicle routing problem.

Graph Model
sTSP: Let V = {v1 ,......, vn } be a set of cities, E = {( r, s ) : r , s ∈ V } be the edge set, and drs = dsr be a cost measure associated with edge ( r , s ) ∈ E.

aTSP: If drs ≠ dsr for at least one ( r, s ) then the sTSP becomes an aTSP. The aTSP and sTSP are defined on different graphs – complete directed and undirected. sTSP can be considered, in many cases, as a subproblem of the aTSP.

mTSP: The mTSP is defined as: In a given set of nodes, let there are m salesmen located at a single depot node. The remaining nodes (cities) that are to be visited are intermediate nodes. Then, the mTSP consists of finding tours for all m salesmen, who all start and end at the depot, such that each intermediate node is visited exactly once and the total cost of visiting all nodes is minimized.

Description
Below is the dynamic programming procedure:

There is an optimization property for TSP: Every subpath of a path of minimum distance is itself of minimum distance.

Compute the solutions of all subproblems starting with the smallest. Whenever computing a solution requires solutions for smaller problems using the above recursive equations, look up these solutions which are already computed. To compute a minimum distance tour, use the final equation to generate the lst node, and repeat for the other nodes. For this problem, we cannot know which subproblems we need to solve, so we solve them all.

Recursive formulation
Number the cities 1, 2,. . ., N and assume we start at city 1, and the distance between city i and city j is dij .Consider subsets S ⊆ {2,. . ., N} of cities and, for c ∈ S, let D(S, c) be the minimum distance, starting at city 1, visiting all cities in S and finishing at city c.

First phase: if S = {c}, then D(S, c) = d1,c. Otherwise: D(S, c) = minx∈S−c (D(S − c, x) + dx,c )

Second phase: the minimum distance for a complete tour of all cities is M = minc∈{2,...,N} (D({2,. . ., N}, c) + dc,1 )

A tour n1,. . . nN is of minimum distance just when it satisfies M = D({2, . . ., N}, nN ) + dn N,1.

Example
Distance matrix: $$C=

\begin{pmatrix} 0 & 2 & 9 & 10 \\ 1 & 0 & 6 & 4 \\ 15 & 7 & 0 & 8 \\ 6 & 3 & 12 & 0 \end{pmatrix}$$

g(2, ∅ ) = c21 = 1 g(3, ∅ ) = c31 = 15

g(4, ∅ ) = c41 = 6

k = 1, consider sets of 1 element: Set {2}: g(3,{2}) = c32 + g(2, ∅ ) = c32 + c21 = 7 + 1 = 8      p(3,{2}) = 2 g(4,{2}) = c42 + g(2, ∅ ) = c42 + c21 = 3 + 1 = 4      p(4,{2}) = 2 Set {3}: g(2,{3}) = c23 + g(3, ∅ ) = c23 + c31 = 6 + 15 = 21    p(2,{3}) = 3 g(4,{3}) = c43 + g(3, ∅ ) = c43 + c31 = 12 + 15 = 27   p(4,{3}) = 3 Set {4}: g(2,{4}) = c24 + g(4, ∅ ) = c24 + c41 = 4 + 6 = 10     p(2,{4}) = 4 g(3,{4}) = c34 + g(4, ∅ ) = c34 + c41 = 8 + 6 = 14     p(3,{4}) = 4 k = 2, consider sets of 2 elements: Set {2,3}: g(4,{2,3}) = min {c42 + g(2,{3}), c43 + g(3,{2})} = min {3+21, 12+8}= min {24, 20}= 20 p(4,{2,3}) = 3 Set {2,4}: g(3,{2,4}) = min {c32 + g(2,{4}), c34 + g(4,{2})} = min {7+10, 8+4}= min {17, 12} = 12 p(3,{2,4}) = 4 Set {3,4}: g(2,{3,4}) = min {c23 + g(3,{4}), c24 + g(4,{3})} = min {6+14, 4+27}= min {20, 31}= 20 p(2,{3,4}) = 3

Length of an optimal tour:

f = g(1,{2,3,4}) = min { c12 + g(2,{3,4}), c13 + g(3,{2,4}), c14 + g(4,{2,3}) } = min {2 + 20, 9 + 12, 10 + 20} = min {22, 21, 30} = 21

Successor of node 1: p(1,{2,3,4}) = 3

Successor of node 3: p(3, {2,4}) = 4

Successor of node 4: p(4, {2}) = 2

Optimal TSP tour: 1 → 3 → 4 → 2 → 1

Pseudocode
function algorithm T SP (G, n)             for k := 2 to n do                 C({1, k}, k) := d1,k end for for s = 3 to n do                for all S ⊆ {1, 2,. . ., n}, |S| = s do                    for all k ∈ S do                        {C(S, k) = minm≠1,m≠k,m∈S [C(S − {k}, m) + dm,k ]} end for end for end for opt := mink≠1 [C({1, 2, 3, . . ., n}, k) + d1,k] return (opt) end

Exhaustive enumeration
This brute-force method starting at any city, enumerate all possible permutations of cities to visit, and find the distance of each permutation and choose one of minimum distance. The total number of possible routes covering all N cities can be given as (N − 1)! and (N − 1)!/2 in aTSP and sTSP respectively. Stirling's approximation: $$N! \approx \sqrt{2 \pi N} ({N \over e})^N$$

Dynamic programming approach
Faster than the exhaustive enumeration but still exponential, and the drawback of this algorithm, though, is that it also uses a lot of space: the worst-case time complexity of this algorithm is $$O(2^n n^2)$$ and the space $$O(2^n n)$$.

Time: the fundamental operations employed in the computation are additions and comparisons. The number of each in the first phase is given by $$(\sum_{k=2}^{n-1}k(k-1)\binom{n-1}{k})+(n-1)-(n-1)(n-2)2^{n-3}+(n-1)$$

and the number of occurrence of each in the second phase is $$\sum_{k=2}^{n-1}k={n(n-1) \over 2} -1$$

Space: $$(\sum_{k=2}^{n-1}k\binom{n-1}{k})+(n-1)=(n-1)2^{n-2}$$

Approximation
Solving even moderate size of the TSP optimally takes huge computational time, therefore there is a room for the development and application of approximate algorithms, or heuristics. The approximate approach never guarantee an optimal solution but gives near optimal solution in a reasonable computational effort. So far, the best known approximate algorithm available is O(n( log2 n )O(c)) where n is problem size of TSP.

Proof of correctness
Algorithms for optimization problems require proof that they always return the best possible solution. Dynamic programming algorithms are only as correct as the recurrence relations they are based on.

Guideline to implement Dynamic Programming 1. Characterize the recursive structure of an optimal solution, 2. define recursively the value of an optimal solution, 3. compute, bottom-up, the cost of a solution, 4. construct an optimal solution. Dynamic programming can be applied to any problem that observes the principle of optimality.Dynamic programming is a technique for efficiently implementing a recursive algorithm by storing partial results as long as the naive recursive algorithm computes the same subproblems over and over again. TSP has such a property which make it possible to be solved by dynamic programming.

Drilling of printed circuit boards
A direct application of the TSP is in the drilling problem of printed circuit boards (PCBs). To connect a conductor on one layer with a conductor on another layer, or to position the pins of integrated circuits, holes have to be drilled through the board. The holes may be of different sizes. To drill two holes of different diameters consecutively, the head of the machine has to move to a tool box and change the drilling equipment. This is quite time consuming. Thus it is clear that one has to choose some diameter, drill all holes of the same diameter, change the drill, drill the holes of the next diameter, etc. Thus, this drilling problem can be viewed as a series of TSPs, one for each hole diameter, where the 'cities' are the initial position and the set of all holes that can be drilled with one and the same drill. The 'distance' between two cities is given by the time it takes to move the drilling head from one position to the other. The aim is to minimize the travel time for the machine head.

Vehicle routing
Suppose that in a city n mail boxes have to be emptied every day within a certain period of time, say 1 hour. The problem is to find the minimum number of trucks to do this and the shortest time to do the collections using this number of trucks. As another example, suppose that n customers require certain amounts of some commodities and a supplier has to satisfy all demands with a fleet of trucks. The problem is to find an assignment of customers to the trucks and a delivery schedule for each truck so that the capacity of each truck is not exceeded and the total travel distance is minimized. Several variations of these two problems, where time and capacity constraints are combined, are common in many real-world applications. This problem is solvable as a TSP if there are no time and capacity.

Other applications
Other applications such as X-Ray crystallography, Computer wiring, The order-picking problem in warehouses, Mask plotting in PCB production are in the category of sTSP while Printing press scheduling problem, School bus routing problem, Crew scheduling problem, Interview scheduling problem, Hot rolling scheduling problem, Mission planning problem and Design of global navigation satellite system surveying networks can be modeled as mTSP.

Precise algorithm for solving the TSP
Besides Dynamic Programming, Linear programming and Branch-bound algorithm are precise algorithms that can solve TSP. Linear programming applies to the cutting plane method in the integer programming, i.e. solving the LP formed by two constraints in the model and then seeking the cutting plane by adding inequality constraint to gradually converge at an optimal solution. When people apply this method to find a cutting plane, they often depend on experience. So this method is seldom deemed as a general method.

Branch-bound algorithm is a search algorithm widely used by people though it's not good for solving the large-scale problem. It controls the searching process through effective restrictive boundary so that it can search for the optimal solution branch from the space state tree to find an optimal solution as soon as possible. The key point of this algorithm is the choice of the restrictive boundary. Different restrictive boundaries may form different branch-bound algorithms. Branch-bound algorithm.

Approximate algorithm for solving the TSP
As the application of precise algorithm to solve problem is very limited, we often use approximate algorithm or heuristic algorithm. The result of the algorithm can be assessed by C / C* ≤ ε. C is the total travelling distance generated from approximate algorithm; C* is the optimal travelling distance; ε is the upper limit for the ratio of the total travelling distance of approximate solution to optimal solution under the worst condition. The value of ε >1.0. The more it closes to 1.0, the better the algorithm is. These algorithms include: Interpolation algorithm, Nearest neighbour algorithm, Clark & Wright algorithm, Double spanning tree algorithm, Christofides algorithm, Hybrid algorithm, Probabilistic algorithm.