Brute Force Algorithms

Brute Force Algorithms refers to a programming style that does not include any shortcuts to improve
performance, but instead relies on sheer computing power to try all possibilities until the solution to a problem
is found.

A classic example is the traveling salesman problem (TSP). Suppose a salesman needs to visit 10 cities across
the country. How does one determine the order in which cities should be visited such that the total distance
traveled is minimized? The brute force solution is simply to calculate the total distance for every possible route
and then select the shortest one. This is not particularly efficient because it is possible to eliminate many
possible routes through clever algorithms.

Another example: 5 digit password, in the worst case scenario would take 105 tries to crack.

The time complexity of brute force is O(n*m) . So, if we were to search for a string of ‘n’ characters in a string
of ‘m’ characters using brute force, it would take us n * m tries.

Brute-Force Search

Brute force search is the most common search algorithm as it does not require any domain knowledge, all that is
required is a state description, legal operators, the initial state and the description of a goal state. It does not
improve the performance and completely relies on the computing power to try out possible combinations.

The brute force algorithm searches all the positions in the text between 0 and n-m whether the occurrence of the
pattern starts there or not. After each attempt, it shifts the pattern to the right by exactly 1 position. The time
complexity of this algorithm is O(m*n). so if we are searching for n characters in a string of m characters then it
will take n*m tries.

Suppose a salesman needs to travel 10 different cities in a country and he wants to determine the shortest
possible routes out of all the possible combinations. Here brute force algorithm simply calculates the distance
between all the cities and selects the shortest one.

Greedy Algorithm

Among all the algorithmic approaches, the simplest and straightforward approach is the Greedy method. In this
approach, the decision is taken on the basis of current available information without worrying about the effect
of the current decision in future.

Greedy algorithms build a solution part by part, choosing the next part in such a way, that it gives an immediate
benefit. This approach never reconsiders the choices taken previously. This approach is mainly used to solve
optimization problems. Greedy method is easy to implement and quite efficient in most of the cases. Hence, we
can say that Greedy algorithm is an algorithmic paradigm based on heuristic that follows local optimal choice at
each step with the hope of finding global optimal solution.

In many problems, it does not produce an optimal solution though it gives an approximate (near optimal)
solution in a reasonable time.

Components of Greedy Algorithm

Greedy algorithms have the following five components −

  A candidate set − A solution is created from this set.
 A selection function − Used to choose the best candidate to be added to the solution.
 A feasibility function − Used to determine whether a candidate can be used to contribute to the
solution.
 An objective function − Used to assign a value to a solution or a partial solution.
 A solution function − Used to indicate whether a complete solution has been reached.

Areas of Application

Greedy approach is used to solve many problems, such as

 Finding the shortest path between two vertices using Dijkstra’s algorithm.
 Finding the minimal spanning tree in a graph using Prim’s /Kruskal’s algorithm, etc.

Where Greedy Approach Fails

In many problems, Greedy algorithm fails to find an optimal solution, moreover it may produce a worst
solution. Problems like Travelling Salesman and Knapsack cannot be solved using this approach.

0-1 Knapsack

In 0-1 Knapsack, items cannot be broken which means the thief should take the item as a whole or should leave
it. This is reason behind calling it as 0-1 Knapsack.

Hence, in case of 0-1 Knapsack, the value of xi can be either 0 or 1, where other constraints remain the same.

0-1 Knapsack cannot be solved by Greedy approach. Greedy approach does not ensure an optimal solution. In
many instances, Greedy approach may give an optimal solution.

The following examples will establish our statement.

Example-1

Let us consider that the capacity of the knapsack is W = 25 and the items are as shown in the following table.

Item A B C D
Profit 24 18 18 10
Weight 24 10 10 7

Without considering the profit per unit weight (pi/wi), if we apply Greedy approach to solve this problem, first
item A will be selected as it will contribute maximum profit among all the elements.

After selecting item A, no more item will be selected. Hence, for this given set of items total profit is 24.
Whereas, the optimal solution can be achieved by selecting items, B and C, where the total profit is 18 + 18 =
36.

Example-2

Instead of selecting the items based on the overall benefit, in this example the items are selected based on ratio
pi/wi. Let us consider that the capacity of the knapsack is W = 60 and the items are as shown in the following
table.
 Item A B C
Price 100 280 120
Weight 10 40 20
Ratio 10 7 6

Using the Greedy approach, first item A is selected. Then, the next item B is chosen. Hence, the total profit is
100 + 280 = 380. However, the optimal solution of this instance can be achieved by selecting items, B and C,
where the total profit is 280 + 120 = 400.

Hence, it can be concluded that Greedy approach may not give an optimal solution.

Dynamic Programming

Dynamic Programming is also used in optimization problems. Like divide-and-conquer method, Dynamic
Programming solves problems by combining the solutions of subproblems. Moreover, Dynamic Programming
algorithm solves each sub-problem just once and then saves its answer in a table, thereby avoiding the work of
re-computing the answer every time.

Two main properties of a problem suggest that the given problem can be solved using Dynamic Programming.
These properties are overlapping sub-problems and optimal substructure.

Overlapping Sub-Problems

Similar to Divide-and-Conquer approach, Dynamic Programming also combines solutions to sub-problems. It is

mainly used where the solution of one sub-problem is needed repeatedly. The computed solutions are stored in a
table, so that these don’t have to be re-computed. Hence, this technique is needed where overlapping sub-
problem exists.

For example, Binary Search does not have overlapping sub-problem. Whereas recursive program of Fibonacci
numbers have many overlapping sub-problems.

Optimal Sub-Structure

A given problem has Optimal Substructure Property, if the optimal solution of the given problem can be
obtained using optimal solutions of its sub-problems.

For example, the Shortest Path problem has the following optimal substructure property −

If a node x lies in the shortest path from a source node u to destination node v, then the shortest path from u to v
is the combination of the shortest path from u to x, and the shortest path from x to v.

The standard All Pair Shortest Path algorithms like Floyd-Warshall and Bellman-Ford are typical examples of
Dynamic Programming.

Steps of Dynamic Programming Approach

Dynamic Programming algorithm is designed using the following four steps −

 Characterize the structure of an optimal solution.

 Recursively define the value of an optimal solution.
 Compute the value of an optimal solution, typically in a bottom-up fashion.
  Construct an optimal solution from the computed information.

Applications of Dynamic Programming Approach

 Matrix Chain Multiplication

 Longest Common Subsequence
 Travelling Salesman Problem

Backtracking

The Backtracking is an algorithmic-method to solve a problem with an additional way. It uses a recursive
approach to explain the problems. We can say that the backtracking is needed to find all possible combination
to solve an optimization problem.

Backtracking is a systematic way of trying out different sequences of decisions until we find one that "works."

In the following Figure:

 Each non-leaf node in a tree is a parent of one or more other nodes (its children)
 Each node in the tree, other than the root, has exactly one parent

a tree for a particular "goal" leaf node.

Backtracking is undoubtedly quite simple - we "explore" each node, as follows:

To "explore" node N:
1. If N is a goal node, return "success"
2. If N is a leaf node, return "failure"
3. For each child C of N,
Explore C
If C was successful, return "success"
4. Return "failure"
Backtracking algorithm determines the solution by systematically searching the solution space for the given
problem. Backtracking is a depth-first search with any bounding function. All solution using backtracking is
needed to satisfy a complex set of constraints. The constraints may be explicit or implicit.

Explicit Constraint is ruled, which restrict each vector element to be chosen from the given set.

Implicit Constraint is ruled, which determine which each of the tuples in the solution space, actually satisfy
the criterion function.

Branch and Bound

Branch-and-bound is a general technique for improving the searching process by systematically enumerating all
candidate solutions and disposing of obviously impossible solutions.

Branch-and-bound usually applies to those problems that have finite solutions, in which the solutions can be
represented as a sequence of options. The first part of branch-and-bound, branching, requires several choices to
be made so that the choices branch out into the solution space. In these methods, the solution space is organized
as a treelike structure.

The branch-and-bound algorithm handles this problem by bounding and pruning. Bounding refers to setting a
bound on the solution quality (e.g., the route length for TSP), and pruning means trimming off branches in the
solution tree whose solution quality is estimated to be poor. Bounding and pruning are the essential concepts of
the branch and-bound technique, because they are used to effectively reduce the search space.

General Method

1. In branch and bound method state space tree is built and all the children of E nodes (a live node
whose children are currently being generated) are generated before any other node can become a
live node.
2. For exploring new nodes either a BFS or D-Serach Techniques can be used.
3. In Branch and bound technique BFS-like state space search will be called FIFO Search. This is
because the list of live nodes is FIFO. On the other hand D-Serach like state space search will be
called LIFO search because the list of live node is LIFO.
4. In this method a space tree of possible solutions is generated. Then partitioning is done at each
node of the tree. We compute lower bound and upper bound at each node. This computation
leads to selection of answer node.
5. Bounding functions are user to avoid the generation of subtree that do not contain an answer
node.

Travelling Salesman Problem

A traveler needs to visit all the cities from a list, where distances between all the cities are known and each city
should be visited just once. What is the shortest possible route that he visits each city exactly once and returns
to the origin city?

Solution
Travelling salesman problem is the most notorious computational problem. We can use brute-force approach to
evaluate every possible tour and select the best one. For n number of vertices in a graph, there are (n - 1)!
number of possibilities.

Instead of brute-force using dynamic programming approach, the solution can be obtained in lesser time, though
there is no polynomial time algorithm.

Let us consider a graph G = (V, E), where V is a set of cities and E is a set of weighted edges. An edge e(u, v)
represents that vertices u and v are connected. Distance between vertex u and v is d(u, v), which should be non-
negative.

Suppose we have started at city 1 and after visiting some cities now we are in city j. Hence, this is a partial tour.
We certainly need to know j, since this will determine which cities are most convenient to visit next. We also
need to know all the cities visited so far, so that we don't repeat any of them. Hence, this is an appropriate sub-
problem.

For a subset of cities S Є {1, 2, 3, ... , n} that includes 1, and j Є S, let C(S, j) be the length of the shortest path
visiting each node in S exactly once, starting at 1 and ending at j.

When |S| > 1, we define C(S, 1) = ∝ since the path cannot start and end at 1.

Now, let express C(S, j) in terms of smaller sub-problems. We need to start at 1 and end at j. We should select
the next city in such a way that

C(S,j)=minC(S−{j},i)+d(i,j)wherei∈Sandi≠jc(S,j)=minC(s−{j},i)+d(i,j)wherei∈Sandi≠j
Algorithm: Traveling-Salesman-Problem
C ({1}, 1) = 0
for s = 2 to n do
for all subsets S Є {1, 2, 3, … , n} of size s and containing 1
C (S, 1) = ∞
for all j Є S and j ≠ 1
C (S, j) = min {C (S – {j}, i) + d(i, j) for i Є S and i ≠ j}
Return minj C ({1, 2, 3, …, n}, j) + d(j, i)

Analysis

There are at the most 2n.n

sub-problems and each one takes linear time to solve. Therefore, the total running time is O(2n.n2)

Example

In the following example, we will illustrate the steps to solve the travelling salesman problem.
From the above graph, the following table is prepared.

1 2 3 4
1 0 10 15 20
2 5 0 9 10
3 6 13 0 12
48 8 9 0

S=Φ

Cost(2,Φ,1)=d(2,1)=5Cost(2,Φ,1)=d(2,1)=5
Cost(3,Φ,1)=d(3,1)=6Cost(3,Φ,1)=d(3,1)=6
Cost(4,Φ,1)=d(4,1)=8Cost(4,Φ,1)=d(4,1)=8

S=1

Cost(i,s)=min{Cost(j,s–(j))+d[i,j]}Cost(i,s)=min{Cost(j,s)−(j))+d[i,j]}
Cost(2,{3},1)=d[2,3]+Cost(3,Φ,1)=9+6=15cost(2,{3},1)=d[2,3]+cost(3,Φ,1)=9+6=15
Cost(2,{4},1)=d[2,4]+Cost(4,Φ,1)=10+8=18cost(2,{4},1)=d[2,4]+cost(4,Φ,1)=10+8=18
Cost(3,{2},1)=d[3,2]+Cost(2,Φ,1)=13+5=18cost(3,{2},1)=d[3,2]+cost(2,Φ,1)=13+5=18
Cost(3,{4},1)=d[3,4]+Cost(4,Φ,1)=12+8=20cost(3,{4},1)=d[3,4]+cost(4,Φ,1)=12+8=20
Cost(4,{3},1)=d[4,3]+Cost(3,Φ,1)=9+6=15cost(4,{3},1)=d[4,3]+cost(3,Φ,1)=9+6=15
Cost(4,{2},1)=d[4,2]+Cost(2,Φ,1)=8+5=13cost(4,{2},1)=d[4,2]+cost(2,Φ,1)=8+5=13

S=2

Cost(2,{3,4},1)=⎧⎩⎨d[2,3]+Cost(3,{4},1)=9+20=29d[2,4]+Cost(4,{3},1)=10+15=25=25Cost(2,{3,4},1)
{d[2,3]+cost(3,{4},1)=9+20=29d[2,4]+Cost(4,{3},1)=10+15=25=25
Cost(3,{2,4},1)=⎧⎩⎨d[3,2]+Cost(2,{4},1)=13+18=31d[3,4]+Cost(4,{2},1)=12+13=25=25Cost(3,{2,4},1)
{d[3,2]+cost(2,{4},1)=13+18=31d[3,4]+Cost(4,{2},1)=12+13=25=25
Cost(4,{2,3},1)=⎧⎩⎨d[4,2]+Cost(2,{3},1)=8+15=23d[4,3]+Cost(3,{2},1)=9+18=27=23Cost(4,{2,3},1)
{d[4,2]+cost(2,{3},1)=8+15=23d[4,3]+Cost(3,{2},1)=9+18=27=23

S=3

Cost(1,{2,3,4},1)=⎧⎩⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪d[1,2]+Cost(2,{3,4},1)=10+25=35d[1,3]+Cost(3,
{2,4},1)=15+25=40d[1,4]+Cost(4,{2,3},1)=20+23=43=35cost(1,{2,3,4}),1)d[1,2]+cost(2,
{3,4},1)=10+25=35d[1,3]+cost(3,{2,4},1)=15+25=40d[1,4]+cost(4,{2,3},1)=20+23=43=35
The minimum cost path is 35.

Start from cost {1, {2, 3, 4}, 1}, we get the minimum value for d [1, 2]. When s = 3, select the path from 1 to 2
(cost is 10) then go backwards. When s = 2, we get the minimum value for d [4, 2]. Select the path from 2 to 4
(cost is 10) then go backwards.

When s = 1, we get the minimum value for d [4, 3]. Selecting path 4 to 3 (cost is 9), then we shall go to then go
to s = Φ step. We get the minimum value for d [3, 1] (cost is 6).

Bin Packing Problem (Minimize number of used Bins)

Given n items of different weights and bins each of capacity c, assign each item to a bin such that number of
total used bins is minimized. It may be assumed that all items have weights smaller than bin capacity.

Example:

Input: wieght[] = {4, 8, 1, 4, 2, 1}

Bin Capacity c = 10
Output: 2
We need minimum 2 bins to accommodate all items
First bin contains {4, 4, 2} and second bin {8, 2}

Input: wieght[] = {9, 8, 2, 2, 5, 4}

Bin Capacity c = 10
Output: 4
We need minimum 4 bins to accommodate all items.

Input: wieght[] = {2, 5, 4, 7, 1, 3, 8};

Bin Capacity c = 10
Output: 3

Lower Bound
We can always find a lower bound on minimum number of bins required. The lower bound can be given as :

Min no. of bins >= Ceil ((Total Weight) / (Bin Capacity))

In the above examples, lower bound for first example is “ceil(4 + 8 + 1 + 4 + 2 + 1)/10” = 2 and lower bound in
second example is “ceil(9 + 8 + 2 + 2 + 5 + 4)/10” = 3.
This problem is a NP Hard problem and finding an exact minimum number of bins takes exponential time.
Following are approximate algorithms for this problem.

 Applications
1. Loading of containers like trucks.
2. Placing data on multiple disks.
3. Job scheduling.
4. Packing advertisements in fixed length radio/TV station breaks.
5. Storing a large collection of music onto tapes/CD’s, etc.