Module - 2

Vandana U
Assistant Professor
Dept. of AI & DS
 Contents of the Module

BRUTE FORCE APPROACHES (contd..): Exhaustive Search

(Travelling Salesman probem and Knapsack Problem).

DECREASE-AND-CONQUER: Insertion Sort, Topological

Sorting.
DIVIDE AND CONQUER: Merge Sort, Quick Sort, Binary Tree
Traversals, Multiplication of Large Integers and Strassen’s
Matrix Multiplication.
Exhaustive Search:
Introduction:
Some problems arise in situations that involve— explicitly or implicitly —combinatorial objects
such as permutations, combinations, and subsets of a given set. Many such problems are
optimization problems: they ask to find an element that maximizes or minimizes some desired
characteristic such as a path length or an assignment cost

Many problems can be thought as search problems. A search problem involves some set of
possibilities and we are looking for one or more of the possibilities that satisfy some property.
Example:
Searching for shortest path from Goa to Mangalore.
Searching for the best cake shop in Cochin.

For some problems we need to understand and perhaps generate the set of all possibilities over
which we will search.
Exhaustive Search:
Method for exhaustive search of all possibilities:
•Generate a list of all potential solutions to the problem.
•Evaluate potential solutions one by one disqualifying unfeasible ones and, for an optimization
problem , keeping track of the best one found so far.

Definition : Exhaustive search is simply a brute-force approach to combinatorial problems. It

suggests generating each and every element of the problem domain, selecting those of them that
satisfy all the constraints, and then finding a desired element(e.g., the one that optimizes some
objective function).

We illustrate exhaustive search by applying it to three important problems:

Travelling Salesman Problem

Knapsack Problem
Assignment Problem
Algorithm 1 : Travelling Salesman
Problem
Travelling Salesman Problem: Introduction:
The traveling salesman problem (TSP) has been intriguing researchers for the last 150 years by
its seemingly simple formulation, important applications, and interesting connections to other
combinatorial problems.
Problem Statement : The problem asks to find the shortest tour through a given set of n cities
that visits each city exactly once before returning to the city where it started.
The problem can be conveniently modeled by a weighted graph, with the graph’s vertices
representing the cities and the edge weights specifying the distances.
Travelling Salesman Problem: Introduction:
Then the problem can be stated as the problem of finding the shortest Hamiltonian circuit of the
graph.(A Hamiltonian circuit is defined as a cycle that passes through all the vertices of the graph
exactly once. It is named after the Irish mathematician Sir William Rowan Hamilton (1805–1865),
who became interested in such cycles as an application of his algebraic discoveries.)

It is easy to see that a Hamiltonian circuit can also be defined as a sequence of n+1 adjacent
vertices vi0, vi1…….,vin-1 , vi0 , where the first vertex of the sequence is the same as the last one
and all the other n − 1 vertices are distinct. Further, we can assume, with no loss of generality,
that all circuits start and end at one particular vertex (they are cycles after all, are they not?).
Thus, we can get all the tours by generating all the permutations of n − 1intermediate cities,
compute the tour lengths, and find the shortest among them.
Travelling Salesman Problem: Problem 1:
Example 1 : Solve the below Travelling salesman problem using exhaustive search:

Note : Diagram represents cities : a,b,c and d with their distances.

3 Solution : If the starting city of salesman is city ‘a’ then all the
a b possible routes with the total distances are given below:
7 Route 1 : a → b → c → d → a , Distance : 13
4 5
9 Route 2 : a → b → d → c → a , Distance : 20

d c Route 3 : a → d → c → b → a , Distance : 13
1
Route 4 : a → d → b → c → a , Distance : 25
Route 5 : a → c → b → d → a , Distance : 25 Route 6 : a → c → d→ b → a , Distance : 20
Travelling Salesman Problem:
Generating all possible routes from ‘a’ , we observe that first and third routes gives us the
optimal distance of ’13’. Hence salesman can opt any one of the two routes.
Here we see that, we generate (n-1)! permutations or routes for ‘n’ cities in general. So the
time complexity of TSP is of the order (n-1)! as we select optimal routes only after finding all
the (n-1)! routes. Thus c(n) ∈ Θ((n-1)! )
Practice Question : Solve the below Travelling Salesman problem using Exhaustive Search. Also,
mention its time complexity.
1 2 0 10 15 20
5 0 9 10
6 13 0 12
8 8 9 0
3 4
Applications of Travelling Salesman Problem:
The Traveling Salesman Problem (TSP) is a well-known optimization problem that has numerous
real-world applications in various fields. Here are some of the most notable applications:
Vehicle Routing: The TSP is widely used in the optimization of delivery and transportation routes.
In this application, the salesman is represented by a delivery vehicle, and the cities correspond to
delivery locations. The goal is to find the shortest possible route that visits each location exactly
once and returns to the starting point. This problem is critical in the context of delivery companies,
as it helps to minimize fuel consumption, reduce the number of vehicles required, and ensure the
timely delivery of goods.
Circuit Design: In the field of electronics, the TSP is used to design efficient circuits. In this
application, the cities represent the components of a circuit, and the salesman is the electrical
signal that must travel through each component exactly once. The goal is to find the shortest
possible path that minimizes the total resistance and capacitance of the circuit.
Protein Folding: The TSP is also used in the field of biology, specifically in the study of protein folding.
In this application, the cities represent different amino acids in a protein, and the salesman is the
path that the protein takes to fold into its final three-dimensional shape. The goal is to find the
shortest possible path that minimizes the energy required for the protein to fold into its final shape.
Algorithm 2 : Knapsack Problem
Knapsack Problem: Introduction:
Problem Statement : Given n items of known weights w1,w2,...,wn and values v1,v2,...,vn and a
knapsack of capacity W, find the most valuable subset of the items that fit into the knapsack.

i,e, maximize Vi . Ii Subjected to the constraint W i . Ii ≤ W

The exhaustive-search approach to this problem leads to generating all the subsets of the set of n
items given, computing the total weight of each subset in order to identify feasible subsets (i.e., the
ones with the total weight not exceeding the knapsack capacity), and finding a subset of the largest
value among them
Time Complexity : Since the number of subsets of an n-element set is 2n, the exhaustive search leads
to a Ω(2n) algorithm, no matter how efficiently individual subsets are generated.
Thus, for both the traveling salesman and knapsack problems considered above, exhaustive search
leads to algorithms that are extremely inefficient on every input. In fact, these two problems are the
best-known examples of so called NP-hard problems. No polynomial-time algorithm is known for
any NP hard problem. Moreover, most computer scientists believe that such algorithms do not exist,
although this very important conjecture has never been proven.
Knapsack Problem: Example 1 : Apply Exhaustive search to obtain an optimal solution to
the knapsack problem where knapsack capacity = 30
Objec A B C
t
Weig 18 8 22
Solution : Subsets Total ht
Weight Total Value(Profit) Feasible (Yes/No)

{ } Profit0 41 28 500 Yes

{ A} 18 41 Yes
{ B} 8 28 Yes
{ C} 22 50 Yes
{ A,B} 18+8=26 41+28=69 Yes
{ A,C} 18+22=40 41+50=91 No
{ B,C} 8+22=30 28+50=78 Yes
{ A,B,C} 18+8+22=48 41+28+50=119 No
Here the weights for subsets {A,C} and {A,B,C} exceeds the capacity of knapsack 30. Hence, they are
not feasible solutions. Remaining are feasible, but the subset which gives maximum value is {B,C}
Knapsack Problem:
Conclusion : For any knapsack problem, with ‘n’ items, we have to find out the possible subsets
not exceeding the knapsack capacity and having maximum profit.

Practice Question:
Example 1 : Apply Exhaustive search to obtain an optimal solution to the knapsack problem where
knapsack capacity = 15
Applications of Knapsack Problem:
Cryptography, Applied mathematics, Networking, and Operations research .
Optimization of food orders, where the problem is modeled as a multi-objective unbounded knapsack problem .
job scheduling for cloud computing, where the online knapsack problem is used to allocate resources efficiently
Resource allocation: The knapsack algorithm can be used in resource allocation problems where you have a limited set of
resources and you need to maximize the value of items that can be carried within those constraints. This can be seen in industries
like finance, logistics, and manufacturing.
Portfolio optimization: In finance, the knapsack algorithm can be applied to optimize investment portfolios. Here, the items
represent different investment options with varying returns and risks, and the goal is to maximize the return on investment while
staying within the constraints of the available funds.
Scheduling: The knapsack algorithm can be used in scheduling problems where tasks with varying durations and priorities need to
be assigned to resources with limited capacity. This can be applied in project management, job scheduling, and other similar
contexts.
Data compression: In data compression algorithms, the knapsack problem can be used to efficiently select a subset of data to be
transmitted or stored, while minimizing the overall size of the compressed data.
Subset sum problem: The knapsack algorithm can be used to solve the subset sum problem, where given a set of numbers, the
goal is to find a subset that sums up to a target value. This has applications in cryptography, number theory, and combinatorial
optimization.
DNA sequencing: In bioinformatics, the knapsack algorithm can be used for DNA sequencing problems where the goal is to
assemble the sequence of DNA fragments by maximizing the coverage of the original DNA sequence while considering constraints
such as read lengths and overlaps.
Resource planning in cloud computing: The knapsack algorithm can be applied to optimize resource allocation in cloud computing
environments where virtual machines with different resource requirements need to be assigned to physical servers with limited
Algorithm 3 : Assignment Problem
Assignment Problem: Introduction:
Problem Statement : There are n people who need to be assigned to execute n jobs, one person
per job. (That is, each person is assigned to exactly one job and each job is assigned to exactly
one person.). The cost that would accrue if the ith person is assigned to the jth job is a known
quantity C[i, j] for each pair i, j = 1, 2,...,n. The problem is to find an assignment with the
minimum total cost.

Hence the problem is stated as : i,e, maximize cij . xij

Subjected to the constraints : xij =

The problem can be conveniently modeled by a weighted graph, with the graph’s vertices
representing the cities and the edge weights specifying the distances.
Assignment Problem: Example 1 : Solve the following assignment
problem
J1 J2 J3 J4
Note: Here, J1,J2,J3,J4 are jobs and P1,P2,P3,P4 are persons.
P1 7 10 13 8
P2 9 2 5 12 Solution:
P3 3 15 4 9 The exhaustive search says that we have to find out all
P4 13 5 10 7 possible n-tuples like J1,J2,..,Jn) for problem of ‘n’ jobs.
Then select one feasible tuple which has minimum cost.
P1 P2 P3 P4
Here we have 4 jobs, hence 4! = 24 different tuples can be
(J1 J2 J3 J4) = 7+ 2+4 + 7 = 20 generated as shown above. After all 24 tuples are
(J1 J2 J4 J3) = 7+ 2+9+10=28
generated we have to find the tuple with minimum cost

(J1 J3 J2 J4) = 7+ 5+15+7=34

Hence in general , for any assignment problem with ‘n’ jobs
, we have n! number of permutations or n-tuples.
(J1 J3 J4 J2) = 7+ 5+9+5=26 Therefore, time complexity is O(n!)
…..
…..
Applications of Assignment Problem:
1. Logistics and Transportation: One of the most common applications of the assignment problem is in logistics and
transportation management. For example, consider a delivery company that needs to assign a set of delivery routes to its fleet of
vehicles. By formulating this as an assignment problem, the company can optimize the allocation of routes to vehicles based on
factors such as distance, time constraints, and vehicle capacity. This ensures that deliveries are made in the most efficient way
possible, minimizing costs and maximizing customer satisfaction.
2. Workforce Scheduling: Another area where the assignment problem finds extensive use is in employee scheduling. For instance,
a hospital may need to assign nurses to different shifts based on their availability, skills, and workload requirements. By modeling
this as an assignment problem, the hospital can optimize the allocation of nurses to shifts, ensuring adequate coverage while
considering individual preferences and constraints.
3. Resource Allocation: The assignment problem also plays a crucial role in resource allocation scenarios. For instance, consider a
manufacturing plant that needs to assign different tasks to its machines based on their capabilities and efficiency. By formulating
this as an assignment problem, the plant can optimize the allocation of tasks to machines, maximizing productivity and minimizing
downtime.
4. Project Assignment: In project management, assigning team members to specific tasks is a critical aspect of ensuring project
success. By using the assignment problem framework, project managers can optimize task assignments based on factors such as
skill sets, availability, and workload distribution. This helps in balancing workloads among team members and maximizing overall
project efficiency.
5. Matching Algorithms: The assignment problem also finds applications in matching algorithms, where the goal is to match
individuals or entities based on their preferences and characteristics. For example, online dating platforms use assignment
problem algorithms to match users based on their interests, location, and compatibility scores. Similarly, job matching
platforms use similar algorithms to match job seekers with suitable job opportunities.
6. Sports Scheduling: The assignment problem is often employed in sports scheduling to determine the optimal allocation of
THANK YOU
 Module-2
Chapter-2
Decrease and
Conquer
By,
Vandana
Assistant Professor
Dept. of AI/DS
Decrease and Conquer :
•The decrease-and-conquer technique is based on exploiting the relationship between a
solution to a given instance of a problem and a solution to its smaller instance.
•Once such a relationship is established, it can be exploited either top down or bottom
up.
•Top down leads naturally to a recursive implementation, although, as one can see from
several examples in this chapter, an ultimate implementation may well be non-recursive.
•The bottom-up variation is usually implemented iteratively, starting with a solution to the
smallest instance of the problem; it is called sometimes the incremental approach.
Diagramatic Representation:
Problem
Divide

Sub-problem1 Sub-problem 2 …………….. Sub-problem n

Conquer
Extend Extend the solution to other instances of the problem.
Solution to Sub-
problem 1 Solution to Sub- Solution to Sub-
problem 2 …………….. problem n

Combine and get the overall Solution

There are 3 variants of Decrease and
Conquer:
●Decrease by a Constant
●Decrease by a constant factor
●Variable size decrease
1. Decrease by a Constant:
•In the decrease-by-a-constant variation, the size of an instance is reduced by the same
constant on each iteration of the algorithm.
•Typically, this constant is equal to one , although other constant size reductions do happen
occasionally.
Example : the exponentiation problem of computing an
where a≠0 and n is a nonnegative integer

Solution : The relationship between a solution to an

instance of size n and an instance of size n − 1is obtained
by the obvious formula
an = an −1.a. So the function f(n) = an can be computed
either “top down” by using its recursive definition.
2. Decrease by a Constant factor:
•The decrease-by-a-constant-factor technique suggests reducing a problem instance by the
same constant factor on each iteration of the algorithm. In most applications, this constant
factor is equal to two.

Example : the exponentiation problem of computing an

where a≠0 and n is a nonnegative integer

Example : Find 25 using the technique of Decrease by a

Constant factor.
3. Variable size decrease:
•Here, the size-reduction pattern varies from one iteration of an algorithm to another
Example : Finding GCD of 2 numbers using Euclidean Division method.

gcd(m,n)= gcd(n, m mod n)

Though the value of the second argument is always smaller on the right-hand side than on the
left-hand side, it decreases neither by a constant nor by a constant factor.
Advantages and Disadvantages of Decrease and Conquer:
Advantages Disadvantages
Simplicity: Decrease-and-conquer is often Problem-Specific: The technique is not
simpler to implement compared to other applicable to all problems and may not be
techniques like dynamic programming or suitable for more complex problems.
divide-and-conquer.
Efficient Algorithms: The technique often leads Implementation Complexity: The technique
to efficient algorithms as the size of the input can be more complex to implement when
data is reduced at each step, reducing the time compared to other techniques like divide-and-
and space complexity of the solution. conquer, and may require more careful
planning.
Problem-Specific: The technique is well-suited
for specific problems where it’s easier to solve
a smaller version of the problem.

Applications : Binary search, Finding the maximum or minimum element in an array, and Finding
the closest pair of points in a set of points.
Example 1: Insertion Sort
Insertion Sort:
•This sort follows the first variant of Decrease and Conquer . ie, Decrease by a Constant.
Problem : To sort the given array a[0,…….n-1] in ascending order.

Design :
Step 1: Divide the array into 2 parts.
- Sorted Array
- Unsorted Array.
Step 2 : Here, kth element can be inserted into any position from 0 to j in the sorted list.
Step 3 : After every iteration the unsorted list will be decreasing by one element.
Step 4: After n-1 iterations all elements will be in sorted array.
Insertion Sort:
•Starting with A[1] and ending with A[n −1],A[i] is inserted in its appropriate place among the
first i elements of the array that have been already sorted (but, unlike selection sort, are
generally not in their final positions).
Mathematical Analysis
1. Decide on parameters indicating input
size: n the basic operation :
2. Identify
Comparison , While j>=0 ad A[j]>v do

3. Check whether the number of times

the basic operation is executed depends
only on the size of an input. If it also
depends on some additional property, the worst-case, average-case, and, if necessary,
best-case efficiencies have to be investigated separately.
4. Setup a sum expressing the number of times the algorithm’s basic operation is
executed.
5. Using standard formulas and rules of sum manipulation, either find a closed form
formula for the count or, at the very least, establish its order of growth.
For this algorithm , since we can have best and worst time complexity, Step 3, 4 and
5 can be applied together.
Example 2: Topological Sorting