01 Knapsack using Dynamic Programming

0-1 KNAPSACK
USING
DYNAMIC PROGRAMMING
MADE BY:-
FENIL SHAH
15CE121
CHARUSAT UNIVERSITY

What is Dynamic Programming ?
• Dynamic programming is a method for solving
optimization problems.
The idea: Compute the solutions to the sub-
problems once and store the solutions in a table,
so that they can be reused (repeatedly) later.
• It is a Bottom-Up approach.
• Coined by Richard Bellman who described
dynamic programming as the way of solving
problems where you need to find the best
decisions one after another

Properties
• Two main properties of a problem suggest
that the given problem can be solved using
Dynamic Programming.
• These properties are overlapping sub-
problems and optimal substructure.

Steps of Dynamic Programming Approach
• Dynamic Programming algorithm is designed
using the following four steps −
1. Characterize the structure of an optimal
solution.
2. Recursively define the value of an optimal
solution.
3. Compute the value of an optimal solution,
typically in a bottom-up fashion.
4. Construct an optimal solution from the
computed information.

Applications of Dynamic Programming
Approach
• Matrix Chain Multiplication
• Longest Common Subsequence
• Travelling Salesman Problem
• Knapsack Problem

The 0-1 Knapsack Problem
(Rucksack Problem)
• Given: A set S of n items, with each item i having
– wi - a positive weight
– bi - a positive benefit (value)
• Goal: Choose items with maximum total benefit but with
weight at most W (weight of Knapsack).
– In this case, we let Tdenote the set of items we take
– Objective: maximize
– Constraint:
Ti
ib


Ti
i Ww

Why not Greedy?
• Greedy approach provides the optimal
solution to the fractional knapsack.
• 0-1 Knapsack cannot be solved by Greedy
approach.
• It does not ensure an optimal solution to the
0-1 Knapsack

EXAMPLE:
• Capacity 200
• Items :
X1 : v1/w1 = 12, weight : 50
X2 : v2/w2 = 10, weight : 55
X3 : v3/w3 = 8, weight : 10
X4 : v4/w4 = 6, weight : 100
 Value of Greedy : 50 * 12 + 55 *10 + 8* 10= 1230
 Optimal : 8 * 100 + 55*10 + 8*10= 1430
 SO, Greedy Approach does not ensure an optimal solution
to the 0-1 Knapsack
To solve 0-1 Knapsack, Dynamic Programming approach is
required.

Recursive formula for sub-problems:
3 cases:
1. There are no items in the knapsack, or the weight of the knapsack is 0 -
the benefit is 0
2. The weight of itemi exceeds the weight w of the knapsack - itemi cannot be
included in the knapsack and the maximum benefit is V[i-1, w]
3. Otherwise, the benefit is the maximum achieved by either not including
itemi ( i.e., V[i-1, w]),
or by including itemi (i.e., V[i-1, w-wi]+bi)
otherwise}],1[],,1[max{
wwif],1[
0or0for0
],[ i









ii bwwiVwiV
wiV
wi
wiV

0-1 Knapsack Algorithm
for w = 0 to W // Initialize 1st row to 0’s
V[0,w] = 0
for i = 1 to n // Initialize 1st column to 0’s
V[i,0] = 0
for i = 1 to n
for w = 0 to W
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w

Let’s run our algorithm on the
following data:
n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
Example

Example (1)
0
1
2
3
4 50 1 2 3
4
iW

for w = 0 to W
V[0,w] = 0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW
Example (2)

for i = 1 to n
V[i,0] = 0
0
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW
Example (3)

V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
0
i=1
bi=3
wi=2
w=1
w-wi =-1
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW
0
0
0
Example (4)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
300
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=1
bi=3
wi=2
w=2
w-wi =0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
Example (5)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
300
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=1
bi=3
wi=2
w=3
w-wi =1
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
3
Example (6)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
300
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=1
bi=3
wi=2
w=4
w-wi =2
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
3 3
Example (7)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
300
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=1
bi=3
wi=2
w=5
w-wi =3
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
3 3 3
Example (8)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=2
bi=4
wi=3
w=1
w-wi =-2
3 3 3 3
0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
Example (9)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=2
bi=4
wi=3
w=2
w-wi =-1
3 3 3 3
3
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
0
Example (10)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=2
bi=4
wi=3
w=3
w-wi =0
3 3 3 3
0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
43
Example (11)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=2
bi=4
wi=3
w=4
w-wi =1
3 3 3 3
0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
43 4
Example (12)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=2
bi=4
wi=3
w=5
w-wi =2
3 3 3 3
0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
73 4 4
Example (13)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=3
bi=5
wi=4
w= 1..3
3 3 3 3
0 3 4 4
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
7
3 40
Example (14)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=3
bi=5
wi=4
w= 4
w- wi=0
3 3 3 3
0 3 4 4 7
0 3 4 5
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
Example (15)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=3
bi=5
wi=4
w= 5
w- wi=1
3 3 3 3
0 3 4 4 7
0 3 4
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
5 7
Example (16)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=4
bi=6
wi=5
w= 1..4
3 3 3 3
0 3 4 4
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
7
3 40
70 3 4 5
5
Example (17)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=4
bi=6
wi=5
w= 5
w- wi=0
3 3 3 3
0 3 4 4 7
0 3 4
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
5
7
7
0 3 4 5
Example (18)

We’re DONE!!
The max possible value that can be carried in
this knapsack is $7

• This algorithm only finds the max possible
value that can be carried in the knapsack
– i.e., the value in V[n,W]
• To know the items that make this maximum
value, we need to trace back through the
table.

How to find actual Knapsack Items
• All of the information we need is in the table.
• V[n,W] is the maximal value of items that can
be placed in the Knapsack.
• Let i=n and k=W
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 // Assume the ith item is not in the
knapsack

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=4
k= 5
bi=6
wi=5
V[i,k] = 7
V[i1,k] =7
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7
Finding the Items

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=4
k= 5
bi=6
wi=5
V[i,k] = 7
V[i1,k] =7
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7
Finding the Items (2)

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=3
k= 5
bi=5
wi=4
V[i,k] = 7
V[i1,k] =7
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=2
k= 5
bi=4
wi=3
V[i,k] = 7
V[i1,k] =3
k  wi=2
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7
7

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW i=1
k= 2
bi=3
wi=2
V[i,k] = 3
V[i1,k] =0
k  wi=0
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7
3

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7
i=0
k= 0
The optimal
knapsack
should contain
{1, 2}

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
00
0
0
0
0 0 0 0 000
1
2
3
4 50 1 2 3
4
iW
3 3 3 3
0 3 4 4 7
0 3 4
i=n, k=W
while i,k > 0
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1
5 7
0 3 4 5 7
The optimal
knapsack
should contain
{1, 2}
7
3

for w = 0 to W
V[0,w] = 0
for i = 1 to n
V[i,0] = 0
for i = 1 to n
for w = 0 to W
< the rest of the code >
What is the running time of this algorithm?
O(W)
O(W)
Repeat n times
O(n*W)
Running time

Applications of Knapsack Problem
• Resourse allocation with financial constraints
• Construction and Scoring of Heterogenous
test
• Selection of capital investments

01 Knapsack using Dynamic Programming

Recommended

More Related Content

What's hot (20)

Similar to 01 Knapsack using Dynamic Programming (17)

Recently uploaded (20)

01 Knapsack using Dynamic Programming