05 - OR2 - Dynamic Programming (Deterministic) PDF
05 - OR2 - Dynamic Programming (Deterministic) PDF
DETERMINISTIC
DYNAMIC PROGRAMMING
1
16/09/2020
• At stage n the process will be in some state sn. Making policy decision x n then moves
the process to some state sn+1 at stage n+1.
• The contribution thereafter to the objective function under an optimal policy has
been previously calculated to be f*n+1(sn+1).
• The policy decision xn also makes some contribution to the objective function.
• Combining these two quantities in an appropriate way provides fn(sn, xn), the
contribution of stages n onward to the objective function.
• Optimizing with respect to xn then gives fn*(sn) = fn(sn, xn*).
• After xn* and fn*(sn) are found for each possible value of sn, the solution procedure
is ready to move back one stage.
3
Example-1:
Distributing Medical Teams to Countries
• The WORLD HEALTH COUNCIL is devoted to improving health
care in the underdeveloped countries of the world.
• It now has five medical teams available to allocate among
three such countries to improve their medical care, health
education, and training programs.
• Therefore, the council needs to determine how many teams
(if any) to allocate to each of these countries to maximize the
total effectiveness of the five teams.
• The teams must be kept intact, so the number allocated to
each country must be an integer.
2
16/09/2020
Example-1:
Distributing Medical Teams to Countries
• The measure of performance being used is additional
person-years of life. (For a particular country, this measure
equals the increased life expectancy in years times the
country’s population.)
• Table 1 gives the estimated additional person-years of life (in
multiples of 1,000) for each country for each possible
allocation of medical teams.
• Question: Which allocation maximizes the measure of
performance?
Example-1:
Distributing Medical Teams to Countries
Table 1. Data for the World Health Council problem
3
16/09/2020
Example-1: Formulation
Example-1: Formulation
4
16/09/2020
Example-1: Formulation
Example-1: Formulation
10
5
16/09/2020
Example-1: Formulation
11
Example-1: Formulation
• So that,
12
6
16/09/2020
Example-1: Formulation
• Therefore,
13
Example-1: Formulation
14
7
16/09/2020
15
8
16/09/2020
• Stage 1 (n = 1).
• In this case, the only state to be considered is the starting
state of s1 = 5. To allocating x1 medical teams to country 1
leads to a state of 5 – x1 at stage 2.
x1
s1 f1(s1, x1) = p1( x1) + f2*(s1- x1) f1*(s1) x1*
0 1 2 3 4 5
5 160 170 165 160 155 120 170 1
17
18
9
16/09/2020
19
Example-2:
Distributing Scientists to Research Teams
• A government space project is conducting research on a
certain engineering problem that must be solved before
people can fly safely to Mars.
• Three research teams are currently trying three different
approaches for solving this problem.
• The estimate has been made that, under present
circumstances, the probability that the respective teams—
call them 1, 2, and 3—will not succeed is 0.40, 0.60, and
0.80, respectively.
• Thus, the current probability that all three teams will fail is
(0.40)(0.60)(0.80) = 0.192.
20
10
16/09/2020
Example-2:
Distributing Scientists to Research Teams
• Because the objective is to minimize the probability of
failure, two more top scientists have been assigned to the
project.
• Table 2 gives the estimated probability that the respective
teams will fail when 0, 1, or 2 additional scientists are added
to that team.
• Only integer numbers of scientists are considered because
each new scientist will need to devote full attention to one
team.
• The problem is to determine how to allocate the two
additional scientists to minimize the probability that all three
teams will fail.
21
Example-2:
Distributing Scientists to Research Teams
22
11
16/09/2020
Example-2: Formulation
23
Example-2: Formulation
24
12
16/09/2020
Example-2: Formulation
Example-2: Formulation
• and when n = 3,
26
13
16/09/2020
Example-2: Formulation
27
• n=2 X2
s2 f2(s2, x2) = p2( x2) . f3*(s2- x2) f2*(s2) x2*
0 1 2
0 0,48 0,48 0
1 0,30 0,32 0,30 0
2 0,18 0,20 0,16 0,16 2
28
14
16/09/2020
• n=1
x1
s1 f1(s1, x1) = p1( x1) . f2*(s1- x1) f1*(s1) x1*
0 1 2
2 0.064 0,060 0,072 0,060 1
29
30
15
16/09/2020
31
32
16
16/09/2020
33
, k= 2, 3
34
17
16/09/2020
35
36
18
16/09/2020
37
38
19
16/09/2020
Optimal Solution
39
20