MOHD FAISOL MD.

SALLEH BEEQ2013 A201 TOPIC 5 PART 1

9 5.0 OPTIMIZATION WITH i. Differentiate between free optimization problem with  Chap 12.1,
EQUALITY CONSTRAINT optimization with constraint. 12.2
5.1 Solving optimization with ii. Identify objective function and constraint function.
equality constraint problem
5.2 Interpretation of Lagrange iii. Differentiation between two methods to solve constrained
multiplier optimization: substitution method and Lagrange multiplier
method.
3
iv. Explain reasons why the Lagrange multiplier method may
have advantages over the substitution method.
v. Write a complete Langrangian function.
vi. Solve problem using Lagrange multiplier method.
vii. Interpret Lagrange multiplier.

Free-constraint vs constraint optimization and the effect of a constraint

Reality check indicates, the best outcome is essentially, the culmination between wants and resources. As such, what is
considered best under the free constraint scenario may no longer the optimum result, given constraint(s). The presence of
constraint(s) put a limitation on what the outcome would be. Apparently, the constrained outcome is lesser in magnitude as
compared to the free-constraint outcome. Nevertheless, the process of locating the best possible outcome (using Lagrange
Multiplier Method) is very much follows similar steps with some variants.

1
 MOHD FAISOL MD. SALLEH BEEQ2013 A201 TOPIC 5 PART 1

The steps are simplified as:

Step 1: Mathematically, specify the objective function and the constraint(s) function.

Step 2: Represent both functions into a consolidated function that unifies the two different functions into a single function.
The newly transformed function is so unique that it retains the substance of the original functions despite changes
in its form. This newly transformed function in known as Lagrangian function as a result of the application of
Lagrange Multiplier method.

Step 3: Apply the first oder partial differentiation to the Lagrangian function with respect to its multiplier and every single
choice variables.

Step 4: Solve for the critical number of the choice variables and the multiplier by solving the system of first-order partial
derivatives after imposing the F.O.C to them.

Step 5: Find the stationary value(s) for the objective function (or the transformed Lagrangian function) using the critical
numbers.

Step 6: Verify the status of the stationary value(s) for extremum (relative minimum and/ or maxmum) using S.O.C expressed
in terms of Bordered Hessian (|𝐻 ̅ |)determinantal test for extremum.

2
 MOHD FAISOL MD. SALLEH BEEQ2013 A201 TOPIC 5 PART 1

Finding stationary values: Method of substitution vs Lagrange Multiplier Method

Example: Method of substitution

Step 2: Combine the constraint with the objective function through substitution. That is we are eliminating variable 𝑥2 by so
doing. 𝑈 = 𝑥1 (30 − 2𝑥1 ) + 2𝑥1 = 32𝑥1 − 2𝑥12
Step 3: Take F.O.C on the new function and solve for 𝑥1
𝑑𝑈
= 32 − 4𝑥1 = 0. 𝑆𝑜, 𝑥1∗ = 8 𝑎𝑛𝑑 𝑠𝑢𝑏𝑠𝑡𝑖𝑡𝑢𝑡𝑒 𝑓𝑜𝑟 𝑥2∗ = 14
𝑑𝑥1
Step 4: Substitute these critical values in the original objective function or the re-expressed objective function for stationary
value.
𝑈 ∗ = 128
Step 5: Check for the sign of S.O.C for relative maximum i.e verify that the stationary value constitutes a (constrained)
maximum of U.
𝑑2𝑈
= −4 < 0 (𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑚𝑎𝑥𝑖𝑚𝑢𝑚 𝑜𝑓 𝑈)
𝑑𝑥12

3
 MOHD FAISOL MD. SALLEH BEEQ2013 A201 TOPIC 5 PART 1

Point worth noting with regard to Substitution Method to solve for constrained optimisation
When the constraint is itself a complicated function or when there are several constraints to consider, however, the technique
of substitution and elimination of variables could become a burdensome task. More importantly, when the constraint comes
in a form such that we cannot solve it to express one variable as an explicit function of the other, the elimination method
would in fact be of no avail. Even if the variable were known to be an implicit function of the other (even if the conditions of
the implicit-function theorem were satisfied). This problem can be handled well by an alternative method known as Lagrange
Multiplier Method. This explains why the Lagrange multiplier method may have advantages over the substitution method.

Example: Langrange Multiplier Method

Step 1: Mathematically specify both the objective and constraint function

Step 2: Form the Langrange function for the problem

Step 3: Apply the first oder partial differentiation to the Lagrangian function with respect to its multiplier and every single
choice variables.
𝐿1 = 𝑥2 + 2 − 4𝜆
𝐿2 = 𝑥1 − 2𝜆
𝐿𝜆 = 60 − 4𝑥1 − 2𝑥2

Step 4: Then let them equal zero (i.e F.O.C) and solve for the critical numbers or values of the variables using appropriate
technique learnt in the topic 1 and 2.
𝐿1 = 𝑥2 + 2 − 4𝜆 = 0

4
 MOHD FAISOL MD. SALLEH BEEQ2013 A201 TOPIC 5 PART 1

𝐿2 = 𝑥1 − 2𝜆=0
𝐿𝜆 = 60 − 4𝑥1 − 2𝑥2 = 0

Give us 𝑥1∗ = 8, 𝑥2∗ = 14, 𝜆∗ = 4 and use these values for the stationary value of 𝐿∗ (𝑥1∗ = 8, 𝑥2∗ = 14, 𝜆∗ = 4) =
128. 𝑇ℎ𝑒 𝑟𝑒𝑠𝑢𝑙𝑡 𝑓𝑜𝑟 𝑡ℎ𝑒 𝑠𝑡𝑎𝑡𝑖𝑜𝑛𝑎𝑟𝑦 𝑣𝑎𝑙𝑢𝑒 𝑖𝑠 𝑖𝑑𝑒𝑛𝑡𝑖𝑐𝑎𝑙 𝑤𝑖𝑡ℎ 𝑈 ∗ 𝑓𝑜𝑢𝑛𝑑 𝑢𝑠𝑖𝑛𝑔 𝑠𝑢𝑏𝑠𝑡𝑖𝑡𝑢𝑡𝑖𝑜𝑛 𝑚𝑒𝑡ℎ𝑜𝑑 𝑒𝑎𝑟𝑙𝑖𝑒𝑟.
Step 6: Verify the status of the stationary value(s) for extremum (relative minimum and/ or maxmum) using S.O.C expressed
in terms of Bordered Hessian (|𝐻 ̅ |)determinantal test for extremum.

We will detail it out later in our lecture.

In brief, we can conclude that given a general 2-choice variables objective and constraint function

Step 1: Specify both the objective and constraint function

𝑍 = 𝑓(𝑥1 , 𝑥2 ) subject to g(𝑥1 , 𝑥2 ) = 𝑐

Step 2: Write a Lagrange function for the problem

Other steps will be discussed later when the time comes.