School of Civil Engineering

FACULTY OF ENGINEERING

Design Optimisation – MEng & MSc

CIVE5024M & CIVE5971M
Lecture 02

By Dr Amirul I. Khan
Copyright © 2019 University of Leeds UK. All rights reserved
Learning outcomes

• Define local and global minima (maxima) for unconstrained

and constrained optimization problems
• Write optimality conditions for unconstrained problems
• Write optimality conditions for constrained problems

Copyright © 2019 University of Leeds UK. All rights reserved

Recap Graphical Method
STEP 6: Identification of the optimum
solution
We now simply read the
coordinates of point D
(4, 12) to obtain the
optimum design, having
a maximum value for
the profit function as
P=8800.
Thus, the best strategy for
the company is to
manufacture 4 A and
P=400x1+600x2
12 B machines to
Copyright © 2019 University of Leeds UK. All rights reserved
Multiple optimum solutions

• A situation can arise in which a constraint

is parallel to the cost function. If the
constraint is active at the optimum, there
are multiple solutions to the problem
• Minimise
f(x) = -x1-0.5x2
• Subject to:
2x1+3x2 ≤12; 2x1+x2 ≤ 8;
-x1≤0; -x2 ≤ 0
• In this problem, the second constraint is
parallel to the cost function. Therefore,
there is a possibility of multiple optimum
designs.
• Graphical solution shows that line BC
gives an optimum design, giving the
problem infinite number of optimum
solutions

Copyright © 2019 University of Leeds UK. All rights reserved

PROBLEM WITH
UNBOUNDED SOLUTIONS

• Some design problems may not have a bounded

solution.
• This situation can arise if we forget a constraint
or incorrectly formulate the problem.
• To illustrate such a situation, consider the
following design problem:
• Minimise
f(x) = -x1+2x2
• Subject to:
-2x1+x2 ≤ 0; -2x1+3x2 ≤ 6;
-x1≤ 0; -x2 ≤ 0
• The feasible set for the problem is shown in the
Figure with several cost function contours.
• It is seen that the feasible set is unbounded.
• Therefore, there is no finite optimum solution,
and we must re-examine the way the problem
was formulated to correct the situation.
• The Figure shows that the problem is
underconstrained.

Copyright © 2019 University of Leeds UK. All rights reserved

Infeasible problem

• If we are not careful in formulating it, a design

problem may not have a solution, which
• happens when there are conflicting requirements
or inconsistent constraint equations.
• There may also be no solution when we put too
many constraints on the system; that is, the
constraints are so restrictive that no feasible
solution is possible.
• These are called infeasible problems. To illustrate
them, consider the following:
• Minimise
f(x) = x1+2x2
• Subject to:
3x1+2x2 ≤ 6; 2x1+3x2 ≥ 12;
x1≤ 5; x2 ≤ 5; x1 ≥ 0; x2 ≥ 0
• Basically, the first two constraints impose
conflicting requirements. The first requires the
feasible design to be below the line AG, whereas
the second requires it to be above the line CF.
• Since the two lines do not intersect in the first
quadrant, the problem has no feasible region.

Copyright © 2019 University of Leeds UK. All rights reserved

2D example, inactive
constraints

• Minimise
f(x,y)=(x-4)2+(y-6)2
• Subject to
g1=x+y-12≤0
g2=x-8 ≤ 0
g3=-x ≤ 0
g4=-y ≤ 0

Copyright © 2019 University of Leeds UK. All rights reserved

Constrained and
unconstrained problems

Almost all realistic problems of design optimization

are constrained problems
There could be some limitations on the
performance characteristics of an engineering
system.
Constraints that represent limitations on the behaviour or
performance of the system are termed behaviour or
functional constraints.
Constraints that represent physical limitations on design
variables, such as availability, fabricability, and
transportability, are known as geometric or side
constraints.
Copyright © 2019 University of Leeds UK. All rights reserved
It is still, important to learn how to solve efficiently
an unconstrained problem because
Some of the optimization techniques treat a
general constrained problem as an equivalent
sequence of simpler unconstrained problems.
Also, inverse problems can often be formulated
as unconstrained problems.
Examples:
Numerical model calibration using experimental results
Material parameter identification

Copyright © 2019 University of Leeds UK. All rights reserved

What is common here?

Severely constrained design space

Copyright © 2019 University of Leeds UK. All rights reserved
Japanese K-car

K-car has to satisfy the following Cost item (for Ordinary car, K-car
requirements: two years), yen  1500 kg
Length  3400mm Road tax 34,500 7,200
Width  1480mm
Environment 37,800 8,800
Height  2000 mm
tax
Engine  0.66 litre
Compulsory 22,470 18,890
Main benefit: in Japan one insurance
doesn’t have to have a Total, yen 94,770 = £655 34,980 =
private parking space to buy £242
a K-car

Copyright © 2018 University of Leeds UK. All rights reserved

McDonnell XF-85 Goblin

XF85 Goblin (FF 23 August 1948) had to fit into the bomb bay of Convair B-36
bomber as a defensive "parasite fighter“. The design was entirely driven by these
constraints. The B-36 was supposed to carry up to three Goblins. The project was
cancelled a year later primarily due to advances in air refuelling.
• Length: 4.5 m
• Wingspan: 6.4 m (foldable wings)
• Height: 2.5 m
• Loaded weight: 2,050 kg
• Maximum speed: 1,069 km/h
• Service ceiling: 14,600 m
• 4 × 12.7 mm machine guns

10-engined Convair B-36 Peacemaker: 6

piston engines (pusher propellers) and 4
jet engines - "six turnin' and four burnin' “.
 Length: 49.40 m
 Wingspan: 70.10 m
 Height: 2.5 m
 Max takeoff weight: 190,000 kg
 In service: 1948-1958
Copyright © 2018 University of Leeds UK. All rights reserved
Jettied buildings

Jettying is a building technique used in medieval timber frame buildings in which

an upper floor projects beyond the dimensions of the floor below. This has the
advantage of increasing the available space in the building without obstructing
the street. This practice flourished in the 16th century and later declined due to
reduced availability of oak.

Goslar, Germany
Copyright © 2018 University of Leeds UK. All rights reserved
How do you find the summit if
you can’t see the landscape?

Copyright © 2019 University of Leeds UK. All rights reserved

Classification of methods for
continuous design variables

Copyright © 2019 University of Leeds UK. All rights reserved

Optimum Design Concepts

• Optimality criteria methods: These are the conditions a

function must satisfy at its minimum point.
• Optimization methods seeking solutions (perhaps using
numerical methods) to the optimality conditions are often called
optimality criteria or indirect methods.
• Search methods : Sometimes called direct methods are based
on a different philosophy. Here we start with an estimate of the
optimum design for the problem.
• Usually the starting design will not satisfy the optimality criteria;
therefore, it is improved iteratively until they are satisfied. Thus,
in the direct approach we search the design space for optimum
points.

Copyright © 2019 University of Leeds UK. All rights reserved

Optimality criteria method

 The classical methods of optimization are useful in finding the optimum

solution of continuous and differentiable functions.
 These methods are analytical and make use of the techniques of
differential calculus in locating the optimum points.
 Since some of the practical problems involve objective functions that
are not continuous and/or differentiable, the classical optimization
techniques have limited scope in practical applications.
 However, a study of the calculus methods of optimization forms a basis
for developing most of the numerical techniques of optimization.
 We assume that the variables are continuous and that all problem
functions are continuous and at least twice continuously differentiable.
 A thorough knowledge of optimality conditions is important for an
understanding of the performance of various numerical (search)
methods.

Copyright © 2019 University of Leeds UK. All rights reserved

What is an optimum?
• What are the necessary and sufficient conditions in locating the optimum
solution of an optimization problem.
• We must first define what is meant by an optimum.
• A design optimization problem is always converted to the minimisation of a cost
function
• The cost function could be a single or multi-variable function with no constraints,
and a multi-variable function with equality and inequality constraints?
• The feasible set Ω or constraint set, feasible region, or feasible design space.
consists of all of the search space inside of which the optimal solution can be
found.
• Unconstrained problems, have no constraints or restrictions.
• Therefore the entire design space 𝛀 = ℝ𝑛 can be used to search for the
optimum.
• The optimization problem is to find a point in the feasible design space that
gives a minimum value to the cost function.
Copyright © 2019 University of Leeds UK. All rights reserved
• Suppose that 𝑓: ℝ𝑛 → ℝ is a real-valued function defined on some set Ω ⊂ ℝ𝑛 → ℝ.
• A point 𝒙∗ ∈ Ω is a local minimiser of 𝑓 over Ω if there exists 𝜀 > 0 such that

𝑓 𝒙 ≥ 𝑓 𝒙∗ ∀ 𝒙 ∈ Ω ∖ {𝒙∗ } 𝑎𝑛𝑑 ∥ 𝒙 − 𝒙∗ ∥< 𝜀.

• A point 𝑥 ∗ ∈ Ω is a global minimiser of 𝑓 over Ω if

𝑓 𝒙 ≥ 𝑓 𝒙∗ ∀ 𝒙 ∈ Ω ∖ {𝒙∗ }

• If in the definitions above we replace " ≥ " with ">," then we have a strict local
minimiser and a strict global minimiser, respectively.
• A global minimum point is the one where there are no other feasible points with
better cost function values.
• A local minimum point is the one where there are no other feasible points “in the
vicinity” with better cost function values.

Copyright © 2019 University of Leeds UK. All rights reserved

1D example of local and global
optimum
• Consider graphs of a function f(x) of one
variable, as shown in Figure.
• In Part (a) of the figure, where (-∞ ≤ x≤ ∞),
points B and D are local minima since the
function has its smallest value in their
neighbourhood.
• Similarly, both A and C are points of local
maxima for the function.
• There is, however, no global minimum or
maximum for the function since the domain
and the function f(x) are unbounded.
• If we restrict x to lie between -a and b, as in
Part(b) of Figure , then point E gives the
global minimum and F the global maximum
for the function.
• Both of these points have active constraints,
while points A, B, C, and D are
unconstrained.
Copyright © 2019 University of Leeds UK. All rights reserved
2D example, inactive
constraints

• Minimise
f(x,y)=(x-4)2+(y-6)2
• Subject to
g1=x+y-12≤0
g2=x-8 ≤ 0
g3=-x ≤ 0
g4=-y ≤ 0
• Unconstrained points: To locate minimum points we
use f(x*,y*) ≤ f(x, y) at a candidate feasible point (x*,y*)
in it’s small feasible neighbourhood. Point E(4,6) is the
local minimum.
• Constrained points: We check the local minimum at
some other points such as A where f(0,0)= 52 is not
local minimum since it violates f(0,0) ≤ f(x,y) for any
feasible point around (0,0).
• Similarly for B, C, D and G are not local minimum
• Hence E is local and global minimum.
• Note: Constraints have no role in determining the
minimum. But this not always true.

Copyright © 2019 University of Leeds UK. All rights reserved

2D example, active constraints

• Minimise 7

f(x,y)=(x-3)2+(y-3)2 6

5
• Subject to
4

g1=x+y-4≤0; x,y>0

x2
3

(2, 2) satisfies g1 and is 2

minimum 1

0
0 2 4 6 8
-1
x1

Copyright © 2019 University of Leeds UK. All rights reserved

2D example, active constraints

• Minimise
f(x,y)=(x-10)2+(y-8)2
• Subject to
g1=x+y-12≤0
g2=x-8 ≤ 0
g3=-x ≤ 0
g4=-y ≤ 0
• It is seen that point G, with coordinates (7, 5)
and f=18, has the smallest distance from
point E.
• At this point, the constraint g1 is active.
• Thus, for the present objective function, the
constraints play a prominent role in
determining the minimum point for the
problem.

Copyright © 2019 University of Leeds UK. All rights reserved

Multiple optimum

• Maximise
2 2
𝑓 𝑥, 𝑦 = 𝑥 − 4 + 𝑦−6
• Subject to
𝑔1 = 𝑥 + 𝑦 − 12 ≤ 0
𝑔2 = 𝑥 − 8 ≤ 0
𝑔3 = −𝑥 ≤ 0
𝑔4 = −𝑦 ≤ 0
• The feasible region for the problem is ABCD, as
shown in Figure
• The objective function is an equation of a circle
with center at point E(4, 6). Two objective
function contours are shown in the figure.
• It is seen that point D(8, 0) is a local maximum
point because any feasible move away from the
point results in reduction of the objective function.

Copyright © 2019 University of Leeds UK. All rights reserved

Multiple optimum

• Point C(8, 4) is not a local maximum point

since a feasible move along the line CD
results in an increase in the objective
function,
• Thus violating the definition of a local max-
point
𝑓 𝑥 ∗ , 𝑦 ∗ ≥ 𝑓(𝑥, 𝑦)
• It can be verified that points A and B are
also local maximum points and point G is
not. Thus, this problem has the following
three local maximum points:
• Point A(0, 0): 𝑓 0, 0 = 52
• Point B(0, 12): 𝑓 0, 12 = 52
• Point D(8, 0): 𝑓 8, 0 = 52
• There is no strict global maximum point.

Copyright © 2019 University of Leeds UK. All rights reserved

Criteria for global existence
• Weierstrass Theorem—Existence of a Global Minimum
• If 𝑓(𝑥) is continuous on a nonempty feasible set Ω that is closed and bounded,
then 𝑓(𝑥) has a global minimum in Ω = {𝑥 | 𝑎 ≤ 𝑥 ≤ 𝑏},
• Note that when conditions of the Weierstrass theorem are satisfied, the
existence of a global optimum is guaranteed.
• It is important, however, to realize that when they are not satisfied, a global
solution may still exist; that is, it is not an “if-and-only-if” theorem. The
theorem does not rule out the possibility of a global minimum . The difference is
that we cannot guarantee its existence.
• For example, minimizing 𝑓 𝑥 = 𝑥 2 subject to the constraints −1 < 𝑥 < 1.
• Since the feasible set is not closed, conditions of the Weierstrass theorem are
not met; therefore, it cannot be used. However, the function has a global
minimum at the point 𝑥 = 0.
• Note also that the theorem does not provide a method for finding a global
minimum point even if its conditions are satisfied; it is only an existence
theorem.
Copyright © 2019 University of Leeds UK. All rights reserved
Concept of necessary and
sufficient conditions for optimum

• Necessary Conditions: The conditions that must be

satisfied at the optimum point are called necessary.
Stated differently, if a point does not satisfy the
necessary conditions, it cannot be optimum.
• Sufficient Condition: If a candidate optimum point
satisfies the sufficient condition, then it is indeed an
optimum point.
The optimality conditions for unconstrained or constrained
problems can be used in two ways:
1. They can be used to check whether a given point is a
local optimum for the problem.
2. They can be used to solve for the local optimum points.

Copyright © 2019 University of Leeds UK. All rights reserved

• Necessary & Sufficient conditions without constraints
• Find
𝑥1
𝑥2
𝑿= ⋮ 𝑤ℎ𝑖𝑐ℎ 𝑚𝑖𝑛𝑖𝑚𝑖𝑠𝑒𝑠 𝑓(𝑿)
𝑥𝑛−1
𝑥𝑛

𝑥𝑖𝐿 ≤ 𝑥𝑖 ≤ 𝑥𝑖𝑈 𝑜𝑟 𝑥𝑖 ≥ 0; 𝑖 = 1, 2, … , 𝑛

Copyright © 2019 University of Leeds UK. All rights reserved

Necessary condition

• Suppose that f :ℝn→ℝ is a real-valued function defined on

some set Ω ⊂ ℝn
• If f has a local extremum (minimum or maximum) at X=X* then:
 
 f X*
 
 x1 * 
 f X   
  f X*  
f X*  
f X* 
T

 
f X   x
*
   0
 2
  x1 x2 xn 
  
 f X *

 xn 

• This is also called 1st order necessary condition

• The condition does not say what happens if a minimum or
maximum occurs at a point X* where the derivative fails to exist
hence the theorem is not applicable.

Copyright © 2019 University of Leeds UK. All rights reserved

Sufficient condition

• Suppose that f :ℝn→ℝ is a real-valued function defined on some set Ω ⊂ ℝn

• If f has a local extremum (minimum or maximum) at X=X* then the Hessian

 
matrix H(X*):
H( X* )  D 2 f X*  0 ( ve definite) or
D2 f X   0( ve definite )
*

 f 2 f 2 f 2 
 2  
 x12 x1 x2 x1 xn 
 f f 2 f 2 
 
where D 2 f X*   x2 x1

x22

x2 xn 

 2   
 f f 2 f 2 
 x x 
 n 1 xn x2 xn2 
• This is also called 2nd order sufficient condition
Copyright © 2019 University of Leeds UK. All rights reserved