MECH 3042: Numerical Techniques

Optimization

Woo Kyun Kim, PhD

Associate Professor
Department of
Mechanical and Materials Engineering
University of Cincinnati
 Objectives
The primary objective of this chapter is to introduce you to how
optimization can be used to determine minima and maxima of both
one-dimensional and multidimensional functions. Specific objectives
and topics covered are
• Understanding what optimization means in engineering and
scientific problem solving.
• Recognizing the difference between one-dimensional and multi-
dimensional optimization and distinguishing between global and
local optima.
• Locating the optimum of a single-variable function with the golden-
section search and parabolic interpolation.
• Knowing how to apply the fminbnd function to determine the
minimum of a 1-D function.
• Visualizing 2-D functions with MATLAB contour and surfc commands
and understanding how to apply fminsearch to minimize a
multidimensional function.
MECH 3042 – Numerical Techniques 2
 Optimization
• Optimization is the process of creating something that is as
effective as possible.
Example 1: The daily cost of a manufacturer when
producing units is given by
2 5000
𝐶 ( 𝑥 ) =0.0001 𝑥 −0.08 𝑥 +65+
𝑥
What is the minimum daily cost?
Example 2: Find the largest rectangle that fits inside the
graph of the parabola below the line with the top side of the
rectangle on the horizontal line .

MECH 3042 – Numerical Techniques 3

 Optimization
Example 3: The bonding between two atoms can be
modeled by the Lennard-Jones potential.

[( ) ( ) ]
12 6
𝜎 𝜎
𝐸 (𝑟 )=4 𝜀 −
𝑟 𝑟
and are parameters, e.g., for argonand .

MECH 3042 – Numerical Techniques 4

 Mathematical Description
• From a mathematical perspective, optimization deals with
finding the maxima and minima of a function that depends on
one or more variables.
• Most practical optimization problems require numerical,
computer solutions.

MECH 3042 – Numerical Techniques 5

 One-Dimensional vs. Multidimensional
• One-dimensional problems involve functions that depend on a
single dependent variable—for example, f(x).
• Multidimensional problems involve functions that depend on
two or more dependent variables—for example, f(x,y).
• The process of finding a maximum is essentially identical to
finding a minimum because the same value x* both minimizes
f(x) and maximizes –f(x).

MECH 3042 – Numerical Techniques 6

 Global vs. Local
• A global optimum represents the very best solution while a
local optimum is better than its immediate neighbors. Cases
that include local optima are called multimodal.
• Generally, we desire to find the global optimum.

MECH 3042 – Numerical Techniques 7

 One-Dimensional Minimization - Bracketing
• For a continuous function a minimum is bracketed when there
is a triplet of points, , such that and . Then, the function has a
minimum in .
• According to “Numerical Recipes”, the initial bracketing of a
minimum is an essential part of any one-dimensional
minimization.
X
X 𝑓 (𝑥 𝑚) If , then becomes the next triplet.
𝑓 (𝑥¿¿3)¿
𝑓 (𝑥¿¿1)¿ X
𝑓 (𝑥¿¿2)¿ If , then becomes the next triplet.
𝑥1 𝑥2 𝑥𝑚 𝑥 3 x
X
X 𝑓 (𝑥 𝑚) If , then becomes the next triplet.
𝑓 (𝑥¿¿3)¿
𝑓 (𝑥¿¿1)¿ X
𝑓 (𝑥¿¿2)¿ If , then becomes the next triplet.
𝑥1 𝑥𝑚 𝑥2 𝑥3 x
MECH 3042 – Numerical Techniques 8
 1-D Minimization (1): Golden-Section Search
What is the golden ratio?

1+ √ 5
𝜙= =1.618 …
2
𝑙1 +𝑙 2 𝑙 1
= =𝜙
𝑙1 𝑙2

MECH 3042 – Numerical Techniques 9

 Golden-Section Search - Method
How does the golden-section search method work?
X
If , then X
X
𝑥𝑚 − 𝑥2
=𝑤 𝑥𝑚 = 𝑥 2 + 𝑤 ( 𝑥 3 − 𝑥 2 ) 𝑥1 𝑥2 𝑥3
𝑥3 − 𝑥2 𝑥𝑚
X
If , then X
X
𝑥 2 − 𝑥𝑚
=𝑤 𝑥𝑚 = 𝑥 2 −𝑤 ( 𝑥2 − 𝑥 1)
𝑥2 − 𝑥1 𝑥1 𝑥𝑚 𝑥 2 𝑥3

• Error quantification:
(𝑖) (𝑖 ) ( 𝑖) ( 𝑖) (𝑖 )
𝐸 =max ( 𝑥 − 𝑥 , 𝑥 − 𝑥 )
𝑎 2 1 3 2

𝜀(𝑎𝑖) =
| |𝐸(𝑎𝑖)
𝑥 (𝑖 )
2
× 100 %

MECH 3042 – Numerical Techniques 10

 Golden-Section Search - Discussions
• If the triplet is already in the golden ratio, all the subsequent
triplets remain in the golden ratio. If not, they converge to the
golden ratio.
• Once the triplet is in the golden ratio, the length of the next
triplet is always reduced to whichever decision is made.

MECH 3042 – Numerical Techniques 11

 1-D Minimization (2): Parabolic Search
• Another algorithm uses parabolic interpolation of three points
to estimate optimum location.
• The location of the maximum/minimum of a parabola defined
as the interpolation of three points (, , and ) is:
1 ( 𝑥 2 − 𝑥 1 ) 2 [ 𝑓 ( 𝑥 2 ) − 𝑓 (𝑥 3) ] − ( 𝑥 2− 𝑥 3 ) 2 [ 𝑓 ( 𝑥 2 ) − 𝑓 (𝑥 1) ]
𝑥𝑚 =𝑥 2 −
2 ( 𝑥 2− 𝑥 1 ) [ 𝑓 ( 𝑥 2 ) − 𝑓 (𝑥 3) ] − ( 𝑥 2− 𝑥 3 ) [ 𝑓 ( 𝑥 2 ) − 𝑓 (𝑥 1) ]

MECH 3042 – Numerical Techniques 12

 MATLAB BUILT-IN FUNCTION: fminbnd
• fminbnd is designed for one-dimensional minimization, which
is based on Brent’s method; a hybrid approach combining the
slow, dependable golden-section search with the faster, but
possibly unreliable, parabolic interpolation.
• Example usages:
>> [xmin fval]=fminbnd(function, x1, x2)
or
>> options=optimset(‘display’,’iter’);
>> [x fx]=fminbnd(function,x1,x2,options)
or
>> options=optimset(‘tolx’, 1.0e-3)
>> [x fx]=fminbnd(function,x1,x2,options)
MECH 3042 – Numerical Techniques 13
 Multi-Dimensional Minimization – Line Search
• In line search method, we start from a point (initial guess) in
N-dimension and proceed from there in some vector
direction , then any function of N variables can be minimized
along the line by one-dimensional methods.
𝑓 ( 𝒙 + 𝛼 𝒏) =𝑔 (𝛼)

• Various line search methods

(e.g. steepest descent method,
conjugate gradient method)
consist of such line
minimizations and different 𝒙1 𝒙 2 𝒏1
methods will differ only by how,
at each stage, they choose the 𝒏 3 𝒙 3𝒏 2
next direction to try.

MECH 3042 – Numerical Techniques 14

 Multi-Dimensional Visualization Using MATLAB
• contour(): creates a contour plot.
• surfc(): creates a 3-D surface plot with a contour
underneath.

MECH 3042 – Numerical Techniques 15

 MATLAB BUILT-IN FUNCTION: fminsearch
• fminsearch can be used to determine the minimum of
multidimensional function. It is based on Nelder-Mead
method, which is a direct-search method that uses only
function values and does not require derivatives.
• Example usages:
>> [xmin fval]=fminsearch(function, x0)
or
>> options=optimset(‘display’,’iter’);
>> [x fx]= fminsearch(function,x0,options)
or
>> options=optimset(‘tolx’, 1.0e-3)
>> [x fx]= fminsearch(function,x0,options)
MECH 3042 – Numerical Techniques 16
 Questions?

MECH 3042 – Numerical Techniques 17