Scribd
Upload
77 views

Linear vs. Nonlinear Objective Functions

The document discusses linear and nonlinear optimization problems. For nonlinear optimization problems, local optima may exist in the interior or on the boundaries of constraints, and local optima are not necessarily global. The document proves that for convex functions on convex sets, local minima are also global minima.

Uploaded by

Saghir Ahmad
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
77 views

Linear vs. Nonlinear Objective Functions

The document discusses linear and nonlinear optimization problems. For nonlinear optimization problems, local optima may exist in the interior or on the boundaries of constraints, and local optima are not necessarily global. The document proves that for convex functions on convex sets, local minima are also global minima.

Uploaded by

Saghir Ahmad
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Nonlinear Optimization: Introduction

Linear vs. nonlinear objective functions


Linear Nonlinear

When objective function is linear Optimum always attained at constraint boundaries A local optimum is also a global optimum When objective function is nonlinear Optima may be in the interior as well as at boundaries of constraints A local optimum is not necessarily a global optimum
September 12, 2007 (1 : 6)

Nonlinear Optimization: Introduction

Convexity and optimality


f(x)

A convex function f on a convex set S : f is below or on the linear interpolant between any two points in the set x1 , x2 S
)+ (x 1 f
x1

( )f 1

x 2)

x2

f x1 + (1 )x2 f (x1 ) + (1 )f (x2 ) [0, 1] Examples of convex functions on R n (a) f (x ) = c T x (also concave!) T (b) f (x ) = + c T x + 1 2 x Qx (Q positive semidenite matrix)
September 12, 2007 (2 : 6)

Nonlinear Optimization: Introduction

S Global minimizer: x S such that f (x ) f (x ) x S Local minimizer: x S such that, for some > 0, f (x ) f (x ) x S B (x ; ) where B (x ; ): ball of radius

Rn

f(x)
Local Local

Global

centered at x

Theorem. For convex functions on convex sets holds that each local minimizer is a global minimizer
Example: min f (x ) such that
x

Ax b , where f (x ) = c T x (linear program) or T f (x ) = + c T x + 1 2 x Qx , Q positive semidenite (quadratic program)


September 12, 2007 (3 : 6)

Nonlinear Optimization: Introduction

Proof. Let x be a local minimizer. Then > 0 s. t. f (x ) f (x ) x S B (x ; ). Let x S arbitrarily. Will show f (x ) f (x ). S convex x + (1 )x = x S x local min f (x ) f (x ) = f x + (1 )x [f convex] f (x ) + (1 )f (x ), that is, f (x ) f (x ) f (x ) f (x )

[0, 1]

Choose > 0 such that x S B (x ; )

>0

September 12, 2007

(4 : 6)

Nonlinear Optimization: Introduction

Unconstrained optimization

Will start to consider unconstrained optimization min f (x ) n or, equivalently, Find x R n such that f (x ) f (x ) x R n

x R

Function f is nonlinear in x . Unconstrained optimization meaningless for linear f , since linear f on R n are unbounded or constant Most common application for unconstrained optimization: inverse problems, parameter estimation

September 12, 2007

(5 : 6)

Nonlinear Optimization: Introduction

Unconstrained optimization: example application


At time intervals t1 , t2 , . . . , tm a physical process generates a time sequence of m observations (measurements) b1 , b2 , . . . , bm A model of the process says that bk b (tk ) where b (t ) = x1 + x2 e x3 t + x4 e x5 t Model not exact: measuring errors (noise), modeling errors We want to nd the coecients x1 , . . . , x5 that best matches the observations Dene the residual rk = bk (x1 + x2 e x3 tk + x4 e x5 tk ) and solve the unconstrained optimization problem
m x1 ,...,x5

min

2 rk k =1

September 12, 2007

(6 : 6)

You might also like