Computational Optimization and Applications 5,97-138 (1996)

@ 1996 Kluwer Academic Publishers. Manufactured in The Netherlands.

A Class of Smoothing Functions for Nonlinear

and Mixed Complementarity Problems*
CHUNHUI CHEN [email protected]
O.L. MANGASARIAN [email protected]
Computer Sciences Department, Universiry of Wisconsin, 1210 West Dayton Street, Madison, WI 53706

Received October 27,1994; Revised February 21, 1995

Abstract. We propose a class of parametric smooth functions that approximate the fundamental plus function,
(x)+ =max(O. x). by twice integrating a probability density function. This leads to classesof smooth parametric
nonlinear equation approximations of nonlinear and mixed complementarity problems (NCPs and MCPs). For
any solvable NCP or MCP, existence of an arbitrarily accurate solution to the smooth nonlinear equation as well as
the NCP or MCP, is established for sufficiently large value of a smoothing parameter o. Newton-based algorithms
are proposed for the smooth problem. For strongly monotone NCPs, global convergence and local quadratic
convergence are established. For solvable monotone NCPs, each accumulation point of the proposed algorithms
solvesthe smooth problem. Exact solutions of our smooth nonlinear equation for various values of the parameter CL,
generate an interior path, which is different from the central path for interior point method. Computational results
for 52 test problems compare favorably with those for another Newton-based method. The smooth technique is
capable of solving efficiently the test problems solved by Dirkse and Ferris [6], Harker and Xiao [ 111 and Pang &
Gabriel [28].

Keywords: smoothing, complementarity problems

1. Introduction

The complementarity condition

where x and y are vectors in R” and the symbol I denotesorthogonality, plays a funda-
mental role in mathematical programming. Many problems can be formulated by using
this complementarity condition. For example, most optimal@ conditions of mathematical
programming [24] as well as variational inequalities [4] and extended complementarity
problems [9, 21, 38-Jcan be so formulated. It is obvious that the vectors x and y satisfy
complementarity condition if and only if

x=(x-y)+,

*This material is based on research supported by Air Force Office of Scientific Research Grant F49620-94-l-0036
and National Science Foundation Grant CCR-9322479.
98 CHEN AND MANGASARIAN

where the plus function (.)+ is defined as

(O+ = md!i, 01,

for a real number 6. For a vector x, the vector (x)+ denotes the plus function applied to each
component of X. In this sense, the plus function plays an important role in mathematical
programming. But one big disadvantage of the plus function is that it is not smooth because
it is not differentiable. Thus numerical methods that use gradients cannot be directly
applied to solve a problem involving a plus function. The basic idea of this paper is to
use a smooth function approximation to the plus function. With this approximation, many
efficient algorithms, such as the Newton method, can be easily employed.
There are many Newton-based algorithms for solving nonlinear complementarity prob-
lems, variational inequalities and mixed complementarity problems. In [lo] a good sum-
mary and references up to 1988 are given. Generalizations of the Newton method to
nonsmooth equations can be found in [32, 331 and [34]. Since then, several approaches
based on B-differentiable equations were investigated in [ 11, 261 and [27]. In addition,
an algorithm based on nonsmooth equations and successive quadratic programming was
given [28], as well as a Newton method with a path following technique [6,30], and a trust
region Newton method for solving a nonlinear least squares reformulation of the NCP [22].
With the exception of [22], a feature common to all these methods is that the subproblem
at each Newton iteration is still a combinatorial problem. In contrast, by using the smooth
technique proposed here, we avoid this combinatorial difficulty by approximately reformu-
lating the nonlinear or mixed complementarity problem as a smooth nonlinear equation.
Consequently, at each Newton step, we only need to solve a linear equation. This is much
simpler than solving a mixed linear complementarity problem or a quadratic program.
Smoothing techniques have already-been applied to different problems, such as, Zt-
minimization problems [19], multi-commodity flow problems [29], nonsmooth program-
ming [ 18, 391, linear and convex inequalities [3], and linear complementarity problems
[2,3] and [15]. These successful techniques motivate a systematic study of the smoothing
approach. Questions we wish to address include the following. How to generate new
smoothing functions? What is a common property of smoothing functions?
In Section 2, we relate the plus function through a parametric smoothing procedure,
to a probability density function with a parameter /I. As the parameter B approaches
zero, the smooth plus function approaches the nonsmooth plus function (.)+. This gives
us a tool for generating a class of smooth plus functions and a systematic way to develop
properties of these functions. In Section 3, we approximate the NCP by a smooth parametric
nonlinear equation. For the strongly monotone case, we establish existence of a solution
for the nonlinear equation and estimate the distance between its solution and the solution of
original NCP For a general solvable NCP, existence of an arbitrarily accurate solution to the
nonlinear equation, and hence to the NCP, is established. For a fixed value of the smoothing
parameter (Y = $, we give a Newton-Armijo type algorithm and establish its convergence.
In Section 4, we treat the MCP, the mixed complementarity problem (21). For the case of
a solvable monotone MCP with finite bounds I, u E R”, we prove that if the smoothing
parameter p is sufficiently small, then the smooth system has a solution. An efficient smooth
algorithm based on the Newton-Armijo approach with an adjusted smoothing parameter is
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 99

also given and convergence is established. In Section 5 we show that exact solutions of
our smooth nonlinear equation, for various values of the smoothing parameter /3 generate
an interior path to the feasible region, different from the central path of the interior point
method [ 171. We compare the two paths on a simple example and show that our path gives
a smaller error for the same value of the smoothing parameter B. In Section 6, encouraging
numerical testing results are given for 52 problems from the MCPLIB [7] which includes
all the problems attempted in [6, 111 and [28]. These problems range in size of up to 8192
variables. These examples include the difficult von Thiinen NCP model [28, 371 which is
solved here to an accuracy of 1.Oe - 7.
A few words about our notation. For f: R + R and x E R”, the vector f(x) in R” is
defined by the components(f (x))i = f (Xi), i = 1, . . . , n. The support set of f(x), which
is the set of points such that f(x) # 0, will be denoted by supp(f (x)}. The set of m-by-n
real matrices will be denoted by RmMI. The notation 0 and 1 will representvectors with all
components0 and 1 respectively, of appropriate dimension. The infinity, 1, and l2 norms
will be denoted by ]I . Iloo, I] . 111and I] . 112
respectively. The identity matrix of arbitrary
dimension will be -denoted by I. For a differentiable function f: R” + Rm, V f will
denote the m x n Jacobianmatrix of partial derivatives. If F(x) hasLipschitz continuous
first partial derivatives on R” with constant K > 0, that is

IlVF(x) - VF(y)II 5 Kllx - yll, Vx, y E R”,

we write F(x) E LCk(Rn).

2. A class of smoothing functions

We consideraclassof smoothapproximationsto the fundamentalfunction (x)+ = max{x, 0)

Notice first that (x>+ = I_“, u (y)dy, where e(x) is the stepfunction:

a(x) =
11 ifx>O
0 ifxF0

The step function a(x) can in turn be written as, a(x) = [:,6(y) dy, where 6(x) is the
Dirac delta function which satisfiesthe following properties
+CO
S(x) 10, S(y) dy = 1.
s --oo

Figures 1to 3 depict the above functions. The fact that the plusfunction is obtainedby twice
integrating the Dirac delta function, prompts us to propose probability density functions
as a meansof smoothing the Dirac delta function and its integrals. Hence we consider
the piecewise continuous function d(x) with finite number of pieces which is a density
function, that is it satisfies
00
d(x) 10 and d(x) dx = 1. (1)
J --bo
 10

-2
-10 4 644 0 2 4 6 8 10

Figure 1. The plus function (x)+ = max(x, 0).

2
44 = S wcy -
1.5 *

0.5 -

0.

-1
-104 d 4 -2 0 2 4 8 8 10

Figure 2. The step function 13(x) = 1 if x z 0.0 if x 5 0.

1.2 - w - -

l-

0.4 -

o- * a * .
-104 6 4 -2 0 2 4 6 0 10

Figure 3. The Dirac delta function S(x) .

NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 101

To parameterize the density function we define

,. 1 x
t(x, PI = $ s (2)
0

where p is a positive parameter. When B goesto 0, the limit of i(x, jl) is the Dirac delta
function 6(x). This motivates a classof smooth approximations asfollows:
x
i(t,j?)dtx ff(x)
Sk B>= s--bo
and

(3)

Therefore, we can get an approximate plus function by twice integrating a density function.
In fact, this is the sameasdefining

x
ixx, B) = +O”(x - t)+i(t, /3) dt = (x - t)i(r , p) dt. (4)
/ --co J-cm
This formulation wasgiven in [18] and [ 13, p. 121for a density (kernel) function with finite
support. We will give our resultsin termsof a density function with arbitrary support. This
includes the finite support density function asa specialcase.

Proposition 2.1. Let d(x) be a probability densityfunction and i(x, B> = id($), where
#I is a positive parameter: Let d(x) satisfy the following assumptions:
(Al) d(x) is piecewise continuous withBnite number ofpieces and satisfies (1).
W) W4lcqx, = ./-‘,” 1x1d(x) dx < +CQ.
Then the dejinitions of $(x, @) given by (3) and (4) are consistent.

Proof: By the definition (2) and assumption(A2), we have that p(x, /3) defined by (4)
satisfies
r z
j%x,B)=x s-co’ d(t) dt -p
s ’ td(t) dt
--oo
(5)

By direct computation,

5
P’CX,
B>= J-co d(t) dt
x
= t^(t, /?) dt = sI(x, #?) (6)
s -m
102 CHEN AND MANGASARIAN

Hence the derivatives of 5(x, B) defined by (3) and (4) are the same and the difference
between the two representations of j?(x, p) is a constant, say c. If we let x approach -oo
in both (3) and (4), then j(x, /I) approaches 0 in both, and hence c = 0. Therefore the
definitions of j(x, /3) given by (3) and (4) are consistent. 0

Now we give properties of j(x, ,5) that show that it is an accurate approximation of the
plus function (x)+ as p approaches zero.

Proposition 2.2 (Properties of fi(x, jS>,j? >O). Let d(x) and i(x, p) be as in Proposition
2.1, and let d(x) satisfy (Al) and (A2). Then Z?(x, /3) has thefollowing properties:
(1) 5(x, /I) is continuously differentiable. If, in addition, d(x) is k-times continuously
differentiable, 6(x, t?I) is (k + 2)-times continuously differentiable.
(2) -028 5 Ij(x, B) - (xl+ 5 DIP, where

0
D1 = Ix Id(x) dx (7)
s -co

+bo
D2 = max xd(x) dx, 0 (8)
IS-m
(3) 5(x, p) is nondecreasing and convex. In addition, let d(x) satisfy

(A3) supp {d(x)) = R.

Then $(x, /I) is strictly increasing and strictly convex in x for a@ed B P- 0.
(4) 0 5 $‘(x, /3) 5 1 and if(A3) is satisfied, then 0 -C j’(x, #?) c 1.
(5) Let (A3) be satisfied and D2 = 0, then

Proof:

(1) By Eq. (6) in the proof of last proposition, the conclusion follows.
(2) If x 2 0, by using (5), we have that
I. d
BCX, B) - (x>+ = x ’ d(t) dt - /9 ’ td(t) dt -x
s -co s -co
co 00 ce
= -x d(t) dt + /3 td(t) dt - /3 td(t) dt
s s s

=Z$i(t-;)d(t)ci-/!?~~td(t)~w
P
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 103

Therefore

b(x,
B>
-(x>+
2-B
s03 -cm
td(t) dt 2 -f?Dz

and

ItId dt = BDl.

Otherwise, x < 0, then

& 1
@(x, /!3) - (x)+ = x ’ d(t) dt +/3 ’ JtJ d(t) dt
J-m f --a0
0
rS ItI d(t) dt = 0,s
s -CO

and

(3) By Eq. (6) and the fact that d(x) > 0,

B)=s%
y<x, --co
d(t) dt 2 0

and

(i’k B> - b’(y, #0)(x - y) = (x - y) lx d(t) dt 2 0

Y

Therefore b(x, /?) is nondecreasingand convex. If, in addition, (A3) is true, then

b’(x, B> > 0 and <$‘<x, B> - $‘(y, /9)(x - y) > 0, for x # y

So 6(x, /3) is strictly increasingand strictly convex.

(4) By formula (6), it is easy to seethat 0 5 fi’(x, p) 5 1. If (A3) is true, d(x) > 0 and
c/(x, /3) is strictly increasing. Therefore 0 < j’(x, /3) < 1.
104 CHEN AND MANGASARIAN

(5) By (A3), we have

JC-;) d(r) dt > 0, x I 0

and

d(r) dt > 0, x < 0

By the similar proof of (2), we have

/Xx, B) - (xl+ > -DzS = 0

Therefore

Now we have a systematicway for generatinga classof smoothplus functions. Given any
probability density function d(x) satisfying (Al) and (A2) we define i(x, /I) and b(x, fi)
asin (2) and (3) respectively. The smoothfunction j(x) /3) approximatesthe plus function
with increasing accuracy asthe parameter#l approaches0. The properties of the function
j(x, p) are given in Proposition 2.2 above.
We now give examplesof smoothplus functions. The first example, which will be used
throughout this paper, is basedon the sigmoid function of neural networks [3, 12, 201 and
defined asfollows:
1
s(x, a) = a!>0
1 + e--ax ’

This function approximatesthe stepfunction o (x) as(II tendsto infinity. Since the derivative
with respect to x of this function tendsto the Dirac delta function as(Ytends to infinity, it
follows that a! plays the role of $ and we shall therefore take

1
a! = -.
B
Example 2.1 (Neural Networks SmoothPlus Function [3]). Let

e-I
d(x) =
(1 + e+)2

Here DI = log2, 02 = 0 and supp{d(x)) = R, where D1 and D2 are defined by (7)

and (8). Integrating $d( F) twice gives

j(x, /I) = x + #3log (1 + e-“)

NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 105

Letting IX = i, we have

= (Ys(x, a>(1 - S(X, a))

Figures 4 to 6 depict the functions p(x, 5), s(x, 5) and t (x, 5) respectively.

10 F -

P(G 4
5

-10 -5 6 4 -2 0 2 4 5 5 10

Figure 4. The p function p(x, (I) = x + d log( 1 + e+“) with a! = 5.

2
8(W) --
8(W)
1.5 '

1 -

0.5 -

00.

0.5 .

-1L
-1 - a * *
-10
-10 .5
d 4 -4 -2 0 2 4 5 5 10

Figure 5. The sigmoid function s(x, (u) = & with cr = 5.

106 CHEN AND MANGASARIAN

1.2

0.6

0.6

0.4

0.2

e
--lo d -6 -4 -2 0 2 4 6 8 10

Figure 6. The f function t(x, 01) = C,~C~~jz witha = 5.

Following are severalother smoothplus functions basedon probability density functions

proposedby other authors.

Example 2.2 (Chen-Harker-Kanzow-Smale Smooth Plus Function 121, [15] and 1361).
Let

2
d(x) =
(x2 + 4)f

Here Dr = 1, 02 = 0, supp{d(x)) = R and

Example 2.3 (Pinar-Zenios SmoothPlus Function 1291). Let

1 if05xll
d(x) =
0 otherwise

Here D1 = 0, 02 = i, supp{d(x)} = [0, 11and

0 ifx CO
b(x,B)= 5 ifOlx (fi
i x- $ ifx>/3

This function can also be obtained by applying the Moreau-Yosida regularization [13,
p. 131to the plus function.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 107

Example 2.4 (Zang SmoothPlus Function 1391). Let

1 if-isxs$
d(x) =
0 otherwise

Here DI = i, DZ = 0, supp{d(x)} = [-i, i] and

1
0 if xc-g
jxx, B) = $j(x + $)” if 1x15 $
x if x>$

Note that in Examples 2.3 and 2.4 above, the density function d(x) hascompact support
while the smoothfunction F(x, B) is only oncecontinuously differentiable. In Examples 2.1
and 2.2, d(x) hasinfinite support while the functions p(x, cr) and b(x, B) are differentiable
infinitely often.
We summarizethe various functions introduced asfollows:

(x>+ -L a(x) A- 6(x)

1 1
pb, a) = @ x, A ,I s(x,a) =i x,- -A- t(x,cx) =; x,-
( > ( (Y> ( a>

Becauseof our favorable experience with the function p(x, o) [3] on linear complemen-
tarity problems and linear and convex inequalities, we chose it for our numerical experi-
ments. Further comparisonsusing different approximations to the plus function are left for
future work.

3. The nonlinear complementarity problem

In this section we consider the nonlinear complementarity problem (NCP) of finding an x

in R” such that

0 5 x I F(x) 2 0 (9)
108 CHEN AND MANGASARIAN

Here F(x) is a differentiable function from R” to R”. By using the smoothfunction 3(x, B)
introduced in last section, we considerthe smoothnonlinear equation

R(x) =x - $(x - F(X), /I) = o (10)

as an approximation to the following nonsmoothequivalent reformulation of the NCP

x = (x - F(x))+ (11)

We first show that a natural residual for the NCP is easily bounded by a corresponding
residual for the nonlinear Pq. (10).

Lemma 3.1.

lb - (x - F(x))+llp I Ix - i<x - F(x), B>II/J

+ yp maxI&, &}fi, p = 1,2,oo, (12)

whereyl =n, B=yt;landy,= 1. The constants D1 and D2 dependon the density

function usedand are defined in (7) and (8).

Proof:

lb - (x - Fb)>+llp 5 IIX - fib - F(x), B) + m - F(x), B) - (x - W))+llp

L lb - h - F(x), B>llp+ II% - F(x), B) - (x - F(x))+II,
5 llx - i%x - F(x), B)llp + vp max{Dt, DdB. 0

The above result is also true for any monotone norm [25].
We first consider the strongly monotone NCP, that is there exists a k > 0 such that for
anyx, y E R”

(F(x) - F(Y)>~(x - Y) 2 kllx - Y II2 (13)

Since the NCP is strongly monotone, it has a unique solution [lo]. The following error
bound for the strongly monotone NCP is given asTheorem 3.2.1 in [31].

Lemma 3.2. Let the NCP be strongly monotoneand let F(x) be Lipschitz continuous.
Thenfor any x E R”

lb - Xllp 5 C,Ilx - (x - F(x))+(I,, p = 1,2,oo, (14)

where X is the unique solution of the NCP and C, is a condition constant of F independent
ofx.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 109

Now, we give an error bound for the NCP by using the smooth function j?(x, ,3). By
Lemma 3.1 and Lemma 3.2, it is easy to get the following lemma.

Lemma 3.3. Let the NCP be strongly monotone and let F(n) be Lipschitz continuous.
Then for any x E R”

lb - Xllp 5 c,w -b<x - F(x),B)llp+r!,max{ol,D~}B>, p = 1,2,co, (15)

where X and C, are defined in Lemma 3.2, and y!, and the constants Dt, 02 are dejined in
Lemma 3.1.

Let the residual f(x) of the nonlinear Eq. (10) be defined as follows

f(x) = ;WTW4 = $1~ - F(x - F(x), iOIl; (16)

We now prove that if x is a stationary point of f(x) for a monotone F(x), then x must be a
solution of the nonlinear Eq. (lo), and hence by (15) x is an approximate solution of the NCP.

Proposition 3.1. Suppose that d(x) satisjes (Al)-(A3) and p(x, /?) is defined by (4).
For any monotone NCP, we have that V R(x) is positive definite. In addition, let x be a
stationary point of f(x), then x mustbe a solution of the nonlinear Eq. (IO).

Proof: By definition,

VR(x) = diag@‘(x - F(x), B))(diag(j’-‘(x - F(x), /3>- 1) + VF(x))

By (4) of Proposition 2.2, we have 0 c j’(x, B) < 1 and hencethe diagonal matricesabove
are positive definite. Since V F (x) is positive semidefinite,it follows that V R(x) is positive
definite. Let x be a stationary point of f(x), then

Vf(x) = VR(x)rR(x) = 0.

Since VR(x) is nonsingular, R(x) = 0, it follows that x satisfies(10). 0

When F(x) is strongly monotone and Lipschitz continuous, then the level setsof f(x)
are compact. We state this result asthe following proposition.

Proposition 3.2. Consider the strongly monotoneNCP with Lipschitz continuous F(x).
Then f(x) defined by (16) has compact level sets.

Proof: Supposenot, then there exists a sequence{xk} c R” and a positive number M such
that llxk 112
--+ co ask + co, and llxk - $(xk - F(xk), 8)112_( M. Then by Lemma 3.3,

11.~- .f II2 5 G.W + ~2maxI&, &MC

110 CHEN AND MANGASARIAN

where X is the unique solution of the NCP. Let k ----f 00, the left hand side of the above
inequality goesto 00 and the right hand side staysfinite. This is a contradiction. Hence the
level setsof f(x) are compact. 0

We now show that, for the strongly monotone NCP with Lipschitz continuous function
F(x), the nonlinear Eq. (10) always hasa unique solution.

Theorem 3.1. Supposethat d(x) safisJes(Al)-(A3), j(x, /?) is deBned by (4), F(x) is
strongly monotone and Lipschitz continuous. Then the nonlinear Eq. (10) has a unique
solution.

Proof: By Proposition 3.2, the level setsof f(x) are compact. So minXeRaf(x) must
have a solution X, which is a stationary point of f(x). By Proposition 3.1 we get that x
satisfies(10). If y is another solution of (lo), then

0 = (x - y)(R(x) - R(y)) = (x - y) j+;’ VR(x + t(y -x)1 dt(x - y>

for somet E [0, 11. Since VR is positive definite by Proposition 3.1, it follows that x = y.
Therefore Eq. (10) has a unique solution. 0

Let x(p) be a solution of (10). Then x(p) = @(x(B) - F(x@)), p). By Lemma 3.3, we
have the following theorem which boundsthe distancebetween the solution x(/l) of (10)
and the solution point of the original NCP (9).

Theorem 3.2. Consider a strongly monotone NCP with Lipschitz continuous F(x). Let
x(B) be a solution of (10). Then, for the solution X of the NCP (9), we have that

b(B) -ZIP 5 C,Y, maxi&, 02U3, p = 1,&m

Here C, is the condition constant defined in Lemma 3.2, yp and D1, 02 are constants
de$ned in Lemma 3.1.

By the above result, we know that if p is sufficiently small, a solution of (10) can
approximate the solution of NCP to any desiredaccuracy. Hence we can solve (10) to get
an approximate solution of the NCP.
For most part of this remaining section, we consideronly the function p(x, 01)defined in
Example 2.1. We explore further the property of real numbersx and y that approximately
satisfy the following equation

x=p(x-Yea)

which is related to Eq. (10) that generatesan approximate solution of the NCl? We claim
that suchx and y will approximately satisfy
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 111

In order to prove this fundamental fact we establishthe following lemma,the proof of which
is relegatedto the Appendix 1.

Lemma 3.4. Let h(x) be deBnedasfollows:

h(x) = -x log(1 + S - emX), 6 1 0

(i) If0 c 6 c 1, then

max h(x) 5 2.
rqo,--logSI

(ii) If8 2 1, then

n-m h(x) 5 max (Slog26, i].

XG[-log&O]

(iii) ZfS = 0, then

max h(x) 5 2.
XC[O,~) e

We will now show that if an x and y satisfy

-2 5 x - p(x - y, a) 5 0,
a!
where 61 > 0, then the complementarity condition 0 5 x I y 2 0 is approximatdy

satisfied for large (Y, in the following sense

C(h)
t-x)+ 5 19 C-y)+ I ;, (XY)+ I ---p

where C(&) is the constant defined in Proposition 3.3. Note that asu + 00, the comple-
mentarity condition 0 I x I y 1. 0 is exactly satisfied.

Proposition 3.3. Let x, y E R satisfy

--615x-p(x-y,a)10,
CY
where 61 > 0. Then

CC&)
t-x>+ I ;, t-Y)+ 5 2, by)+ I Q2’

where

C(&) = max(2, (es1- 1) log2(es1- l)}.

112 CHEN AND MANGASARIAN

PrOOf: Let 6 = e-” + emay - 1, since

-1 ( x - (x - y) - L log(1 + e-+y)) _( 0,
a! (II
we have

1 ( emax + e-‘“Y 5 es1

Hence 0 5 6 _( es1 - 1. Since

e --(Ix 5 emax + emcrY= 1 + 6 5 es],

we have x p - %. Hence

61
(-xl+ 5 -.
a
Similarly,
C-y)+ 5 ;.

Now we considerthe estimateof (xy)+. Since edcrx+ eeay = 1 + S, we have

y = --J log(l + S - e--ax),

Therefore

y > 0W 1 + 6 - eeax 5 1 * e-a’ -> 6 (17)

and

(18)

CaseI. 0 -c 6 -c 1.
If y > 0, then by (17),

x ( -‘log 6.
a!
Hence

by>+ 5 max -rlog(l+6-ePX)

xc[O,-!!g a!

=-j2 yg==;gslMY) (Let Y = ax)

2
<-
- (By (i) of Lemma 3.4)
cz?
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 113

Otherwise y 5 0, then

x > +ogs 20.

Hence ny I 0, (xy)+ = 0.
Cuse2. s 11.
If y 2 0, then by (17),

x&ogs50.
a!

Hencexy 5 0, (xy)+ = 0. Otherwise y 5 0, then x 5 -i logs. If x I 0, then (xy)+ = 0.

Now we only consider the case x < 0. Therefore

t-v)+ 5 max -2 log(1 + 6 - e-ax) = Q2

L rc~~~8,0,~t~) (Let y = W.
XE[-y,O] CY

By (ii) of Lemma 3.4,

Case 3. 6 = 0.
In this case, we have x 1 0 and y > 0. By using (iv) of Lemma 3.4,

1
(XY)+ i - max h(x) 5 -$a
a2 xaca)

Combining the above three cases, we get

C(h)
(XY)+ I -. q
a!2

Even in the caseof a solvable monotone nonlinear complementarity problem (e.g. 0 5

x I F(x) > 0, F(x) := 0), the nonlinear Eq. (10) may not necessarilyhave a solution.
However, for all Si L D1, and 82 2 D2, the following systemof inequalities

always has a solution for #l > 0. In particular, for the p(x, o) defined in Example 2.1 we
have that for all 61 > log 2, the following system of inequalities

-315 x -p(x - F(x),a!) 5 0, (20)

a!
114 CHEN AND MANGASARIAN

always has a solution. Hence by Proposition 3.3, a solution of (20) will approximately
satisfy the NCP condition

x 2 0, F(x) 2 0, XTF(X) = 0.
Consequently, Proposition 3.3 can be used to establish the following useful theorem.

Theorem 3.3. Consider a solvable nonlinear complementarity problem (9). Let 61 L

log 2 and u > 0. There exists x satisfying (20), and consequently the NCP conditions are
approximately satisfied as follows:

c-x>+ 5 -1,
61 (-F(x))+ 5 -1,
f-5 (XTF(X))+ 5 9,
a! (Y

where C(&) is defined in Proposition 3.3.

We now specify our computational algorithm for solving the NCP by smoothing. The
algorithm consists of a Newton method with an Armijo line search with parameters 6 and
asuchthatO<S<landO<a<i.

Algorithm 3.1 (Newton NCP Algorithm). Given x0 E R” and let k = 0.

(1) If IlVf WII -= c, stop.
(2) Direction dk

dk = -VR(x$‘R(xk)

(3) Stepsize hk (Armij,)

xk+l = xk + &dk, & = KlaX{l, 8, a*, . . .), s.t.

f (xk) - f @k+l) >_ oAkjd;Vf (xk)]

k = k + 1 go to step (I).

The above algorithm is well defined for a monotone NCP with a continuously differen-
tiable F(x). We will state the following convergence theorem [5]. We omit the proof that
is similar to the proof of Theorem 4.3.

Theorem 3.4. Consider a solvable monotone nonlinear complementarity problem (9)

with F(x) E LCk(R”). Then
(1) The sequence {xk} defined in Algorithm 3.1 is well defined.
(2) Any accumulation point of the above sequence solves the nonlinear Eq. (10).
(3) If an accumulation point exists, the whole sequence {xk} converges to x quadratically.
(4) If, in addition, F is strongly monotone and Lipschitz continuous, then the sequence
(xk} converges to X, the solution of (lo), at a quadratic rate.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 115

4. The mixed complementarity problem

The mixed complementarity problem (MCP) is defined as follows [6]:

Given a differentiable F: R” -+ R”, 1, u E I?‘, 1 -e u, where k = R U (+cq --cm},
findx, w, v E R”, suchthat

F(x)-w+u=O
05x-ZlwzO (21)
o~ulu-x~o

This MCP model includes many classes of mathematical programming problems, such
as nonlinear equations, nonlinear programming, nonlinear complementarity problems and
variational inequalities.
By using the smooth function j(x, /3) instead of the plus function, we reformulate the
MCP approximately as follows.
Fori = l,...,n:

Case 1. li = --OO and Ui = 00:

F;:(x) = 0

Case 2. Zi > --OO and Ui = 00:

Xi - li - $(-Xi - li - Fi(x), p) = 0

Case 3. li = --OO and Ui < CO:

Xi - ui + $(Ui -xi + I;;:(~),fi) = 0 (22)

Case 4. li > -Wand ui < 00:

I;;:(x)-wi+ui=O
Xi - li - ,G(Xi - li - Wi, @) = 0
ui-Xi-~(Ui-Xi-Ui,B)=O.

We will denote the above 4 cases collectively by the nonlinear equation

R(x,w,u)=O (23)

Note that the natural residual for the MCP is given by the left hand side of above relation
with the jj function replaced by the plus function. We denote collectively this natural
residual by

r(x, w, u) (24)
116 CHEN AND MANGASARIAN

Now we give a lemmathat boundsthe naturalresidualof the MCP by the residualof Eq. (23)
and vice versa. The proof is a simple application of the properties of the p function.

Lemma 4.1. Let N be number of equationsin (23) and p(x, a) de@ed in Example 2.1.
Then

alog
IINx, w ~)I12 5 lb-(x, w, ~)I12 +
a0

lb-(x, w U)ll2 5 IIRk w, ~)I12 + fi1og2

a0

Let f(x, w, u) be the residual function of the nonlinear Eq. (23) defined as follows

f (x, w, u) = ;R(x, w, u)=R(x, w, u) (25)

Now we state an existenceresult for the monotoneMCP with 1, u E R”.

Theorem 4.1. Supposethat d(x) sati@es(Al)-(A3) and b(x, /?) is dejined by (4). Con-
sider a solvable mixed complementarity problem (21) with monotone F(x) and 1, u E R”.
The nonlinear Eq. (23) has a solutionfor suflciently small /%

Proof: We shall prove that a level set of f(x, w, u) is nonempty and compact. First we
will prove that the set X = {x I f(x, w, u) 5 C} is compact for all C E R. Since f is
continuous, the level setX is closed. Hence we only needto showthat the set X is bounded.
Supposenot, then there exists ink} E X and there exists 1 5 i 5 n such that xi” goesto
+oo or -co. Without lossof generality, we assumethat xi” goesto +oo. Then the residual
corresponding to the following equation approaches00:

Ui - Xi - $(Ui - Xi - Vi, B) = 0.

This contradicts the fact that xk E X. Let C = fn max{Di, Dz}~~~, where Di and DZ
are nonnegative constants defined in (7) and (8). It is easy to show that the level set
Levc(f) = {(x, w, u) 1 f(x, w, u) 5 C} is not empty. Now we will prove that Levc(f)
is compact for

IIliIll<i<n(Ui - li)
B<
,/Zmai{Dl, DzJ’

We have proven that the x part must be bounded. Therefore, if the level set Levc(f) is
unbounded,thereexists (xk, wk, u’) E Levcf f) suchthat (wk, uk) are unbounded. Without
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 117

loss of generality, we assume xk + X and there exist 1 I i 5 n such w: + +oo or

-oc as k + 03. If wf + -00, the residual corresponding to the equation

Xi - li - j(Xi - li - Wi, B) = 0.

goesto 00 ask + 00. But (xk, wk, uk) E Levc(f), which is a contradiction. Otherwise,
wf + +CS. By the equation

Fi(X) - Wi + Vi = 0

and the fact that xk is bounded,we get that ask -+ 00, I$ + +m. Hence, ask -+ cm,

f(Xk, Wk, U’) 2 i((*i -Xf - b(Uf - Xf - $9 fi))* + (Xf - li - j(Xf - Zi - Wf, 8))“)

--+ k((Ui - ifi)* + (Xi - li)*) 2 f (Ui - li)* > in max{Di, D*}*p* = C

for all

This contradicts (xk, wk, uk) E Levc(f). Hencethere existsalevel set of f(x, w, u) which
is nonempty and compact. Therefore the problem

min f(x, w, u)
x.w,u

must have a minimum, which satisfies

Vf(x, w, u) = VR(x, w, u)Q(x, w, u) = 0.

Let hi = diag(@‘(x - 1 - w, /3)) and A2 = diag($(u - x - u, /l)), then

VR(x,w,u)= [KY!; $ iJ

Notice that

and 0 c y(x - f - w, /3)), j?‘(u - x - u, 8)) c 1. When F is monotone, the Jacobian

VR(x, w, u) is nonsingular. Therefore

Vf(x, w, u) = 0 d R(x, w, u) = 0. 0
118 CHEN AND MANGASARIAN

From now on, we only consider the function p(x, cr) defined in Example 2.1. The
following theorem is a direct application of Proposition 3.3. that can be proved by applying
the Proportion 3.3 to each equation.

Theorem 4.2. Consider a solvable mixed complementarity problem (21). Let 61 1 log 2
and a > 0. Then the following system of inequalities, i = 1, . . . , n:
Case 1. lj = --oO and Ui = Co:

-1 < Fi(X) = 0 5 1
a!- cd

Case 2. li > -CO and Ui = CO:

-'_(Xj-li-~(Xj-lj-Fj(X),B)iO
cd

Case3. li = --OOand ui < 00:

05 Xi - Uj + j(Ui - Xi + Fi(X), B) 5 $ (26)

Case4. li > --OOand ui < 00:

&
-- 61 5 F;:(X)-Wi+Vj I-
o! 01

-- 61 5 Xi - lj - b(Xj - lj - Wi, fi) 5 0

cd
-'IUi-Xi-~(Ui-Xi-Ui,B)I:O,
a!

where p(x, a) is defined in Example 2.1, always has a solution (x, w, v) which satisjies
the MCP conditions approximately in thefollowing sense,for i = 1, . . . , n:
Case1. li = -CXJand ui = 00:

Case2. li > -CCI and ui = CO:

(lj-Xi>+
I z7 (--Fi(x)>+
5z, ((Xi -li>Fi(X))+ I: %$9
Case3. li = --oO and Ui < 00:

61
(Xi - Ui)+ I -9 (4(X))+ I ZT (-(Ui - Xi)Fi(X))+ 5 y,
ff
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 119

Case4. li > -mandui < 00:

Cli -Xi>+ 5 z7 (-Wi>+ I ZV ((Xi - Zi)Wi)+ _< F,

C(b)
(xi -Ui>+ I $t (-vi>+ I z, ((Ui -Xi>%>+ i -
cd2 ’

where C(Sl) is defined in Proposition 3.3.

Now we state the smooth method for the mixed complementarity problem based on the
Newton Algorithm 3.1 in which the smoothing parameter will be adjusted. In the algorithm,
we adjust the smoothing parameter a! inverse proportion to the natural residual r(x, w, u)
of (24) for the MCP in the following way. Let N be the total number of nonlinear equations
in (23) and let (x, UI, V) be current point. Let

Il~bw.vh if Ilr(x, w, u)ll2 < fi

u(x, w, u) = (27)
Jis
& . . otherwise

The following smooth algorithm generates an E-accurate solution for the MCP, in the sense
that the natural residual r(x, w, u) of (24) satisfies Ilr(x, w, u)llm 5 6.
In order to get an r-accurate solution for the MCP. We need (Y sufficient large. We will
establish a simple lemma before we get such o.

Lemma 4.2. Let real numbers a and b satisfy

(-a)+ I 1, (-b)+ F : and (ab)+ I T,

then

maxI&, XT&J,
bin@, b)l I t
a!
where C(&) is defined in Proposition 3.3.

Proof: Without loss of generality, we assume that a 5 b. If a 2 0, then (ab)+ > a2.
Therefore

(min(a, b)( = a 5 ,/m 5 q.

Ifa (: 0,

61
Jmin(a, b)J = -a = (-a)+ 5 -.
a!
120 CHEN AND MANGASARIAN

Combining the above two cases, the conclusion follows. 0

Therefore to satisfy Imin(a, b)l 5 E, we choose 012 maxt61q6m1. By using 6, = log 2,

we obtain from the definition of C(St), when a! 2 omax p $, that Imin(a, 6)( 5 E.

Algorithm 4.1 (Smooth Algorithm for MCP). Input tolerance E, parameter u1 2 v > 1
and initial guess x0 E R”:
(1) Initialization. For1 5 i 5 nofCase4of(22),letw~ = (Ft(xo))+, I$ = (-Ft(xg))+,
k = 0 and (110= a(xo, wg, ~0). Choose amax 2 $
(2) If Ilr(xk, wk, ~k)iico 5 c, stop.
(3) Newton Armijo step. Find (xk+l , wk+l , Ukfl) by a Newton-Armi,io step applied to

R(x, w, u) = 0.

(4) Parameter update. If U(xk+l, !.&+I, uk+l) 2 vffk, set

*k+l = a@k+l, wk+l, uk+lh

&W-Wise if llvf (xk+l, wk+l, uk+1)112 5 et set

Ifmk+l > a-, set ff]lk+l = a!,. Let k = k + 1, go to step (2).

Let Z denote the index set of the Fi of Case 1, J of Case 2, K of Case 3 and L of Case 4
of (22). In order to characterize the nonsingularity of VR, we now give a definition of a
regular MCP Note that the monotone NCP is regular. More generally, an NCP with a PO .
Jacobian matrix is regular.

Definition 4.1. An MCP is called regular if

is nonsingular, for all positive diagonal matrices DJ , Dk and DL, that have the dimensions
of I .Z1, I K I and IL I respectively.

Theorem 4.3. Consider a solvable regular mixed complementarity problem (21) with
F(x) E LCk(R”). Then
(1) The sequence {xk, wk, uk} defined in Algorithm 4.1 exists.
(2) Any accumulation point of the above sequence is an e-accurate solution of the MCP
(21).
NONLINEAR AND MIXED COMPLEMENTARI-R’ PROBLEMS 121

(3) Zf an UCCUmUhtiOn point eXiStS, the whole sequence{xk, wk, vk} converges to an
E-accurate solution quadratically.

(4) Zf; inn&$ion, the level set {(x, w, v) 1 11r (x3 w, @II2 5 Ilr(x0, ~0, v0>112 +
v-l
g ) is compact, the sequence{xk, wk, vk} convergesto an E-accurate solution
at a qu2fratic rate.

Proof: we denote (xk , wk , vk) by yk and (.f , zz,,5) by j.

(1) Let AJ = diag(p'(xJ - 15 - FJ(x),a)), AK = diag(p’(ug - XK - FK(X),~)),

AL, = diag(p’(xL - IL - w, (Y)) and A Lo = diag(p’(uL - XL - v, a)), then

AJ

Z -AL, AL,

I- ALA-1 AL2

O
I-AJ
Z-AK
+
0
0
0
122 CHEN AND MANGASARIAN

Therefore

VF,(x) 0’
VF.I(X> 0
VFK(X) 0
+
VFL(X) 0
0 0
0 0. 1 AL? - I ALL _

Hence V R(x) w, u) is nonsingular if the MCP is regular. If F E LC; (R”), then

{ (xk, wk, uk)) are well defined.
(2) For an accumulation point j, we have yk, + j. Since the parameter a! can be
changed only finite many times, then (ilk = d, for all k 2 k. Therefore, without loss
generality, we consider the sequence{yk, } for fixed 5. In casethat f(j) = 0, for 5,
we have & = CX,,,~.Otherwise, since ]]Vf(yk,)llz -+ 0, there exists an i such that
]]Vf(y~>]]2 i E. By (4) of Algorithm 4.1, (21will changeto vi&. That contradicts the
definition of 5. Hence R(j) = 0 for a! = (Y-, y- is an e-accurate solution of the MCP
(21). The other caseis that f(j) > 0 for 6. Since F E LCk(R”), for a compact set
S whoseinterior contains {yk,} and j, we have that R(y) E LC;, (S) for someRI. By
Quadratic Bound Lemma [25, p.1441,we have

Ilf(yk, + Ak,dki) - f(yk,) - vf(yk,)Thk,dk, 112 5 2[lAk$k, 11;.

Since VR(y) is nonsingular,on the compact S, there exists K(S) and k(S) such that

x~VR-~(~)VR-‘(~)X 5 K(S)xTx, Vy E S,x E R”

and
~~VR-‘(y)vR-~(y)x > k(S)xTx, Vy E S,x E R”.

Consequently

fok) - .f(Yk, + Ak$k,)

1 -hkiVf(yk,)Tdk, - ‘$‘ii R(yki)TvR(yk,)-TvR(yk,),-lR(yk,)

2 -hk,Vf(yki)Tdk, - T@(Yk,)‘R(yk,)
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 123

= ah ’ - TAk,) Ivf(yk,)‘dki 1

2(1 -a)
z hk,a(Vf(yki)Td(Yk,)l, if kk, 5 KIK(S)

By the rule of choosing hk,, we have kki 2 Sw, where S is the constant used in
Armijo stepsize. Therefore

-f-hi) - f(Yki + Akidki) 2 20~&&~vf(ykL)Td(ykL)\

= 2as ~~v~(Yk,)‘vR(Yki)-l~~(yki)-~~~(yk~)~

Since yk, - j, we have Of(y) = 0. Thus R(y) = 0 and f(J) = 0. This contradicts
the assumptionf(j) > 0. This casecannot occur.
(3) By the analysis in (2), we have yk, + j, R(y) = 0 and R(y) E LCK, (S). Therefore

IIR(Y + 4 - R(Y) - wY~T412 5 $- IMlf

for y, y + d E S. Ford = VR(y)-‘R(y), we have

lIR(~)ll; - IIRCY +d)II; ? IIR(r)II; -

2JG
if IlR(~)ll2 5 KIKcSj

Hence, if y is close enoughto y, the Newton step accepted. According to the standard
result of local quadratic convergence for Newton Method, Theorem 5.2.1 in [S], the
conclusion follows.
(4) IAd’i =o, l,..., be the sequenceof different parametersrx usedin Algorithm 4.1.
Let{kj}, i =O, l,..., with kc = 0, be the indicessuchthat the parametercxchanges,
that is for ki 5 k 5 ki+l - l,ok=&. For~eandya,byJkmna4.1,wehave

IIR(~o)llz 5 Il~(yo)ll2 + fi1og2

a0
124 CHEN AND MANGASARIAN

For ko 5 k 4 kl, since f(yk) I f(yu) with ~0,

filog2
IIWYk)llZ I IIWYo)ll2 5 Il~(Y0)112 + do

By Lemma 4.1,

filog2 2Jmog2
ll~(Yk>llZ 5 IIWYdll2 + I ll~(YO>ll2 +
a0 a0

For& and ykl-i, byLemma4.1,

JElog2 2+/Tlog2 + filog2

IIR(Yk,-*)I12 I Ilr(Yk,-I)112 + 5 IldYo) II2 +
a1 a0 vu0

For ki 5 k < k2, since f(yk) 5 f(yk,-1) with czl,

2filog2 + filog2
llwk)112 I II~(Yk~-l)ll2 5 Il~(Yo)ll2 + (yo
vu0

By Lemma4.1,

filog 2 2fi log 2 2filog2

Il~(Yk)llZ 5 IIWYdll2 + I Il~(YO)ll2 + +
a1 a0 vu0

Inductively, for CX~

and yki-l,

2filog2 + 22/iIilog2 2filog2 + Olog2

p(YkI-1)112 5 Ilr(Yo)ll2 + +...+
a0 vu0 U+-XO via0

forki 5 k < ki+ly

2filog2 2fi log 2 + ...+ 2filog2 + filog2

II~(Yk)ll2 I Il~(Yo)llz + +
(110 vao I+‘IX0 v’ao

2&Vlog2 2%/x log 2 2&v log 2 2filog2

Ilr(Ydll2 5 IHY0)ll2 + + +*a*+ +
a0 vu0 U+XJ lAYtO

Therefore, for all k we have

2+/T log 2 v 2filog2

Ilr(Ydll2 I Ilr(Yo)ll2 + 1+;+-$+... 5 Ilr(Yo>ll2 + x
(110 ( > a0

If the level set (y 1 Ilr(y)ll2 5 Ilr(yo)ll2 + &2<F2) is compact, there exists an accu-
mulation point. By (2) and (3), the whole sequenceconvergesto an e-accurate solution of
MCP (21). D
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 125

We shall give our numerical test results for Algorithm 4.1 in Section 6, after relating our
smooth approach to the central path of the interior point method [ 171 in Section 5.

5. Relation to the interior point method

In this section, we consider the NCP (9). Let the density function d(x) satisfy (Al)-(A3)
and 02 = 0, and let j?(x, B) be defined by (4). If x solves the nonlinear Eq. (10) exactly, then

x = jqx - F(x), p> > x - F(x),

where the last inequality follows from the fact that j(t, fi) > e, from (5) of Proposition
2.2. Hence

x > 0 F(x) > 0,

and x belongs to the interior of the feasible region {x 1 F(x) 2 0, x >_ 0) of the NCP Hence
an exact solution of (10) is interior to the feasible region. However the iterates of the smooth
method, which are only approximate solutions of (lo), are not necessarily feasible. For the
function j defined in Example 2.2 [2, 15,361 , the exact solution x of the Eq. (10) satisfies

x > 0, F(x) > 0, XiFi(X)=h’, ix l,..., n.

which is precisely the central path of the interior point method for solving NCP. Methods
that trace this path but allow iterates to be exterior to the feasible region have been proposed
in [2, 15, 361. In [ 161, the relation between Smale’s method 1361 and the central path was
pointed out. For our function b defined in Example 2.1, the solution x of the nonlinear Eq.
(lo), for different values of 8, constitutes another path in the interior of the feasible region
that satisfies:

x > 0, F(x) > 0, XiFi(X) 5 2p2, i = 1,. . . ,n.

We now compare our path and the central path of the interior point method by using a very
simple example.

Example 5.1. Let F(x) = Mx + q, where

M=[; :I, 4=[::]

The unique solution is (1,O). Figure 7 depicts the central path of the interior point method
as well as the smooth path generated by an exact solution of the smooth nonlinear Eq. (10).
Figure 8 depicts the error along the central path and along our smooth path as a function
of the smoothing parameter /?. The error is measured by the distance to the solution point.
For this example, the error along our smooth path is smaller than that along the central path
for the same value of the parameter fi.
126 CHEN AND MANGASARIAN

0.8 -

0.8 - con8~l Path

c10.5-

c- Smooth Path

0.1 -

8’.8 1 1.2 1.4

Xl
1.8 1.8 2

Figure 7. Comparison of the interior smooth path generated by an exact solution of the smooth nonlinear Eq.
(16versus the c&al path for Example 5.1.

0.9 -

0.8 -

centml Path Emof /

0.2 -

0.2 0.4 0.8 1.2 1.4 1.8 1.8

Figure 8. Error comparison for the central path versus the smooth path for Example 5.1.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 127

6. Numerical results

In this section, we give our computational experience with the smooth Algorithm 4.1 for
the MCP. We implemented the smooth Algorithm 4.1 with an SOR preprocessor if all
diagonal elements of the Jacobian matrix are positive. An initial scaling of the function
Fi (x), inversely proportional to the absolute value of the diagonal elements of the Jacobian
matrix, is performed if 1Vi Fi (xc) 1 2 100. The details of implementing the smooth algorithm
are given in Appendix 2. For comparison, we also give the results for the PATH solver [6].
Both algorithms were run on a DECstation 5000/125. Among the 52 test problems, which
includes all the problems attempted in [ 11,281 and [6], 5 1 problems are from the MCPLIB
[7], and one is the generalized von Thtinen model from [28] and [37]. Our smooth algorithm
was run using one set of default parameters and so was the PATH solver. The smooth
algorithm is written in the C language and implemented by using the GAMSKPLIB [8].
A MINOS routine [23] was used to perform a sparse LU decomposition for solving sparse
linear equations. Both algorithms use the same convergence tolerance of E = 1.Oe - 6.
Table 1 gives a simple description of the test problems [7].

Table 1. MCPLIB problems.

Model origin GAMS file Size

Distillation column modeling (NLE) hydroc20.gms 99

Distillation column modeling (NLE) hydroc06.gms 29
Distillation column modeling (NLE) methanOl.gms 31
NLP problem form Powell (NLP) powellmcp.gms 8
NLP problem form Powell (NLP) powell.gms 16
NLP test problem form Colville (NLP) colvncp.gms 15
Dual of Colville problem (NLP) colvdual.gms 20
Obstacle problem (NLP)(6 cases) obstacle.gms ( 5625
Obstacle Bratu problem (NLP)(6 cases) bratugms 5 5625
W-P) cycle.gms 1
NJ’) josephy.gms 4
(NW kojshin.gms 4
WP) explcp.gms 16
Elastohydrodynamic lubrication (NCP) ehlkostgms 101
Nash equilibrium (VI) nash.gms 10
Nash equilibrium (VI) choigms 13
Spatial price equilibrium (VI) sppe.gms 21
Spatial price equilibrium (VI) tobingms 42
Walrasian equilibrium (VI)(2 cases) mathi*.gms 4
Walrasian equilibrium (VI)(2 cases) scarfa*.gms 14
Walrasian equilibrium (VI)(2 cases) scti*.gms 40
Traffic assignment (VI) gafni.gms 5
Traffic assignment (VI) bertsekasgms 15
Traffic assignment (VI) freebertgms 15
Invariant capital stock (VI) hanskoop.gms 14
Project Independence energy system (VI) piesgms 42
Optimal control (Extended LQP)(6 cases) opt-cont.gms 5 8192
Optimal control from Bertsekas (MCP)(6 cases) bett-oc.gms 5 5000
128 CHEN AND MANGASARIAN

Figure 9. Smooth versus PATH for small MCP.

The average CPU times taken by PATH solver and smooth algorithm for all small prob-
lems are depicted in Figure 9. Figures 10, 11 and 12 depict the CPU times for all remaining
problems except the von Thtinen model. We note that the PATH solver [6] is faster than
Josephy’s Newton method [14] and Rutherford’s GAMS [8] mixed inequality and linear
equation solver (MILES) [35] which is also Newton-based. Figures 9 to 12 indicate that
our smooth algorithm is faster than PATH solver for the larger problems, whereas PATH
solver is faster on smaller problems.
The newest version of PATH (PATH 2.7) that uses a Newton method on the active set [l]
as a preprocessor, improves solution times on the larger problems. Our smooth method can
be similarly improved by adding the projected Newton preprocessor. We have compared
PATH and SMOOTH with a Newton preprocessor on a Sun SPARCstation 20. The results
are given in Figures 13 to 16. It can be seen that with a Newton preprocessor, the solution
times are very similar for PATH and SMOOTH for larger problems, whereas PATH is still
better for the smaller problems.
As mentioned in [28], the generalized von Thtinen model is an NCP with 106 variables.
This is a very difficult problem that has challenged many of the recently proposed algorithms
[28, 371. In order to guarantee that the function F(x) is well defined, we added a lower
bound of 1.Oe-7 to variables xi to x26 as suggested by Jong-Shi Pang. We used three starting
points. In the first, we set all variables to 1, as suggested by Michael C. Ferris; the second
one is a starting point suggested in [37], while the third is the point suggested in [37] and
modified by Jong-Shi Pang. SMOOTH, with or without the Newton preprocessor, solved
the problem from all the three starting points. Solution times did not change by adding
the Newton preprocessor. We report times for SMOOTH with the preprocessor. Starting
with the first point, SMOOTH took a long time, 95.44 seconds to solve the problem. From
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 129

Figure 10. Smooth versus PATH for optimal control problem(beIt-oc.gms).

2soo

WOO-

lsao-

1
I!!
'WOO-

SOO-
smodl Hms

1ooo2ooosooo4ooosoOOwo07wo8ooogoM)
n

Figure II. Smooth versus PATH for optimal control problem(optront.gms).

130 CHEN AND MANGASARIAN

Figure 12. Smooth versus PATH for obstacle problems.

14 16 fdhd path e-
=1+ + * + ’ ,

1.8 -
I
I
1.6-

1.4-

0.6
IJ
0.4 t ,t

0.2 I, I’
1 A___* I”\ss- ,,’
’ :\ ,
.--------
_---
-
___---

path time
ol Ad +
0 10 21 D 30 40 60 80 70 60 60 loo
n

Figure 13. Smooth with Newton preprocessor versus PATH 2.7 for small MU?
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 131

Figure 14. Smooth with Newton preprocessor versus PATH 2.7 for optimal control problem(bert-oc.gms).

0 loo0 2ooo 3ooo

”

Figure 15. Smooth with Newton preprocessor versus PATH 2.7 for optimal control problem(opt-cont.gms).
132 CHEN AND MANGASARIAN

0
0 1000 WOO aooo 4000 5wo woo
n

Figure 16. Smooth with Newton preprocessor versus PATH 2.7 for obstacle problems.

the secondpoint, we obtained a solution in 36 iterations and 3.70 secondsand from the
third point, we obtained a solution in 49 iterations and 7.01 seconds.PATH 2.7 solved the
problem 7 times out of 10 from the first starting point, 6 times out of 10 from the second
starting point, and 5 times out of 10 from the third starting point. The average times of
the successfulPATH runs were 2.59, 3.94 and 3.21 secondsfor the first, secondand third
starting points respectively. We note that when the artificially imposedlower boundson x1
to X26were removed, PATH failed from all starting points and SMOOTH failed from the
first and secondstarting points. However from the third starting point, SMOOTH generated
a solution with somesmall negative componentswhich, when setto 1.Oe-9,gave a solution
accurate to 1.79539e-7.
Summing up the numerical experiments with PATH and SMOOTH, we believe that
comparisonsbetween the two methodswithout a Newton preprocessoris more indicative
of their relative effectiveness. With the Newton preprocessor,a lot of the work for the larger
problemsis performed by the Newton preprocessorandhencethe nearly equalperformance
of the two methodson theseproblems.

7. Conclusion

Smoothing is an effective approximate reformulation for a wide range of complementarity

problemsto any desireddegreeof accuracy. Newton-type methodsfor solving smoothprob-
lemsconstitute a powerful computational approachfor solving theseproblems. Paralleliza-
tion of smooth methods for large-scaleproblems and their extension to other nonconvex
and nonsmoothproblems, are two promising researchareasworth investigating.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 133

Appendix 1

In order to prove Lemma 3.4, we need the following lemma.

Lemma 7.1.
(i) Let’t (x) = xemX, then

max t(x) smax(t(a),r(b),J-].

xeb,bl

(ii) Let

x2
g(x) = (1 + J)eX - 1’ 6 L 0, - lo&l+ S) 6 [a, bl

(iii) Let h(x) be de@ed in Lemma 3.4, - log(1 + 6) 6 [u, b], f/Zen

x~~lh(x) 5 max h(ah h(b), s(4, g(b), &p), &t(b), & .

1 e 1

Proof:

(i) By definition, t’(x) = e-I - xemX = (1 - x)eVX, hence I’(X) = 0 implies x = 1.

Therefore

ma f(x) 5 max f(u),f(b),f(l)

xchbl

(ii) Notice - log(1 + S) $ [a, b] and 6 > 0, we know

2x((l + 6>eX - 1) -x2(1 + S)eX

g’(x) =
((1 + 8)eX - 1)2 ’

Hence g’(x) = 0 implies x = 0 or

1+6
(1 + 6)e” - 1 = yxeX.
134 CHEN AND MANGASARIAN

By (i) and g(0) = 0

(iii) Since

h’(x) = - lOg(1 + 6 - e-‘) - 1 +yTemx,

the point x is a stationary point of h(x) if and only if

--x
x
log(l + S - e-“) = - xe
1 + S - eBx = - (1 + 6)eX - 1’

Therefore, by (ii),

2 2
5 max h(u),h(D),g(u),g(b),~t(~),- -
i 1+6 t(b)y (1 +qe I * cl
Proof of Lemma 3.4:

(i) IfO<S< l,-log(l+6)$[0,-log6].By(iii)ofLemma7.1and

h(0) = 0, h(- log@ = 0, g(0) = 0,

g(-log6) =610g%,t(0)=0,t(-10g6)=-610g&

we have
2
max h(x) 5 max 61og*S, -& log& - .
xG[O,-logs] i (1 + 6)e I

It is easy to get

2s
maxSlog*S5:, -1ogs 5 2.
6dO.11 s~~1-1+6

Combining the above inequalities, we get the conclusion.

(ii) If 6 1 1, - log(l + 6) # I- logs, 01. Similarly to (i), we know

h(0) = 0, h(- log6) = 0, g(0) = 0,

g(-logS)=Slog*6,t(O)=O,t(-log6)=-Slog6.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 135

By (iii) of Lemma 7.1,

26
max h(x) I max Slog2 S, -- ‘) jmax{Slog2S,f].
xG[-log&O] i 1+6 log” (1 + J)e

(iv) If 6 = 0,

Jimoh = 0, .emh(x) = O,_l@og(x) = 0, l&twg(x) = 0,

_lin+io
t (x) = 0, lim t(x) = 0,
x-00

For any c > 0, we have 0 = - log(1 + S) $ [c, t-co). And there exists EO> 0 such
that

h(c) -c f, g(r) < E, t(c) < f, for0 < e -z 60

Therefore, for 0 < e < CO

max h(x) 5 2
XE[E,DO) e

Let E approach0, then

max h(x) I z.
X~[O.co)

Appendix 2

Here is the actual implementation of the smoothAlgorithm 4.1. In the following algorithm
(Xk) t”k, uk) is simply denoted by yk.

Algorithm 7.1 (Smooth Algorithm for MCP). Input tolerance E = l.Oe - 6, and initial
guessx0 E R”

(1) Initialization. For 1 5 i 5 n of Case4 of (22), let wi = (E;;:(xg))+, u; = (-Fi(Xo))+,

k = 0 and ~0 = cx(yo).
(2) If Ilr(yO)llm 5 c, stop.
(3) Newton direction dk

dk = --VR(yd-‘R(Yd

In order to avoid nonsingularity, for Cases 24 of (22), if Vi Ri(yk) < l.Oe - 9, let
ViRi(yk) = l.Oe - 9.
136 CHEN AND MANGASARIAN

(4) Stepsize ik (Armijo)

yk+l = yk + &dk, hk = maX{ 1,6, s2, . . .}, S.t.

f(Yk+l) 5 f(yk)

where 6 = 0.75.
(5) Parameter update. !?fdyk+l) > ak9 set

ak+l = dYk+lh

ak+l = kk.

Letk=k+l,gotostep@).

For someof the test problems, somefunctions are not well defined outside the feasible
region. In such cases,the line searchstep (5) may fail. If this occurs, we will try to push
the next point inside the feasible region by setting the smoothparametera! to a very small
value, such as l.Oe - 10.

Acknowledgment

We are grateful to Michael C. Ferris and Steven P. Dirkse for letting us use their CAMS
interface routines, their PATH solver for comparison purposesand their Newton prepro-
cessorthat we incorporated into a later version of SMOOTH. We are also grateful to Jong-
Shi Pang for providing us with the generalized von Thiinen model and an initial starting
point.

References

1. D.P Bertsekas, “Projected Newton methods for optimization problems with simple constraints,” SIAM Journal
on Control and Optimization, vol. 20, pp. 221-246.1982.
2. B. Chen and PT. Harker, “A non-interior-point continuation method for linear complementarity problems,’
SIAM Journal on Matrix Analysis and Applications, vol. 14, pp. 1168-l 190, 1993.
3. C. Chen and O.L. Mangasarian, “Smoothing methods for convex inequalities and linear complementarity
problems,” Technical Report 1191, Computer Sciences Department, University of Wisconsin, Madison, Wis-
consin, 53706, November 1993. (To appear in Mathematical Programming, vol. 71, 1995.)
4. R.W. Cottle, E Giannessi, and J.-L. Lions, Variational Inequalities and Complementarity Problems, John
Wiley & Sons: New York, 1980.
5. J.E. Dennis and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations,
Prentice-Hall: Englewood Cliffs, N.J., 1983.
6. S.P. Dirkse and MC. Ferris, “The path solver: A non-monotone stabilization scheme for mixed complemen-
tarity problems,” Optimization Methods and Software, vol. 5, pp. 123-156, 1995.
NONLINEAR AND MIXED COMPLEMENTARITY PROBLEMS 137

7. S.P. Dirkse and M.C. Ferris, “MCPLIB: A collection of nonlinear mixed complementarity problems,” Opti-
mization Methods and Software, vol. 5, pp. 319-345, 1995.
8. S.P. Dirkse, M.C. Ferris, P.V. Preckel, and T. Rutherford, “The GAMS callable program library for variational
and complementarity solvers,” Manuscript, University of Wisconsin, 1993.
9. M.S. Gowda, “On the extended linear complementarity problem,” Technical Report, Department of Mathe-
matics & Statistics, University of Maryland Baltimore Country, Baltimore, Maryland, 1994, Mathematical
Programming, to appear.
10. P.T. Harker and J.-S. Pang, “Finite-dimensional variational inequality and nonlinear complementarty prob-
lems: A survey of theory, algorithms and applications,” Mathematical Programming, vol. 48, pp. 161-220,
1990.
11. P.T. Harker and B. Xiao, “Newton’s method for the nonlinear complementarity problem: A B-differentiable
equation approach,” Mathematical Programming, vol. 48, pp. 339-357, 1990.
12. J. Hertz, A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley:
Redwood City, California, 199 1.
13. J.-B. Hiriart-Urruty and C. Lemarechal, Convex Analysis and Minimization Algorithms I, Springer-Verlag:
Berlin, 1993.
14. N.H. Josephy, “Newton’s method for generalized equations,” Technical Summary Report 1965, Mathematics
Research Center, University of Wisconsin, Madison, Wisconsin, 1979.
15. C. Kanzow, ‘Some tools allowing interior-point methods to become noninterior,” Technical Report, Institute
of Applied Mathematics, University of Hamburg, Germany, 1994.
16. M. Kojima and N. Megiddo, “The relation betweeh the path of centers and smale’s regularization of the linear
programming problem,” Linear Algebra and Its Applications, vol. 152, pp. 135-139, 1991.
17. M. Kojima, S. Mizuno, and A. Yoshise, “A polynomial-time algorithms for a class of linear complementarity
problems,” Mathematical Programming, vol. 44, pp. l-27, 1989.
18. J. Kreimer and R.Y. Rubinstein, “Nondifferentiable optimization via smooth approximation: General analyt-
ical approach,” Annals of Operations Research, vol. 39, pp. 97-l 19, 1992.
19. K. Madsen and H.B. Nielsen, “A finite smoothing algorithm for linear II estimation,” SIAM on Optimization,
vol. 3, no. 2, pp. 223-235, May 1993.
20. O.L. Mangasarian, “Mathematical programming in neural networks,” ORSA Journal on Computing, vol. 5,
no. 4, pp. 349-360, 1993.
2 1. O.L. Mangasarian and J.-S. Pang, “The extended linear complementarity problems,” SIAM Journal on Matrix
Analysis and Applications, vol. 16, pp. 359-368, 1995.
22. Jorge J. More, “Global methods for nonlinear complementarity problems,” Technical Report Preprint MCS-
P429-0494, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois,
1994.
23. B.A. Murtagh and M.A. Saunders, “MINOS 5.0 user’s guide:’ Technical Report SOL 83.20, Stanford Uni-
versity, December 1983.
24. K.G. Murty, Linear Complementarity, Linear and Nonlinear Programming, Helderman-Verlag: Berlin, 1988.
25. J.M. Ortega, Numerical Analysis, A Second Course, Academic Press, 1972.
26. J.-S. Pang, “Newton’s method for B-differentiable equations,” Mathematics of Operations Research, vol. 15,
pp. 331-341.1990.
27. J.-S. Pang, “A B-differentiable equation based, globally and locally quadratically convergent algorithm for
nonlinear programs, complementarity and variational inequality problems,” Mathematical Programming,
vol. 51, pp. 101-131, 1991.
28. J.-S. Pang and S.A. Gabriel, “NE/SQP: A robust algorithm for the nonlinear complementarity problem,”
Mathematical Programming, vol. 60, pp. 295-337, 1993.
29. M.C. Pinar and S.A. Zenios “On smoothing exact penalty functions for convex constrained optimization,”
SIAM Journal on Optimization, vol. 4, pp. 486511, 1994.
30. D. Ralph, “Global convergence of damped Newton’s method for nonsmooth equations via the path search,”
Mathematics of Operations Research, vol. 19, pp. 352-389, 1994.
3 1. J. Ren, “Computable error bounds in mathematical programming,” Technical Report 1173, Computer Sciences
Department, University of Wisconsin, Madison, Wisconsin 53706, August 1993.
138 CHEN AND MANGASARIAN

32. SM. Robinson, “Generalized equations and their solution: Part I: Basic theory,” Mathematical Programming
Study, vol. 10, pp. 128-140, 1979.
33. SM. Robinson, “Strongly regular generalized equations,” Mathematics of Operations Research, vol. 5, pp. 43-
62, 1980.
34. S.M. Robinson, “Newton’s method for a class of nonsmooth functions,” Working Paper, Department of
Industrial Engineering, University of Wisconsin, Madison, Wisconsin, 1988.
35. T.F. Rutherford, “MILES: A mixed inequality and nonlinear equation solver,” Working Paper, Department of
Economics, University of Colorado, Boulder, 1993.
36. S. Smale, “Algorithms for solving equations,” In Proceedings of the International Congress of Mathematicians,
pp. 172-195, Ameri. Math. Sot., Providence, 1987.
37. B. Xiao, “Global Newtons methods for nonlinear programs and variational inequalities,” Technical Report,
Ph.D. Thesis, Department of Decision Sciences, The Wharton School, University of Pennsylvania, Philadel-
phia, PA, 1990.
38. Y. Ye, “A fully polynomial-time approximation algorithm for computing a stationary point of the general
linear complementarity problem,” Mathematics of Operations Research, vol. 18, pp. 334-345, 1993.
39. I. Zang, “A smoothing-out technique for mitt-max optimization,” Mathematical Programming, vol. 19, pp.
61-77, 1980.