6.

Numerical methods for mathematical programs with

complementarity constraints

Armin Nurkanović

Winter School on Numerical Methods for Optimal Control of Nonsmooth Systems

École des Mines de Paris
February 3-5, 2025, Paris, France
Outline of the lecture

1 Overview of MPCC methods

2 Regularization and penalty methods

3 Active-set methods

4 Numerical benchmarks

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 1/37
Numerical methods for MPCCs

2
MPEC f (w)
1.5 +
min f (w) (1a) w$
w∈Rn
s.t. g(w) = 0, (1b) 1

h(w) ≥ 0, (1c)

w2
0.5
0 ≤ w1 ⊥ w2 ≥ 0, (1d)
0

-0.5
w = (w0 , w1 , w2 ) ∈ Rn , w0 ∈ Rp , w1 , w2 ∈ Rm ,
-1
Ω = {x ∈ Rn | g(w) = 0, h(w) ≥ 0, 0 ≤ w1 ⊥ w2 ≥ 0}, -1 0 1 2
w1
▶ Standard NLP methods solve the KKT conditions.
▶ MPECs violate constraint qualifications, and the KKT conditions may not be necessary.
▶ There are many stationary concepts for MPECs, and not all are useful.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 2/37
Numerical methods for MPCCs

2
MPEC f (w)
1.5 +
min f (w) (1a) w$
w∈Rn
s.t. g(w) = 0, (1b) 1

h(w) ≥ 0, (1c)

w2
0.5
0 ≤ w1 ⊥ w2 ≥ 0, (1d)
0

-0.5
w = (w0 , w1 , w2 ) ∈ Rn , w0 ∈ Rp , w1 , w2 ∈ Rm ,
-1
Ω = {x ∈ Rn | g(w) = 0, h(w) ≥ 0, 0 ≤ w1 ⊥ w2 ≥ 0}, -1 0 1 2
w1
▶ Standard NLP methods solve the KKT conditions.
▶ MPECs violate constraint qualifications, and the KKT conditions may not be necessary.
▶ There are many stationary concepts for MPECs, and not all are useful.
▶ Workaround/main idea: solve a (finite) sequence of more regular problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 2/37
Two families of MPCC methods

1. Regularization, smoothing, penalty methods: solve a sequence of regular nonlinear

programs parametrized by a parameter σ k > 0.
▶ Pro: easy to implement.
▶ Con: ill-conditioning, weaker theoretical properties.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 3/37
Two families of MPCC methods

1. Regularization, smoothing, penalty methods: solve a sequence of regular nonlinear

programs parametrized by a parameter σ k > 0.
▶ Pro: easy to implement.
▶ Con: ill-conditioning, weaker theoretical properties.

2. Combinatorial, active-set, pivoting methods: solve a sequence of piece MPCCs

(TNLP, BNLP), until a piece problem is found that solves also the MPCC.
▶ Pro: strong theoretical properties, can be combined with 1.
▶ Con: more difficult to implement, worst-case combinatorial complexity.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 3/37
Two families of MPCC methods

1. Regularization, smoothing, penalty methods: solve a sequence of regular nonlinear

programs parametrized by a parameter σ k > 0.
▶ Pro: easy to implement.
▶ Con: ill-conditioning, weaker theoretical properties.

2. Combinatorial, active-set, pivoting methods: solve a sequence of piece MPCCs

(TNLP, BNLP), until a piece problem is found that solves also the MPCC.
▶ Pro: strong theoretical properties, can be combined with 1.
▶ Con: more difficult to implement, worst-case combinatorial complexity.

Both classes of methods are available in nosnoc’s mpccsol() function:

github.com/nosnoc/nosnoc
To get started see: github.com/nosnoc/nosnoc/tree/main/examples/generic_mpcc

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 3/37
Outline of the lecture

1 Overview of MPCC methods

2 Regularization and penalty methods

3 Active-set methods

4 Numerical benchmarks

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 4/37
Scholtes’ global relaxation method
The easiest to implement and the most efficient regularization method [Scholtes, 2001].

Reg(σ k )
min f (w)
w∈Rn
s.t. g(w) = 0,

w2
h(w) ≥ 0,
w1 , w2 ≥ 0,
w1,i w2,i ≤ σ k , i = 1, . . . , m.

w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 5/37
Scholtes’ global relaxation method
The easiest to implement and the most efficient regularization method [Scholtes, 2001].

Reg(σ k )
min f (w)
w∈Rn
s.t. g(w) = 0,

w2
h(w) ≥ 0,
w1 , w2 ≥ 0,
w1,i w2,i ≤ σ k , i = 1, . . . , m.

w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 5/37
Scholtes’ global relaxation method
The easiest to implement and the most efficient regularization method [Scholtes, 2001].

Reg(σ k )
min f (w)
w∈Rn
s.t. g(w) = 0,

w2
h(w) ≥ 0,
w1 , w2 ≥ 0,
w1,i w2,i ≤ σ k , i = 1, . . . , m.

w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 5/37
Scholtes’ global relaxation method
The easiest to implement and the most efficient regularization method [Scholtes, 2001].

Reg(σ k )
min f (w)
w∈Rn
s.t. g(w) = 0,

w2
h(w) ≥ 0,
w1 , w2 ≥ 0,
w1,i w2,i ≤ σ k , i = 1, . . . , m.

w1

Theorem ( [Scholtes, 2001, Hoheisel et al., 2013])

Let {σ k } ↓ 0 and let wk be a stationary point of Reg(σ k ) with wk → w∗ such that
MPEC-MFCQ holds at w∗ . Then w∗ is a C-stationary point of the the MPEC (1).

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 5/37
C-stationarity allows first-order descent directions

5
Consider the two-dimensional MPEC:
4
min (w1 − 1)2 + (w2 − 1)2
w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0. 3

w2
2
∗
▶ The origin w = 0 is an C-stationary point with the
1
optimal multipliers ν = −2, ξ = 2.
S-stationary
0
C-stationary
-1
0 2 4
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 6/37
C-stationarity allows first-order descent directions

5
Consider the two-dimensional MPEC:
4
min (w1 − 1)2 + (w2 − 1)2
w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0. 3

w2
2
∗
▶ The origin w = 0 is an C-stationary point with the
1
optimal multipliers ν = −2, ξ = 2.
S-stationary
▶ There exists two descent direction d = (1, 0) and 0
d = (0, 1) C-stationary
▶ The origin is not B-stationary, w = (1, 0) and -1
0 2 4
w = (0, 1) are S-stationary.
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 6/37
Smoothing method have drawbacks
No benefit in using them. Have elaborate convergence theory [Ralph and Wright, 2004].

Instead of relaxing w1 , w2 ≥ 0 w1 w2 ≤ σ, use smoothing w1 , w2 ≥ 0, w1 w2 = σ.

Consider the two-dimensional MPEC:

min (w1 + 1)2 + (w2 + 1)2

w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 7/37
Smoothing method have drawbacks
No benefit in using them. Have elaborate convergence theory [Ralph and Wright, 2004].

Instead of relaxing w1 , w2 ≥ 0 w1 w2 ≤ σ, use smoothing w1 , w2 ≥ 0, w1 w2 = σ.

4

Consider the two-dimensional MPEC: 3

min (w1 + 1)2 + (w2 + 1)2 2

w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.

w2
1

0
▶ Relaxation method converges in one iteration to S-stationary
w∗ = (0, 0). -1

-2
-2 0 2 4
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 7/37
Smoothing method have drawbacks
No benefit in using them. Have elaborate convergence theory [Ralph and Wright, 2004].

Instead of relaxing w1 , w2 ≥ 0 w1 w2 ≤ σ, use smoothing w1 , w2 ≥ 0, w1 w2 = σ.

4

Consider the two-dimensional MPEC: 3

min (w1 + 1)2 + (w2 + 1)2 2

w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0. wk

w2
1

0
▶ Relaxation method converges in one iteration to S-stationary
w∗ = (0, 0). -1
▶ Smoothing needs an infinite sequence for
-2
convergence. -2 0 2 4
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 7/37
Smoothing method have drawbacks
No benefit in using them. Have elaborate convergence theory [Ralph and Wright, 2004].

Instead of relaxing w1 , w2 ≥ 0 w1 w2 ≤ σ, use smoothing w1 , w2 ≥ 0, w1 w2 = σ.

4

Consider the two-dimensional MPEC: 3

min (w1 + 1)2 + (w2 + 1)2 2

w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.

w2
1
wk
0
▶ Relaxation method converges in one iteration to S-stationary
w∗ = (0, 0). -1
▶ Smoothing needs an infinite sequence for
-2
convergence. -2 0 2 4
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 7/37
Smoothing method have drawbacks
No benefit in using them. Have elaborate convergence theory [Ralph and Wright, 2004].

Instead of relaxing w1 , w2 ≥ 0 w1 w2 ≤ σ, use smoothing w1 , w2 ≥ 0, w1 w2 = σ.

4

Consider the two-dimensional MPEC: 3

min (w1 + 1)2 + (w2 + 1)2 2

w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.

w2
1
wk
0
▶ Relaxation method converges in one iteration to S-stationary
w∗ = (0, 0). -1
▶ Smoothing needs an infinite sequence for
-2
convergence. -2 0 2 4
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 7/37
Smoothing method have drawbacks
No benefit in using them. Have elaborate convergence theory [Ralph and Wright, 2004].

Instead of relaxing w1 , w2 ≥ 0 w1 w2 ≤ σ, use smoothing w1 , w2 ≥ 0, w1 w2 = σ.

4

Consider the two-dimensional MPEC: 3

2 2
min (w1 + 1) + (w2 + 1) 2
w∈R2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.

w2
1
wk
0
▶ Relaxation method converges in one iteration to S-stationary
w∗ = (0, 0). -1
▶ Smoothing needs an infinite sequence for
convergence. -2
-2 0 2 4
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 7/37
Other regularization methods
There exist many elaborate ways to relax the L-shaped set. Convergence theory in [Hoheisel et al., 2013]

(a) Scholtes (b) Lin-Fukushima (c) Ste,ensen-Ulbrich (d) Kadrani et al. (e) Kanzow-Schwartz
w2

w2

w2

w2

w2
w1 w1 w1 w1 w1

▶ They have better convergence properties than Scholtes’ method if the NLP’s are solved
exactly.
▶ In practice, they perform better only on easier problems [Nurkanović et al., 2024].
:

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 8/37
Other regularization methods
There exist many elaborate ways to relax the L-shaped set. Convergence theory in [Hoheisel et al., 2013]

(a) Scholtes (b) Lin-Fukushima (c) Ste,ensen-Ulbrich (d) Kadrani et al. (e) Kanzow-Schwartz
w2

w2

w2

w2

w2
w1 w1 w1 w1 w1

▶ They have better convergence properties than Scholtes’ method if the NLP’s are solved
exactly.
▶ In practice, they perform better only on easier problems [Nurkanović et al., 2024].
Convergence results [Hoheisel et al., 2013]:
a) [Scholtes, 2001]: C-stationarity under MPEC-MFCQ.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 8/37
Other regularization methods
There exist many elaborate ways to relax the L-shaped set. Convergence theory in [Hoheisel et al., 2013]

(a) Scholtes (b) Lin-Fukushima (c) Ste,ensen-Ulbrich (d) Kadrani et al. (e) Kanzow-Schwartz
w2

w2

w2

w2

w2
w1 w1 w1 w1 w1

▶ They have better convergence properties than Scholtes’ method if the NLP’s are solved
exactly.
▶ In practice, they perform better only on easier problems [Nurkanović et al., 2024].
Convergence results [Hoheisel et al., 2013]:
a) [Scholtes, 2001]: C-stationarity under MPEC-MFCQ.
b) [Lin and Fukushima, 2005]: C-stationarity under MPEC-MFCQ..

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 8/37
Other regularization methods
There exist many elaborate ways to relax the L-shaped set. Convergence theory in [Hoheisel et al., 2013]

(a) Scholtes (b) Lin-Fukushima (c) Ste,ensen-Ulbrich (d) Kadrani et al. (e) Kanzow-Schwartz
w2

w2

w2

w2

w2
w1 w1 w1 w1 w1

▶ They have better convergence properties than Scholtes’ method if the NLP’s are solved
exactly.
▶ In practice, they perform better only on easier problems [Nurkanović et al., 2024].
Convergence results [Hoheisel et al., 2013]:
a) [Scholtes, 2001]: C-stationarity under MPEC-MFCQ.
b) [Lin and Fukushima, 2005]: C-stationarity under MPEC-MFCQ..
c) [Steffensen and Ulbrich, 2010]: C-stationarity under MPEC-CPLD.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 8/37
Other regularization methods
There exist many elaborate ways to relax the L-shaped set. Convergence theory in [Hoheisel et al., 2013]

(a) Scholtes (b) Lin-Fukushima (c) Ste,ensen-Ulbrich (d) Kadrani et al. (e) Kanzow-Schwartz
w2

w2

w2

w2

w2
w1 w1 w1 w1 w1

▶ They have better convergence properties than Scholtes’ method if the NLP’s are solved
exactly.
▶ In practice, they perform better only on easier problems [Nurkanović et al., 2024].
Convergence results [Hoheisel et al., 2013]:
a) [Scholtes, 2001]: C-stationarity under MPEC-MFCQ.
b) [Lin and Fukushima, 2005]: C-stationarity under MPEC-MFCQ..
c) [Steffensen and Ulbrich, 2010]: C-stationarity under MPEC-CPLD.
d) [Kadrani et al., 2009]: M-stationarity under MPEC-CPLD.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 8/37
Other regularization methods
There exist many elaborate ways to relax the L-shaped set. Convergence theory in [Hoheisel et al., 2013]

(a) Scholtes (b) Lin-Fukushima (c) Ste,ensen-Ulbrich (d) Kadrani et al. (e) Kanzow-Schwartz
w2

w2

w2

w2

w2
w1 w1 w1 w1 w1

▶ They have better convergence properties than Scholtes’ method if the NLP’s are solved
exactly.
▶ In practice, they perform better only on easier problems [Nurkanović et al., 2024].
Convergence results [Hoheisel et al., 2013]:
a) [Scholtes, 2001]: C-stationarity under MPEC-MFCQ.
b) [Lin and Fukushima, 2005]: C-stationarity under MPEC-MFCQ..
c) [Steffensen and Ulbrich, 2010]: C-stationarity under MPEC-CPLD.
d) [Kadrani et al., 2009]: M-stationarity under MPEC-CPLD.
e) [Kanzow and Schwartz, 2013]: M-stationarity under MPEC-CPLD.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 8/37
“The price of inexactness” [Kanzow and Schwartz, 2015]

▶ Most convergence results derived under the assumption that every subproblem for a fixed
σ k is solved exactly.
▶ If NLPs are solved to some ϵ > 0 threshold, than most methods have weaker convergence
properties.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 9/37
“The price of inexactness” [Kanzow and Schwartz, 2015]

▶ Most convergence results derived under the assumption that every subproblem for a fixed
σ k is solved exactly.
▶ If NLPs are solved to some ϵ > 0 threshold, than most methods have weaker convergence
properties.
▶ Steffensen-Ulbrich, Kadrani, Kanzow-Schwartz converge only to weakly stationary points!

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 9/37
“The price of inexactness” [Kanzow and Schwartz, 2015]

▶ Most convergence results derived under the assumption that every subproblem for a fixed
σ k is solved exactly.
▶ If NLPs are solved to some ϵ > 0 threshold, than most methods have weaker convergence
properties.
▶ Steffensen-Ulbrich, Kadrani, Kanzow-Schwartz converge only to weakly stationary points!
▶ Scholtes’ method still converges to C-stationary points.
▶ A nice implementation for Scholtes’ method is in pair it with interior-point methods,
where the barrier τ and homotopy σ parameters are jointly driven to
zero [Raghunathan and Biegler, 2005] (IPOPT-C).

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 9/37
Penalty methods
Closely related to regularization methods, sometimes with a one-to-one correspondence [Leyffer et al., 2006]

Main idea: put difficult part into objective.

The ℓ∞ reformulation
The ℓ1 reformulation
min f (w) + ρs
minn f (w) + ρw1⊤ w2 w∈Rn ,s∈R
w∈R
s.t. g(w) = 0,
s.t. g(w) = 0,
h(w) ≥ 0,
h(w) ≥ 0,
w1 , w2 ≥ 0,
w1 , w2 ≥ 0.
w1,i w2,i ≤ s, i = 1 . . . m,
0 ≤ s ≤ s̄.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 10/37
Penalty methods
Closely related to regularization methods, sometimes with a one-to-one correspondence [Leyffer et al., 2006]

Main idea: put difficult part into objective.

The ℓ∞ reformulation
The ℓ1 reformulation
min f (w) + ρs
minn f (w) + ρw1⊤ w2 w∈Rn ,s∈R
w∈R
s.t. g(w) = 0,
s.t. g(w) = 0,
h(w) ≥ 0,
h(w) ≥ 0,
w1 , w2 ≥ 0,
w1 , w2 ≥ 0.
w1,i w2,i ≤ s, i = 1 . . . m,
0 ≤ s ≤ s̄.

▶ If the penalty parameter ρ large enough, solve single NLP to solve

MPEC [Anitescu, 2005a, Anitescu, 2005b, Ralph and Wright, 2004].
▶ Good implementations update the penalty parameter in a homotopy ρk = σ1k
▶ In practice, usually converges faster than Scholtes’ method, but has a lower success rate.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 10/37
Steering the homotopy parameter to zero
The practical performance depends on the update rate.

Approach: Solve a sequence of regularized NLPs (σ k ), warm start the next iteration with
w∗ (σ k−1 ). Update the homotopy parameter via:
▶ Linear update rule:

σ k+1 = κσ k , κ ∈ (0, 1)

▶ For faster convergence, superlinear rules may also be helpful:

σ k+1 = min(κσ k , (σ k )η ), κ ∈ (0, 1), η > 1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 11/37
Steering the homotopy parameter to zero
The practical performance depends on the update rate.

Approach: Solve a sequence of regularized NLPs (σ k ), warm start the next iteration with
w∗ (σ k−1 ). Update the homotopy parameter via:
▶ Linear update rule:

σ k+1 = κσ k , κ ∈ (0, 1)

▶ For faster convergence, superlinear rules may also be helpful:

σ k+1 = min(κσ k , (σ k )η ), κ ∈ (0, 1), η > 1

Some best practices:

▶ Start with not too large, and not too small σ 0 . For well-scaled problems σ 0 = 1 usually
works well.
▶ Do not update too aggressively nor top conservatively. Good choices are κ = 0.1, κ = 0.2
and η = 1.5.
▶ Update rules are usually monotone. To increase robustness, if an iteration fails, go back
and update with κ+ = γκ, e.g., γ = 2.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 11/37
Outline of the lecture

1 Overview of MPCC methods

2 Regularization and penalty methods

3 Active-set methods

4 Numerical benchmarks

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 12/37
Motivation for active set methods for MPECs

Difficulties:
1. Regularization methods, also under very strong assumptions, converge to points that are
weaker than S-stationary, which are possibly not B-stationary.
2. Regularized NLPs, small or large σ, may be extremely difficult to solve (much more than
TNLPs/BNLPs).

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 13/37
Motivation for active set methods for MPECs

Difficulties:
1. Regularization methods, also under very strong assumptions, converge to points that are
weaker than S-stationary, which are possibly not B-stationary.
2. Regularized NLPs, small or large σ, may be extremely difficult to solve (much more than
TNLPs/BNLPs).

Main idea of active-set methods:

1. Find a feasible point w ∈ Ω of the MPEC (1).
2. Solved the branch NLP with some partition of the index set, e.g. I1 (w), I2 (w)
3. Call an oracle to verify B-stationary or to give a better I1 (w+ ) and I2 (w+ ) (e.g. an
LPEC)
4. If not B-stationary, go to 2.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 13/37
A brief history of active-set methods

▶ Late 1990s, early 2000s: selecting the next TNLP/BNLP based on signs of
multipliers [Fukushima and Tseng, 2002, Giallombardo and Ralph, 2008,
Izmailov and Solodov, 2008, Jiang and Ralph, 1999, Lin and Fukushima, 2006,
Liu et al., 2006, Luo et al., 1996, Scholtes and Stöhr, 1999]. Convergence to
B-stationarity can only be guaranteed under MPEC-LICQ.
▶ 2007: In [Leyffer and Munson, 2007] for the first time suggested to use LPECs as a
stopping criteria and for step computation.
▶ 2022: SQP-type methods with LPECs, developed for MPECs with bound
constraints [Kirches et al., 2022] (B-stationary), extension for general constraints via
augmented Lagrangian [Guo and Deng, 2022] (M-stationary).
▶ 2024-2025: for general MPECs (B-stationary) in [Kazi et al., 2024] and [N. and Leyffer,
2025].

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 14/37
Refresher on the the Branch Nonlinear Program (BNLP)

MPEC active sets Partitioning of the degenerate set I00 (w

I+0 (w) = {i ∈ {1, . . . , m} | w1,i > 0, w2,i = 0}, D1 (w) ∪ D2 (w) = I00 (w),
D1 (w) ∩ D2 (w) = ∅,
I0+ (w) = {i ∈ {1, . . . , m} | w1,i = 0, w2,i > 0},
I1 (w) := I0+ (w) ∪ D1 (w),
I00 (w) = {i ∈ {1, . . . , m} | w1,i = 0, w2,i = 0}. I2 (w) := I+0 (w) ∪ D2 (w),

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 15/37
Refresher on the the Branch Nonlinear Program (BNLP)

MPEC active sets Partitioning of the degenerate set I00 (w

I+0 (w) = {i ∈ {1, . . . , m} | w1,i > 0, w2,i = 0}, D1 (w) ∪ D2 (w) = I00 (w),
D1 (w) ∩ D2 (w) = ∅,
I0+ (w) = {i ∈ {1, . . . , m} | w1,i = 0, w2,i > 0},
I1 (w) := I0+ (w) ∪ D1 (w),
I00 (w) = {i ∈ {1, . . . , m} | w1,i = 0, w2,i = 0}. I2 (w) := I+0 (w) ∪ D2 (w),

Branch NLP (BNLP) at w∗

min f (w)
w∈Rn

w2
s.t. g(w) = 0,
h(w) ≥ 0,
w1,i = 0, w2,i ≥ 0, i ∈ I1 (w∗ )
w1,i ≥ 0, w2,i = 0, i ∈ I2 (w∗ ).
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 15/37
Refresher on the the Branch Nonlinear Program (BNLP)

MPEC active sets Partitioning of the degenerate set I00 (w

I+0 (w) = {i ∈ {1, . . . , m} | w1,i > 0, w2,i = 0}, D1 (w) ∪ D2 (w) = I00 (w),
D1 (w) ∩ D2 (w) = ∅,
I0+ (w) = {i ∈ {1, . . . , m} | w1,i = 0, w2,i > 0},
I1 (w) := I0+ (w) ∪ D1 (w),
I00 (w) = {i ∈ {1, . . . , m} | w1,i = 0, w2,i = 0}. I2 (w) := I+0 (w) ∪ D2 (w),

Branch NLP (BNLP) at w∗

min f (w)
w∈Rn

w2
s.t. g(w) = 0,
h(w) ≥ 0,
w1,i = 0, w2,i ≥ 0, i ∈ I1 (w∗ )
w1,i ≥ 0, w2,i = 0, i ∈ I2 (w∗ ).
w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 15/37
Refresher on LPECs
If there are no complementarity constraints, the LPEC reduces to an LP.

Theorem ([Luo et al., 1996])

Let TΩ (w) = TΩMPEC (w), and w∗ ∈ Ω be a local minimizer of the MPEC (1), then it holds that

∇f (w∗ )⊤ d ≥ 0 for all d ∈ TΩMPEC (w∗ ), (2)

or equivalently, d = 0 is a local minimizer of the following optimization problem:

min ∇f (wk )⊤ d
d∈Rn

s.t. g(wk ) + ∇g(wk )⊤ d = 0,

h(wk ) + ∇h(wk )⊤ d ≥ 0,
k k
w1,i + d1,i = 0, ∀i ∈ I0+ ,
k k
w2,i + d2,i = 0, ∀i ∈ I+0 ,
k k k
0 ≤ w1,i +d1,i ⊥ w2,i +d2,i ≥ 0, ∀i ∈ I00

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 16/37
Refresher on the relation between MPEC pieces

MPEC feasible set relations

\ [
ΩTNLP = ΩBNLP(I1 ,I2 ) ⊂ Ω = ΩBNLP(I1 ,I2 ) .
(I1 ,I2 ) (I1 ,I2 )

(a) TNLP (b) RNLP (c) BNLP(fg;i) (d) BNLP(i;fg)

w2

w2

w2

w2
w1 w1 w1 w1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 17/37
Refresher on the relation between MPEC pieces

MPEC feasible set relations

\ [
ΩTNLP = ΩBNLP(I1 ,I2 ) ⊂ Ω = ΩBNLP(I1 ,I2 ) .
(I1 ,I2 ) (I1 ,I2 )

(a) TNLP (b) RNLP (c) BNLP(fg;i) (d) BNLP(i;fg)

w2

w2

w2

w2
w1 w1 w1 w1

Summary:
▶ The TNLP, RNLP, and BNLPs are regular nonlinear optimization problems.
▶ If we know the right TNLP/BNLP, we can just solve a regular NLP to solve the MPEC.
▶ There are 2|I00 | BNLPs - highlighting the combinatorial nature.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 17/37
MPECopt: An algorithm for computing a B-stationary points

Algorithm MPECopt (simplified)

Input: w0 ,ρ0 ∈ [ρlb , ρub ], γ L ∈ (0, 1), γ U > 1
1 Call Phase I Algorithm to find a feasible point w0 ∈ Ω
for k = 0, . . . do // Major/outer loop
2 Possibly reset trust region radius ρk,0 ∈ [ρlb , ρub ] for l = 0, . . . do // Minor/inner loop
3 Solve LPEC(wk , ρk,l ) for dk,l if dk,l = 0 then
4 terminate // (B-stationary point found)
5 else if Active set changed then
6 Solve BNLP(wk + dk,l ) for wk,l
if f (wk,l ) < f (wk ) then
7 wk+1 = wk,l ; ρk,l+1 = γ U ρk,l ; break // step accepted
k,l+1 L k,l
8 else ρ =γ ρ // reduce TR-radius
k,l+1 L k,l
9 else ρ =γ ρ // reduce TR-radius

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 18/37
Algorithm for computing a B-stationary point of MPECs

LPEC(wk , ρ) - reduced

min ∇f (wk )⊤ d
d∈Rn

s.t. g(wk ) + ∇g(wk )⊤ d = 0,

h(wk ) + ∇h(wk )⊤ d ≥ 0,
k k
w1,i + d1,i = 0, w2,i + d2,i ≥ 0, ∀i ∈ I0+ ,
k k
w1,i + d1,i ≥ 0, w2,i + d2,i = 0, ∀i ∈ I+0
k k k
0 ≤ w1,i +d1,i ⊥ w2,i +d2,i ≥ 0, ∀i ∈ I00 ,
∥d∥∞ ≤ ρ,

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 19/37
Algorithm for computing a B-stationary point of MPECs

LPEC(wk , ρ) - full

min ∇f (wk )⊤ d
d∈Rn

s.t. g(wk ) + ∇g(wk )⊤ d = 0,

h(wk ) + ∇h(wk )⊤ d ≥ 0,

k k
0 ≤ w1,i +d1,i ⊥ w2,i +d2,i ≥ 0, ∀i,
∥d∥∞ ≤ ρ,

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 19/37
Algorithm for computing a B-stationary point of MPECs

LPEC(wk , ρ) - full
BNLP(wk )
k ⊤
min ∇f (w ) d
d∈Rn
min f (x)
s.t. g(wk ) + ∇g(wk )⊤ d = 0, x∈Rn
s.t. g(w) = 0,
h(wk ) + ∇h(wk )⊤ d ≥ 0,
h(w) ≥ 0,
w1,i = 0, w2,i ≥ 0, i ∈ I1k ,
k
0 ≤ w1,i k
+d1,i ⊥ w2,i +d2,i ≥ 0, ∀i, w1,i ≥ 0, w2,i = 0, i ∈ I2k .

∥d∥∞ ≤ ρ,

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 19/37
Algorithm for computing a B-stationary point of MPECs

LPEC(wk , ρ) - full
BNLP(wk )
k ⊤
min ∇f (w ) d
d∈Rn
min f (x)
s.t. g(wk ) + ∇g(wk )⊤ d = 0, x∈Rn
s.t. g(w) = 0,
h(wk ) + ∇h(wk )⊤ d ≥ 0,
h(w) ≥ 0,
w1,i = 0, w2,i ≥ 0, i ∈ I1k ,
k
0 ≤ w1,i k
+d1,i ⊥ w2,i +d2,i ≥ 0, ∀i, w1,i ≥ 0, w2,i = 0, i ∈ I2k .

∥d∥∞ ≤ ρ,
Main steps:
1. Solve LPEC(wk , ρ) to determine the active set for BNLP(wk + d) - or verify
B-stationarity. Don’t use d for iterate update.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 19/37
Algorithm for computing a B-stationary point of MPECs

LPEC(wk , ρ) - full
BNLP(wk )
k ⊤
min ∇f (w ) d
d∈Rn
min f (x)
s.t. g(wk ) + ∇g(wk )⊤ d = 0, x∈Rn
s.t. g(w) = 0,
h(wk ) + ∇h(wk )⊤ d ≥ 0,
h(w) ≥ 0,
w1,i = 0, w2,i ≥ 0, i ∈ I1k ,
k
0 ≤ w1,i k
+d1,i ⊥ w2,i +d2,i ≥ 0, ∀i, w1,i ≥ 0, w2,i = 0, i ∈ I2k .

∥d∥∞ ≤ ρ,
Main steps:
1. Solve LPEC(wk , ρ) to determine the active set for BNLP(wk + d) - or verify
B-stationarity. Don’t use d for iterate update.
2. Solve BNLP(wk + d) for accuracy: if decrease in objective accept step, else: resolve
LPEC with smaller ρ.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 19/37
Full vs reduced LPEC

LPEC(wk , ρ) - full ▶ If wk infeasible, then not clear how to

define index sets in reduced LPEC.
min ∇f (wk )⊤ d ▶ If wk feasible, then d = 0 feasible.
d∈Rn

s.t. g(wk ) + ∇g(wk )⊤ d = 0, ▶ If wk not B-stationary, then

∇f (wk )⊤ d < 0.
h(wk ) + ∇h(wk )⊤ d ≥ 0,
▶ Use wk + d for next active set guess.
k k
0 ≤ w1,i +d1,i ⊥ w2,i +d2,i ≥ 0, ∀i,
▶ More complementarity constraints, more
∥d∥∞ ≤ ρ, computationally expensive? Depends on ρ.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 20/37
Full vs reduced LPEC

LPEC(wk , ρ) - full ▶ If wk infeasible, then not clear how to

define index sets in reduced LPEC.
min ∇f (wk )⊤ d ▶ If wk feasible, then d = 0 feasible.
d∈Rn

s.t. g(wk ) + ∇g(wk )⊤ d = 0, ▶ If wk not B-stationary, then

∇f (wk )⊤ d < 0.
h(wk ) + ∇h(wk )⊤ d ≥ 0,
▶ Use wk + d for next active set guess.
k k
0 ≤ w1,i +d1,i ⊥ w2,i +d2,i ≥ 0, ∀i,
▶ More complementarity constraints, more
∥d∥∞ ≤ ρ, computationally expensive? Depends on ρ.

Lemma
Let w ∈ Ω be a feasible point of the MPEC (1). For all trust region radii that satisfy
k k
0 < ρ < ρ̄ = min{{w1,i | i ∈ I+0 } ∪ {w2,i | i ∈ I0+ }}, (3)

the sets of the local minimizers of the reduced and full LPECs are identical. In the special case
of I00 = {1, . . . , m} the reduced and full LPEC coincide and in that case, ρ̄ = ∞.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 20/37
Example: nonconvexity of LPECs and better/worse minima

Assumption in example: LPEC solved to global optimality (turns out: not so restrictive in
practice).
3
Consider the two-dimensional MPEC:
f (x)
2 2 !rf (7x)
min 4(w1 − 1) + (w2 − 1) d
w∈R2 2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.
1 x
^

x2
▶ Two B-stationary points w̄ = (1, 0) and ŵ = (0, 1), x
7
with f (w̄) = 1 and f (ŵ) = 4. 0
▶ If ρ is sufficiently small, ŵ with f (ŵ) = 4 is verified.

-1
-1 0 1 2 3
x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 21/37
Example: nonconvexity of LPECs and better/worse minima

Assumption in example: LPEC solved to global optimality (turns out: not so restrictive in
practice).

Consider the two-dimensional MPEC: 3

f (x)
2 2 !rf (7x)
min 4(w1 − 1) + (w2 − 1) d
w∈R2 2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.
1 x
^

x2
▶ Two B-stationary points w̄ = (1, 0) and ŵ = (0, 1),
x
7
with f (w̄) = 1 and f (ŵ) = 4. 0
▶ If ρ is sufficiently small, ŵ with f (ŵ) = 4 is verified.
▶ If ρ large, the globally optimal LPEC solution finds
-1
a BNLP with f (w̄) = 1.
-1 0 1 2 3
x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 21/37
Example: nonconvexity of LPECs and better/worse minima

Assumption in example: LPEC solved to global optimality (turns out: not so restrictive in
practice).
Consider the two-dimensional MPEC: 3
f (x)
min2 4(w1 − 1)2 + (w2 − 1)2 !rf (7x)
w∈R d
2
s.t. 0 ≤ w1 ⊥ w2 ≥ 0.

1 x
^

x2
▶ Two B-stationary points w̄ = (1, 0) and ŵ = (0, 1),
with f (w̄) = 1 and f (ŵ) = 4. x
7
0
▶ If ρ is sufficiently small, ŵ with f (ŵ) = 4 is verified.
▶ If ρ large, the globally optimal LPEC solution finds
a BNLP with f (w̄) = 1. -1
▶ Conversely, start w̄, the full LPEC finds a BNLP -1 0 1 2 3
with a larger objective, step rejected, ρ reduced. x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 21/37
On solving LPECs

Solution methods for LPECs:

1. Regularization and penalty methods.
▶ Pro: easy to implement.
▶ Con: not guaranteed to verify d = 0 as B-stationary point of LPEC.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 22/37
On solving LPECs

Solution methods for LPECs:

1. Regularization and penalty methods.
▶ Pro: easy to implement.
▶ Con: not guaranteed to verify d = 0 as B-stationary point of LPEC.
2. Tailored LPEC methods, extensions of the simplex method, branch and cut.
▶ Pro: guaranteed to verify d = 0 as B-stationary point of LPEC.
▶ Con: difficult to implement.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 22/37
On solving LPECs

Solution methods for LPECs:

1. Regularization and penalty methods.
▶ Pro: easy to implement.
▶ Con: not guaranteed to verify d = 0 as B-stationary point of LPEC.
2. Tailored LPEC methods, extensions of the simplex method, branch and cut.
▶ Pro: guaranteed to verify d = 0 as B-stationary point of LPEC.
▶ Con: difficult to implement.
3. Mixed-integer reformulation: reformulate into equivalent mixed-integer linear program
(MILP).
▶ Pro: trivial to implement, find global minimum.
▶ Con: worst case exponential complexity.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 22/37
LPEC as MILP

▶ Trust region ∥d∥ ≤ ρ makes feasible

LPEC(wk , ρ) as MILP
set compact.
min ∇f (wk )⊤ d (4a) ▶ bigM dominated by ρ, just needs to
d∈Rn ,
y∈{0,1}m be large enough for feasiblity
M = max((w1k , w2k )) + ρ.
s.t. g(wk ) + ∇g(wk )⊤ d = 0, (4b)
▶ At feasible points, ρ can be arbitarly
k k ⊤
h(w ) + ∇h(w ) d ≥ 0, (4c) small - very tight relaxations.
0≤ w1k + d1 ≤ yM, (4d) ▶ BNLP index sets:
0≤ w2k + d2 ≤ (e − y)M, (4e) I1 (wk + d) = {i | yi = 0},
∥d∥∞ ≤ ρ. (4f)
I2 (wk + d) = {i | yi = 1}.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 23/37
LPEC as MILP

▶ Trust region ∥d∥ ≤ ρ makes feasible

LPEC(wk , ρ) as MILP
set compact.
min ∇f (wk )⊤ d (4a) ▶ bigM dominated by ρ, just needs to
d∈Rn ,
y∈{0,1}m be large enough for feasiblity
M = max((w1k , w2k )) + ρ.
s.t. g(wk ) + ∇g(wk )⊤ d = 0, (4b)
▶ At feasible points, ρ can be arbitarly
k k ⊤
h(w ) + ∇h(w ) d ≥ 0, (4c) small - very tight relaxations.
0≤ w1k + d1 ≤ yM, (4d) ▶ BNLP index sets:
0≤ w2k + d2 ≤ (e − y)M, (4e) I1 (wk + d) = {i | yi = 0},
∥d∥∞ ≤ ρ. (4f)
I2 (wk + d) = {i | yi = 1}.

Lemma
Let w∗ ∈ Ω be a S-stationary point of the MPEC (1). For a sufficiently small ρ > 0, a global
minimizer of the relaxed MILP (4) is d = 0, and any y ∈ {0, 1}m such that (4d) and (4e) hold.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 23/37
Phase II - finding feasible points

Crossover strategy
1. Use regularization or penalty based method with σ k .
2. If ∥diag(w1 )w2 ∥∞ < ρ0 , solve LPEC(w∗ (σ k ), ρ0 ).
3. Solve BNLP(w∗ (σ k ) + d), if successful, feasible point found.
4. If not, reduce σ k and go to 1.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 24/37
Phase II - finding feasible points

Crossover strategy
1. Use regularization or penalty based method with σ k .
2. If ∥diag(w1 )w2 ∥∞ < ρ0 , solve LPEC(w∗ (σ k ), ρ0 ).
3. Solve BNLP(w∗ (σ k ) + d), if successful, feasible point found.
4. If not, reduce σ k and go to 1.

Theorem (Feasibility (informal), N. & Leyffer, 2025 )

Under suitable technical assumptions, if w(σ k ) is close enough to a feasible point of the
MPEC, then for a sufficently large ρ, every feasible point d of LPEC(w∗ (σ k ), ρ0 ) predicts a
feasible BNLP.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 24/37
Phase II - finding feasible points

Crossover strategy
1. Use regularization or penalty based method with σ k .
2. If ∥diag(w1 )w2 ∥∞ < ρ0 , solve LPEC(w∗ (σ k ), ρ0 ).
3. Solve BNLP(w∗ (σ k ) + d), if successful, feasible point found.
4. If not, reduce σ k and go to 1.

Theorem (Feasibility (informal), N. & Leyffer, 2025 )

Under suitable technical assumptions, if w(σ k ) is close enough to a feasible point of the
MPEC, then for a sufficently large ρ, every feasible point d of LPEC(w∗ (σ k ), ρ0 ) predicts a
feasible BNLP.

Theorem (Convergence (informal), N. & Leyffer, 2025)

Under suitable technical assumptions, given a feasible point w0 ∈ Ω, the MPECopt algorithm
finds a B-stationary point in a finite number of iterations.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 24/37
Example: identifying a feasible BNLP
Combination of LPECs with regularization methods.

Consider the two-dimensional MPEC:

4
min2 − 2w1 + w2 f (x)
w∈R c(x) = 0
3 x1 x2 = =
s.t. − w1 − (w2 − a)2 + 1 ≥ 0
0 ≤ w1 ⊥ w2 ≥ 0 2

x2
1 x$ (= )
▶ In the example, we set a = 1.1.

-1
-1 0 1 2 3 4
x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 25/37
Example: identifying a feasible BNLP
Combination of LPECs with regularization methods.

Consider the two-dimensional MPEC:

4
min2 − 2w1 + w2 f (x)
w∈R !rf (x$ (= ))
3 c(x) = 0
s.t. − w1 − (w2 − a)2 + 1 ≥ 0
c(x$ (= ))+rc(x$ (= ))> d = 0
0 ≤ w1 ⊥ w2 ≥ 0 2
d

x2
1 x$ (= )
▶ In the example, we set a = 1.1.
▶ If σ is not small enough, LPEC selects an infeasible
0
BNLP.
-1
-1 0 1 2 3 4
x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 25/37
Example: identifying a feasible BNLP
Combination of LPECs with regularization methods.

Consider the two-dimensional MPEC:

4
min2 − 2w1 + w2 f (x)
w∈R c(x) = 0
3 x1 x2 = =
s.t. − w1 − (w2 − a)2 + 1 ≥ 0
0 ≤ w1 ⊥ w2 ≥ 0 2

x2
1
▶ In the example, we set a = 1.1.
x$ (= )
▶ If σ is not small enough, LPEC selects an infeasible
0
BNLP.
▶ For smaller τ LPEC predicts correct BNLP.
-1
-1 0 1 2 3 4
x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 25/37
Example: identifying a feasible BNLP
Combination of LPECs with regularization methods.

Consider the two-dimensional MPEC:

4
min − 2w1 + w2 f (x)
w∈R2
2
!rf (x$ (= ))
s.t. − w1 − (w2 − a) + 1 ≥ 0 3 c(x) = 0
0 ≤ w1 ⊥ w2 ≥ 0 c(x$ (= ))+rc(x$ (= ))> d = 0
d
2

x2
▶ In the example, we set a = 1.1. 1
▶ If σ is not small enough, LPEC selects an infeasible x$ (= )
BNLP. 0
▶ For smaller τ LPEC predicts correct BNLP.
▶ In practice, often for large σ the LPEC finds a -1
feasible BNLP. -1 0 1 2 3 4
x1

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 25/37
Example: identifying a feasible BNLP
Combination of LPECs with regularization methods.

Consider the two-dimensional MPEC:

4
min − 2w1 + w2 f (x)
w∈R2
2
!rf (x$ (= ))
s.t. − w1 − (w2 − a) + 1 ≥ 0 3 c(x) = 0
0 ≤ w1 ⊥ w2 ≥ 0 c(x$ (= ))+rc(x$ (= ))> d = 0
d
2

x2
▶ In the example, we set a = 1.1. 1
▶ If σ is not small enough, LPEC selects an infeasible x$ (= )
BNLP. 0
▶ For smaller τ LPEC predicts correct BNLP.
▶ In practice, often for large σ the LPEC finds a -1
feasible BNLP. -1 0 1 2 3 4
▶ Moreover, often the solution of this BNLP coincides x1
with the solution of the MPEC.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 25/37
Outline of the lecture

1 Overview of MPCC methods

2 Regularization and penalty methods

3 Active-set methods

4 Numerical benchmarks

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 26/37
Comparison of Phase I methods on macMPEC

1 1

0.8 0.8
Fraction solved (Phase I)

Fraction solved (Phase I)

0.6 0.6

0.4 0.4

Reg-LPEC Reg-LPEC
0.2 Reg-Simple Reg-Simple
Pen-`1 -LPEC
0.2
Pen-`1 -LPEC
Feasibility-`1 Feasibility-`1
Feasibility-`1 Feasibility-`1
0 0
10!4 10!2 100 102 104 0 5 10 15
Wall time (s) 2x times best
(a) Absolute timings. (b) Relative timings.

Figure: Evaluating different Phase I algorithms in MPECopt on the MacMPEC test set.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 27/37
Comparison of LPEC methods on macMPEC (1/3)

1 1

0.8 0.8
Fraction solved

Fraction solved
0.6 0.6

0.4 0.4

0.2 Gurobi-MILP Gurobi-MILP

HiGHS-MILP
0.2
HiGHS-MILP
Reg-MPEC Reg-MPEC
`1 -MPEC `1 -MPEC
0 0
10!4 10!2 100 102 104 0 5 10 15 20 25 30 35
Total LPEC wall time (s) 2x times best for LPEC
(a) Absolute timings. (b) Relative timings.

Figure: Evaluating different LPEC algorithms in MPECopt on the MacMPEC test set.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 28/37
Comparison of LPEC methods on macMPEC (2/3)

104
Gurobi-MILP
HiGHS-MILP
Reg-MPEC
Maximal LPECs wall time

102 `1 -MPEC

100

10!2

10!4
0 20 40 60 80 100 120 140 160
Problem instance

Figure: Maximal solution times for different LPEC algorithms in MPECopt on the MacMPEC test set.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 29/37
Comparison of LPEC methods on macMPEC (3/3)

Gurobi-MILP HiGHS-MILP Reg-MPEC `1 -MPEC

LPEC wall time

100 100
100 100

10!2 10!2

100 100 100 100

NLP wall time NLP wall time NLP wall time NLP wall time

Figure: Comparison of total NLP and LPEC computation times on the MacMPEC.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 30/37
Active-set vs regularization methods on macMPEC

1 1

0.8 0.8
Fraction solved

Fraction solved
0.6 0.6

0.4 0.4

MPECopt-Reg-Gurobi MPECopt-Reg-Gurobi
0.2 MPECopt-Reg-HiGHS MPECopt-Reg-HiGHS
MPECopt-`1 -Gurobi
0.2
MPECopt-`1 -Gurobi
Reg Reg
Pen-`1 Pen-`1
0 0
10!2 100 102 104 0 5 10 15
Wall time (s) 2x times best
(a) Absolute timings. (b) Relative timings.

Figure: Evaluating different MPEC solution methods on the MacMPEC test set in terms of finding a
stationary point.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 31/37
Number of NLP and LPEC solves

12 MPECopt-Reg-Gurobi Reg
MPECopt-Reg-HiGHS Pen-`1
MPECopt-`1 -Gurobi
10
NLPs solved

0
0 20 40 60 80 100 120 140 160 180 200
Problem instance
Figure: Number of NLPs (top plot) and NLPs (bottom plots) solved in MacMPEC on all problem
instances.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 32/37
Number of NLP and LPEC solves

6
MPECopt-Reg-Gurobi
MPECopt-Reg-HiGHS
5 MPECopt-`1 -Gurobi
Reg
Pen-`1
4
LPECs solved

0
0 20 40 60 80 100 120 140 160 180 200
Problem instance
Figure: Number of NLPs (top plot) and NLPs (bottom plots) solved in MacMPEC on all problem
instances.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 32/37
Distribution of stationary points

MPECopt-Reg-Gurobi
180
not B stationary
160 B stationary

140

120

100
Count

80

60

40

20

0
A C M S W Failure
Stationary point

Figure: Distribution of stationary points on MacMPEC for different solution methods. Failure counts the
number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 33/37
Distribution of stationary points

MPECopt-`1 -Gurobi
180
not B stationary
160 B stationary

140

120

100
Count

80

60

40

20

0
A C M S W Failure
Stationary point

Figure: Distribution of stationary points on MacMPEC for different solution methods. Failure counts the
number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 33/37
Distribution of stationary points

Reg
150
not B stationary
B stationary

100
Count

50

0
A C M S W Failure
Stationary point

Figure: Distribution of stationary points on MacMPEC for different solution methods. Failure counts the
number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 33/37
Distribution of stationary points

Pen-`1
140
not B stationary
B stationary
120

100

80
Count

60

40

20

0
A C M S W Failure
Stationary point

Figure: Distribution of stationary points on MacMPEC for different solution methods. Failure counts the
number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 33/37
Active set vs regularization methods on random MPECs

1 1

0.8 0.8
Fraction solved

Fraction solved
0.6 0.6

0.4 0.4

MPECopt-Reg-Gurobi MPECopt-Reg-Gurobi
0.2 MPECopt-Reg-HiGHS MPECopt-Reg-HiGHS
MPECopt-`1 -Gurobi
0.2
MPECopt-`1 -Gurobi
Reg Reg
Pen-`1 Pen-`1
0 0
10!2 10!1 100 101 102 103 0 5 10 15
Wall time (s) 2x times best
(a) Absolute timings. (b) Relative timings.

Figure: Evaluating different MPEC solution methods on the synthetic test set in terms of finding a
stationary point.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 34/37
Number of NLP and LPEC solves

12 MPECopt-Reg-Gurobi Reg
MPECopt-Reg-HiGHS Pen-`1
MPECopt-`1 -Gurobi
10
NLPs solved

0
0 20 40 60 80 100 120 140 160 180 200
Problem instance
Figure: Number of NLPs (top plot) and NLPs (bottom plots) solved in the synthetic test set on all
problem instances.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 35/37
Number of NLP and LPEC solves

3.5

3
LPECs solved

2.5

1.5

1
MPECopt-Reg-Gurobi Reg
0.5 MPECopt-Reg-HiGHS Pen-`1
MPECopt-`1 -Gurobi
0
0 20 40 60 80 100 120 140 160 180 200
Problem instance
Figure: Number of NLPs (top plot) and NLPs (bottom plots) solved in the synthetic test set on all
problem instances.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 35/37
Distribution of stationary points

MPECopt-Reg-Gurobi
200
not B stationary
B stationary

150
Count

100

50

0
A M S W Failure
Stationary point

Figure: Distribution of stationary points on the synthetic test set for different solution methods. Failure
counts the number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 36/37
Distribution of stationary points

MPECopt-`1 -Gurobi
100
not B stationary
B stationary

80

60
Count

40

20

0
A C S W Failure
Stationary point

Figure: Distribution of stationary points on the synthetic test set for different solution methods. Failure
counts the number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 36/37
Distribution of stationary points

MPECopt-`1 -Gurobi
200
not B stationary
B stationary

150
Count

100

50

0
A M S W Failure
Stationary point

Figure: Distribution of stationary points on the synthetic test set for different solution methods. Failure
counts the number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 36/37
Distribution of stationary points

Reg
120
not B stationary
B stationary
100

Count 80

60

40

20

0
A M S W Failure
Stationary point

Figure: Distribution of stationary points on the synthetic test set for different solution methods. Failure
counts the number of infeasible problems.
6. Numerical methods for mathematical programs with complementarity constraints
A. Nurkanović 36/37
Summary

▶ MPEC methods solve (approximately) a sequence of more regular NLP.

▶ Regularization/penalty methods: easy to implement, but may converge to spurious
stationary points.
▶ In practice, they do not converge often to spurious stationary points.
▶ However, if MPEC-LICQ does not hold we can only know if we solve an LPEC.
▶ Active-set methods solve a sequence of branch NLPs. Converge always to B-stationary
points.
▶ LPECs of reasonable size can be efficiently solved as MILPs, because small ρ means not so
much branching.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 37/37
Cited references I

Anitescu, M. (2005a).
Global convergence of an elastic mode approach for a class of mathematical programs with
complementarity constraints.
SIAM Journal on Optimization, 16:120–145.
Anitescu, M. (2005b).
On using the elastic mode in nonlinear programming approaches to mathematical
programs with complementarity constraints.
SIAM Journal on Optimization, 15(4):1203–1236.
Fukushima, M. and Tseng, P. (2002).
An implementable active-set algorithm for computing a b-stationary point of a
mathematical program with linear complementarity constraints.
SIAM Journal on Optimization, 12(3):724–739.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 38/37
Cited references II

Giallombardo, G. and Ralph, D. (2008).

Multiplier convergence in trust-region methods with application to convergence of
decomposition methods for mpecs.
Mathematical Programming, 112(2):335–369.
Guo, L. and Deng, Z. (2022).
A new augmented lagrangian method for mpccs—theoretical and numerical comparison
with existing augmented lagrangian methods.
Mathematics of Operations Research, 47(2):1229–1246.
Hoheisel, T., Kanzow, C., and Schwartz, A. (2013).
Theoretical and numerical comparison of relaxation methods for mathematical programs
with complementarity constraints.
Mathematical Programming, 137(1):257–288.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 39/37
Cited references III

Izmailov, A. F. and Solodov, M. V. (2008).

An active-set newton method for mathematical programs with complementarity
constraints.
SIAM Journal on Optimization, 19(3):1003–1027.
Jiang, H. and Ralph, D. (1999).
Qpecgen, a matlab generator for mathematical programs with quadratic objectives and
affine variational inequality constraints.
Computational Optimization and Applications, 13:25–59.
Kadrani, A., Dussault, J.-P., and Benchakroun, A. (2009).
A new regularization scheme for mathematical programs with complementarity constraints.

SIAM Journal on Optimization, 20(1):78–103.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 40/37
Cited references IV

Kanzow, C. and Schwartz, A. (2013).

A new regularization method for mathematical programs with complementarity constraints
with strong convergence properties.
SIAM Journal on Optimization, 23(2):770–798.
Kanzow, C. and Schwartz, A. (2015).
The price of inexactness: convergence properties of relaxation methods for mathematical
programs with complementarity constraints revisited.
Mathematics of Operations Research, 40(2):253–275.
Kazi, S. R., Thombre, M., and Biegler, L. (2024).
Globally convergent method for optimal control of hybrid dynamical systems.
IFAC-PapersOnLine, 58(14):868–873.
Kirches, C., Larson, J., Leyffer, S., and Manns, P. (2022).
Sequential linearization method for bound-constrained mathematical programs with
complementarity constraints.
SIAM Journal on Optimization, 32(1):75–99.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 41/37
Cited references V

Leyffer, S., López-Calva, G., and Nocedal, J. (2006).

Interior methods for mathematical programs with complementarity constraints.
SIAM Journal on Optimization, 17(1):52–77.
Leyffer, S. and Munson, T. S. (2007).
A globally convergent filter method for mpecs.
Preprint ANL/MCS-P1457-0907, Argonne National Laboratory, Mathematics and
Computer Science Division.
Lin, G.-H. and Fukushima, M. (2005).
A modified relaxation scheme for mathematical programs with complementarity
constraints.
Annals of Operations Research, 133(1):63–84.
Lin, G.-H. and Fukushima, M. (2006).
Hybrid approach with active set identification for mathematical programs with
complementarity constraints.
Journal of optimization theory and applications, 128(1):1–28.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 42/37
Cited references VI

Liu, X., Perakis, G., and Sun, J. (2006).

A robust sqp method for mathematical programs with linear complementarity constraints.
Computational Optimization and Applications, 34(1):5–33.
Luo, Z., Pang, J., and Ralph, D. (1996).
Mathematical Programs with Equilibrium Constraints.
Cambridge University Press, Cambridge.
Nurkanović, A., Pozharskiy, A., and Diehl, M. (2024).
Solving mathematical programs with complementarity constraints arising in nonsmooth
optimal control.
Vietnam Journal of Mathematics, pages 1–39.
Raghunathan, A. U. and Biegler, L. T. (2005).
An interior point method for mathematical programs with complementarity constraints
(mpccs).
SIAM Journal on Optimization, 15(3):720–750.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 43/37
Cited references VII

Ralph, D. and Wright, S. J. (2004).

Some properties of regularization and penalization schemes for mpecs.
Optimization Methods and Software, 19(5):527–556.
Scholtes, S. (2001).
Convergence properties of a regularization scheme for mathematical programs with
complementarity constraints.
SIAM Journal on Optimization, 11(4):918–936.
Scholtes, S. and Stöhr, M. (1999).
Exact penalization of mathematical programs with equilibrium constraints.
SIAM Journal on Control and Optimization, 37(2):617–652.
Steffensen, S. and Ulbrich, M. (2010).
A new relaxation scheme for mathematical programs with equilibrium constraints.
SIAM Journal on Optimization, 20(5):2504–2539.

6. Numerical methods for mathematical programs with complementarity constraints

A. Nurkanović 44/37