Conic Programming in GAMS

Armin Pruessner,
Michael Bussieck, Steven Dirkse, Alex Meeraus

GAMS Development Corporation

INFORMS 2003, Atlanta
October 19-22
 Direction

• What this talk is about

– Overview: the class of conic programs
– Conic constraints and examples using GAMS
– Solvers in GAMS and numerical results
– CONE World - a forum for conic programming

• What this talk is not about:

– Conic programming algorithms
– Detailed applications

2
 Overview

What are conic programs?

• Generalized linear programs with the addition
of nonlinear convex cones
• Class includes, for example,
• Linear program (LP)
• (Convex) Quadratic program (QP)
• (Convex) Quadratically constrained QP (QCQP)

• Recently much activity in this area!

3
 Areas and Applications
Conic programming used for:
• Engineering
– Truss topology design
– FIR filter design
• Finance
– Portfolio optimization
• Statistics and Numerical Linear Algebra
– Robust linear programming
– Norm minimization problems

7th DIMACS Implementation Challenge on SDP and

Second Order Cone Programming (SOCP)
4
 Modeling and Solving

Model Generality
Less More
General General

(Convex) (Convex)
LP SOCP SDP
QP QCQP

Less More
Difficult Difficult
Solution Difficulty

5
 Cone Programs
• General form of conic program

min f T x
s.t. Ax ≤ b
x ∈ C, x ∈ l x , u x [ ]
where C is a second order cone (dim=k):

 x1:( k −1)  
Ck =   : x1:( k −1) ≤ xk 
  xk 
2

6
 Example (quadratic cone)
Quadractic cone C sometimes also called
Lorentz cone (or ice cream cone)

Trivial Quadratic Cone:

2 2
z ≥ x +y

7
 Second Order Cone Programs
min f Tx
s.t . C i x + d i ≤ a x + bi ,
T
i i = 1,..., N

x ∈ ℜ n f , a i ∈ ℜ n C i ∈ ℜ ( k i −1)× n d i ∈ ℜ k i −1 bi ∈ ℜ
Equivalent to conic program

• Linear constraints: cone dimension k=1

• Cone constraints: change of variables
(vector) y = Ci x + d i , z = aiT x + bi
8
 Types of Cones
• Quadratic Cone
2
xi ≥ ∑ xi
j ≠i
• Rotated Quadratic Cone
2
2x x
i j
≥ ∑x k
k ≠i ,k ≠ j

Sometimes preferable for modeling quadratic

inequalities
9
 Rotated Quadratic Cone

• Show equivalence to quadratic cone:

2
2 xy ≥ z 2
Rotated quadratic cone

xy ≥ ∑ z − xy 2
i +x +y2 2

 z 
(x + y ) ≥   Quadratic cone
( x − y ) 
10
 General Transformation

• If x ∈ C ⇔ Ax ∈ C
r q
where

• C r rotated quadratic cone

• C q quadratic cone
 1 1 
 0 L 0
2 2
 1 1 
 − 0 L 0
A=  2 2 
 0 0 1 0
 
 M M O M
 0 0 0 L 1
11
 Conic Constraints in GAMS

In the GAMS modeling language:

• Conic constraints denoted by =C=

• Conic programs result in “linear

programs” in GAMS

12
 Quadratic Cone in GAMS
Set s /s1,s2,…,sn/;
Set t(s) / s2,…,sn/;
Variable x(s);

Conic “LP” formulation

x(‘s1’) =C= sum(t(s), x(s));

Equivalent NLP formulation

x(‘s1’) =G= sqrt[sum(t(s),sqr(x(s)))];

Æ Note: Summation on right hand side 13

 Rotated Quadratic Cone
Set s /s1,s2,s3,…,sn/;
Set t(s) / s3,…,sn/;
Variable x(s);

Conic “LP” formulation

x(‘s1’)+x(‘s2’) =C= sum(t(s), x(s));

Equivalent NLP formulation

2*x(‘s1’)*x(‘s2’)=G= sum[t(s),sqr(x(s))];
Æ Note: Summation on right hand side
14
 Example: Complex L1 Norm

Complex L1 Norm Minimization

• Minimize ||Ax-b||1 where A,x,b are complex

valued

• We can write this as

Re( A) − Im( A) Re( x)   Re(b) 

min     −  
 Im( A) Re( A)  Im( x)   Im(b)  2

15
 Example (Continued)
Complex L1 Norm Minimization

min ∑ t (i)
i

 z re(i)  Re( A) − Im( A) Re( x)   Re(b) 

s.t.   =     −  
 zim(i)  Im( A) Re( A)  Im( x)   Im(b) 
t (i ) ≥ z re2 (i ) + zim
2
(i ) (quadratic cone)

16
 GAMS Model (Quadratic Cone)
Objective.. obj =E= sum(i, t(i));

reseq_re(i).. res_re(i) =E= sum(j, A_re(i,j)*x_re(j))

- sum(j,A_im(i,j)*x_im(j))
- b_re(i);

reseq_im(i).. res_im(i) =E= …

coneeq(i).. t(i) =C= res_re(i) + res_im(i);

Model conemodel /objective, reseq_re, reseq_im, coneeq/;

Solve conemodel using lp minimizing obj;

17
 Portfolio Optimization
∑ x jσ j , j ' x j ' = α Dx Objective is to
2
min
j, j'
2 minimize variance
(risk), subject to an
s.t. ∑xj
j = 1, xi ≥ 0 expected return
σj,j’ = covariance
∑p x
j
j j ≥ rmin
H=1 / (numdays-1)

xj = % of investment in stock j
pj = price change (return) for stock j
rmin = minimum expected return
Dj,d = Deviation per day d of stock j wrt to mean return
18
 Portfolio Optimization (Cont.)
2
Can rewrite: min α Dx 2
By introducing intermediate variables p,q, and w:

minimize α 2r
subject to w(d ) = ∑ D( j , d ) x( j )
j

q =1
≥ ∑ w(d)
2
2qr
d
19
 GAMS Model (Rotated Cone)
Objective.. obj =E= a*(2*r);

Budget.. sum(j, x(j)) =E= 1;

Return.. Sum(j, p(j)*x(j)) =G= rmin;

Wcone(days).. w(d) = sum(j, D(j,d)*x(j);

cone_eq1.. q =E= 1;

cone_eq2.. q + r =C= sum(d, w(d));

Model conemodel / all /;

Solve conemodel using lp minimizing obj;

20
 GAMS/MOSEK
Solving Conic Models in GAMS:

• Newest addition is MOSEK

– LP (simplex or interior point)
– MIP (branch and bound)
– Conic Programs (conic interior point):
– Convex NLP

• Solver CONOPT also accepts conic constraints

21
 Numerical Examples

DIMACS Challenge Models (SDP and SOCP)

• Chose subset of models from DIMACS (only
SOCP models)

• SOCP models:
– Sum of norms
– Antenna array weight design
– Scheduling problems

Æ MOSEK solves all SOCP models and is the most

efficient
22
 Numerical Examples

DIMACS Benchmarks by Hans Mittelmann

• Solvers: MOSEK 2.5.1 (ext MPS), LOQO

6.03, SDPT3 3.01, SeDuMi 1.05R4

• 18 SOCP problems
– In SeDuMi (MATLAB) format
– MOSEK: extended QPS format (based on MPS)

23
 DIMACS Results (H. Mittelmann)
Problem LOQO MOSEK SDPT3 SeDuMi
Nb 11 3 11 9
Nb_L1 9 3 20 11
Nb_L2 16 8 18 24
Nb_L2_Bessel 7 2 11 12

Nql30 11 2 4 4
Nql60 151 9 14 9
Nql180 MM 182 232 459

Qssp30 15 2 6 3
Qssp60 221 9 29 18
Qssp180 MM 355 504 780

Sched_50_50_orig 5 1 6 6
Sched_100_50_orig 22 2 17 13
Sched_100_100_orig 94 4 28 29
Sched_200_100_orig 409 10 95 138

Sched_50_50_orig 7 1 6 5
Sched_100_50_orig 28 2 13 9
Sched_100_100_orig 107 5 22 32
Sched_200_100_orig 445 10 68 75

MM: memory problems

24
 DIMACS Results (Cont.)

Use Performance Profiles (Dolan and Moré,

2002) to visualize results:

• Cumulative distribution function for a

performance metric

• Performance metric: ratio τ of current solver

time over best time of all solvers for “success”

• Intuitively: probability of success if given τ

times fastest time (τ=ratio)
25
 Profiles (Data: H. Mittelmann)

Figure 1 26
 CONE World
• An online forum for
discussion and
information on cone
programming

• CONELib library of
models
– GAMS cone format
– NLP formulation

• Conic programming
solvers

• Links and lists

27
ConeLib – Conic Models in GAMS

28
ConeLib – Conic Models in GAMS

29
 Cone Versus NLP Formulation

• Modeling conic constraints can be tedious

• NLP form of quadratic cone is more natural
• Rotated quadratic cone for QP is cumbersome
BUT
• Potentially significant computational advantages

• Compare cone vs. NLP formulation on CONELib

(currently 28 models)
• NLP formulation: substitute into conic constraint (converted
using CONVERT utility in GAMS)
• In practice “smarter” NLP formulations may exist depending
on model
30
 Cone Vs. NLP (Efficiency)
Performance Profile
100
90
Percent Of Models Solved

80
70
60
50
40
30
MOSEK
20 NLP-1
NLP-2
10 NLP-3
0
1 10 100 1000
Time Factor
Figure 2 31
 Reproducibility of Results

• Reproducibility of results is key to validity in scientific

disciplines (but sometimes neglected)
– Open data source (models, solvers, solver options)
– Should be easily reproducible

• As part of the GAMS World, we provide downloadable

scripts to reproduce results

• Currently only for NLP/Cone models (Global World)

– See www.gamsworld.org/global/reproduce

32
Reproducibility of Results

33
Reproducibility of Results

34
 Conclusions
• Addition of conic programming capability within
GAMS

• MOSEK as state-of-the-art conic programming

solver
• Although modeling still cumbersome, significant
computational advantages using cones
• CONELib: growing collection of conic
programming models

• Presentation will be available under

http://www.gams.com/presentations 35
 References
• Lobo, M.S., Vandenberghe, L., Boyd, S. and Lebret, H.
Applications of Second Order Cone Programming,
Linear Algebra and its Applications, 284:193-228,
November 1998.
• MOSEK Optimization Tools Help Desk, Version 2, online
at www.mosek.com/documentation, 2003.

• H.D. Mittelmann, An independent benchmarking of SDP

and SOCP solvers, Math Program., Series B, 2002
(appeared electronically).

• E. D. Dolan and J. J. More, Benchmarking optimization

software with performance profiles, Math. Programming, 91 (2),
201-213, 2002. 36