See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/235012188

An Algorithm for the Generalized Matrix Eigenvalue Problem Ax = Lambda Bx

Article · October 1971

CITATIONS READS

79 1,584

2 authors, including:

Cleve Moler
The MathWorks, Inc
133 PUBLICATIONS 13,457 CITATIONS

SEE PROFILE

All content following this page was uploaded by Cleve Moler on 02 January 2014.

The user has requested enhancement of the downloaded file.

 CNA-32

MU,
AN ALGORITHM FOR THE GENERALIZED MATRIX

EIGENVALUE PROBLEM Ax = XBx

by

C. B. Moler
Stanford University

G. W. Stewart

October, 1971 CNA-32

Supported in part by an NSF grant at Stanford University

and by an ONR contract at the University of Michigan.

Supported in part by NSF grant GP-23655 at The University

of Texas in Austin.

This report is issued joint.,i as STAN-CS-232-71 at Stanford

University.

jDJ3TiT~~h SA~ii1r A
o r .:u*-,ic rcle0se;
Reprocluced by U.- C e;
NATIONAL TECHNICAL £r;. " '..i...
INFORMATION SERVICE
UDpartmort
of Commerce CENTER FOR NUMERICAL ANALYSIS
Springfild VA 22151

THE UNIVERSITY OF TEXAS AT AUSTIN

sl"OOO/c%$7.d o/0 2-
 AN ALGORITHM FOR THE GENERALIZED

MATRIX EIGENVAfLJE PROBLEM

Ax =%Bx

by

C. B. Moler
Computer Science Department
Stanford University
Department of Mathematics
University of Michigan

and

G. W. Stewart
Departments of Computer Science and Mathematics
University of Texas at Austin

Abstract

*A new method, called the QZ algorithm, is presented for the

solution of the matrix eigenvalue problem Ax = %Bx with general

square matrices A and B • Particular attention is paid to the

degeneracies which result when B is singular. No inversions of B

or its submatrices are used. The algorithm is a generalization of the

QR algorithm, and reduces to it when B = I . A Fortran program and

some illustrative examples are included.

i
 AN ALGORITHM FOR THE GENERALIZED

MATRIX EIGENVALUE PROBLE4

Ax = \Bx

C. B. Moler:/
-
G. W. Stewartf.

1. Introduction

We shall be concerned with the matrix eigenvalue problem of

determining the nontrivial solutions of the equation

Ax = Bx,
where A and B are real matrices of order n . When B is nonsingular

this problem is formally equivalent to the usual eigenvalue problell

-lx
B Ax = x

When B is singular, however, such a reduction is not possible,

and in fact the characteristic polynomial det(A-4B) is of degree less

than n , so that there is not a complete - ',of eigenvalues for the

problem. In some cases the missing eigenvalues may be regarded as

"infinite". In other cases the entire problem may be poorly posed. The

term infinite eigenvalue is justified by the fact that if B is perturbed

slightly so that it is no longer singular, there may appear a number of

large eigenvalues that grow unboundedly as the perturbation is reduced to

zero. However, if det(A-%B) vanishes identically, say when A and B

have a common null space, then any % may be regarded as an eigenvalue.

Such problems have unusuaily pathological features, and we refer to them

as "ill-disposed" problems.

SI/ Computer Science Department, Stanford University, and

Department of Mathematics, University of Michigan.
-- Departments of Computer Science and Mathematics, University of
Texas at Austin.

1
 In numerical work the sharp distinction between singular and non-

singular matrices is blurred, and the pathological features associated

with singular B carry over to the case of nearly singular B • The

object of this paper is to describe an algorithm for computing the

eigenvalues and corresponding eigenvectors that is unaffected by nearly

singular B . The algorithm, the heart of which we call the QZ-algorit*.,

is essentially an iterative method for computing the decomposition

contained in the following theorem [10].

Theorem. There are unitary matrices Q and Z so that WAZ and QBZ

are "oth upper triangular.

We say that the eigenvalue problems QAZy = XQBZy and Ax = %Bx are

unitarily equivalent. The two problems obviously have the same eigenvalues,

and their eigenvectors are related by the equation x = Zy .

The algorithm proceeds in four stages. In the first, which is a

generalization of the Householder reduction of a single matrix to

Hessenberg form [4,5], A is reduced to upper Hessenberg form and at the

same time B L. reduced to upper triangular form. In the second step,

which is a generalization of the Francis implicit double shift QR algorith-

[3,8], A is reduced to quasi-triangular form while the triangular form

of B is maintained. In the third stage the auasi-triangular matrix is

effectively reduced to triangular form and the eigenvalues extracted. ..

the fourth stage khe eigenvectors are ob ined from the triangular matrices

and then transformed back into the original coordinate system.

The transformations used in reducing A and B are applied in such

a way that Wilkinson's g(:neral analysis of the roundoff errors in unitary

2
transformations [11] shows that the computed matrices are exactly unitarily

equivalent to slightly perturbed matrices A+ E and B+ F • This means

that the computed eigenvalues, which are the ratios of the diagonal elements

of the final matrices, are the exact eigenvalues of the perturbed proble5

(A+E)x = X(B+F)x . If an eigenvalue is well conditioned in the sense that

it is insensitive to small pertu,

rbations in A and B (see [10] for a

detailed analysis), then it will be computed accurately. This accuracy

is independent of the singularity or nonsingularity of B .

The use of unitary transformations in the reduction also sbnp.ifies

the problem of convergence: a quantity may be set to zero if a perttutraticn

of the same size can be tolerated in the original matrix.

Our computer program does not actually- produce the eigenvalues ,X.

but instead returns a.' and i , the diagonal elements of the triangular

matrices QAZ and QBZ . The divisions in X. =xi/pi

c become the

responsibility of the program's user. We emphasize this point because

the ai and 0i contain more information than the eigenvalues themselves.

Since our algorithm is an ex-tnsion of the QR algorithm, the well

known properties of the QR algorithm apply to describe the behavior of

our algorithm.

In their survey article [9], Peters and Wilkinson describe another

approach for the case when B is nearly singular. In their method onp

computes an approximate null space for B and removes it from the problem..

The tecnnique is reapplied to the deflated problem, and so on until a well

conditioned problem is obtained. The method has the crucial drawback that
 one must determine the rank of B If a wrong decision is reached,

the well-conditioned eigenvalues may be seriously affected.

The special case where A is symmetric and B is positive definite

has been extensively treated. For the case of well-conditioned B the

"Cholesky-Wilkinson" method [ 6 ] enjoys a well deserved popularity.

A mcdification of this algorithm for band matrices is given by Crawford[ i i.

. variant of the Peters-Wilkinson method for nearly semidefinite B has

been given by Fix and Heiberger [ 2 1. Although our method does not

preserve symmetry and is consequently more time consuming than these

algorithms, its stability may make it preferable when B is nearly

semidefinite.

ih
2. Reduction to Hessenberg-Triangular Form

In this section we shall give an algorithm whereby A is reduced

to upper Hessenberg form and simultaneously B is reduced to triangular

form. While a treatment of the reductions in this and the following sections

can be given in terms of standard plane rotations and elementary

Her-itian matrices, we find it convenient from a computational point of

view to work exclusively with a modified form of the elementary Hermitians.

A:cordingly, we introduce the following notation.

By Vr(k) we mean the class of symmetric, orthogonal matrices of

the form
I+ vu
T
where v u = -2 , v is a scalar multiple of u , only components
k,k+l,...,ktr-i of u are nonzero, and u = 1 . Given any vector x ,

it is easy to choose a member Q of V r(k) so that

QX x + (u X)v

has its k+l,...,k+r-1 components equal to zero, its k-th component changed

and all other components unchanged. Since uk = 1 7 the computation of

Qy for any y requires only 2r-1 multiplications and 2r-1 additions.

(In particular, use of a matrix in 2 requires only 5 multiplications

instead of the 4 required by a standard plane rotation.)

or the mot part, we shall use only matrices in V2 and Qf3 "

When a matrix Q in 73 (k) premtltiplies a matrix A only rows

k , k+l , and k+2 in QA are changed. Tf the elements k , k+l ,

and K+2 in a col.umn of A are zero, they remain zero in QA . Likewise,

if Zey (k) , only columns k , k+"- , arid k+2 are changed in AZ • if

some row has elements k , k*1 , and k+2 zero, then they remain zero

in AZ . Similar considerations hold for the class '(2

 All our transformations will be denoted by Q's and Z's with

i
various subscripts. The Q's will always be premultpliers, that is

row operations. The Z's will always be postmultipliers, or column

operations. The letter Q is being used in its traditional role to

denote orthogonal matrices. The letter Z was chosen to denote orthogonal

matrices which introduce zeros in strategic locations.

The first step in the reduction is to reduce B to upper triangular

form by -remultiplication by Householder reflections. The details of

this reduction are well known (e.g. see [4,11]) and we confine ourselves
to a brief description to illustrate our notation. At the k-th stage of

the reduction (illustrated below for k -5 and n 5 ), the elements

below the first k-1 diagonal elements of B are zero.

x x x x x
0 x x x x
S0 0 x x x
i0 0 x1 x x
O 0 x x x

Each x represents an arbitrary nonzero element. Each x1 represents

an element to be annihilated in the next step. A matrix Qk(%n k

is chosen to annihilate bk+l,k, bk+2,k, ... ,bn, k , and B is overwritten

by QB , giving a matrix of the form illustrated below.

x x x x x
0 x x x x
0 0 0 x x
0 0 0 xI x

This process is repeated until k = n-l . Of course A is overwritten

by6
 After this reduction, A and B have the form

A B

x x x x x x x x x
x x x x x 0 x x x x
x x x x x 0 0 x x

x x x x x 0 0 0 x
1
x x x x x 0 0 0 0 x

The problem now is reduce A to upper Hessenberg form while

preserving the triangularity of B . This is done as follows (for k =5 ).

First Qev 2 (4) is determined to annihilate a5 1 ' The matrices QA and

QB , which overwrite A and B then have the form

x x x x x x x x x x
x x x x x 0 x x x x
x x x x x 0 0 x x x
x x x x x 0 0 0 x x
0 x x x x 0 0 0 xi

The transformation has introduced a nonzero element on the (5,4)-position

of B . However, a Zey 2 ( 4 ) can be used to restore the zero without

disturbing the zero introduced in A .

This step is typical of all the others. The elements of A are

annihilated by Q's in the order illustrated below.

x x x x x
x x x x x
X3 x5 x x x
x2 x x x x
x6 x x

7
 As each element of A
is annihilated, it introduces
a nonzero element
on the subdiagonal of B , which
is immediately annihilated by
a suitably
chosen Z . The entire algorithm,
including the Householder triangulari-
zation of B may be summed up
as follows.

1) For k = 1,2,n-1

1) Choose Qkeyn~k+l(k) to annihilate

bk+l, k bk+2 ,k,..
k .,
2) B-Q k
2) For k = 1,2,...,n-2
1) For I = n-ln-2,...,pk+1
1) Choose QkIE2(l)
to annihilate al+l,
k

.2) A QkfA , B - Qki B

3) Choose Zkr'V
2 (e) to annihilate b

4~) B - BZk1 , A- 7k

The complete reduction requires

17 ad i i n an 2 s3a e o tabout 17 n3 multiplications,
additions and n square roots. If eigenvectors
are also to be
computed, the product of the
Zs must be accumulated. This
requires
an additional 3 n3 multiplications and
22 - n3 additions. The product
of the Q's is not required for the computation
of eigenvectors.

8
3. The Explicit QZ Step

In this and the next section we assume that A is upper Hessenberg

and B is upper triangular. In this section we shall propose an iterative

technique for reducing A to upper triangular form while maintaining the

triangularity of B . The idea of our approach is to pretend that B is

nonsingular and examine the standard QR algorithm for C = AB

manipulations are then interpreted as unitary equivalences on A and 3•

Specifically suppose that one step of the QR algorithm with shift

is applied to C . Then Q is determined as an orthogonal transfo-7a: 2o-

such that the matrix

(3.1) R = Q(C-kI)
is upper triangular. The next iterate C' is defined as

C' = RQT + kI = QCQT

If we set

At = QAZ
and

B' = QBZ

where Z is any unitary matrix, then

AB' T C

The matrix Q is determined by the requirement that R be upper

triangular. We choose Z so that A' is upper Hessenberg and B' is

upper triangular. This insures that the nice distribution of zeros,

introduced by the algorithm of Section 2, is preserved by the QZ step.

Thus a tentative form of our algorithm might read

'I7
iiA
 1) Determine Q so that QC is upper triangular,

2) Determine Z so that QAZ is upper Hessenberg and QBZ is upper

triangular,

3) A-WQAZ, B-QBZ.

The problem is then to give algorithms for computing Q and Z which do

not explicitly require C = ABeo( a

The determination Q is relatively easy. For from (3.1) and the

definition of C it follows that

(3.2) Q(A-kB) = RB = S

Since R and B are upper triangular, so is S . Thus Q is the unitary

matrix that reduces A-kB to upper triangular form. Since A-kB is

upper Hessenberg, Q can be expressed in the form

= . ,Q 1
(5.3)¢ n-lQn-. . '

where Q e (k).

To calculate Z we apply Q in its factored form (3.3) to B and

determine Z in a factored form so that B stays upper triangular.

Specifically QIB has the for- (k = 5)

xx x x x
1
x x x X X
0 0 x x x
0 0 0 x x
0 0 0 0 x

If QI B is postmultiplied by a suitable Z1 01 2 (1) the nonzero element

below the diagonal can be removed. Similarly Q2QBZI has the form.

10
 0 x x x x
0 x x x x
0 x1 x x x
0 0 0 X x
0 0 0 0 x

and the offending nonzero element can be removed by a ZEV2 (2)

Proceeding in this way, we construct Z in the form

Z = ZIZ 2...Zn_1

where ZkE '2 (k)

Although QBZ is upper triangular, it is not at all clear that

QAZ is upper Hessenberg. To see that it is, rewrite equation (3.2)

in the form

(3.4) QAZ = SZ + kQBZ

From the particular form of Z and the fact that S is upper triangular,

it follows that SZ is upper Hessenberg. Thus (3.4) expresses QAZ as

the sum of an upper Hessenberg and an upper triangular matrix. In

fact (3.4) represents a computationally convenient form for computing QAZ

We summarize as follows.

1) Determine Q = Qn-i-n-2.. Q1 (Qk E 2(k)) so that

S = Q(A-kB) is upper triangular.

2) Determine Z = ZlZ 2 ...Zn.1 (ZkCV 2 (k)) so that BI = qBZ

is upper triangula -

3) A: = SZ+ kB'

If this algorithm is applied iteratively with shifts klk 2 , ...

there result sequences of matrices A1 ,A2 ... , B1 ,B 2 ,... , and

ClC 2 ,... satisfying

ll
 A QAVZ B QB Z
V~l l- V

T
CV+ 1 = QCVQ Cv = AVBV

provided B1 is nonsingular. The matrices Av are upper Hessenberg

and the B are upper triangular. The C are the upper Hessenberg

matrices that would result from applying the QR algorithm with shifts

klk 2 , . to C1 . As Cv tends to upper triangular form, so must AV

since B is upper triangular.

V
Most of the properties of the QR algorithm carry over to the QZ

algorithm. The eigenvalues will tend to appear in descending order as

one proceeds along the diagonal. The convergence of a ( v) to zero

n, n-i
may be accelerated by employing one of the conventional shifting strategies.
Once a(V) becomes negligible one can deflate the problem by working
n,n-i
with the leading principal submatrices of order n-i . If some other
(v) beoenelgbeoncn
subdiagonal element of AV , say a 1 ,_ I , becomes negligible, one can
effect a fTrther savings by working with rows and columns I through n

Because we have used unita-y transformations, an element of AV or BV

can be regarded as negligible if a perturbation of the same size as the

element can be tolerated in A1 or B1 *

The algorithm given above is potentially unstable. If k is large

compared with A and B , the formula (3.4) will involve subtractive

cancellation and A' will be computed inaccurately. Since the shift

approximates the eigenvalue current]y being found and the problem may

have very large eigenvalues, there is a real possibility of encountering

a large shift. Fortunately the large eigenvalues tend to be found last

so that by the time a large shift emerges the small eigenvalues will have

been computed stably. (The large eigenvalues are of course ill-conditioned

12
and cannot be computed accurately.) To be safe one might perform, the

first few iterations with a P ro shift in order to give the larger

eigenvalues a chance to percolate to the top.

13
 4. Implicit Shifts

The potential instability in the explicit algorithm results from

* the fact that we have used formula (3.4) rather than unitary equivalences

to compute A' . One way out of this difficulty is to generalize the

implicit shift method for the QR algorithm to the QZ algorithm so that

both A' and B' are computed by unitary equivalences. The implicit
shift technique has the additional advantage that it can be adapted to

perform two shifts at a time. For real matrices this means that a double

shift in which the shifts are conjugate pairs can be performed in real

arithmetic.

Since we are primarily interested in real matrices, we will concentrate

on double shifts. The method is based on the following observation.

Suppose that A is upper Hessenberg and B is upper triengular and

nonsingular. Then if Q and 7Z are unitary matrices such that QAZ

is upper Hessenberg and QBZ is upper triangular, then Q is determined

by its first row. In fact, AB-1 and QABQH are both upper Hessenberg,

so that, by the theorem on page 352 of [11], Q is determined by its

first row.

Thus we must do two things. First, find the first row of 1Q•

Second, determine Q and Z so that Q has the correct first row,

AZ is upper Hessenberg, and QBZ is upper triangular. The first part

is relatively easy. The first row of Q is the first row that would be

obtained from a double shifted QR applied to AB- 1 . Since A is

upper Hessenberg and B upper triangular, it is easy to calculate the

first two columns of AB . aut these, along with the shifts, completely

determine the first row of Q . Only nonsingularity of the upper 2-by-2

14i
submatrix of B is actually required here. If either bll or b22

is too small, so that this submatrix is nearly singular, a type of

deflation can be carried out. We will return to this point later.

The second part is a little more difficult, and is really the crux

of the algorithm since it retains the Hessenberg and triangular forms.

Only the first three elements of the first row of Q are nonzero. Thus,

if is a matrix in VI (1) with the same first row of Q , then A

and Q 1 B have the following form (when n 6 )

x x x x x x x x x x x x
2
x x x x X X x x x x x x
1 1
x x x x x x x x x x x x
O 0 x x x x C 0 0 x x x
0 0 0 x x x 0 0 0 0 x x
0 0 0 0 x x 0 0 0 0 0 x

As in the standard implicit shift QR algorithm, it is convenient to think

of Q1 as the reflection which annihilates two of the three nonzero

elements in a fictitious "zeroth" column of A .

We must reduce Q 1A to upper Hessenberg and Q 1B to upper triangular

by unitary equivalences. However, we may not premultiply by anything which

affects the first row. This is done as follows. The matrix Q1 B has

three nonzero elements outside the triangle. These can be annihilated

by two Z's , a Z, in V 3(1) which annihilates the (3,1) and (5,2)

elements and then a Z' which annihilates the resulting (2,1) element.

Let Z1 ZlZ" Then QIBZ is upper triangular. Applying Z to

01A C4Iraq I)A7 + n+ f11 e

-Pn g -Pn-
(?i,

15
 x x x x x

x1 x x x x x
x x x x x x
0 ) 0 x x x
0 0 0 0 x x

This is multiplied by Q2 in W,(2) that annihilates the (3,1) and

(4,1) elements. Then Q2 Q1 AZ1 and QQIBZ 1 have the forms

x x x x x x x x x X x x
x x x x x x 0 x x x x x
0 x x x x x 0 x2 x x x x
0 x x x x x 0 xI x1 x x x
0 0 0 x x x 0 0 0 0 ii x
0 0 0 0 x x 0 0 0 0 0 x

The first columns are now in the desired form. The nonzero elements

outside the desired structure have been "chased" into the lower 5-by-5

submatrices.

Now, postmultiply by Z2 , a product of a matrix in W.3(2) and a

matrix in W2(2) that reduces the current B to tringular form. Then

premultiply by Q5 in VD) to annihilate two elements outside the

Hessenberg structure of the resulting A

The process continues in a similar way, chasing the unwanted nonzero

elements towards the lower, right-hand corners. It ends with a slightly

simpler step which uses Qn-2 in 2 (n-l) to annihilate the (n,n-2)

element of the current A , thereby producing a Hessenberg matrix, and

Zn-2 in W 2 (n-1) which annihilates the (n,n-1) element of the

current B , producing a triangular B but not destroying the Itessenberg A

16
 The fictitious zeroth column of A is determined in part by the

shifts. In analogy with the implicit double shift algorithm, we take the

shifts kI and 'k

2 to be the two zeros of the two-by-two problem

det(A-kB) 0

where

n -l n - I n
nln
,n-lfn- n1,n
aa b
an,n-i an, n 0b n, n

It is not desirable to compute k1 and k2 explicitly, or even to find

the coefficients in the quadratic polynomial det(A-kB) . Instead,

following the techniques used in " hqr2 " [8 ], we obtain ratios of

1
the three nonzero elements of the first column of (AB-1 -k) (AB- -k 2 I)

directly from formulas which involve only the differences of diagonal

elements. This insures that small, but ncn-negligible, offdiagonal

elements are not lost in the shift calculation. The formulas are

(m n-1)
aa a a a a a b al •(bl)
mm nn 11 nn mm mmn

22 n
ll 22 1
n nn bb nn
M2
22 al 1 a a all a__) )

a2. a aa

i2i.
a -)- ~
a32

M_ U)+
141
+ a30l

1 7 ~
 We are now in a position to summarize the double implicit shift

method. It is understood that A and B are to be overwritten by

the transformed matrices as they are generated.

1) Compute alO, a2 0 , and a30 by (h.1).

2) For k = l,2,...,n-2

a) Determine Qk q3(k) to annihilate ak+lk-i and ak 4, k-I

b) Determine teW.(k) to annihi!.lte b and b

3 ~k+2,k+l k-,
c) Determine 2Zj"e- 2 (k) to annihilate bk+l,k

5) Determine QnEY(
2 (n-l) to annihilate a
n-l 2n, n-2
4) Determine Zn_! ?2 (n-I) to annihilate bn,n_ 1

For each k , determination of Qk requires a few multiplications

and one square root. Application of Qk to both A and B requires

about lO(n-k) multiplications. The work involved with each Z is

the same. Application of 7j" requires only about 6(r.-k) multiplications.

The number of additions is about the samie. Summing these for k from

1 to n-i gives a total of about l3n multiplications, ln 2 additions

and 3n square roots per double iteration.

By way of comparison, for the duble shift QR algorithm as implemented

in ,k in
" hqZ bcoessipl
b k and
an Z"
k is not used. Furthermore, the

transformations are carried out on only one matrix. Consequently, each

o2
double iteration requires about 5n- multiplications, 5n2 additions and

n square roots. Thus the QZ algorithm applied on two matrices can be

expected to require roughly 2.6 times as much work pec' iteration as

the QR elgori-hm on a single matrix.

a.18
 In order to obtain eigenvectors, the Q's are ignored and the Z's
2 2
accuinulated. This requires about 8n more multiplications and 8n

more additions per double iteration. I

There is one difficulty. The formulas for alO , a2 0 , and a0i

are not defined when b and b are zero. Moreover, as b and

1122 11
b22 approach zero the terms that determine the shift (terms involving

an , b , etc.) become negligible compared to the other terms, so that

nnl nn
the effect of the shift is felt only weakly.

Part of the solution to th.is difficulty is to deflate from the top.

If b is negligible it may be set to zero to give the forms for A

and B (n= 4)

x x x x 0 x x x
x x x x 0 x x x
0 x x x 0 0 x x
C 0 x x 0 0 0 x

A Q in 2(2(l) can then be used to annihilate the (2,1) e-leme.nt -f :

which deflates the problem.

The rest of the solution lies in recognizing that there J *'O Unr-

of a problem. If b and b22 are small then the problem has 3]So-e

eigenvalues. We have already observed that the larwer ei genl-.!es tcnd

to emerge at the upper left, and the larger the figenvalne, t,'e swi'ter

its emergence. Moreover the speed will not be affzc~ed by a 8mall

This means that whenever the implicit shift is dilltei by a small b

or b2 2 , the algorithm is none the less prof:itably e-.i,lovee in fjndi-4

a large eigenvalue.

19
 1/

5. Furt.,e. I eduction of the Quasi-Triangular Form

The result of the algorithm described so far is in an upper

triangular matrix B and a quasi-upper triangular matrix A in which

no tao consecutive subdiagonal elements are nonzero. This means that

the original problem decomposes into one by one and two by two subproblems.

The eigenvalues of the one by one problems are the ratios of the corres-

ponding diagonal elej..ents of A and B . The eigenvalues of the two by

two problems might be calculated as the roots of a quadratic equation,

and may be complex even for real A and B •

There are two good reasons for not using the quadratic directly,

but instead reducing the two by two problems. First, when A and I3

are real, the calculation of eigenvectors is greatly facilitated if all

the real eigenvalues are contained in one by one problems. A more

nport.-t second reason is that the one by one problems contain more

information then the eigenvalues alone. For example, if all and bl

are small then the eigenvalue X, = all/bl is ill conditioned, however

reasonable it may appear. This reason obviously applies to complex

eigenvalues as well as real ones. Accordingly, we recommend that the

two by two problems be reduced to one by one problems and that t-.e

diagonal elenents, rather than the eigenvalues, be reported.

Without loss of generality we may consider the problem of reducin-

two by two matrices A and B simultaneously to upper triangular form

by unitary equivalences. For our purposes we may assume that, B is

'Ipper triangular.

Two special cases may be disposed of immediately. If b is zero,

then a QcW1
2 (1) may be chosen to reduce a21 to zero. The ze!ro elements

20
 of QB are not disturbed. Similarly, if b22 is zero, a ZE 2 (1)

may be chosen to reduce a21 to zero without disturbing b21 *

In the general two-by-two case. it is not difficult to write down

formulas for the elements of A' = QAZ and B' = QBZ for any Q and Z

* Moreover, these formulas can be arranged so that numerically one of a21

or b'
23. is effectively zero. It is not obvious, however, that the other

element is numerically zero, and the effect of assuming that it is by

setting it to zero could be disastrous. Consequently, we must consider

a somewhat more complicated procedure.

The theoretical procedure for reducing A to triangular form may be

described as follows. Let X be an eigenvalue of the problem and form

the matrix E = A-XB . C'-iose a Ze; 2(1) to annihilate either ell

or e . Since the rmrs of E are parallel, it follows that whichever

of ell or e2 1 is annihilated the other must also be annihilated.

Now choose Qe 2(1) so that either QAZ or QBZ is upper triangiular.

Since the first column of QEZ is zero and QEZ = QAZ -XQBZ , it follows

that, however Q is chosen, both QAZ and QBZ must be upper triangular.

In the presence of ruinding error the method of computing X and

the choice of Z and Q are critical to the stability of the process.

A rigorous rounding error analysis will show that, under a reasonable

assumption concerning the ;omputed X , the process described below is

stable. However, to avoid excessive detail, we only outline the aalysis.

We assume that all computations are done in floating point arithmetic with

t base 3 digits and that the problem has been so scaled that underflows

and overflows do not occur. 'Wef-ther assume that a21 is not negligible

in the sense that ja2 1 j < 3-tilAil , where 11-1 denotes, say, the row

sum norm.

21

Al................................................................................
 The algorithm for computing X amounts to making an appropriate

origin shift and computing an eigenvalue from the characteristic equation.

It goes as follows.
all
=bl

12 = a - bl12

a22 a22 b22

22
p = ( b1

a21a12
q =b11b22

2
r = p +q

(5-1) X p + p + sign(p) .vr (complex if r < )

We must now assume that the computed X satisfies the equation

det(A' -XB') = 0

where IIA-A'II < aAMI and IIB-B'11 -<a B 1B 11 with cA and aB small

constants of order -t. Define

E' = A' -%B'

and let E denote the computed value

E = f(A-XB)

Then

E, = E+H

with <, o maxfjjA_, 1Xj 11-3I) with a of order

We claim that, approximately,

(5-2) W 1 3-t tax[!,Al!,

I IBIl .

22
First we note that

(5.3) IEII ? je21 = ja211 _ IJ-A ,

by the assumption that a21 is significant. Now assume that

IEll < P-t jB

I . Then subtractive cancellation must occur in the

computation of ell , e2 , and e2 2 . Thus a,_Xb, , al 2 Xb] 2

and a22 Xb Hence we have IAII > X IIBI! , and, from (5.5),
22 22
IEl > 0-t 1 IIBI! , a contract-ion.
Now

0= det(EI) = det(E)+ (el+hlh2 2 - (e1+h12)h 2 1 + hle2 2 - h1 2 e2 1 , j

Hence

jdet(E) I < PlI!EII max(lIA1I, I%I BIJ }+ p2[m -IfIAIl, JXI II!BII
-t
where p, and p2 axe.of order p From (5.2) it then follows that

ldet(E) I < plIEll n=IIAII, lXI IIBI'

-t :
where p is of order .

Now consider the determination of Z • Assume that the second row

of E is larger than the first. Then Zc2(2 (l) is chosen to annihilate e21

Let F = EZ . Then f is essentially zero. Furthermore, since Z is

f21
unitary

'fllf22I = ldet(E) 1 _<PH Eli maxtllAIA, IXIIBI]"

But If 2 2 1 - Iell11
2 and, since e2 was assumed to be the larger row,

lle 21l = Ell - Hence we have aproximately

fll < p mtillj, lxl jBllI •

To choose Q , let

C=AZ , D=BZ ,

23
 and let fl cl and dI be the first columns of F, C ,and D•
T1
Let q. denote the second row of Q . If JAil _ 1 I1JIBII , we choose Q

to annihilate d . Numerically this means that

21

A l dlj < ail~il ,

TT

where a is a constant on the order of -. We must show that q2 cl

is negligible. But

q Tc1 qT f + XqT l
2 ~2 c1

:S 1IfilI + N 11q, d111

:S p maxQI.A~j, IXIIIBIU + a1XIIUBIJ

< (p + a)ljAlI

If, on the other hand, IXI IIIJ > IIII, we choose Q so that

q2 Cl11 olIAII•

It then follows that

iq2 dll :I.f-q cTi/ H'

21 1 X 1
< 1i " max[(IAII, lVIBII) + aIXl-lhjjl

< (p + a)"l"

In summary, X is computed using (5.1), Z is chosen to annihilate

the first element of the larger of the two rows of A-%B and Q is chosen

to annihilate the (P2I) element of the smaller of the two matrices AZ

and %BZ . In this way, we can be sure that the computed (2,1) elements

of both QAZ and QBZ are negligible.

In practice with matrices of any ordei, if the ,;ransformations are

real, they are applied to the entire matrices. If the transformations are

24
......... -___-_- -I
rz MM1PT -- p F7

complex, they are used to compute the diagonal elements that would result,

but are not actually applied. We thus otain a quabi-triangular problem

in which each two-by-two block is known to correspond to a pair of complex

eigenvalues.

The generalized eigenvectors of this reduced problem can be found by

a back-substit:tion process which is a straightforward extension of the

methock used in " hqr2 " 8 ]. The vectors of the original problem are

then found by applying the accumulated Z's

25
6. Some Numerical Results

The entire process described above has been implemented in a Fortran

program [7 . There are four main subroutines: the initial reduction to

Ressenberg-triangular form, the iteration itself, the computation of the

final diagonal elements, and the computation of the eigenvectors. The

complete program contains about 600 Fortran statements, although this

c-auld be reduced somewhat at the expense of some clarity.

The numerical properties observed experimentally are consistent with

the use of unitary transformations. The eigenvalues are always found to

whatever accuracy is justified by their condition. If an eigenvalue and

eigenvector are not too "ill-disposed", then they produce a small relative

residual.

Similar numerical properties can not generally be expected from any

algorithm which inverts B or any submatrix of B • This is even true of

2-by-2 submatrices, as illustrated by the following example due to

Wilkinson.

A .1 .2 B .1i 1
A= . 0 P

!:ere p is about the square root of the machine precision, that is, L is
2
not negligible compared to 1 , but 12 is. There is one eigenvalue

near -2 . Small relative changes in the elements of the matrices cause

only small relative changes in this eigenvalue. The other eiGenvalue

becomes infinite as p approaches zero. Great care must be taken in

solving this problem so that the mild instability of the one eigenvalue

does not cause an inaccurate result for the other, stable eigenvalue.

26
 Of course, the use of unitary transformations makes our technique

somewhat slower than others which might be considered. But the added

cost is not very great. In testing our program, we solve problems of

order 50 regularly. A .ew problems of orders greater than l0- have

been run, but these become somewhat expensive when they are merely tests.

One typical exacuple of order 50 requires 45 seconds on Stanford's

IBM 360 model 67. Of this, 13 seconds are spent in the initial reduction

29 seconds are used for the 61 double iterations required, and 3 seconds

are needed for the diagonal elements and eigenvectors. If the eigenvectors

are not needed and so the transformations not saved, the total time is

reduced to 27 seconds. By way of comparison, formation of B-1A

a la Peters and Wilkinson [9] and use of Fortran versions (12] of "lortl.es"l
"
[5] and " hqr2 [8] requires a total of 27 seconds for this example.

(All of these times are for code generated by the IBM Fortran IV comnniler,

H level, with the optimization parameter set to 2 .)

In the examples we have seen so far, the total number of double

iterations required is usually &bout 1.2 or 1.3 times the order of

the matrices. This figure is fairly constant, although it is not difficult

to find examples which require many fewer or many more iterations. As a

rule of thumb, for a matrix of order n the time required on the model -7

is about .36 n milliseconds if vectors are computed, .22 n' 'illi-

seconds if they are not.

27
 The example in Table 1 is not typical, but it does illustrate

several interesting points. It was generated by applying non-orthogonal

rank one modifications of the identity to direct sums of companion matrices.

The companion matrices were chosen so that the resulting problem has

three double roots,

3 5 2

4 6T

The double root at c results from the fact that B has a double zero
eigenvalue. All three roots are associated with quadratic elementar-j

divisors; i.e., each root has only one corresponding eigenvector. The
computed diagonals of the triangularized matrices are given in the table.

Note that the four finite eigenvalues are obtained with a relative accuracy

of about 10-8 . This is about the square root of the machine precision

and is the expected behavior for eigenvalues with quadratic elementary

divisors. The singularity of B does not cause any further deterioration

in their accuracy. Furthermore, the infinite eigenvalues are obtained from

the reciprocals of quantities which are roughly the square root of the

machine precision times the norm of B . Consequently we are somewhat

justified if we claim to have computed the square root of infinity.*/

This prompts us to recall the limericK which introduces George Gamow's

One, Two, Three, Infinity:
There was a young fellow from Trinity
Who tried !
But the number of digits
Gave him such fidgits
That he gave up Math for Divinity.

28
 50 -60 50 -27 6 6 16 5 5 5 -6 5

38 -28 27 -17 5 5 5 16 5 5 -6 5

27 -17 27 -17 5 5 5 5 16 5 -6 5
A= B-
27 -28 38 -17 5 5 5 5 5 16 -6 5
27 -28 27 -17 16 5 5 5 5 5 -6 16
27 -28 27 -17 5 16 6 6 6 6 -5 6

a
25- (68670843143 .2637605112.10-6
-12.821841071323 .1312405807.106
5.814535434181 + 10.071071345641 i 11.629071028730
5.800765071150 - 10.047220375909 i 1. 601530502268
5.736511506410 + 9.935928843473 i 11.473022854605
5.510879468089 - 9.545122710676 i 11.021758784186

0.976972281.108
-0.976972290.108
0.49999999310489 + 0.86602543924271 i
0.49999999310489 - 0.66602543924271 i
0.5000000689511 + 0.86602536832617 i
O. 50000000689511 - 0.86602536832617 i

Table 1

29
-a *

Acknowledgments

Most of Moler's work was done during a visit to Stanford University

where he received support from the Ccmputer Science Department, the

Stanford Linear Accelerator Center and National Science Foundation

grant GJ-1158. Partial support was also obtained at the Universit. o

of Michigan from the Office of Naval Research, contract NR-O44-377.

Stewart received support at the University of Texas from NSF grant GP-2:':-5.

W. Kahan and J. H. Wilkinson made several helpful comments.

Linda Kaufman of Stanford -has recently shown how to carry out the

corresponding generalization of the LR algorithm and has written a

program that accepts general complex matrices.

30
 References

[1S Charles Crawford, "The numerical solution of the generalized

eigenvalue problem," Ph.D. Thesis, University of Michigan, 1970.
Also to appear in Comm. ACM.

[2] G. Fix and R. l1eiberger, "Ar algorithm for the ill-conditined

generalized eigenvalue problem," to appear in Numer. Math.

[5] J. G. F. Francis, "The QR transformation -- Faunitary analogue 6c,

the LR transformation," Computer Journal 4, 265-271, 352->45 (19(!-'.).

[41 A. S. Householder, "Unitary triangularization of a nonsymmetric matri:,"

J. Assoc. Comput. Mach. 5, 339-342 (1958).

(51 R. S. Martin and J. H. Wilkinson, "Similarity reduction of a general

matrix to Hessenberg form," Numer. Math. 12, 349-368 (1968).

[01 R. S. Martin and J. H. Wilkinson, "Reduction of the symmetric eigen-

problem Ax = %Bx and re'ated problems to standard form," Nuer.
Math. 11, 99-110 (1968).

[7] C. B. Moler and G. W. Stewart, "The QZ algorithm for Ax = %B<

see Part II of this report.

[8] G. Peters and J. H. Wilkinson, "Eigenvectors of real and complex

m.trices by LR and Q triangularizations," Numer. Math. 16, !81-C~h,
(2-9o).
[9] G. Peters and J. H. Wilkinson, " Ax = XBx and the generalized
eigenproblem," SIA4 J. Numer. Anal. 7, 479-h92 (1970).

[10] G. W. Stewart, "On the sensitivity of the eigenvalue problem

Ax =mXBx ," submitted for publication.

[11] J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University

Press, 1965.
[12] Fortran translations of the Algol procedures in [5] and [8] may be
obtained from the NATS Project, Applied Mathematics Division,
Argonne National Laboratories, Argonne, Illinois.

51
 Part II

FORTRAIr PROGRAM

32
 SUBROUTINE OZ(NOtNABtEPSALFRALFIBETAITERgWANTXX)
DIMENSION A(NDND)tB(NDND),ALFR(N),ALFI(N),BETA(N,XND.ND)
UIMENSION ITER(N)
LOGICAL WANTX
Sc
C A AND 8 ARE N-BY-N REAL MATRICES, STORED IN ARRAYS WITH ND ROWS.
SC EPS IS THE RELATIVE PRECISION OF ELEMENTS OF A AND B.
C FINDS N PAIRS OF SCALARS, (ALFA(M),BETA(M)) SO THAT
C BETA(M)*A - ALFA(M)*B IS SINGULAR.
C THE EIGENVALUES OF A*X - LAMBDA*B*X CAN BE OBTAINED BY
C DIVIOING ALFA(M) BY BETA(M), EXCEPT BETA(M) MIGHT RE ZERO.
C IF (wANTX) ALSO FINDS CORRESPONDING EIGENVECTORS.
C USES ONLY UNITARY TRANSFORMATIONS, NO INVERSES,
C SO EITHER A OR B (OR BOTH) MAY BE SINGULAR.
C
C BETA(M) IS REAL.
C ALFA(M) IS COMPLEX, REAL AND IMAGINARY PARTS IN ALFR(M) AND ALFI(M).

C COMPLEX PAIRS OCCUR WITH ALFA(M)/BETA(M) AND ALFA(M*I)/BETA(M.I)

C COMPLEX CONJUGATES EVEN THOUGH ALFA(M) AND ALFA(M.I) ARE NOT
C NECESSARILY CONJUGATE.
C USES ONLY REAL ARITHMETIC.
C IF A AND B WERE REDUCED TO TRIANGULAR FORM BY UNITARY EQUIVALENCES,
C ALFA AND BETA WOULD BE THE DIAGONALS.
C A AND B ARE ACTUALLY REDUCED ONLY TO QUASI-TRIANGULAR FORM WITH
C I-BY-i AND 2-BY-2 BLOCKS ON DIAGONAL OF A.
C IF ALFA(M) IS NOT REAL, THEN BETA(M) IS NOT ZERO.
C ITER IS TROUBLE INDICATOR AND ITERATION COUNTER.
C IF (ITER(I).EQ.O) EVERYTHING IS OK.
* C ITER(M) IS NUMBER OF ITERATIONS NEEDED FOR M-TH EIGENVALUE.
LC IF (ITER(1) THRU ITER(M) .EO. -1) THEN ITERATION FOR M-Tt
C EIGENVALUE DID NOT CONVERGE AND ALFA(W) THRU ALFA(M) AND
!
C BETA(l) THRU BETA(M) ARE PROBABLY INACCURATE.
C IF (WANTX) X(.,tM) IS THE M-TH REAL EIGENVECTOR.
C X(.,M) AND X(oM*I) ARE THE REAL AND IMAGINARY PARTS
C OF THE M-TH COMPLEX EIGENVECTOk.
C x(.,M) AND -X(.,M+l) AND THE REAL AND IMAGINARY PARTS
C OF THE (M*I)-ST COMPLEX EIGENVECTOR.
C VECTORS NORMALIZED SO THAT LARGEST COMPONENT IS 1. OR I..O.I
C
C USES FOUR PRIMARY SUBROUTINES, QZHES, QZIT, QZVAL AND QZVEC,
C USES FOUR AUXILLIARY SUBROUTINES, HSH3, .iSH2, CHSH2 AND CDIV.
C USES TwO STANDARD FUNCTIONS, SORT AND ABS.
C AUTHGRS: C. B. MOLER, STANFORD, AND G. W. STEWART, U. OF TFXAS
C THIS VERSION DATED 7/19/71.
C
CALL (ZHES(ND,N,A,B,WANTXX)
CALL ZIT (ND,NA,B,EPS,EPSAEPSBITERWANTX.X)
CALL QZVAL(NDN,AgEEPSAeEPSBALFRAL.FIBEIAWANTXX)
IF (WANTX) CALL QZVEC(ND,N,A,dEPSAEPSBALFRvALFI,RETAX)
RtTURN
END
C
C

33

-J
 SUBROUTINE QZHES(NDNA,8,WANTXX)
DIMENSION A(NDND),B(NDND),XtNOND)
LOGICAL WANTX
C
C INITIALIZE Xg USED TO SAVE TRANSFORMATIONS
C
IF (oNOT.WANTX) GO TO 10
00 3 1=19N
DO 2 J=lN
X(IIJ) =0.
*2 CONTINUE
x(I)=1
3 CONTINUE

C REDUCE 8 T0 UPPER TRIAN~GULAR

C
10 NM1:N-1
00 100 L=19NMI

S =0.
DO 20 I=LltN
IF (ASc8(IL))*GTS) S ABS(B(1,L)
20

iIF
CONTINUE
IF (S*EQo0.) GO TO 100
(ABS(B(LtL))*GT*S) S
=0.
DO 25 I=LvN
=ABS(B(LgL))

8(1,L) 8 (IL)/S
R =R 8 (1,L)**2
25 CONTINUE
R = SQRT(R)
IF (P(LqL)*LTo0*p R =-R
8(L*L) = B(LqL) * R
RHO = R*RfLL)
DO 50 J=LIN
T =0.
00 30 I=LN
T = T * 19*81J
30 CONTINUE
T =-T/RHO
DO 40 I=LN
8(19j) = B(I*J) *T0(1L)
40 CONTINUE
50 CONTINUE
DO 80 J=19N
T = 0.

I.60 *
00 60 I=LoN
T = T * 8(1,L)*A(1,J)
CONTINUE
T =-T/RHO
00 70 I=L*N
A(I*J) = A(IJ) + T*8(1,L)
*70 CONTINUE
80 CON~TINUE
H~(L#L) = -~
(C)90) =LlN
8(I9L) = 0.
90 CONTINUE

34
 C100 CONTINUE
C REDUCE A TO UPPER HESSENBERG, KEEP B TRIANGULAR
IF (N*LE.2) GO TO 170
NM2=N-2
DO 160 K=iNM2
Ki = t(I
NKI N-K-i
0O 150 LB=iNKi

CALL HSI2(A(LK),A(LiK),UitU2,ViV2)
IF (UidNF~lo) 6O TO 125
DO 110 J=KoN
T = A(L9J) * U2*A(LI#J)
A(LoJ) A(L9J) * T*Vi
A(L1,J) =A(LiJ) * T*V2
110 CONTINUE
A(LiK) = 0.
DO 120 J=L*N
T = B(LqJ) * U2*B(LlJ)
B(LtJ) =B(L*J) * T*Vi
B(LI9J) =8(L1,J) + T*V2
120 CONTINUE
* 125 CALL HSt2(B(LlLl),B(LiL),UlU2,ViV2)
IF (UI.NE*i.) GO TO 150
DO 130 I=I*Ll
T = 8(IqL1) * U2*B(1,L)
8(I,11) B (1911) * T*Vi
8(19L) 0 (19L) * T*V2
130 CONTINUE
t3(LI*L) = 0.
DO 140 1=19N
T = A(I9LI) * U2*A(IL)
A(I9LI) =A(IqLI) * T*Vl
A(IL) =A(IL) * T*V2
140 CONTINUE
IF (*NOT.WANTX) GO TO 150
DO 145 1=19N
T = X(IqLi) * U2*X(IL)
X(I*Ll) =X(I#Ll) +T*Vl
X( 19 L) X(IL) *T*V2
145 CONTINUE
150 CONTINUE
160 CONTINUE
170 CONWINUE I

RETURN
END
C
C

35
 SUBROUTINE WZIT (NDNA,8,EPSEPSAEPSB, ITERWANTX.X)
DIMENSION A(NOND),8(ND9ND),X(NDND)

[ C
DIMENSION ITER(N)
LOGICAL WANTXqMID
C INITIAL1717 ITER9 COMPUTE EPSA9EPSB
C
ANORM =0,
8NORM = 0.
00O 185 1=19N
ITEkHj) =0
ANI = 0.
IF (I.NE.1) ANI =ABS(A(J.I-1)
HN1 = 0.
00 180 J=I*N
ANI = ANI + ASS(A(I,J))
ONI = BNI + ABS(BU,9J))
180 CONTINUE
IF (ANI.GT*ANORM) ANORM = AN!
IF (8N1.GTo8NORM) BNOiRM = ONI
185 CONTINUE
EPSA =EPS*ANORM
EPSH = EPS*BNORM
C
C REDUCE A TO QUASI-TRIANGULAR, KEEP B TRIANGULAR
C
M =N
200 IF (M.LE*2) GO TO 390
C
C Ch-ECK FOR CONVERGENCE OR REDUC161LITY

DO 220 LH=19M
L = M+1-LB
IF (LoEQ.1) GO TO 260
IF (A8S(A(LL-1)) *LE* EPSA) GO TO 230
220 CONTINUE
230 A(LqL-1) = 0.
IF (L.LT.M-1) GO TO 260
M L-I
GO TO 200
C
C CI-ECK FOR SMALL TOP OF 8
C
260 IF (ABS(B(LL))*GT*EPSO) GO TO 300
8(LgL) =0.
Li = .1
CALL HSI2(A(LL),A(Ll.L),UIU2,V1,V2)
IF (U1.NE.1.) 60 TO 280
00 270 J=L9N
7 = A(LJ) * U2*A(LlqJ)
A(LvJ) =A(L9J) + T*Vl
A(LltJ) =A(Ll9J) # T*V2
T= B(LqJ) + U2*8(LlgJ)
8(Lqj) =B(Ltj) + T*Vl
H(LI*J) B (L19J) + TOV2
270 CONTINUE
280 L =Li
6O T0 230

36
* C
C BEGIN ONE QZ STEP, ITERATION STRATEGY
c
300 Ml = M I

IITER(M)
LiI

IF
IF
1
CONST =0.75
= ITER(M) I
(ITER(M).EG*1) GO TO 305
(ABS(A(t4M-1))LTeCONST*OLDI) GO TO 305
IF (ABS(A(M-lM-2))*LT*CONST*0LU2) GO TO 305
IF (ITER(M).EQ.lO) GO TO 310
IF (ITER(M).GT*30) GO TO 380

C ZEROTH COLUMN OF A

4 305 8ill = ,"'L)

822 =8(L1,11)
IF (A8S(B22)*LT.EPS8) 822 = EPS8
833 = B(MjMI)
IF (ABS(833)*LTeEPSB) B33 =EPSO
844 = B(M*M)
IF (AB5(B44).LT.EPSB) 844 = EPS8
All =A(LL)/811
A12 = A(LtLl)/822
A21 = A(LltL)/Bl

A22 = A(L.2Ll)/B22
A33 = A34A43lgA384*A/8)A3

IF3 ( qwdNT) MOR

=
AF4W4X)Or N)/4

M812
=BK.NE.Ml2
834 = BKI9)l 4

I 2 A1 -K.261
A30 = K.32L)/2
GO TO 37
 IF (K3.GT.fI) K3 M
KMI K-1
IF (KM1.L7.L) KM1 =I
IF (K.EU.L) CALL HSH3(AlOA20,A3catUlU2,U3,yIV2,y
IF (K.GT.L.AND.K.LT-M1) 3)
CALL H'SH3(A(K9MI)AKlKMI),A(K2,KMl),UltU2,U
IF (KEQ.M 3 9 Vl V2 V3)
1 ) CALL HSH2(AiK9KMI),A(KlK14l),IIU
2 9vvl# 2 )
IF (Ul.NE.I.) GO TO 325
* DO 320 J=KM1,MORN
T = A(KJ) + U2*A(KIJ)
IF (MID) T =T *U3*A(K2,J)
A(KvJ) A(K*J) *T*VI
A(KIJ) =A(KlJ) *T*V2
IF (MID) A(K29J) A(K29J) *T*V3
T = B(Ktj) + U2*8(Kl,j)
IF (MID) T = T U3*8(K29J)
B(KgJ) B (KtJ) *T*VI
B(KlJ) =B(KlJ) + T*V2
IF (MID) 8(K2,J) = (K2,PJ) +' T*V3
320 CONTINUE
IF (K.EQ*L) GO TO 325
A(KIK-1) =0.
IF (MID) A(K2,K-1) = 0.
325 IF (K.FQ.M1) GO TO 340
CALL HSH3(8(K2,K2),B(K2,KI) ,9K2,K),ul9U29U
3 9 VlVtV3)
IF (U1.NE.I.) GO TO 340
00 330 I=LORI9K3
T = A(IK2) + U2*A(IKl) +U3*A(1,K)
A(IK2) = A(IK2) + *V
A(IKI) =A(I9Kl) + T*V,2
A(1,K) = A(IK) * T*V3
f = B(IK2) * U2*8(iKl) *U3*B(I1t9)
8(lK2) = (lK2) * T*VI
B(I*Kl) = t8(IKl) + T*V2
8(19K~) = B(IK) + J*V3
330 CONTINUE
H(K2*KJ) =0.
IF (.NOT.WANTA) GO TO 340
0)0 335 1=1,tJ
T = X(IK2) + U2*X(IKI) + U3*X(I*K)
x(IleK2) = X(IK2) + T*VI
X(IK1) =X(IKI) * T*V2
X(IK) = XdIK) * T*V3
335 CONTINUE
340 CALL HSH2(B(KIKl),B(KlK),ultu2,vl9V2)
IF (UlhNE.1.) GO TO .360
0)0 350 I=LOHI.K3
T = A(1,Kl) + L2*A(1,K)
A(I*Kl) A(I,Kl) * T'*Vl
A(1,K) =A(IK) * T*V2
T = B(I,Kl) + U2*b(I,K)
B3(19KJ) B(I,~l) * T*V1
83(19K) 8(1#r:) * T*V?
350 CONTINUE
R(KlK) =0.
IF (.NOT.WANTX) GO TO 36(;
DO 355 IzIoiN
 T = (IKI) .U2*X(JK)
X(I*Kfl X(I*Kl) . T*Vl
X(IqK) =X(~,K) . *V?
355 CONTINUE
360 CCNTINUE
C
C END MAIN LOOP
c
GO TO 200

C END ONE QZ STEP

C
380 DC 385 1=19M
ITER(I) -
385 CONlINUE
390 CONTINUF
RETURN
END
C
C

39
 SUBROUTINE QZVAL(NDNABEPSAEPS8,ALFRALFIBETAWANTXX)
DIMENSION A(NDND),B(NDND),ALFR(N),ALFI(N),BETA(N),X(NCND)
LOGICAL wANTXqFLIP

C FIND EIGEN4VALUES OF QUASI-TRIANGULAR MATRICES

C
C D0 400 THRU 490 FOR M = N STEP (-I OR -2) UNTIL 1
C
M = N
400 CONTINUE
IF (M.EO.1) GO TO 410
IF (A(MvM-I).N~e.o) GO TO 420
C
C ONE-BY-ONE SU8MATR IX9 ONE REAt. ROOT
c
410 ALFH(M) = A(MvM)
tJEIA(M) = F3(H.M)
4LFI(M? = 0.

GO TO 490

C TWO-BY-Two SUBMATRIX
C
420 L =M-1
IF (ABS(B(L9L))oGT.EPSB) GO TO 425
3(LtL) = 0o
CALL HSH2(A(LL )9A(ML)9U1,U2,VlV2)
GO TO 460
425 IF (A8S(8(MvM)).GToEPSB) GO TO 430
t3(mom) = 0.
CALL HSh2(A(MM),A(ML),UIU2,VlV2)
HN= 0.
GO TO 435
430 AN = ABS(A(LL) ).ABS(A(LM) ).ABS(A(ML) ).ABS(A(MM))
BN =ABS(B(LL)),ABS(B(LM))+ABS(B(MM))
Al) =A(L*L)/AN
A12 = A(LqH)/AN
A21 =A(M*L)/AN
A22 = A(M*M)/AN
811 = 8(L*L)/BN
d12 = B(Lg,!)/BN
b22 = 8(MgM)/BN
C = (AI1'B22 *A22*Bll - A21*812)/2,
D9 (A22*Bll Al1*B22 - A2!*B12)**2/4o
1 * A21*822*(A12*Bll - AI1R812)
IF 0.oLT.0.) GO TO 480
C
C TWO REAL ROOTS
C ZERO BOTH A(MoL) AND B(M9L)

IF (C.GE.0.) E = (C + SORT(D))/(811*b22)
IF (C*LT*0.) E = (C - SORT(D))/8~11*822)
All =All - E*Rll
A12 = AlIZ - E*H12
A22 =A22 - E*B22
FLIP = !AHS(All)*AHS(A12))*(iE*(AbS(A21)A3S(A22))
17 (FLIP) CALL H-SH2(A2AllUIU2#VlV2)
IF (.NOT.FLIP) CALL HSH2(A22,A21,U1,U2,VlV2)

440
mA435 IF (U1.NE*.1. GO TO 450-
DO 440 I=l,M
T = ..(19M) t.U2*A(19L)
A(IA) = A(IM) * V1*T
A(19L) =A(I9L) +V2*T
T = (19 M) * U2*6(IL)
B(1,M) = 83(19M) * Vl'*T
13(1,L) = 8(19L) + V2*r

* --
440 CONTINUE
IF (oNOToWANTX) GO TO 450
00 445 1=19N
T =X(IM) * U2*X(I1,L)
X(IM) = X(h-M) *V1*T
X(IL) = X(IL) + V2*I
445 CONTINUE
450 IF (HN.EU.0.) GO TO 475
FLIP = AN *GE. ABS(E)*BN
IF (FLIP) CALL HSH2(B(LqL),13(ML),UI9U2,VlV2)
IF (.NOT*FLIP) CALL HSH2(A(LvL)9A(M9L)9U1,U2,VlvV2)
460 IF (Ul.NE.919) GO TO 475
DO 470 J=LqN
T = A(L*J) + U2*A(MJ)
A(LJ) = A(LqJ * V*
A(MJ) =A(MJ) * V2*T
T = B(L*J) + U2013(MJ)
t3(LqJ) = I3(L9J) + V1'*T
B(MJ) =B(MJ) + V2*T
470) CONTINUE
475 A(MvL) = 0.
"(t)= 0.
ALFR(L) = A(LvL)
ALFR(M) =A(MM)
BETA(L = B(L*L)
BETA(M) = 8 (MeM)
ALA 1(M) =0.
ALFI(L) =0.
m = M-2

C TwO COMPLEX HOOTS

490 Ew C/ Ctl IIr22)

LI =SORT(-D)/(8ll*s?2)
Ailk< = All - ER*Bll
A111 = EI*Fll
A12R = A12 - ER*81?
A121 = EI*812
A21H = A21
A211 0.
A22H A22 - LR*8322
h22 Ei*,22
IC'FLIP .GL.
=(ABS(Al1R).AB3S(AIII).AHS(Al2R).ABS(AI21))
1 (ABS (A21R) .A8S(A2?H)+AI3S(A221))
IF (FLIP) CALL CHSH2(Al2RAl2l,-A11R,-AlllCZSZPSZI)
IF (.NOT.FLIP) CALL CHSH?(A22HA221,-A2lR,-A2lICZSZRSZI,
FLIP = AN .G.(A8S(EN)+A8S(EI))*BN
IF IFLIP) CALL CHSH2(CZ.*HIl*SZR*8129 SZI*612,
I SZR*6229 SZI*R22, C0, SOR, SQl)

41
 IF (.NOT.FLIP) CALL CHSH2(CZ*AIISZR*A12, SZI*A129

I
I CZ*A21+SZR*A22, SZI*A22, CO, SOR9 S01)
SSR =SQR*SZR *SQI*SZI
SS1 = SQR*SZI -SQI*SZR
CQ*CZ*All + CO*SZR*AI2 + SQR*CZ*A21
= *SSR*A22
TI = CQ*SZI*A12 - SQI*CZ*A21 + SSI*A22
BOR = CQ*CZ*Bl1 + CQ*SZR*B12 * SSR*822
BDI = CQ*SZI*B12 + SSI*822
R = SURT(BOR*BDR + BDI*BDI)
BETAL) = BN*R
ALFR(L) = AN*(TR*BOR + TI*8D1)/R
ALEj'.- = AN*(TR*BDI - TI*BDR)/R
TR = SSR*All - SQR*CZ*A12 - CQ*SZR*A21 + CQ*CZ*A22
TI = - SSI*All - SGl*C-Z*A12 + CQ*SZI*A21
8DR =. SSR*81l - SQR*CZ*812 * CQ*CZ*B22
t3DI = - SSI*BIl -SQI*CZ*812
R = SQRT(BDR*B0R *BDI*BDI)
BETA(M) = BN*H
ALFR(M) = AN*(TR*BDR + TI*BDI)/R
ALFJ(M) = Ai-,(TR*BDI -TI*BDR)/R
M =M-

490 IF ('4.GT.0) GO TO 400

RETURN
E ,.0

* 42
 SUBROUTINE QZVEC (NDNABEPSAEPSBALFRALFI ,BETAX)
DIMENSION A(NOND),B(NDND),ALFR(N),ALFI(N),BETA(N),X(NOND)
LOGICAL WANTXFLIP
C
C FINU EIGENVECTORS OF QUASI-TRIANGULAR MATRICES
C USE 8 FOR INTERM'EDIATE STORAGE

t
C,
c DC 500 THRU 590 fOR M = N STEP (-I OR -2) UNTIL 1
C to =N
500 CONTINUE
IF (ALFI(M).NE0 o.) GO TO 550

CC REAL VECTOR
ALFM = ALFR(M)
BETM = t3ETA(M)
IF (ABS(ALFM)*LT*EPSA) ALFM = 0.
4 IF (ABS(8ETM).LTLEPSB) BETM =0.

c
~1C 00 510 THRU 540 FOR L M-1 STEP (-I OR -2) UNTIL I
IF (L.EQ.0) GO TO 540
510 CONTINUE

SL = 0.
00 515 J=LI.M
SL = SL + (BETM*A(LJ)-ALFM*B(LJ))*B(JoM)
515 CONTINUE
IF (L*EQ*1) GO TO 520
IF (A(LvL-1)*NE.0.) GO TO 530
520 D, BETM*A(LL)-ALFM*B(LgL)
IF (D.EQ,0,) D =(EPSA.EPSB)/2,
b(LM) = -SL/D
L =L-I
60 TO 540

530 K =L-1
=> 0 .
0)0 535 J=LloM
SK=SK + (BETM*A(KJ)-AL-FM*B(KJ))*B(JM)
5315 CCNTINUL
TKK = BETM4'A(KtK) - ALFM*B(KK)
TrL_ = BETM*A(KgL) - ALFM4*8(KrL)
TLK = BETM*A(L,K)
TLL = BETM*A(LgL) - ALFM*BCLtL)
D TKK*TLL - TKL*TL(
IF UX.EC.C.) D0 (EPSA+EPSB)/?*
H(LtM) =(TLK*SK - TKK*SL)/D
FLIP = ABS(TKK) .GE. ABS(TLK)
IF (FLIP) B(KtM) = -(SK + TKL*B(LM))/TKK
IF (.NOT.FLIP) H(KoM) = -(SL + TLL*B(LoM))/TLK

540 IF (L.GT.O) GO TO 510

M M-1
GO T0 590

43
 C
C COMPLEX VECTOR
C
5'90 ALMR = ALFR(M-1)
ALMI = ALFI CM-i)
BErM = BLTA(M-I)
MR = M-1
MI =M
C
C NORMALIZE SO THAT M-TH COMPONENT =,-I~
C (M-I)ST = (8ETM*A(MM)-ALFM*B(M,M))*(M-Yh)/(BETM*A(M9M-1))

B(M-1.MR) = ALMI*B(MgN)/ibEfM*A(MM-1))
B(M-1,MI) = (BETM*A(MM)-ALMR*B(MM))/(BETM*A(MM-1))
i4(MMR) = 0.
H(mMI) = -I.
C
C DO 560 THRU 585 FOR L =M-2 STEP (-I OR -2) UNTIL 1
C
L =M-2
IF (L.EO.O) GO TO 585
960 CONTINUE

iiSLI
- ITR
SLR =0.

DO 565 J=LlM
0.
= BETM*A(LJ) -ALMR*BCLJ)

T; = -ALMIt3(LgJ)
SLR = SLR + TR*8(JgMH) -Tl*B(JMl)

511 = SLI + Tk*f3(JqMl) *TI*B(JMR)

565 CONTINUE
IF (L.EO*l) GO TO 570
IF (A(LL-1).NE.0.) GO TO 575
5?0 DR = BET'4IA(LtL) - ALMR*83(1,L)
01 = -ALMI*B(LgL)
CALL CDIV(-SLR9 -SLI, DR9 Dl, 13(LqMR), B(LMI))
L =L-1
60oTO ce5
c, 75K =L-1
SKk = 0.
SKI = O
DO 580 J=LiM
TH = BETM*A(t(,J) -ALMR*B(KJ)

TI = -ALMI*B(K*J)
SKR = SKR * TR*B(JMR) -TI*B(JMl)

SKI = SKI + TP*B(JoKI) *TI*B(JMR)

C80 CONTINUE
TKKR = 8ETM*A(KK) - ALMR*B(KK)
TKKI = ALMI*8(K*K)
TKLR = dETM*A(K*L) - ALMR*B(KtL)

I.TLKR rKLI = ALMI*B(KtL)

= BtETM*A(L*K)
TLK I = 0.
TLLR = BETM*A(LL) - ALMN*B(LtL)
TLLI = -ALMI*B(L#L)
OR =TKKR*TLLR -TKKI*TLLI -TKLR*TLKW

1 TKKR*TLLI *TKKI*TLLR -TKLI*TLKR

 IF(DRoEQ.0. oANDo DI.EQ.0&) DR = (EPSA+EPSB)/2.
CALL CDIV(TLKR*SKR-TKKR*SLRTKKI*SLI,

N2 I ~TLKR*SK I TKKR*SL I-TKK I*SLR9

DR, DIv H(LMR)g R(LMI))
FLIP = (ABS(TKKR)+A8S(TKKI)) oGE. ABS(TLKR)
IF (FLIP) CALL CDIV(-SKN.-TKLR*B(L,MR),TKLI*8(LMl),
I -SKI-TKLR*HuLvMI)-TKLI*H(LMR).
2 TKKR, TKKI, F3(IKMR), 8(KtMI))
IF (*NOT* FLIP) CALL CDIV(-SLR-TLLR*B(LMR),TLLI*8(LtMI),
I -1SLI-TL.LR*8(LMI)-TLLI*B(LMH?),
2 TLKR, TLK1, H(KMR), f3(KMl))
L=
585 IF (L.GTO0) GO TO 560
m M-2

C TRNSFOM TOORIGINAL COORDINATE SYSTEM

600 CNTINUE
M =M01

iF (M.t.O)
GO TO60

C NORMALIZE SO THAT LARGEST COMPONENT =1.

C
M =N
6,30 CONTINUE
S = 0.
IF (ALFI(M.NE.0.) GO TO 650
00 635 I=I.N
R ABS(X(IM))
IF (R.LT*S) GO TO 635
S P
D) X(19M)
635 C:ONTINUE
00 640 1=19.4
x(IgM) =X(I*M)/D
640 CONTINUE
M = -I
* 60 TO 690
C
*650 00 655 11.oN
R X(1,M-I)**2 + X(I,M) *2
IF (R.LT.S) GO TO S
S R
DR x(Igm-1)

655 CON4TINUE
00 660 I=1,N
CALL CDIV(X(I9M-1),X(IM),DRDlgX~iM-1 ),X(IM) )
660 CONTINUE

45
 = M-2
f90 IF (M.GT.0) GO TO 630
C
700 RETURN
END

46

..........
''' .. ...
* ...-
- -*"
-," -
" ' ' : =-
-- - -
" '
"'""--4' -*-- =
*
 SUBROUTINE HStH3(AlA2,A3,UI ,U29U3,VlV?,V3)
C
* C FINDS HOUSEHOLDER TRANSFORMATION THAT
WILL ZERO A2 AND A3
C P 1*(V1,V2,V3)*(UlU2,U3)**T

IF (A2-EG.0, .AND. A3.EQ.0.) GO TO 10

S =ABS(AI) * ABS(A2) * ABS(A3)
Ul = Al/S
U2 =42/S
U3 = A3/S
R =SORT (Ul*L'I sU2*U2,U3*U3)
IF (U1.LTO.) P, -R
V2=-U3/R
V3=V3/1
Ul = 1.
U2 =V2/VI

10 U! = 0.
RETURN

47
 SLBROUTINE HSIH2(AlA2oUltU2tVltV2)

C FIMOS HOUSEHOLDER TRANSFORMATION THAT WILL ZERO A2

C P = I (VlV2)*(U1,U2)**T

IF (A2.EQ.0.) GO TO 10
S = ABS(Al) 4 ABS(A2)
Ul = A2/S
U2l= A2/S
R SuRT(U1*Ul+U2*U2)
IF (Ul-LT.O.) R -R
VI = -(Ul *R)/R
V2 = -U2/R
ul = I.
U2 = V21VI
RETURN
10 ul= 0.
RE~T U RN

48
 SU8ROUTINE CHSH2(A1R*AjIA2R9A21,C.SR#SI)

C COMPLEX HOUSEHOLDER THAT WILL ZERO A?

C (C 5*)
C P (S~ -C ) 9 C REAL9 S COMPLEX
C
IF (A2RoEQ*Oo *AND* A219EQ9o.) (30 TO 10
IF (AIR*EQOo eAND, AII*EQ.09) 60 TO 20
R =SQRT(AIR*AIRAlI*AII)

SR= (AxR*A2R.AI'A21)/R
SI = (AIR*A21-A1I*A2R)/R
R SURT(C*C*SR*SR+SI*SI)
C =C/R
SR = S/
SI = SI/R
RE TURN
10 C = 1.
St1 0.
RETURN
20 C =0.
Sp 1.
SI 0.
RET UR N

49
rv efl I .. 1 * 1"9 .

SUBROUTINE CDIV(RXIYRYIZRZI)

C COMPLEX DIVIDE. Z X/Y

4 F (ABS(YR).LT*ABS(YI)) GO TO 10
7WR= XRY
WI = XI/YR
VI = Yl/YR
D) = 1. * VI*VI
Zk = (WR *WI*VI)/D
ZI = (WI -WR'VI)/D
1RETURN

/l ENDV
= -WR

50

View publication stats