Applied Mathematics and Computation 183 (2006) 953–960

www.elsevier.com/locate/amc

On convergence of the symmetric modified successive

overrelaxation method for 2-cyclic coefficient matrices
M.T. Darvishi *, P. Hessari
Department of Mathematics, Razi University, Kermanshah 67149, Iran

Abstract

In this paper, we study the convergence of the symmetric modified successive overrelaxation (SMSOR) iterative method
for real symmetric 2-cyclic and consistently ordered coefficient matrices. We proposed some theorems, which they obtain
results better than similar works. By some numerical examples we show the goodness of our results.
 2006 Elsevier Inc. All rights reserved.

Keywords: SOR method; MSOR method; Symmetric MSOR method; Convergence

1. Introduction

Let us consider a system of n equations

Ax ¼ b; ð1Þ
nn n
where A 2 R and b; x 2 R with unknown vector x. For A satisfying ‘‘property A’’, in particular, for A of
the form
 
D1 H
A¼ ; ð2Þ
K D2
where D1 and D2 are square nonsingular diagonal matrices not necessarily of the same order. Young [5] pro-
posed the modified SOR (MSOR) methods to solve Eq. (1), where one relaxation parameter x1 is used for the
‘‘red’’ equations, which corresponds to D1 and the other relaxation factor x2 is used for ‘‘black’’ ones, which
corresponds to D2. For x1 = x2, the MSOR method reduces to the SOR one and the MSOR method is better
than SOR method.
Convergence of MSOR method is obtained in [3–5] when A is positive definite, strictly diagonally dominant
and so A is an H-, L- or M-matrix.

*
Corresponding author.
E-mail address: [email protected] (M.T. Darvishi).

0096-3003/$ - see front matter  2006 Elsevier Inc. All rights reserved.
doi:10.1016/j.amc.2006.05.121
954 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960

Many researchers worked on symmetric successive (accelerated) methods to solve the general system
Ax = b. Hadjidimos and Yeyios [2] introduced symmetric AOR (SAOR) for solving of square system
Ax = b where A is a nonsingular matrix. Young [5] proposed the symmetric MSOR method to solve
Eq. (1). Darvishi and Khosro-Aghdam [1] developed the symmetric successive overrelaxation (SSOR) method
to find the least square solution of minimal norm to system Ax = b where A is an m · n complex matrix and
rank(A) 6 min{m, n}.
In this paper, we assume the coefficient matrix A in Eq. (1) has form (2), that is A is a 2-cyclic matrix. We
will investigate the convergence of the symmetric MSOR method. Some sufficient and necessary conditions for
convergence will present, when A is a real symmetric positive definite matrix. Finally, we confirm our theoret-
ical results by some numerical examples.
In the following section we describe SMSOR method. Section 3 concerns convergence of the method for
2-cyclic consistently ordered coefficient matrices. Some numerical examples are presented in Section 4. At last,
the paper is concluded in Section 5.

2. Symmetric MSOR method

For the solution of liner system (1), when A is a 2-cyclic matrix of the form (2), we consider the following
splitting:
A ¼ D  CL  CU ; ð3Þ
     
D1 0 0 0 0 H
where D ¼ , CL ¼ and C U ¼ .
0 D2 K 0 0 0
The MSOR method is as follows:
xðmþ1Þ ¼ LX xðmÞ þ c; ð4Þ
where
1
LX ¼ ðD  XC L Þ ½ðI  XÞD þ XC U ; ð5Þ
1
c ¼ ðD  XC L Þ Xb ð6Þ
and
X ¼ diagðx1 I 1 ; x2 I 2 Þ; ð7Þ
where I1 and I2 are unit matrices of order as same as orders of D1 and D2, respectively.
In symmetric MSOR we obtain xðrþ2Þ as follows:
1

xðrþ2Þ ¼ LX xðrÞ þ c;
1
ð8Þ

where LX and c are defined in (5) and (6).

And so by backward MSOR we compute x(r+1) from xðrþ2Þ as follows:
1

xðrþ1Þ ¼ U X xðrþ2Þ þ c0 ;
1
ð9Þ

where

U X ¼ ðD  XC U Þ1 ½ðI  XÞD þ XC L  ð10Þ

and

c0 ¼ ðD  XC U Þ1 Xb: ð11Þ

We delete xðrþ2Þ from (8) and (9) and obtain the symmetric modified SOR method as follows:
1

xðrþ1Þ ¼ J X xðrÞ þ k; ð12Þ

 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960 955

where
1 1
J X ¼ U X LX ¼ ðD  XC U Þ ½ðI  XÞD þ XC L ðD  XC L Þ ½ðI  XÞD þ XC U 
¼ I  ðD  XC U Þ1 ð2I  XÞDðD  XC L Þ1 XA
and
1 1
k ¼ U X c þ c0 ¼ ðD  XC U Þ ð2I  XÞDðD  XC L Þ Xb:

2.1. Symmetric MSOR algorithm

   
ðrÞ y ðrÞ b1
Let x ¼ be the rth approximation of solution of system (1) and b ¼ .
zðrÞ b2
Then the symmetric MSOR algorithm has the following iterations:
8 ðrþ1Þ
<z
> ¼ ð1  x2 Þ2 zðrÞ þ x1 x2 ð2  x2 ÞD1 1
2 KD1 Hz
ðrÞ
þ x2 ð2  x2 Þð1  x1 ÞD1
2 Ky
ðrÞ

þx1 x2 ð2  x2 ÞD1 1 1
2 KD1 b1 þ x2 ð2  x2 ÞD2 b2 ;
>
: ðrþ1Þ 2
y ¼ ð1  x1 Þ y ðrÞ þ x1 D1
1 Hz
ðrþ1Þ
þ x1 ð1  x1 ÞD1
1 Hz
ðrÞ
þ x1 ð2  x1 ÞD1
1 b1 :

3. Convergence

In this section, we study the convergence of the symmetric MSOR method for 2-cyclic consistently ordered
coefficient matrices. We start this by introducing some theorems and lemmas.
Theorem 1. Suppose that A is a consistently ordered matrix and l is an eigenvalue of D1 1
2 KD1 H . If k satisfies

ðk  ð1  x1 Þ2 Þðk  ð1  x2 Þ2 Þ ¼ x1 x2 ð2  x1 Þð2  x2 Þlk; ð13Þ

2 2
then k is an eigenvalue of JX. Conversely, if k is an eigenvalue of JX such that k 5 (1  x1) , k 5 (1  x2) and l
satisfies (13), then l is a nonzero eigenvalue of D1 1
2 KD1 H .
 
z
Proof. Suppose that k and are eigenvalue and eigenvector of JX, respectively, then we have
    y
z z
JX ¼k :
y y
Since
" # " #
ð1  x1 ÞI 1 þ x1 x2 D1 1 1
1 HD2 K x1 ð1  x2 ÞD1 H ð1  x1 ÞI 1 x1 D1
1 H
J X ¼ U X LX ¼ 
x2 D1
2 K ð1  x2 ÞI 2 ð1  x1 Þx2 D1 1 1
2 K ð1  x2 ÞI 2 þ x1 x2 D2 KD1 H
" #
ð1  x1 Þ2 I 1 þ x1 x2 ð1  x1 Þð2  x2 ÞD1 1 2 1 1 1 2 1
1 HD2 K x1 x2 ð2  x2 ÞD1 HD2 KD1 H þ x1 ½ð1  x1 Þ þ ð1  x2 Þ D1 H
¼ :
x2 ð2  x2 Þð1  x1 ÞD1
2 K ð1  x2 Þ2 I 2 þ x1 x2 ð2  x2 ÞD1 1
2 KD1 H

So we have
8 2
> 1 1 2 1 1 1
< kz ¼ ð1  x1 Þ z þ x1 x2 ð1  x1 Þð2  x2 ÞD1 HD2 Kz þ x1 x2 ð2  x2 ÞD1 HD2 KD1 Hy
>
2
þx1 ½ð1  x1 Þ þ ð1  x2 Þ D11 Hy;
>
>
: 2
ky ¼ x2 ð2  x2 Þð1  x1 ÞD2 Kz þ ð1  x2 Þ y þ x1 x2 ð2  x2 ÞD1
1 1
2 KD1 Hy

hence
8 2
>
> ðk  ð1  x1 Þ Þz  x1 x2 ð1  x1 Þð2  x2 ÞD1 1
1 HD2 Kz
<
2
¼ x21 x2 ð2  x2 ÞD1 1 1 1
1 HD2 KD1 Hy þ x1 ½ð1  x1 Þ þ ð1  x2 Þ D1 Hy;
>
>
:
ðk  ð1  x2 Þ2 Þy  x1 x2 ð2  x2 ÞD1 1 1
2 KD1 Hy ¼ x2 ð2  x2 Þð1  x1 ÞD2 Kz;
956 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960

by computing z in the second equation and substituting it in the first equation we obtain
2 2
ðk  ð1  x1 Þ Þðk  ð1  x2 Þ Þy ¼ x1 x2 ð2  x1 Þð2  x2 ÞkD1 1
2 KD1 Hy

as l is an eigenvalue of D1 1
2 KD1 H then we have

ðk  ð1  x1 Þ2 Þðk  ð1  x2 Þ2 Þ ¼ x1 x2 ð2  x1 Þð2  x2 Þkl:

We can prove the second assertion by reversing the process. h

Lemma 1. Consider the quadratic equation x2  bx + c = 0, where b and c are real numbers. Both roots of the
equation are less than one in modulus if and only if jcj < 1 and jbj < 1 + c.

Proof. See Young [5]. h

Theorem 2. If A is a consistently ordered matrix and if l is a real eigenvalue of D1 1

2 KD1 H with jlj < 1, then the
symmetric MSOR method converges for
0 < x1 < 2; 0 < x2 < 2:

Proof. After some manipulations in Eq. (13), we get

2 2 2 2
k2  ½ð1  x1 Þ þ ð1  x2 Þ  x1 x2 ð2  x1 Þð2  x2 Þlk þ ð1  x1 Þ ð1  x2 Þ ¼ 0: ð14Þ
By setting
2 2
b ¼ ð1  x1 Þ þ ð1  x2 Þ  x1 x2 ð2  x1 Þð2  x2 Þl
and
2 2
c ¼ ð1  x1 Þ ð1  x2 Þ
in Eq. (14), from Lemma 1, and relation jcj < 1 we have
2 2
jð1  x1 Þ ð1  x2 Þ j < 1
therefore
jð1  x1 Þð1  x2 Þj < 1; ð15Þ
so, if we have
j1  x1 j < 1; j1  x2 j < 1;
then (15) holds. Therefore
0 < x1 < 2; 0 < x2 < 2:
From jbj < 1 + c in Lemma 1, we have
2 2 2 2
jð1  x1 Þ þ ð1  x2 Þ  x1 x2 ð2  x1 Þð2  x2 Þlj < 1 þ ð1  x1 Þ ð1  x2 Þ ;
therefore
2 2 2 2 2 2
1  ð1  x1 Þ ð1  x2 Þ < ð1  x1 Þ þ ð1  x2 Þ  x1 x2 ð2  x1 Þð2  x2 Þl < 1 þ ð1  x1 Þ ð1  x2 Þ :
This inequality changes to the following relations:
2 2 2 2
ð1  x1 Þ þ ð1  x2 Þ  x1 x2 ð2  x1 Þð2  x2 Þl < 1 þ ð1  x1 Þ ð1  x2 Þ ð16Þ
and
2 2 2 2
1  ð1  x1 Þ ð1  x2 Þ < ð1  x1 Þ þ ð1  x2 Þ  x1 x2 ð2  x1 Þð2  x2 Þl: ð17Þ
After some manipulations in Eqs. (16) and (17) and as jlj < 1 we see that Eqs. (16) and (17) are always true
and the proof is complete. h
 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960 957

3.1. Real symmetric matrices with positive diagonal elements

We assume that A is a real symmetric matrix with positive diagonal elements. In this case we have KT = H
and therefore, C TL ¼ C U . Hence, the splitting (3) can be rewritten as follows:
A ¼ D  C L  C TL : ð18Þ
In this case we have
1 1 1 1
J X ¼ I  ðD  XC U Þ ð2I  XÞDðD  XC L Þ XA ¼ I  ½X1 ðD  XC U Þ M½X1 ðD  XC L Þ A; ð19Þ
where
M ¼ X1 ð2I  XÞD: ð20Þ
To give convergence theorem for symmetric MSOR, first we present two lemmas that we need them in the
sequel.
Lemma 2. Let A be a real symmetric matrix which has form (18) and with positive diagonal elements. Then the
real symmetric matrix M given in (20) is positive definite if and only if
0 < x1 < 2; 0 < x2 < 2: ð21Þ

Proof. The eigenvalues of M are the following expressions:

2  x1 ð1Þ
d ii ; i ¼ 1; . . . ; n1 ð22Þ
x1
and
2  x2 ð2Þ
d ii ; i ¼ 1; . . . ; n2 ; ð23Þ
x2
ð1Þ ð2Þ
where d ii and d ii are diagonal elements of D1 and D2, respectively. Evidently M is positive definite if and only
ð1Þ ð2Þ
if its eigenvalues are positive. Since d ii ; i ¼ 1; . . . ; n1 and d ii ; i ¼ 1; . . . ; n2 , are positive values, then values in
(22) and (23) are positive if and only if
0 < x1 < 2; 0 < x2 < 2:
This completes the proof. h

Lemma 3. Let A and B are two real symmetric n · n matrices, one of them is positive definite. Then the eigen-
values of C = AB are real and positive if and only if both A and B are positive definite matrices.

Proof. See [2]. h

Theorem 3. If A is a real symmetric positive definite matrix which has the form (18), then the parameters x1 and
x2 satisfy (21) if and only if the symmetric MSOR converges.

Proof. First, we assume that x1 and x2 satisfy (21). Similar to JX, LX and UX, we define matrices J 0X , L0X and
U 0X , respectively, in the following manner:
1 1 1 1 1 1
J 0X ¼ A2 J X A 2 ; L0X ¼ A2 LX A 2 ; U 0X ¼ A2 U X A 2 ; ð24Þ
since JX = UXLX, so we have
1 1 1 1 1 1
J 0X ¼ A2 U X LX A 2 ¼ A2 U X A 2 A2 LX A 2 ¼ U 0X L0X : ð25Þ
We can write
1 1
LX ¼ I  ðD  XC L Þ XA ¼ I  ½X1 ðD  XC L Þ A
958 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960

and
1 1
U X ¼ I  ðD  XC U Þ XA ¼ I  ½X1 ðD  XC U Þ A:
Hence
1 1 1
L0X ¼ I  A2 ½X1 ðD  XC L Þ A2 ð26Þ
and
1 1
U 0X ¼ I  A2 ½X1 ðD  XC U Þ1 A2 : ð27Þ
As A is a symmetric matrix, therefore
T
U 0X ¼ ðL0X Þ ;
so
T
J 0X ¼ ðL0X Þ L0X :
Hence, the J 0X is a nonnegative definite matrix. Therefore J 0X has nonnegative eigenvalues and so JX has this
property. On the other hand, by virtue of Lemma 2, matrix M is positive definite, too. From (25)–(27) we have
1 1 1 1
J 0X ¼ I  A2 ½X1 ðD  XC U Þ M½X1 ðD  XC L Þ A2 ;
therefore
1 1
I  J 0X ¼ A2 ½X1 ðD  XC U Þ1 M½X1 ðD  XC L Þ1 A2
1 1 1 T 1 1 1
¼ ½M 2 ½X1 ðD  XC L Þ A2  ½M 2 ½X1 ðD  XC L Þ A2 : ð28Þ
We have
1 1 1 1
Y
n1
ð1Þ
Y
n2
ð2Þ
det½M 2 ½X1 ðD  XC L Þ1 A2  ¼ det M 2 det A2 xn11 xn22 d ii d ii > 0:
i¼1 i¼1
1 1 1 1
So matrix M ½X ðD  XC L Þ A is nonsingular. Therefore I  J 0X is a real symmetric positive definite matrix.
2 2

Consequently, the eigenvalues of J 0X are less than one and so the eigenvalues of JX are less than one, too. Thus
we proved that JX has nonnegative eigenvalues where they are less than one. This implies that q(JX) < 1 which
establishes the convergence of the symmetric MSOR method.
Conversely, suppose that q(JX) < 1. We have
I  J X ¼ ½X1 ðD  XC TL Þ1 M½X1 ðD  XC L Þ1 A ð29Þ
as JX is similar to J 0X which is nonnegative definite and q(JX) < 1, then matrix I  JX in (29) has eigenvalues
which are positive and non greater than one. Now from matrix
½X1 ðD  XC TL Þ½I  J X ½X1 ðD  XC TL Þ1
which is similar to the left hand side of Eq. (29) and therefore has the same eigenvalues as the matrix I  JX
has. In view of (29) the following expression will be equivalent to the product just we define it:
1 1
E ¼ M½X1 ðD  XC L Þ A½X1 ðD  XC TL Þ : ð30Þ
Matrix E in Eq. (30) can be rewritten as
E ¼ MF ; ð31Þ
where
1 1 1 1 T
F ¼ ½X1 ðD  XC L Þ A2 ½½X1 ðD  XC L Þ A2 
is a real symmetric positive definite matrix, because
1 1 1
Y
n1
ð1Þ
Y
n2
ð2Þ
det½½X1 ðD  XC L Þ A2  ¼ det A2 xn11 xn22 d ii d ii > 0:
i¼1 i¼1
 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960 959

In Eq. (29) as E has positive eigenvalues, therefore the matrix product in Eq. (31) has positive eigenvalues, too.
As M and F are real symmetric matrices and F is a positive definite matrix, according to Lemma 3, M is also a
positive definite matrix. Hence, we can apply Lemma 1 which gives the parameters x1 and x2 must satisfy
relation (21). This completes the proof. h

Theorem 4. Let A be a real symmetric matrix which has form (18). If 0 < x1 < 2 and 0 < x2 < 2, then matrix JX
of the symmetric MSOR method has real eigenvalues. Moreover, if q(JX) < 1 then A is a positive definite matrix.

Proof. Since 0 < x1 < 2 and 0 < x2 < 2, therefore by virtue of Lemma 2, M is a positive definite matrix. Hence
JX is similar to the following matrix:
1 1
J X ¼ I  ½M 2 ðX1 ðD  XC L ÞÞ½X1 ðD  XC TL Þ1 MðX1 ðD  XC L ÞÞ1 AððX1 ðD  XC TL ÞÞ1 M 2 Þ
1 1
¼ I  M 2 ðX1 ðD  XC L ÞÞ1 AðX1 ðD  XC TL ÞÞ1 M 2 ;
1
which is symmetric, as A and M 2 are symmetric. So JX has real eigenvalues.
For the second part, we set
1 1
C ¼ ðX1 ðD  XC TL ÞÞ MðX1 ðD  XC L ÞÞ ;
therefore we have
1 1 T 1 1
C ¼ ½M 2 ðX1 ðD  XC L ÞÞ  ½M 2 ðX1 ðD  XC L ÞÞ :
As
1 1 1
Y
n1
ð1Þ
Y
n2
ð2Þ
det½M 2 ðX1 ðD  XC L ÞÞ  ¼ det M 2 xn11 xn22 d ii d ii > 0;
i¼1 i¼1

then C is a positive definite matrix and

I  J X ¼ CA: ð32Þ
If q(JX) < 1, then eigenvalues of JX are less than one and I  JX has positive eigenvalues. Thus CA is positive
definite and Lemma 3 implies that A is a positive definite matrix. This completes the proof. h

4. Numerical examples

In this part we present two examples. For both we obtain the spectral value of the MSOR and SMSOR
iteration matrix. For simplicity, we only introduce the coefficient matrix A of the linear system.
Example 1. The coefficient matrix A is defined as follows:
0 1
2 0 0 1 0
B C
B 0 2 0 1 1 C
B C
A¼B B 0 0 2 0 1 C C:
B C
@ 1 1 0 2 0 A
0 1 1 0 2
Table 1 shows the results.

Table 1
Spectral radius of MSOR and SMSOR methods for Example 1 and arbitrary values of x1 and x2
x1 x2 qMSOR qSMSOR
1 1 0.75 0.75
0.78 0.98 0.8079 0.7622
1.04 0.94 0.7564 0.7513
0.9765 1.045 0.7453 0.7506
960 M.T. Darvishi, P. Hessari / Applied Mathematics and Computation 183 (2006) 953–960

Table 2
Spectral radius of MSOR and SMSOR methods for Example 2 and arbitrary values of x1 and x2
x1 x2 qMSOR qSMSOR
0.5 0.5 0.8202 0.6908
0.7 0.8 0.6935 0.5604
0.8 0.78 0.6726 0.5457
0.97 0.87 0.5752 0.5089

Example 2. In this case the coefficient matrix A has the following form:
0 1
2 0 1
B C
A¼@ 0 2 1 A:
1 1 2
We show the results in Table 2.
For both examples we report the values of spectral radius of the MSOR and SMSOR iterative methods in
Tables 1 and 2, for some arbitrary values of x1 and x2.

5. Conclusion

In this paper, we obtain interesting results for convergence of symmetric MSOR method for a special case
of coefficient matrix of a linear system. Numerical examples show our results are better than other results in
another papers that cited in this paper.

References

[1] M.T. Darvishi, R. Khosro-Aghdam, Symmetric successive overrelaxation methods for rank deficient linear systems, Appl. Math.
Comput. 173 (1) (2006) 404–420.
[2] A. Hadjidimos, A. Yeyios, The symmetric accelerated overrelaxation (SAOR) method, Math. Comput. Simul. XXIV (1982) 72–76.
[3] D. Herceg, M.M. Martins, M.E. Trigo, On convergence of the MSOR method for some classes of matrices, SIAM J. Matrix Anal.
Appl. 14 (1993) 122–131.
[4] M.M. Martins, A note on the convergence of the MSOR methods, Linear Algebra Appl. 141 (1990) 223–226.
[5] D.M. Young, Iterative Solution of Large Linear Systems, Academic press, New York, 1971.