Technische Universität München

Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods
Hans-Joachim Bungartz

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 1
Technische Universität München

3.1 Preliminary Remarks

3.2 Iterative Solvers

3.3 The Multigrid Principle

3.4 AMG

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 2
Technische Universität München

3.1 Preliminary Remarks

3.2 Iterative Solvers

3.3 The Multigrid Principle

3.4 AMG

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 3
Technische Universität München

3.1 Preliminary Remarks

• Multigrid methods and their many relatives (multi-scale,

multi-resolution, multi-something) are the most efficient algorithms for
the iterative solution of large sparse systems of linear equations. Here,
in contrast to classical iterative schemes, the number of iterative steps
that are needed to obtain a certain error reduction does not depend on
the system size n.
• The system matrices typically stem from the discretization of partial
differential equations – hence, there is a geometric context
(neighbouring grid points, coarse and fine grids, high and low error
frequencies). Therefore, the original multigrid solvers developed in that
context are also called geometric multigrid methods.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 4
Technische Universität München

Preliminary Remarks (2)

• However, what to do if such a context is missing? If there is just a

system Ax = b, but no information on the origin of A? If there may have
never been gird points etc.? For that, algebraic multigrid methods
(AMG) have been developed – as highly efficient black box solvers,
which do not know anything more than the matrix entries. This third
chapter shall provide a brief introduction to AMG.
• We start with a concise overview of
• the discretization of PDE (where do the matrices come from?),
• classical iterative solvers (how did people solve these systems in
pre-multigrid times?), and
• the (geometric) multigrid principle.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 5
Technische Universität München

3.1 Preliminary Remarks

3.2 Iterative Solvers

3.3 The Multigrid Principle

3.4 AMG

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 6
Technische Universität München

3.2 Iterative Solvers

Finite Difference Methods for PDE
• given: domain
Ω ⊆ IRd , d ∈ {1, 2, 3}
• on Ω, we define: (regular) grid Ωh
• grid resolution or mesh width

h = (hx , hy , hz )
• replace derivatives with difference quotients:
• first derivatives: forward-, backward-, or central differences
∂u . u(ξ + hx ) − u(ξ) u(ξ) − u(ξ − hx ) u(ξ + hx ) − u(ξ − hx )
(ξ) = , ,
∂x hx hx 2hx
• second derivatives: standard 3-point stencil
∂ 2 u . u(ξ + hx ) − 2u(ξ) + u(ξ − hx )
=
∂x2 h2x
• Laplace operator in 2D or 3D: 5-point or 7-point stencil
• There are broader stencils, too (involving more neighbours).
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 7
Technische Universität München

Finite Difference Methods (2)

• in each inner grid point:

• construct a difference equation
• unknowns (degrees of freedom): discrete approximation to the
function values in the grid points
• one degree of freedom per grid point and per (scalar) unknown
quantity
• in points on or close to the boundary:
• concrete equation depends on boundary conditions:
• Dirichlet: no difference equation in boundary points (there, the function values
are given, hence no unknowns needed)
• Neumann: special appearance of the difference equation at the boundary
(implement the boundary conditions)
• discretization leads to a system of linear equations (SLE)
• sparsely populated, in general
• fast (iterative) solvers are vital to scientific computing

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 8
Technische Universität München

Finite Difference Methods (3)

• simple example: Poisson equation on the unit square

−∆u = f on ]0, 1[2

• equidistant square grid: h = hx = hy = 1/N

• number of degrees of freedom (unknowns): M = (N − 1)2
• standard 5-point stencil results in linear system Ax = b with
• M × M -Matrix A (the pointwise difference equations)
• M -vectors b (right-hand side) and x (unknown values of u )
• A is a sparse matrix with band structure; after multiplication with h2 :
• Dirichlet boundary conditions:
• all diagonal elements are 4, per row 2 to 4 non-zeroes (−1)
• boundary values to the right-hand side
• Neumann boundary conditions (various possibilities):
• diagonal elements are 2 or 3 next to the boundary and 4 elsewhere; a
corresponding number of −1 entries per row
• pairs (1, −1) along the boundary to the right-hand side

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 9
Technische Universität München

Finite Difference Methods (4)

• typically, accuracy of quadratic order:

kucomputed − uexact k = O(h2 ) = O(N −2 )

• curse of dimensionality:
• O(N d ) points are needed in case of d dimensions
• starting points for improvements:
• stencils of higher order (cubic, quartic, ...):
• take more than two neighbouring points into account
• problem: matrix gets denser (less non-zeroes)
• being more economic with grid points:
• use locally refined grids (adaptive grids)
• problem: what to do where two resolutions meet – which values?

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 10
Technische Universität München

Standard Iterative Schemes for SLE

• iterative solution of large (sparse) SLE:

• one of the central tasks in numerical simulation
• they occur at the discretization of ODE (BVP) and PDE
• frequently, direct solvers are not competitive:
• too large number of unknowns (cf. PDE in 3D)
• classical elimination approaches may destroy the (sparse) structure
of the matrix
• why exact solution in case of approximations? (this holds, esp., for
the nonlinear case, where an SLE has to be solved in each iterative
step to treat the nonlinearity)
• goal: performance of the kind ”‘for 3 digits we need 10 steps”’ –
independent of the number of unknowns
• however, the typical reality of classical iterative schemes:
• speed of convergence deteriorates with increasing problem size!

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 11
Technische Universität München

Basics of Iterative Schemes

• consider iterative methods, start from x(0) ∈ IRn and end (hopefully)
close to the solution x of Ax = b:
x(0) → x(1) → . . . → x(i+1) → . . . → lim x(i) = x
i→∞

• speed of convergence:

kx − x(i+1) k < γ · kx − x(i) ks

for a 0 < γ < 1 , order of convergence s
• typical behaviour of simple iterative methods for SLE:

s = 1, γ = O(1 − n−k ), k ∈ {0, 1, 2, ...}

(n : number of grid points)
• strategy: look for methods with
only O(n) arithmetic operations per iterative step (cost; that’s the
•
minimum effort)
• convergence behaviour according to γ < 1 − const. (benefit)
• two big families: relaxation and Krylov subspace methods

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 12
Technische Universität München

Relaxation Methods

• sometimes also called smoothers :

• Richardson iteration
• Jacobi iteration
• Gauß-Seidel iteration
• successive over-relaxation (SOR) or damped methods
• starting points:
• error e of the current approximation (cause): unknown
• residual r of the current approximation (effect): easily accessible

r(i) = b − Ax(i) = Ax − Ax(i) = A(x − x(i) ) = −Ae(i)

(used as error indicator, since nothing better is available)

• how to use the residual for better approximations?
• Richardson: use the residual directly as a correction
• Jacobi/Gauß-Seidel: make one component of r to zero
• SOR/damped: as before, but correction may be to big / small

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 13
Technische Universität München

Important Relaxation Schemes

• Richardson iteration:

for i = 0,1,...
(i+1) (i) (i)
for k = 1,...,n: xk := xk + rk
Here, simply the residual r(i) is taken as correction to that current
approximation x(i) (component-wise).

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 14
Technische Universität München

Important Relaxation Schemes (2)

• Jacobi iteration:

for i = 0,1,...
1 (i)
for k = 1,...,n: yk := akk
· rk
(i+1) (i)
for k = 1,...,n: xk := xk + yk

• In each sub-step k of a step i, a correction yk is computed and

stored.
• Applied immediately, this would make the k-component of the
residual r(i) to vanish (for the moment – to be verified easily by
inserting).
• Thus, equation k would be exactly solved with this approximation to
x – a progress, which would be immediately destroyed in the
following sub-step k + 1, of course.
• However, these component updates are not applied at once, but at
the end of each complete iterative step only (second k-loop).

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 15
Technische Universität München

Important Relaxation Schemes (3)

• Gauß-Seidel Iteration:

for i = 0,1,...
for
P k = 1,...,n:
(i) (i+1) (i)
:= bk − k−1 − n
P
rk j=1 akj xj j=k akj xj

1 (i) (i+1) (i)

yk := akk
· rk , xk := xk + yk

• Here, again the Jacobi correction is computed, but now, and in

contrast to Jacobi’s method, each update is applied immediately, as
soon as it is available, and not at the end of each step of iteration.
• Hence, when updating component k, we already know and use the
modified new values of components 1 to k − 1.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 16
Technische Universität München

Important Relaxation Schemes (4)

• Sometimes, a damping (multiplication of the correction term with some

factor 0 < α < 1) or an over-relaxation (factor 1 < α < 2) improves the
convergence behaviour of the schemes discussed above:
(i+1) (i)
xk := xk + αyk .

• For Gauß-Seidel, the variant with α > 1 is widespread, and we then

speak of SOR methods (Successive Over-Relaxation).
• For Jacobi, however, typically damping is used.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 17
Technische Universität München

Discussion: Additive Splitting of the System Matrix

• For a short convergence analysis of the relaxation schemes introduced

above, we need an algebraic formulation instead of the algorithmic one.
• All four schemes are based on the simple idea to write matrix A as a
sum A = M + (A − M ), where M x = b is very easy to solve and where
the difference A − M should be small with respect to some matrix norm.
• With the help of such a suitable M , Richardson, Jacobi, Gauß-Seidel,
and SOR methods can be written as

M x(i+1) + (A − M )x(i) = b

or, solved for x(i+1) ,

x(i+1) := M −1 b−M −1 (A−M )x(i) = M −1 b−(M −1 A−I)x(i) = x(i) +M −1 r(i)

• Furthermore, we split A in another way, using its diagonal part DA , its

strictly lower triangular part LA , and its strictly upper triangular part UA
as summands:
A =: LA + DA + UA .

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 18
Technische Universität München

Discussion: Additive Splitting of the System Matrix (2)

With that, we can show the following relations:

• Richardson: M := I ,
• Jacobi: M := DA ,
• Gauß-Seidel: M := DA + LA ,
1
• SOR: M := D
α A
+ LA .

• When we consider the algorithmic formulations for Richardson’s and

Jacobi’s methods, the first two relations are obvious:
• With Richardson, the residual is taken as correction directly, which
leads to the identity I as pre-factor.
• With Jacobi, the residual is divided by the diagonal matrix element,
which leads to the inverse of the diagonal part DA as pre-factor.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 19
Technische Universität München

Discussion: Additive Splitting of the System Matrix (3)

• Since the Gauß-Seidel iteration is a special case of the SOR scheme

(α = 1), it suffices to prove the above formula for M for the general SOR
case. The algorithm directly provides
 
k−1
X n
X
(i+1) (i) (i+1) (i)
xk := xk + α bk − akj xj − akj xj  /akk
j=1 j=k
 
(i+1) (i) −1
⇔ x := x + αDA b − LA x(i+1) − (DA + UA )x(i)
1 1
⇔ DA x(i+1) = DA x(i) + b − LA x(i+1) − (DA + UA )x(i)
α
  α  
1 1
⇔ DA + LA x(i+1) + (1 − )DA + UA x(i) = b
α α
⇔ M x(i+1) + (A − M )x(i) = b ,

which proves the assumption for the SOR method.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 20
Technische Universität München

Discussion: General Convergence Behaviour

• Concerning convergence, there are two direct consequences of the

approach
M x(i+1) + (A − M )x(i) = b :
 
• If the sequence x(i) converges at all, then the exact solution x of our
system Ax = b is the limit.
• For an analysis, we assume that the matrix −M −1 (A − M ) (i. e. the
matrix which is applied to e(i) to get e(i+1) ; see below) is symmetric.
Then, its spectral radius ρ (i.e. the largest (modulus!) eigenvalue) is the
quantity that characterizes the convergence behaviour:
 
∀x(0) ∈ IRn : lim x(i) = x = A−1 b ⇔ ρ < 1.
i→∞

To see that, subtract M x + (A − M )x = b from the equation at the

slide’s top:

M e(i+1) + (A − M )e(i) = 0 ⇔ e(i+1) = −M −1 (A − M )e(i) .

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 21
Technische Universität München

Discussion: General Convergence Behaviour (2)

• If all eigenvalues have a modulus smaller than 1 and if, hence, ρ < 1,
then all error components are reduced in each step of the iteration. If
ρ > 1, at least one error component will/may grow.
• When constructing iterative schemes, our goal must, of course, be a
spectral radius that is as small as possible (as close to zero as possible).

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 22
Technische Universität München

Discussion: Convergence Statements

• There are a couple of results on the methods’ convergence. We mention

some important ones:
• For the convergence of the SOR method, a necessary condition is
0 < α < 2.
• If A is positive definite, then both the SOR (for 0 < α < 2) and the
Gauß-Seidel iteration converge.
• If A and 2DA − A are both positive definite, then Jacobi’s method
converges. P
j6=i | aij | for all i),
• If A is strictly diagonally dominant, (i. e. if aii >
then both the Jacobi and the Gauß-Seidel method converge.
• In certain cases, the optimal parameter α can be determined, such
that ρ gets minimal and, hence, the error reduction per step of
iteration gets maximal.
• The Gauß-Seidel iteration is not generally better than Jacobi’s, as one
might suppose due to the immediate updates. There are examples
where the first converges but the latter diverges, and vice versa. In many
cases, however, Gauß-Seidel needs only half the number of steps
Jacobi needs.
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 23
Technische Universität München

On the Spectral Radius of Typical Iteration Matrices

• Obviously, ρ is not only crucial for the question whether the iteration
scheme converges at all, but also for its quality, that is its speed of
convergence: The smaller ρ is, the faster all components of the error e(i)
are reduced in each iterative step.
• In practice, unfortunately, the above results on convergence are of a
more theoretical value only, since ρ is frequently that close to 1 that –
despite convergence – the number of steps to obtain convergence is far
too big.
• An important scenario is the dicretization of partial differential equations:
• Typically, ρ depends on the problem size n and, hence, on the
resolution h of the underlying grid, for example
 
1
ρ = O(1 − h2l ) = O 1 − l
4
with a mesh width hl = 2−l .
• This is a huge drawback: The finer and more accurate our grid is,
the poorer gets the convergence behaviour of our relaxation
methods. Hence, better iterative solvers are a must!
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 24
Technische Universität München

3.1 Preliminary Remarks

3.2 Iterative Solvers

3.3 The Multigrid Principle

3.4 AMG

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 25
Technische Universität München

3.3 The Multigrid Principle

• starting point: Fourier mode analysis of the errors

• decompose the error e(i) = x(i) − x into its Fourier components
(Fourier transform)
• observe how they change/decrease under a standard relaxation like
Jacobi or Gau-Seidel (in a two-band sense):
• The high frequency part (with respect to the underlying grid) is
reduced quite quickly.
• The low frequency part (w.r.t. the grid) decreases only very
slowly; actually the slower, the finer the grid is.
• This behaviour is annoying
• the low frequencies are not expected to make troubles, but we
can hardly get rid of them on a fine grid;
• but also encouraging
• the low frequencies can be represented and, hopefully, tackled
on a coarser grid – there is no need for the fine resolution.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 26
Technische Universität München

A Simple Example

• 1D Laplace equation, u(0) = u(1) = 0 (exact solution 0)

• equidistant grid, 65 points, 3-point stencil, damped Jacobi method with
damping parameter 0.5
• start with random values in [0, 1] for u in the grid points
• After 100 (!) steps, there is still a maximum error bigger than 0.1 due to
low-frequency components!
• therefore the name smoothers for relaxation schemes:
• They reduce the strongly oscillating parts of the error quite
efficiently.
• They, thus, produce a smooth error which is very resistent.
• the idea: work on grids of different resolution and combine the effects in
an appropriate way

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 27
Technische Universität München

A Simple Example (2)

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 28
Technische Universität München

Coarse Grid Correction

• sequence of equidistant grids on our domain: Ωl , l = 1, 2, . . . , L, with

mesh width hl = 2−l
• let Al , bl denote corresponding matrix, right-hand side, ...
• combine work on two grids with a correction scheme:

smooth the current solution xl ;

form the residual rl = bl − Al xl ;
restrict rl to the coarse grid Ωl−1 ;
provide a solution to Al−1 el−1 = rl−1 ;
prolongate el−1 to the fine grid Ωl ;
add the resulting correction to xl ;
if necessary, smooth again ;

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 29
Technische Universität München

Coarse Grid Correction (2)

• the different steps of this 2-grid algorithm :

• the pre-smoothing: reduce high-frequency error components,
smooth error, and prepare residual for transfer to coarse grid
• the restriction: transfer from fine grid to coarse grid
• injection : inherit the coarse grid values and forget the others
• (full) weighting : apply some averaging process
• the coarse grid correction: provide an (approximate) solution on
the coarse grid (direct, if coarse enough; some smoothing steps
otherwise)
• the prolongation: transfer from coarse grid to fine grid
• usually some interpolation method

• the post-smoothing: sometimes reasonable to avoid new

high-frequency error components
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 30
Technische Universität München

The V-Cycle

• recursive application of 2-grid scheme leads to multigrid methods

• there, the coarse grid equation is solved by coarse grid correction, too;
the resulting algorithmic scheme is called V-cycle :

smooth the current solution xl ;

form the residual rl = bl − Al xl ;
restrict rl to the coarse grid Ωl−1 ;
solve Al−1 el−1 = rl−1 by coarse grid correction ;
prolongate el−1 to the fine grid Ωl ;
add the resulting correction to xl ;
if necessary, smooth again ;

• on the coarsest grid: direct solution

• number of smoothing steps: typically small (1 or 2)

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 31
Technische Universität München

Further Multigrid Cycles

• the V-cycle is not the only multigrid scheme:

• the W-cycle: after each prolongation, visit the coarse grid once
more, before moving on to the next finer grid

• W-cycle is sometimes advantageous with respect to speed of

convergence

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 32
Technische Universität München

Nested Iteration and Full Multigrid

• two more famous multigrid schemes:

• the nested iteration: start on coarsest grid Ω1 , smooth, prolongate
to Ω2 , smooth, prolongate to Ω3 , and so on, until finest grid is
reached; now start V-cycle
• full multigrid: replace ‘smooth steps above by ‘apply a V-cycle;
combination of improved start solution and multigrid solver

• multigrid idea is not limited to rectangular or structured grids: we just

need a hierarchy of nested grids (works for triangles or tetrahedra, too)
• also without underlying geometry: algebraic multigrid methods

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 33
Technische Universität München

Basic Convergence Results

• Cost (storage and computing time):

• 1D : c · n + c · n/2 + c · n/4 + c · n/8 + . . . ≤ 2c · n = O(n)
• 2D : c · n + c · n/4 + c · n/16 + c · n/64 + . . . ≤ 4/3c · n = O(n)
• 3D : c · n + c · n/8 + c · n/64 + c · n/512 + . . . ≤ 8/7c · n = O(n)
• i.e.: work on coarse grids is negligible compared to finest grid
• Benefit (speed of convergence):
• always significant acceleration compared with pure use of smoother
(relaxation method)
• in most cases even ideal behaviour γ = O(1 − const.)
• effect:
• constant number of multigrid steps to obtain a given number of digits
• overall computational work increases only linearly with n

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 34
Technische Universität München

3.1 Preliminary Remarks

3.2 Iterative Solvers

3.3 The Multigrid Principle

3.4 AMG

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 35
Technische Universität München

3.4 AMG

• As in the geometric case, we want to solve the system Ax = b for some

real, symmetric, and positive definite matrix A and a right-hand side b.
We do not make any further assumptions on structure or origin of A.
• For a multigrid algorithm, we, again, need
• a smoother or relaxation scheme and
• a coarse grid correction, consisting of the components
restriction, coarse grid solver, and prolongation.
• These notions are still geometry-related (“coarse“, etc.), the algorithms,
however, won’t have any geometric context any more. In particular, we
have to do without our frequency analogy from now on.
• AMG defines coarse grids, operators for grid transfer, and coarse grid
equations solely based on the matrix entries ai,j .
• Today, a lot of variants are available – for the following, we concentrate
on the original AMG algorithm, C-AMG.
• The underlying algorithmic pattern (2-grid scheme, V-cycle) strongly
resembles the one from geometric multigrid (see the previous section).

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 36
Technische Universität München

AMG and Geometry

• Attention: Though AMG does not need any geometric context, AMG is
also frequently used for partial differential equations (which, of course,
have a geometric context). That’s why we will meet geometric analogies
again and again.
• This gets apparent when we consider the adjacency graph of A:
• nodes represent unknowns, i. e. components xi of the solution
vector x;
• edges are present, if ai,j 6= 0.
• If A stems from a 2 D PDE grid, then grid and graph frequently look
the same.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 37
Technische Universität München

AMG and Geometry (2)

5x5 grid and adjacency graph for the Laplace equation

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 38
Technische Universität München

Algebraic Smoothness

• What does smooth mean, if there isn’t any relation to geometry nor
frequency?
• geometrically smooth: the usual idea of smoothness (not
oscillating etc.)
• algebraically smooth: simply everything the smoother produces
(in an AMG context, typically, Gauß-Seidel or related methods) –
which might also imply something geometrically non-smooth
• An algebraically smooth error typically mainly consists of components of
eigenvectors to small eigenvalues (so-called small eigenmodes):
• This means that smoothing especially damps the largest
eigenmodes.
• The smallest eigenmodes – the ”almost-kernel” of A – thus require
the largest attention when doing the coarse grid correction.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 39
Technische Universität München

Algebraic Smoothness (2)

• A simple example problem:

−auxx − buyy = f on [0, 1]2 , Dirichlet boundary ,

where a = b in the left half (x < 0.5) and a  b elsewhere

• Constant functions lie in the almost-kernel of A (matrix structure!).
• Hence, geometric smoothness and almost-kernel coincide in this
example.
• Therefore, we expect AMG to coarsen in the direction of geometric
smoothness (see the next slide).

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 40
Technische Universität München

An Example for Algebraic Smoothness

Algebraic smoothness. (a) Error for our sample problem, after seven Gauß-Seidel
sweeps. (b) The error is geometrically smooth in x-direction, (c) but oscillates in
y-direction in the right half. (d) In this example, AMG coarsens the grid in the direction
of geometric smoothness. Bigger squares correspond to coarser grids.
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 41
Technische Universität München

Basic Concepts in Classical AMG (C-AMG)

• heuristics for smoothness: smooth errors only change slowly in the

direction of relatively large (negative) matrix entries.
• For small eigenmodes v (length normalized to 1), it can be shown
that X
v T Av = (−ai,j ) · (vi − vj )2  1 .
i<j

If −ai,j is large, vi and vj can not differ strongly.

•
• Connection strength: For a given threshold θ, 0 < θ ≤ 1, the variable xi
strongly depends on xj , if
−ai,j ≥ θ · max{−ai,k } .
k6=i

• That means that the connection strength is measured relatively to

the largest negative off-diagonal element in the row; positive
off-diagonal elements are always considered as weak connections.
• Despite A = AT , it is possible that xi strongly depends on xj , but
xj only depends weakly on xi (there are AMG variants where this is
not possible; here, we assume symmetric connection strength).
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 42
Technische Universität München

C-AMG: Coarse Grid Selection

• The coarse grid always is a subset of the fine grid.

• Main principle underlying the coarse grid selection: Coarsen in the
direction of strong matrix connections.
• Structure of the algorithm:
1. get As from A by removing all weak connections or corresponding
entries, resp.
2. pass 1: determine a set C of coarse grid points and a set F (all
other variables), starting from As
3. pass 2: tune and optimize, if the interpolation should require that
• example:
• 2 D Laplace equation
• finite element discretizazion on uniform grids
• standard 9-point stencil (8 in the diagonal, −1 off-diagonal)
• independent of θ, all connections are of the same strength, hence
As = A

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 43
Technische Universität München

Illustration of the C-AMG Coarse Grid Selection (Pass 1)

(a) each node gets the number of its off-diagonal connections as weight
(number of off-diagonal non-zeroes, after removing small (and irrelevant)
entries)
(b) a point of maximum weight is selected as a C-point
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 44
Technische Universität München

(c) all neighbours of the new C-point that have not yet been assigned to
either F or C become F -points
(d) each new F -point causes an additional weight of 1 to all neighbours that
have not yet been assigned
(...) bottom row: go on until all points are either C-points or F -points
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 45
Technische Universität München

Interpolation
• starting point again: algebraically smooth error means dominance of
small eigenmodes v
• because of r T r = eT A2 e ≈ v T A2 v = λ2  1 (length of v normed to 1!),
this also implies small residuals
• To understand interpolation, assume r = Av = 0 or r = Ae = 0, resp.
(with an error e essentially consisting of small eigenmodes). For an
F -point i, we get
X X X
ai,i ei = − ai,j ej − ai,j ej − ai,j ej ,
j∈Ci j∈Fis j∈Fiw

where
• Ci denotes the C-points with a strong relation to i (interpolation in i
will be based on these only),
• Fis denotes the F -points with strong connection to i, and
• Fiw denotes all points with weak connection to i.
• The crucial point is now to add the error components ej from the second
and third partial sum above either to the points from Ci or directly to
point i.
• 2 examples:
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 46
Technische Universität München

Prolongation in C-AMG – Example 1

standard 9-point finite element stencil

(a) standard 9-point finite element stencil (or matrix relations, resp.)
(b) strongly connected F -points are interpolated from the neighbouring
C-points (weights according to the matrix entries)
(c) then, the strong connections to the F -points are added to the relations to
the C-points (according to the interpolation weights)
(d) from that, the collpased stencil results ...
(e) ... and from that the interpolation rule
Hans-Joachim Bungartz: Algorithms of Scientific Computing II
3. Algebraic Multigrid Methods, 47
Technische Universität München

Prolongation in C-AMG – Example 2

anisotropic 9-point finite element stencil

(a) anisotropic 9-point finite element stencil (or matrix relations, resp.)
(b) weak relations are included into the diagonal element
(c) from that, the collpased stencil results ...
(d) ... and from that the interpolation rule

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 48
Technische Universität München

Restriction and Coarse Grid Operator in C-AMG

Here, not that much has to be done:

• restriction: is defined as transposed matrix of the prolongation operator
in C-AMG, i. e.
R := P T
• coarse grid operator:
• Here, there are less options than in the geometric multigrid case,
where the given PDE can just be discretized on the coarse grid
again. In contrast to that, we do not know the origin of A and,
therefore, cannot transfer its generation process to other
resolutions.
• That’s why, in C-AMG, always the so-called Galerkin operator is
used, i. e.
Ac := P T AP .
With that, the set of algorithmic components is complete, and we can
construct different multigrid cycles – as before in the geometric case.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 49
Technische Universität München

Performance Features

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 50
Technische Universität München

References

This chapter’s figures and tables have been taken from:

Robert D. Falgout. An introduction to algebraic multigrid computing.
Computing in Science and Engineering, 8(6):24-33, 2006.

Hans-Joachim Bungartz: Algorithms of Scientific Computing II

3. Algebraic Multigrid Methods, 51