1 Poisson Equation

Our first boundary value problem will be the steady-state heat equation, which in two dimensions has the
form
 
∂ ∂T ∂ ∂T
− k + k = q 000 (x), plus BCs. (1)
∂x ∂x ∂y ∂y
If the thermal conductivity k > 0 is constant, we can pull it outside of the partial derivatives and divide
both sides by k to yield the 2D Poisson equation
 2
∂ u ∂2u

− + = f (x), plus BCs. (2)
∂x2 ∂y 2
with f := q 000 /k and u := T . (We typically use u for scalar fields throughout the course for notational
convenience. Later, it will pair well with the set of test functions, denoted as v.)

The short-hand notation for the Poisson equation (2) is

−∇2 u = f (x) in Ω, u = ub on ∂ΩD , ∇u · n̂ = g on ∂ΩN ,
which applies in any number of space dimensions d. Here, we’ve indicated a mixture of boundary conditions:
Dirichlet on ∂ΩD , where u is prescribed, and Neumann on ∂ΩN , where the normal component of theSgradient
is prescribed. We take n̂ to be the outward pointing normal on the domain boundary ∂Ω = ∂ΩD ∂ΩN .

We will work with the Poisson equation and extensions throughout the course. At this point we want
to introduce some simple cases in order to understand optimal solution strategies in the 3D case, which is
arguably the most important in terms of compute cycles consumed throughout the world.

1.1 Finite Differences in 1D

The basic idea behind the finite difference approach to solving differential equations is to replace the differ-
ential operator with difference operators at a set of n gridpoints. In 1D, it is natural to order the points
sequentially, as illustrated in Fig. 1. Here, we consider the two-point boundary value problem
d2 u
− = f (x), u(0) = u(1) = 0. (3)
dx2
We use the second-order centered difference approximation to the second derivative, which for uniform grid
spacing h := xj − xj−1 is
!
uj+1 − 2uj + uj−1 d2 u h2 d4 u 4
− = − + + O(h ) (4)
h2 dx2 xj 12 dx4 xj
h2 d4 u
= f (xj ) − − O(h4 ) (5)
12 dx4 xj
≈ f (xj ). (6)
Note that (4)–(6) comprises three steps. First, we use the Taylor series expansion to express our difference
formula in terms of the desired derivative plus higher-order corrections. Second, we replace the desired
derivative by the data (f ), given by the differential equation. Finally, we incur the our truncation error by
dropping the higher-order terms in the Taylor series. We equate the results to arrive at our discete system
for the numerical approximation, uj ,
uj+1 − 2uj + uj−1
− = fj , j = 1, . . . , n. (7)
h2

The set of equations (7) represents our discretization of the original differential equation and is an
algebraic system consisting of n equations in n unknowns, uj , j=1,. . . ,n. Each equation j relates uj−1 , uj ,
and uj+1 to fj . For this reason, the resulting matrix system is tridiagonal,
 uj−1 uj
uj+1

0 =: x0 x1 x2 ··· xj−1 xj xj+1 ··· xn+1 := 1

Figure 1: Finite difference grid on Ω := [0, 1].

    
2 −1 u1 f1
 −1 2 −1  u2   f2 
.. ..
    
1  .. ..    
 −1 . .  .  =  . , (8)
h2 
 .. ..

 ..



 ..


 . . −1   .   . 
−1 2 un fn
| {z }| {z } | {z }
Ax u f
which has the shorthand Ax u = f , where u is the vector of unknowns and f the vector of data values, as
indicated in (8). We will often simply use A instead of Ax for notational convenience. Here, we explicitly
call out the x-coordinate association in preparation for the 2D development coming in the next section.

We list several attributes of A = Ax that carry over to higher space dimensions.

• A is symmetric, which implies it has real eigenvalues and an orthonormal set of eigenvectors satisfying
Asj = λj sj , sTj si = δij , where the Kronecker δij equals 1 when i = j and 0 when i 6= j.
• A is also positive definite, which means that xT Ax > 0 for all x 6= 0. It also implies λj > 0. Symmetric
positive definite (SPD) systems are particularly attractive because they can be solved without pivoting
using Cholesky factorization, A = LLT , or iteratively using preconditioned conjugate gradient (PCG)
iteration. (For large sparse systems, PCG is typically the best option.)
• A is sparse. It has a fixed maximal number of nonzeros per row (3, in the case of Ax ), which implies
that the total number of nonzeros in A is linear in the problem size, n. We say that the storage cost
for A is O(n), meaning that there exists a constant C independent of n such that the total number of
words to be stored is < Cn.
• A is banded with bandwidth w = 1, which implies that kij = 0 for all |i − j| > w. A consequence
is that the storage bound for the Cholesky factor L is < (w + 1)n. For the 1D case with w=1, the
storage for L is thus O(n). As we shall see, the work to compute the factors is O(w2 n).
• A−1 is completely full. We know this from the physics of the problem. Consider f ≡ 0 save for one
point, where fj = 1 (i.e., fj is the nth column of the n × n identity matrix). This case corresponds to
a point heat source at xj and, as we know, the temperature will be nonzero everywhere except at the
endpoints. In fact, it will exhibit a linear decay from xj to the boundaries. This is the exact Green’s
function for both the continuous and the discrete case. It’s an easy exercise to show that, for any
matrix A = (a1 a2 . . . an ) we have aj = Aej when ej is the jth column of I. The preceding arguments
establish that A−1 must be completely full.
1.2 Poisson Equation in lR2

Our principal concern at this point is to understand the (typical) matrix structure that arises from the 2D
Poisson equation and, more importantly, its 3D counterpart. The essential features of this structure will
be similar for other discretizations (i.e., FEM, SEM), other PDEs, and other space dimensions, so there is
merit to starting with this relatively simple system.

The steady-state heat equation in two dimensions is:

−∇ · k∇T = q 000 (x, y), plus BCs. (9)

For constant thermal conductivity k this equation reduces to the standard Poisson equation in Ω := [0, 1]2 ⊂
lR2 , which we usually express in terms of u for notational convenience:

−∇2 u = f (x, y), plus BCs (10)

 2
∂ u ∂2u

= − +
∂x2 ∂y 2
 2
δ u δ2 u

= − + + O(h2 ),
δx2 δy 2

where we have substituted the finite difference approximations, assumed to be about the point xij := (xi , yj ),

δ2 u ui+1,j − 2ui,j + ui−1,j

:= (11)
δx2 ∆x2
δ2 u ui,j+1 − 2ui,j + ui,j−1
:= ,
δy 2 ∆y 2
with the further assumption of uniform grid spacing, ∆x = ∆y = h. We’ll also consider homogeneous
Dirichlet boundary conditions, that is, u(x, y)|∂Ω ≡ 0. The respective unknowns and data in this case are
uij and fij , governed by the following system of equations
 
ui+1,j − 2ui,j + ui−1,j ui,j+1 − 2ui,j + ui,j−1
− + = fij , (12)
∆x2 ∆y 2

for i, j ∈ [1, . . . , N ]2 .

Assuming a lexicographical ordering in which the i- (x-) index advances fastest, the system takes on the
following matrix structure for ∆x = ∆y = h.
 4 −1 −1
 
 −1 4 −1 −1 u11 f11
   

 
 .. .. ..  u21   f21 

 −1 . . . 
 ..
 
  ..


.. .. .. . .
    
    

 . . −1 . 
 ..
 
  ..



 −1 4 −1 
 .  
  . 


 ..

 uM 1  
  fM 1 

 −1 4 −1 .    
  u12   f
12

 .. 
u22
 
  f

−1 −1 −1 .
  
 4    22 

 .. .. .. ..

 ..  
  .. 


 . −1 . . . 
 .  
  . 


.. .. .. ..
 ..   .. 
. .
    
 . . . −1 .    
uM 2   fM 2
    
 ..  
.
    
1  −1 −1 4  ..   .. 
.. .. =
    
h2  . .
 . . 
 −1    
  ..   .. 
 .. .. 
.
 
.

. . −1
    
    
  ..   .. 
 .. .. .. 
.
 
.

 . . .    
.. ..
    
.. .. ..
    

 . . .

 .  
  . 

  ..   .. 
 .. .. 
.
 
.

. . −1  
    
   
u1N   f1N
    
 −1 4 −1  
u2N   f2N
    
 ..  
.
    
 −1 −1 4  ..   .. 
. .
    
 .. .. .. ..    

 . . . . 
 ..
 
  ..



 .. .. ..

 .   . 
 . . . −1  uM N fM N
−1 −1 4 | {z } | {z }
| {z } u f
A2D

Note that A2D can be expressed as the sum of two systems, one associated with Ax coming
2 2
from δδxu2 , and one associated with one associated with Ay coming from δδyu2 . Specifically, we can
write

A2D = (Iy ⊗ Ax ) + (Ay ⊗ Ix ), (13)

where we have introduced the Kronecker (or tensor) product, ⊗. For two matrices A and B, their
Kronecker product C = A ⊗ B is defined as the block matrix
 
a11 B a12 B · · · · · · a1n B
 a21 B a22 B · · · · · · a2n B 
C :=  . (14)
 
.. .. ..
 . . . 
am1 B am2 B · · · · · · amn B

We will soon explore a few properties of this form, but for now simply note that it allows a clean
expression of the discretized Poisson operator in 2D. Consider the following splitting of A2D .
 −1
 
2
 −1 2 −1 
 
 .. .. 

 −1 . . 

 
 .. .. 

 . . −1 


 −1 2 


 2 −1 


 −1 2 −1 

.. ..
 
 

 −1 . . 

 .. .. 
. .
 

 −1 

 −1 2 
 .. 
.
 
1  
A2D =  
h2  .. 

 . 

 
 .. 

 . 

..
 
 
 . 
 
 .. 
.
 
 
 

 2 −1 

 .. 
.
 

 −1 2 

 .. .. .. 

 . . . 

 
 .. .. 
 . . −1 
−1 2

 2 −1 
 2 −1 
 
 .. .. 

 . . 

 .. .. 
. .
 
 
 

 2 −1 

..
 
 
 −1
 2 . 

 .. 
−1 .
 
 2 
 
 .. .. .. 

 . . . 

.. .. ..
 
 

 . . . 

 .. 
−1 .
 
1  2 
+  
h2 
 .. ..


 . . −1 
 
 .. .. 

 . . −1


 
 .. .. .. 

 . . . 

.. .. ..
 
 

 . . . 

 .. .. 
. . −1 
 

 

 −1 2 


 −1 2 

 .. .. 
. .
 
 
 
 .. .. 
 . . 
−1 2
  
2Ix −Ix
 
Ax
Ax
 .. 
  1  −Ix 2Ix . 
A2D =   + 2
 
.. h  .. ..

 .   . .

−Ix 
Ax −Ix 2Ix

= (Iy ⊗ Ax ) + (Ay ⊗ Ix )
δ 2
We see that we can express A2D as (Iy ⊗ Ax ) + (Ay ⊗ Ix ). The first term is nothing other than δx 2
δ 2
being applied to each row (j) of uij and the second term amounts to applying δy2 to each column
(i) on the grid.

1.3 Matlab Kronecker Product Demos

close all; format compact; Iy ⊗ Ax

% Kronecker Product Demo 0

%
% NOTE: It is important to use SPARSE matrices throughout. 5

%
% Otherwise, your run times will be very long and 10

% you will likely run out of memory!

15

Lx=2; Ly=1;
nx=15; ny=3; % Number of _interior_ points 20

dx=Lx/(nx+1); dy=Ly/(ny+1); 25

% USE help spdiags 30

e = ones(nx,1); Ax = spdiags([-e 2*e -e], -1:1, nx, nx)/(dx*dx);

35
e = ones(ny,1); Ay = spdiags([-e 2*e -e], -1:1, ny, ny)/(dy*dy);

Ix=speye(nx); Iy=speye(ny);
40 Iy ⊗ Ax
45
A = kron(Iy,Ax) + kron(Ay,Ix); %%% FINITE DIFFERENCE STIFFNESS MATRIX 0 5 10 15 20 25 30 35 40 45
nz = 129

% A couple of demo cases without the 1/(dx*dx) scaling.

nd= 5;
e = ones(nd,1); Ad = spdiags([-e 2*e -e], -1:1, nd, nd);
T = kron(Iy,Ad); full(T) Ay ⊗ Ix
0

nd= 15;
e = ones(nd,1); Ad = spdiags([-e 2*e -e], -1:1, nd, nd);
T = kron(Iy,Ad); spy(T)
title(’I_y \otimes A_x’,’fontsize’,16) 50

set(gcf,’PaperUnits’,’normalized’);set(gcf,’PaperPosition’,[0 0 1 1])
print -dpdf iyax.pdf

pause; figure 100

nd= 5;
e = ones(nd,1); Ad = spdiags([-e 2*e -e], -1:1, nd, nd);
T = kron(Ad,Ix); full(T)
150
nd= 15;
e = ones(nd,1); Ad = spdiags([-e 2*e -e], -1:1, nd, nd);
T = kron(Ad,Ix); spy(T)
title(’A_y \otimes I_x’,’fontsize’,16)
set(gcf,’PaperUnits’,’normalized’);set(gcf,’PaperPosition’,[0 0 1 1]) 200
Ay ⊗ Ix
print -dpdf ayix.pdf
0 50 100 150 200
nz = 645

Note that our finite-difference stiffness matrix in matlab would be written as

A = kron(Iy,Ax) + kron(Ay,Ix)
where Ax and Ay are formed using the matlab spdiags command (help spdiags), and Iy and Ix
are formed using speye.

It is important to use sparse matrices in matlab for these higher-dimensional (2D and 3D)
problems or you will run out of memory and it will take very long to solve these problems.

This problem is known in scientific computing and the curse of dimensionality.

1.4 Poisson Equation in lR3

We now extend the 1D and 2D concepts to the most important 3D case. The short story is that
the 3D stiffness matrix takes the wonderfully symmetric form

A3D = (Iz ⊗ A2D ) + (Az ⊗ I2D ) (15)

= (Iz ⊗ Iy ⊗ Ax ) + (Iz ⊗ Ay ⊗ Ix ) + (Az ⊗ Iy ⊗ Ix ).

and the discrete system is as before A3D u = f . This of course is the form that arises for a finite
difference discretization of −∇2 u = f in Ω = [0, 1]3 , u = 0 on ∂Ω, or, more explicitly,
 2
δ2u δ2u

δ u
− + + = f (xi , yj , zk ), (16)
δx2 δy 2 δz 2

with
δ2u uij,k+1 − 2uijk + uij,k−1
:= , (17)
δz 2 ijk ∆z 2

δ2 u δ2 u
and equivalent expressions for δx2
and δy 2
.

Note that, with (15), we really have not restricted ourselves to uniform mesh spacing in each
direction. We could have different grid spacing ∆x, ∆y, and ∆z and number of mesh points Nx ,
Ny , and Nz , in each of the space directions. Then, Ix would be the Nx × Nx identity matrix and
Ax would be the corresponding stiffness matrix, as would also be the case for y and z.