Received: 3 July 2019 Revised: 10 March 2020 Accepted: 2 April 2020

DOI: 10.1002/fld.4842

RESEARCH ARTICLE

Efficient stochastic finite element methods for flow in

heterogeneous porous media. Part 2: Random
lognormal permeability

Luca Traverso1,2 Timothy Nigel Phillips2

1
School of Earth and Ocean Sciences,
Cardiff University, Cardiff, UK Summary
2
School of Mathematics, Cardiff Efficient and robust iterative methods are developed for solving the linear sys-
University, Cardiff, UK tems of equations arising from stochastic finite element methods for single
phase fluid flow in porous media. Permeability is assumed to vary randomly in
Correspondence
T. N. Phillips, School of Mathematics, space according to some given correlation function. In the companion paper,
Cardiff University, Cardiff CF24 4AG, UK. herein referred to as Part 1, permeability was approximated using a truncated
Email: [email protected]
Karhunen-Loève expansion (KLE). The stochastic variability of permeability is
modeled using lognormal random fields and the truncated KLE is projected onto
a polynomial chaos basis. This results in a stochastic nonlinear problem since
the random fields are represented using polynomial chaos containing terms that
are generally nonlinear in the random variables. Symmetric block Gauss-Seidel
used as a preconditioner for CG is shown to be efficient and robust for stochastic
finite element method.

KEYWORDS
AMG, conjugate gradient, groundwater flow, MINRES, permeability, preconditioners, stochastic
finite element method, stochastic mixed finite element method

1 I N T RO DU CT ION

This article is concerned with the numerical simulation of flow through porous media. This is an example of a diffu-
sion problem in which the properties of the medium, such as the permeability, are not known in any precise way due
to the paucity and accuracy of available measurements. Quantifying the uncertainty of the model parameters and mod-
eling them as random variables means that the governing partial differential equation (PDE) is stochastic rather than
deterministic. When the stochastic PDE (SPDE) is equipped with suitable boundary conditions, which can be defined
as stochastic processes, then its solution is also a stochastic process. In this case the mean and variance provide useful
statistical information about the solution.
One of the most popular approaches for solving SPDEs is the Monte Carlo Method (MCM). This approach is based
on selecting an ensemble of realizations for the random input parameters from a given distribution. A deterministic
problem corresponding to each realization from the ensemble is solved and statistical quantities are computed from the
set of solutions. This approach is simple to implement since it can be based on the use of existing PDE solvers. However,
MCM is computationally very expensive since it converges sublinearly at a rate independent of the stochastic dimension.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the
original work is properly cited.
© 2020 The Authors. International Journal for Numerical Methods in Fluids published by John Wiley & Sons Ltd.

1626 wileyonlinelibrary.com/journal/fld Int J Numer Meth Fluids. 2020;92:1626–1652.

 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1627

Consequently, MCM requires a large number of realizations to create meaningful statistics. This limitation has motivated
researchers to investigate acceleration techniques for MCM to improve its performance.
Multilevel Monte Carlo combines multigrid concepts with traditional MCM. The acceleration in convergence is guar-
anteed as most of the MCM simulations are carried out on coarse grids while only a very limited amount of time is spent
on finer grids. When applied to the solution of PDEs1,2 multilevel MC methods have been shown to be incredibly effi-
cient for problems with rough coefficients (ie, spatial random fields with large variance or small correlation lengths).
These types of problems, common to radioactive waste disposal applications, require a large number of random variables
(typically in excess of 100 modes in a Karhunen-Loève expansion [KLE]) in probability space to accurately represent the
variability of the spatial random field.
An alternative class of methods to MCM is based on the discretization of SPDEs by the stochastic finite element
method (SFEM). However, in contrast to MCM, this approach can be computationally expensive since the discrete system
is significantly larger with the dimension of the problem growing factorially with the number of random variables used
to describe the input spatial random field. This phenomenon is known as the curse of dimensionality.
Solution strategies for SFEM depend on the choice of basis functions for the discretization of the stochas-
tic space. There are two popular choices. The first choice uses global tensor product polynomials.3 The advan-
tage of this approach is that it allows the global Galerkin system to be decoupled. However, this is restricted
to problems for which the permeability is approximated by normal or uniform random fields.4 There is no evi-
dence that when the permeability is approximated by a lognormal random field (a very common assumption in
the groundwater modeling community), the global Galerkin system can be decoupled. Furthermore, this approach
has the disadvantage of the size of the stochastic space growing more rapidly than alternative methods such as
polynomial chaos. Solution strategies for this choice involve iterative solvers based on Krylov subspace recycling
techniques.4
The second choice is based on the polynomial chaos (PC) method as outlined in the original work of Ghanem and
Spanos.5,6 In this approach, spectral representations of uncertainty in terms of multi-dimensional Hermite polynomi-
als or polynomial chaos expansions are used to approximate both the model parameters and the solution. This enables
the stochastic equations to be replaced by deterministic systems of PDEs which are then truncated and discretized. The
original PC method was motivated by the Wiener chaos expansion in which Hermite polynomials are used to represent
Gaussian random processes. The PC approach was later extended to generalized PC (gPC) where other sets of orthogo-
nal polynomials are used to generate improved representations of more general random processes. This allows efficient
methods to be constructed for problems for which the uncertainty in the model parameters cannot be represented using
Gaussian random processes. The PC expansion is a projection of the input random variables onto the space spanned by
the orthogonal polynomials. Therefore, the rate of convergence depends on the smoothness of the solution as a func-
tion of the input random variables. The optimal choice of polynomials, in the L2 sense, is the set that is orthogonal
with respect to the probability density function of the random variables that appear in the input variables. As MCM
requires the solution of many PDEs over the same computational domain, PC methods generate large coupled systems
of equations.
The large and highly structured linear system generated by the PC approach has to be solved using Krylov sub-
space iterative solvers. An efficient implementation of SFEM, which does not require the assembly of the global stiffness
matrix uses a block-diagonal preconditioner (subsequently referred to as “mean-based preconditioner”) for CG based
on an incomplete factorization of the mean stiffness matrix.7,8 Powell and Elman9 replaced the incomplete factoriza-
tion with a black-box algebraic multigrid (AMG) solver. Ernst et al10 extended the implementation of the mean-based
preconditioner to the solution of stochastic mixed finite element method (SMFEM) systems. Ullmann4 proposed a Kro-
necker product preconditioner for the stochastic linear (Gaussian/uniform random fields) and nonlinear (lognormal
random field) cases. The implementation of the Kronecker preconditioner was recently extended to the SMFEM11 and
significantly reduced the number of CG and MINRES iterations resulting in faster convergence. However, it is more
expensive to implement than mean-based preconditioners. A review of a large number of iterative solvers, including
one-level iterative methods, multigrid methods, and multilevel methods (for the stochastic discretization), can be found in
Rossell and Vanderwalle.12
In this article, the PC approach is adopted. The derivation of the global Galerkin system is described and solution
strategies that take full advantage of its characteristic block structure are proposed. Numerical experiments in Part 1
showed that a symmetric block Gauss-Seidel preconditioner for CG provides a competitive alternative to traditional
mean-based preconditioners. Since information associated with the off-diagonal blocks of A is incorporated into the pre-
conditioned system, the conditioning is improved. As a result, CG requires few iterations to converge. This approach is
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1628 TRAVERSO and PHILLIPS

particularly efficient for those cases in which the off-diagonal blocks of A hold significant information on the permeability,
that is, problems with large values of 𝜎.
It is evident from the literature and from our computational analysis that mean-based preconditioners cannot be
robust with respect to the permeability since they only include, in the preconditioned system, information associated with
the mean value of the spatial random field. The mean information is included in the blocks of the leading diagonal of the
global stochastic system, whilst variations (representing the variability of the spatial random field) about the mean are
contained in the off-diagonal blocks. When the latter contributions become important the mean-based preconditioner
performs poorly simply because this information is not included in the preconditioned system. For the stochastically
nonlinear case this situation is exacerbated by the fact that every block of the global system has non-zero entries.
To overcome this important limitation, we propose an alternative preconditioner for SFEM in which the off-diagonal
blocks of the global system are included in the preconditioned system using a block symmetric Gauss-Seidel (bSGS)
algorithm. The computational analysis clearly shows that block Gauss-Seidel algorithms used either as a preconditioner
for CG or as standalone solvers are more efficient than mean based preconditioners for the stochastically nonlinear
case and deliver considerable CPU savings. Although for some of the test cases considered, the standalone standard
Gauss-Seidel solver was the best performing solver, its performance deteriorated at a faster rate for test cases with large
standard deviation than preconditioned CG. Therefore, the main finding of this work is that CG equipped with a bSGS
preconditioner should be used to solve SFEM systems.

2 T H E MAT H E M AT ICAL MODEL

The steady-state flow of water in porous media, whose material parameters are assumed to be unknown, is described
by a scalar second-order stochastic partial differential equation (SPDE). The parameter that is known with the greatest
degree of uncertainty is the permeability or hydraulic conductivity, , which describes the ease with which a fluid can
move through pore spaces or fractures. The permeability is modeled as a random field.
Let u denote the pressure head or mean potential and q the velocity or flux. We assume that the medium occupies a
bounded domain D in R2 , with Lipschitz continuous boundary Γ. Let Γ = ΓD ∪ ΓN with ΓD ≠ ∅ and ΓD ∩ ΓN = ∅, where ΓD
and ΓN denote the portions of Γ where Dirichlet and Neumann boundary conditions on u are prescribed, respectively. The
tensor  is a 2 × 2 symmetric and uniformly positive definite tensor that represents permeability. Since the coefficients in
this tensor can never be completely known at every point in a heterogeneous porous medium, we assume that  = (x, 𝜔)
is a random field. Furthermore, only statistical properties of  are assumed.
Let Ω be the set of random events, ℑ the minimal 𝜎-algebra of the subsets of Ω, and P an appropriate probability
measure, then (Ω, ℑ, P) denotes a probability space. Then (x, 𝜔) ∶ D × Ω → R is a family of random fields indexed by x
such that for a fixed x ∈ D, (⋅, 𝜔) is a realization of the permeability and for a fixed realization 𝜔 ∈ Ω, (x, ⋅) is a random
variable.
We define the space of square integrable random variables with respect to P, L2P (Ω), as follows:

L2P (Ω) = {𝜉 ∶ Ω → R; ⟨𝜉 2 ⟩ < ∞},

where
⟨𝜉 2 ⟩ = 𝜉 2 (𝜔) dP(𝜔).
∫R

Let 𝜉 be a real-valued random variable belonging to (Ω, ℑ, P) and suppose there exists a probability density function
𝜌 ∶ R2 → R such that the expected value of 𝜉 is given by

⟨𝜉⟩ = 𝜉(𝜔)𝜌(𝜔) d𝜔.

∫R

The mean value of the random field  at x ∈ D is ⟨(x, ⋅)⟩, the covariance of  at x, y ∈ D is

cov(x, y) = ⟨((x, 𝜔) − ⟨(x)⟩)((y, 𝜔) − ⟨(y)⟩)⟩ = 𝜎 ∈ 𝜚(x, y),

√
and the standard deviation is 𝜎(x) = var(x) where var(x) = cov(x, x).
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1629

2.1 Primal formulation

In this formulation, we seek a random field solution u(x, 𝜔) ∶ D × Ω → R such that P-almost surely

−∇ ⋅ (x, 𝜔)∇u(x, 𝜔) = f (x) x ∈ D,

u(x, 𝜔) = g(x) x ∈ ΓD ,
(x, 𝜔)∇u(x, 𝜔) ⋅ n = 0 x ∈ ΓN , (1)

where n denotes the unit outward normal vector to ΓN , g(x) is the prescribed constant pressure head on ΓD , and f(x) is
the source or sink term. Note that f(x) and g(x) could also be random fields but in this paper they are assumed to be
deterministic functions. The solution of (1) enables the mean and standard deviation of u to be determined everywhere
in D. Once u has been determined, the velocity field q can be derived using Darcy's Law.
Define the solution space W and the test space W0 to be the tensor product spaces of deterministic functions defined
on D and stochastic functions defined on the probability space as follows

W = X ⊗ L2P (Ω), W0 = X0 ⊗ L2P (Ω), (2)

respectively, where

X = {v ∈ H 1 (D) ∶ v = g on ΓD }, X0 = {v ∈ H 1 (D) ∶ v = 0 on ΓD }.

Then the weak formulation of the primal problem is: find u ∈ W such that

⟨a(u, w)⟩ = ⟨L(w)⟩, ∀w ∈ W0 (3)

where
⟨ ⟩
⟨a(u, w)⟩ = (x, 𝜉)∇u(x, 𝜉) ⋅ ∇w(x, 𝜉)dx ,
∫D
⟨ ⟩
⟨L(w)⟩ = f (x)w(x, 𝜉)dx . (4)
∫D

If the permeability tensor is strictly positive and bounded, that is,

0 < k1 ≤ (x, 𝜔) ≤ k2 < ∞ a.e. in D × Ω, (5)

then the Lax-Milgram lemma can be used to establish that there exists a unique solution to the problem (3).

2.2 Mixed formulation

An alternative formulation that allows us to derive accurate approximations for the fluxes is to reformulate the primal
problem (1) as a first-order system by explicitly introducing Darcy's Law. In this formulation, the solution u(x, 𝜔) ∶ D ×
Ω → R, q(x, 𝜔) ∶ D × Ω → R2 is sought of the problem

 −1 (x, 𝜔)q(x, 𝜔) + ∇u(x, 𝜔) = 0 x ∈ D,

∇ ⋅ q(x, 𝜔) = f (x) x ∈ D,
u(x, 𝜔) = g(x) x ∈ ΓD ,
q(x, 𝜔) ⋅ n = 0 x ∈ ΓN . (6)

The solution of (6) gives the mean potential (or pressure) and normal fluxes together with information about their
mean and standard deviation everywhere in D.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1630 TRAVERSO and PHILLIPS

The solution spaces are the tensor product spaces Y = L2 (D) ⊗ L2P (Ω) and

V = {v ∈ H(div; D) ⊗ L2P (Ω) ∶ v ⋅ n = 0 on ΓN × Ω}, (7)

where

H(div; D) = {v ∶ v ∈ L2 (D)2 , and ∇ ⋅ v ∈ L2 (D)}. (8)

The weak formulation of the mixed problem is: find (u, q) ∈ Y × V such that

⟨a(q, v)⟩ + ⟨b(v, u)⟩ = ⟨(g, n ⋅ v)ΓD ⟩, ∀v ∈ V

⟨b(q, w)⟩ = − ⟨(f , w)⟩, ∀w ∈ Y (9)

where
⟨ ⟩
⟨a(q, v)⟩ =  (x, 𝜉)q(x, 𝜉) ⋅ v(x, 𝜉)dx ,
−1
∫D
⟨ ⟩
⟨b(v, w)⟩ = ∇ ⋅ v(x, 𝜉)w(x, 𝜉)dx ,
∫D
⟨ ⟩
⟨(g, n ⋅ v)ΓD ⟩ = g(x)(n ⋅ v(x, 𝜉))dx ,
∫Γ
⟨ D ⟩
⟨(f , w)⟩ = f (x)w(x, 𝜉)dx . (10)
∫D

There exists a unique solution to this problem provided that the bilinear forms are continuous and coercive and the
inf-sup inequality is satisfied.13 The permeability field is required to satisfy (5).

3 PERMEABILITY APPROX IMATION

To convert the stochastic primal and mixed formulations into deterministic problems, we need to represent the stochastic
variability of the permeability tensor (x, 𝜔) by an appropriate set of independent random variables {𝜉1 (𝜔), … , 𝜉d (𝜔)}.
In Part 1,14 two approaches were described to represent the stochastic variability of (x, 𝜔). The first approach, herein
referred to as coloured noise, assumes that the permeability varies randomly throughout D according to a given correlation
function. In this case, the permeability coefficient is approximated using a KLE.6,9,15,16
The permeability tensor (x, 𝜔) possesses a proper orthogonal decomposition

∑
∞
√
(x, 𝜉(𝜔)) = 𝜇(x) + 𝜎 𝜆i 𝜉i 𝛽i (x), (11)
i=1

where 𝜇(x) = ⟨(x, 𝜔)⟩ and {𝜆i , 𝛽i (x)} is the set of eigenvalues and eigenvectors of the covariance function 𝜚(x, y):

𝜚(x, x′ )𝛽i (x′ )dx′ = 𝜆i 𝛽i (x), i = 1, … (12)

∫D

Since  is nonnegative definite, the eigenvalues are real and we label them in decreasing magnitude 𝜆1 > 𝜆2 > ….
When the correlation function is of exponential type and D is a rectangular domain, there exists closed-form solutions
to the eigenvalue problem (12) (see Ghanem and Spanos,6 for example). In this article, we make full use of the closed-form
solutions and only random fields whose correlation function is of exponential or square-exponential type are considered.
Examples in which the eigenvalue problem is solved numerically can be found in Lu and Zhang17 and a description of
numerical algorithms is provided in Ghanem and Spanos.6 In such cases, the additional computational cost of solving
the eigenproblem (12) needs to be considered.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1631

In the coloured noise approach, (x, 𝜔) is approximated by a truncated KLE, d (x, 𝝃(𝜔)), where d is the number of
terms retained in the the KLE (11). The quadratic mean square convergence of d (x, 𝜉(𝜔)) to (x, 𝜉(𝜔)) is guaranteed as
d → ∞. The choice of d usually depends on the rate of decay of the eigenvalues. For example, in applications where the
eigenvalues decay slowly due to small correlation lengths, d may have to be very large.
Xiu and Karniadakis18 used uniform random variables, hence ensuring that d (x, 𝜉(𝜔)) is bounded between two pos-
itive values with probability 1. A consequence of this approach is that the random variables in (11) are not guaranteed to
be independent, thus this condition needs to be assumed explicitly.19
The second approach, herein referred to as white noise, assumes that the permeability varies randomly and indepen-
dently in D. This approach is often used to approximate parameters such as rainfall or groundwater recharge, which
generally do not show strong spatial correlation. Although the permeability is spatially correlated, in practical applica-
tions, the domain D can be decomposed into subdomains corresponding to hydrogeological units on which piecewise
constant hydraulic properties are assumed to hold. Different statistical parameters can be assigned to different regions of
D, thus reflecting the diverse hydraulic behavior of natural deposits. A coarse subdomain decomposition of D is performed
and a continuous random field dk is associated with each subdomain Dk , k = 1, … , ND , such that

⋃
ND
d (x, 𝜉(𝜔)) = dk (x, 𝜉(𝜔)),
k=1

where ND is the number of subdomains in D. Each subdomain Dk , which may be of irregular shape, is enclosed within
a rectangularly shaped domain D′k , that is, Dk ⊂ D′k , for k = 1, … , ND , chosen to be the smallest rectangle enclosing Dk .
Although a KLE is implemented for each subdomain Dk , the eigenvalue problem (12) is solved with respect to D′k .
When the conductivity coefficient (⋅, 𝜔) is assumed to be a Gaussian process, the random variables 𝜉i in (11) are
normally distributed. In these circumstances, the random variables have the desirable property of being uncorrelated
and independent. However, this also makes problems (1) and (6) ill-posed since the diffusion coefficient is not bounded
below and above by positive constants.9 In fact, it is well known that the permeability is required to satisfy (5). Although
Gaussian functions possess an infinite spectrum, it can be shown that well-defined discrete solutions can be obtained if
a relatively small variance is used.
From a mathematical point of view, the white noise approach possesses significant advantages with respect to sin-
gle domain KLE-based approaches. For example, the linear systems are tridiagonal,20 which means that block diagonal
preconditioners can be constructed to solve these problems efficiently.

4 P O LY N O M IAL CH AOS FO R LOGNORMAL RANDOM FIELD

Although Gaussian processes are commonly used to model uncertainty in engineering problems primarily due to their
simplicity, in some cases, it is preferable to use a lognormal process, particularly when the quantity under consideration
is constrained to be always positive. This is true when modeling the permeability or hydraulic conductivity of a porous
medium. Lognormal random fields are very popular among physical scientists and modelers for various reasons. First,
there are several studies, the data of which are summarized in References 21 and 22, that show that parameters such as
hydraulic conductivity or transmissivity are often lognormally distributed. Second, although the lognormal distribution
possesses an infinite upper bound, it only admits the positive part of the physical spectrum. This is obviously consistent
with the physical properties of these parameters. There is substantial evidence, direct and indirect, that supports the view
that the permeability is described by a lognormal distribution (Freeze23 ).
The random field d (x, 𝜉(𝜔)) is a lognormal random field if the logarithm of d is a Gaussian random field, that is,
lognormal random fields are of the form exp(g(x, 𝜉)) where g is a Gaussian random field. Lognormal random fields can
be characterized by defining the mean field and covariance of the underlying Gaussian random field.
In Section 3, we have seen that a Gaussian random field can be approximated by a truncated Karhunen-Loèaussian
process
(x, 𝜉(𝜔)) = exp((x, 𝜉(𝜔))). (13)

If we discretize (x, 𝜉(𝜔)) using the truncated KLE, d (x, 𝝃(𝜔)), then

d (x, 𝜉(𝜔)) = exp (d (x, 𝝃(𝜔))) . (14)

 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1632 TRAVERSO and PHILLIPS

There are advantages to using truncated KLE in polynomial chaos. For example, the optimal mean square convergence
property of the KLE mentioned earlier means that the number of variables required for a given accuracy is minimized,
hence limiting the system size.
Since d (x, 𝜉(𝜔)) is a random variable, it can be projected onto a polynomial chaos basis of order p

∑
P
d (x, 𝝃(𝜔)) = Lk (x)𝜒k (𝝃), (15)
k=1

where Lk are deterministic functions derived from (14) and for which closed forms can be obtained algebraically (see
References 4,6,24-26) and 𝜒k are multi-dimensional chaos polynomials in d random variables 𝜉1 , … , 𝜉d , of degree less
than or equal to p.

5 STOCHAST IC FINITE ELEMENT APPROX IMATION

The implementation of the spectral stochastic finite element method (SSFEM) for discretizing the weak formulation of the
primal problem (3) involves separate discretizations of the deterministic and stochastic spaces. According to the Galerkin
method, we define the finite-dimensional subspaces Sh ⊂ X0 and Th ⊂ L2 (Ω) such that Wh = Sh ⊗ Th ⊂ W = X0 ⊗ L2 (Ω).
The discrete variational formulation of (3) is: Find uh ∈ Wh such that

⟨a(uh , wh )⟩ = ⟨L(wh )⟩. ∀wh ∈ W h . (16)

The deterministic space X0 is discretized using the classical finite element basis functions 𝜙i (x), i = 1, … , Nu , where
Nu is the number of finite element nodes. These basis functions are piecewise linear on a partition Zh of D defined by
triangular finite elements Ti , i = 1, … , Ne , such that,
⋃
Ne
Zh = Ti ,
i=1

where Ne denotes the number of finite elements. Here h denotes the discretization parameter and describes the size of
the finite elements in Zh . Let Eh be the collection of numbered edges, ei , i = 1, … , Nedge , where Nedge is the total number
of edges in Zh .
The stochastic space L2 (Ω) is discretized by means of polynomial chaos of order less than or equal to P in d random
variables 𝜉i , i = 1, … , d. The polynomial chaos basis for Th contains multidimensional polynomials of degree less than or
equal to P, Th = span{𝜒1 , … , 𝜒P } where
(d + p)!
P= , (17)
d!p!

and d represents the number of random variables (number of terms retained in the KLE expansion). The polynomial chaos
basis is chosen so that the following orthogonality condition is satisfied

⟨𝜒i 𝜒j ⟩ = ⟨𝜒i ⟩2 𝛿i,j . (18)

In this article, the probability measure corresponds to that of a d-dimensional normal distribution. Hence, the basis
for Th consists of d-dimensional Hermite polynomials.

5.1 Linear system

To obtain the discrete linear system associated with the weak formulation (16), the mean potential u is approximated by

∑ ∑
P Nu
∑
P
uh (x, 𝝃) = us,r 𝜙r (x)𝜒s (𝝃) = us (x)𝜒s (𝝃). (19)
s=1 r=1 s=1
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1633

Substituting the expansion (19) for uh in (16) and choosing test functions wh = 𝜙k (x)𝜒l (𝝃), k = 1, … , Nu , l = 1, … , P,
yields the linear system of equations
Au = f, (20)

where A is a sparse matrix of size Nu P × Nu P with a block-structure

⎛ A1,1 A1,2 … A1,P ⎞ ⎛ u1 ⎞ ⎛ f1 ⎞

⎜A A2,2 … A2,P ⎟ ⎜ u2 ⎟ ⎜ f2 ⎟
A = ⎜ ⋮ 2,1 ⋮ ⋱ ⋮ ⎟, u = ⎜⋮ ⎟ f = ⎜⋮ ⎟. (21)
⎜A AP,2 … AP,P ⎟⎠ ⎜u ⎟ ⎜f ⎟
⎝ P,1 ⎝ P⎠ ⎝ P⎠

The diagonal blocks of A are defined as tensor products of the mean stiffness matrix, K0 , and ⟨𝜒i ⟩2 , that is,

Ai,i = ⟨𝜒i ⟩2 ⊗ K0 , i = 1, … , P, (22)

where

(K0 )r,s = 𝜇(x)∇𝜙r (x) ⋅ ∇𝜙s (x)dx, (23)

∫D

and 𝜇 denotes the mean value of the conductivity field (x, 𝜉). The off-diagonal blocks of A are tensor products of the stiff-
ness matrices, Kl , with the coefficients of the polynomial chaos expansion, ci,j,l = ⟨𝜉l 𝜒i 𝜒j ⟩, i, j = 1, … , P, and l = 1, … , d,
that is,

∑
d
[ ]
Ai,j = ⟨𝜉l 𝜒i 𝜒j ⟩ ⊗ Kl , (24)
l=1

where
√
(Kl )r,s = 𝜎 𝜆l 𝛽l (x)∇𝜙r (x) ⋅ ∇𝜙s (x)dx. (25)
∫D

Thus, the coefficient matrix A can be expressed in the tensor product form (see Powell9 )

∑
d
A = G0 ⊗ K0 + Gk ⊗ Kk , (26)
k=1

where the entries of the stochastic matrices are defined by

(G0 )r,s = ⟨𝜒r , 𝜒s ⟩, (Gk )r,s = ⟨𝜉k 𝜒r 𝜒s ⟩.

It is evident that the sparsity of the global stochastic coefficient matrix A is governed by the coefficients of the polynomial
chaos expansion, while the sparsity of the blocks of A is determined by the sparsity of the deterministic finite element
stiffness matrix.
The maximum degree of the polynomial chaos expansions of u and (x, ⋅) should be different. In fact, it can be shown
that for the Galerkin matrix A to be positive definite4,16,27 all polynomials of degree less than or equal to 2P have to be
included in the polynomial chaos expansion of . It is only when this condition is satisfied that a full Galerkin projection
of the polynomial chaos expansion of  is obtained. Following Ullmann,4 the number of chaos polynomials used for the
representation of  is
(d + 2P)!
N= . (27)
d!(2P)!

It can be demonstrated4,27 that the inner product ⟨𝜒k 𝜒i 𝜒j ⟩ is non-zero in only finitely many cases. In fact, ⟨𝜒k 𝜒i 𝜒j ⟩ = 0
for all 𝜒k with total degree greater than 2P. A consequence of this observation is that given a fixed number of random
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1634 TRAVERSO and PHILLIPS

0 0

2 2

4 4

6 6

8 8

10 10

12 12

14 14

16 16

18 18

20 20
0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20
nz = 280 nz = 352

(A) N = 20 Kronecker products (B) N = 35 Kronecker products

0 0

2 2

4 4

6 6

8 8

10 10

12 12

14 14

16 16

18 18

20 20
0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20
nz = 382 nz = 400

(C) N = 56 Kronecker products (D) N = 84 Kronecker products

F I G U R E 1 Block sparsity pattern of A for different values of N. A, N = 20 Kronecker products. B, N = 35 Kronecker products. C,
N = 56 Kronecker products. C, N = 84 Kronecker products [Colour figure can be viewed at wileyonlinelibrary.com]

variables d, the infinite polynomial chaos expansion of  automatically truncates itself as part of the SG method (see
Figure 1D). Hence, since the expansion truncates naturally, no error is incurred in the representation of .
To make this clearer, let us consider the case in which d = 3 and pu = 3. According to (17), the size of the stochastic
space for the approximation of u is P = 20. Now, the number of Kronecker products N can take the value 20 if the same
maximum polynomial order, p , is used for the expansion of the lognormal conductivity coefficient. Alternatively, maxi-
mum polynomial orders of 4, 5, or 6 can be used to give the number of Kronecker products corresponding to 35, 56, or 84,
respectively. Although any value of p can be used, it is only for p = 6 (p = 2pu ), which corresponds to N = 84, that a
full Galerkin projection of the lognormal random field is obtained. Furthermore, only in this case is the global Galerkin
matrix A guaranteed to be positive definite (see remark 2.3.4 of Ullmann4 ).
Figure 1 illustrates the block sparsity pattern of A for different values of N. Note that if polynomials of maximum order
p = 6 are used for the chaos expansion of the permeability tensor, then every block of A is non-zero (see Figure 1D). If
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1635

polynomials of order higher than six are considered, the matrices Gk corresponding to orders higher than 2P will have
only zero entries.

5.2 Implementation and solution strategies

The dimension of the global coefficient matrix A grows rapidly with p and d, which means that it is never completely
assembled due to memory constraints. As originally observed by Ghanem and Kruger,7 it is necessary to store d + 1
matrices of size Nu × Nu corresponding to each Kk , k = 0, … , d, in (26) and the non-zero entries of the stochastic matrices
Gk .
The discrete linear system (20) is solved using the preconditioned conjugate gradient (PCG) method. The first pre-
conditioners that were developed for the iterative solution of this system were based on incomplete factorizations of the
diagonal blocks of A.7,8 We define the block-diagonal preconditioner bdiag and the mean preconditioner mean by

bdiag = G0 ⊗ K0 , mean = I ⊗ K0 , (28)

respectively. Each PCG iteration involves the solution of a system of the form z = r, where r is the residual vector. This
requires the solution of P subsystems of equations each of size Nu × Nu with coefficient matrix K0 .
Powell and Elman9 claimed that any efficient deterministic solver can be used for the solution of the P subsystems
and proposed the use of one V-cycle of black-box AMG. The crucial advantage of using black-box AMG is that the
computational cost of one V-cycle is linearly proportional to the discretization parameter h.
It is observed that, when Gaussian random variables are employed, the preconditioned system is positive definite
only when the variance and the order of the polynomials are small.9 This is a consequence of the infinite support of
the Gaussian distribution and the violation of condition (5) for the permeability tensor. Preconditioned CG breaks down
when this condition is violated. Therefore, the use of Hermite polynomials is limited to problems with small variances.
For the SFEM method to be computationally efficient and competitive with respect to traditional sampling methods,
the CG method needs to be equipped with robust preconditioners that are optimal with respect to h, d, p and especially
. It is well known that the performance of preconditioners deteriorates for problems in which  has a large standard
deviation. This is due to the fact that the off-diagonal blocks of A, which are usually ignored when constructing these
preconditioners, become increasingly significant in this case.
To overcome this deficiency, a new preconditioner that fully exploits the block structure of A is proposed. This precon-
ditioner ensures that important information contained in the off-diagonal blocks is captured when  has a large standard
deviation. This is achieved through the addition of an inner iteration to the preconditioning operation which essentially
implements a full inversion of the global stiffness matrix A using a bSGS algorithm. The preconditioner, which to our
knowledge has not been used in the SFEM context before, takes the form,

∑
d
bSGS = G0 ⊗ K0 + Gk ⊗ Kk . (29)
k=1

Note, however, that the preconditioner bSGS is neither assembled nor inverted directly. Each inner iteration requires
the solution of P subproblems of size Nu × Nu using an appropriate fast solver such as UMFPACK or one V-cycle of AMG
for the deterministic problem under consideration. The symmetry of the preconditioner is guaranteed by performing two
iterative sweeps (forward and backward).
Two stopping criteria are used for the proposed algorithm. The inner iteration is terminated when

‖ k ‖
‖z − zk−1 ‖ < 𝜖,
‖ ‖∞

where 𝜖 = 10−8 unless a maximum number of iterations maxitb is achieved. At this stage, the current approximation for z
is chosen to be the preconditioned residual needed within the current CG iteration. In general, the CG algorithm should
decrease the number of CG iterations and, in particular, it should improve the iteration count for those problems for
which the off-diagonal blocks of A are as significant as the diagonal blocks, that is, problems in which the spatial random
field has a large standard deviation.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1636 TRAVERSO and PHILLIPS

6 STOCHAST IC MIXED FINITE ELEMENT APPROX IMATION (SMFEM)

The approach to SMFEM for solving the mixed variational problem (6) is similar to the one presented in
the previous section. However, the mixed finite element approximation requires the definition of subspaces for
H(div; D) in addition to L2 (D). We consider the Raviart-Thomas space of lowest order RT0 as a suitable space
for the approximation of the velocity solution and M0 (K) is defined to be the space of piecewise constant
functions.
The stochastic space L2 (Ω) is discretized by means of polynomial chaos. The spaces for the stochastic approximation
are consequently given by Vh = Yh ⊗ Th ⊂ V = H(div; D) ⊗ L2 (Ω) and Wh = Xh ⊗ Th ⊂ W = L2 (D) ⊗ L2 (Ω).
The discrete variational formulation of (9) is: find qh ∈ Vh and uh ∈ Wh such that

⟨a(qh , vh )⟩ + ⟨b(vh , uh )⟩ =⟨(g, n ⋅ v)ΓD ⟩, ∀vh ∈ V h

⟨b(qh , wh )⟩ = − ⟨(f , wh )⟩. ∀wh ∈ W h . (30)

6.1 Linear system

The potential uh and flux (or velocity) qh are expressed in terms of the expansions

N
∑ ∑
P Ne
∑
P
∑
edge

u (x, 𝜉) =
h
us,r 𝜙r (x)𝜒s (𝜉), q (x, 𝜉) =
h
qs,r 𝜓r (x)𝜒s (𝜉). (31)
s=1 r=1 s=1 r=1

Substituting for uh and qh using expansions (31) into (30), we obtain the discrete linear system
( )( ) ( )
A BT q g
B 0 u = f , (32)

where A is a sparse symmetric matrix of size Nedge P × Nedge P and B is a sparse block diagonal matrix of size Ne P × Nedge P.
The structure of C where
( )
A BT
C= , (33)
B 0

is governed by the coefficients of the polynomial chaos expansion, while the sparsity of each of the blocks of C corresponds
to the sparsity of the deterministic velocity matrix and the deterministic divergence operator.
The diagonal blocks of A are given by (22) with

1
(K0 )r,s = 𝜓 (x)𝜓s (x)dx, (34)
∫D 𝜇 r

while the off-diagonal blocks are given by (24) with

√
(Kl )r,s = 𝜎 𝜆l 𝛽l (x)𝜓r (x)𝜓s (x)dx. (35)
∫D

The diagonal blocks of B are defined by

Bi,i = ⟨𝜒i ⟩2 ⊗ B0 , (36)

where

(B0 )r,s = 𝜙r (x)∇ ⋅ 𝜓s (x)dx. (37)

∫D
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1637

When the permeability tensor is approximated in terms of a polynomial chaos basis of order p, A reduces to size
Nedge p × Nedge p and is tridiagonal and B reduces to size Ne p × Nedge p. The diagonal blocks of A are given by (22), while the
off-diagonal blocks have the form

Ai,j = ⟨kk 𝜒ik 𝜒jk ⟩ ⊗ K, (38)

where

Kr,s = 𝜓r (x)𝜓s (x)dx. (39)

∫D

6.2 Implementation and solution strategies

As for the SFEM method, the coefficient matrix C is never assembled. In addition to storing d + 1 matrices of size Nedge ×
Nedge and the entries of the stochastic matrices Gk , we also store the matrix B0 of size Ne × Nedge .
The efficient solution of the saddle-point system (32) is an active field of research.10,28,29 The approach adopted
here follows the experience gained from solving the deterministic system. The system is solved using MINRES with a
preconditioner based on the approximation of the Schur complement by sparse direct or AMG methods.
For the stochastic system, we use a preconditioner that follows from its deterministic version given by
( )
Ñ0
= , (40)
0 Ṽ

where
Ñ = diag(A) = G0 ⊗ diag(K0 ), (41)

and
[ ]
Ṽ = BÑ BT = G0 ⊗ B0 diag(K0 )−1 BT0 .
−1
(42)

Following the discussion for the second-order problem in Section 5.2, the preconditioner (40) is expected to be efficient
only for problems in which the spatial random field is characterized by small or moderate standard deviation. A robust
preconditioner with respect to 𝜎, for the mixed stochastic formulation remains an active area of research.

7 CO MPARISON OF SMFEM AND MCM

In this section, the SMFEM method is compared with MCM when lognormal distributions are used to describe the per-
meability tensor. The aim is to validate SMFEM against the established MCM approach. The simulations have all been
carried out in serial using MATLAB 7.4 on a laptop PC with 4GB of RAM.
Consider test problem 2 defined in Part 1. This problem, defined in the computational domain [0, 1]2 , is similar to one
considered by Powell and Elman.9 Dirichlet conditions u(0, y) = 1 and u(1, y) = 0 are imposed on the vertical boundaries
and homogeneous Neumann conditions ∇u ⋅ n = 0 are imposed on the horizontal boundaries. Thus, the dominant flow
direction is from left to right. The first order problem is solved in this section corresponding to the system of equations
described in (6).
The spatial discretization uses a triangular mesh with h = 1∕64, thus the number of unknowns given by the mixed
formulation is the sum of the number of elements, Ne = 8192, and number of edges, Nedg = 12 416. The stochastic space
is discretized in a similar fashion to the one described for the SFEM case, that is, polynomial chaos up to order pu = 4 are
used for both u and q.
Solution profiles for various values of pu and Nr are shown in Figure 2. For the mean velocity (x-component) solution,
the profile presented is along y = 0.5 and for the variance solution is along x = 0.5. An in-depth convergence study for a
sampling point having coordinate (0.5, 0.5) is reported in Table 1.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1638 TRAVERSO and PHILLIPS

1 0.315
0.296 0.296
0.296 0.298
0.298
0.9 0.298 0.3
0.3 0.31
0.8
0.3
0.7 0.305
0.3

Variance of qx solution
0.298
0.6 0.298
0.298
0.3
0.5
0.298
0.295
0.4 0.298
0.298
0.3 0.3
0.3
0.29 MCM (N = 10,000)
r
MCM (Nr = 20,000)
0.2 MCM (Nr = 40,000)
0.3 0.285 SMFEM (p = 2)
u
0.1 0.298 0.3
0.298 SMFEM (pu = 3)
0.298
0.296 SMFEM (pu = 4)
0.296 0.296
0 0.28
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Y-direction

(A) Variance qx solution (B) Solution profiles of variance qx solution

F I G U R E 2 Variance solutions for test problem with lognormal distribution (SMFEM). A, Variance qx solution. B, Solution profiles of
variance qx solution [Colour figure can be viewed at wileyonlinelibrary.com]

T A B L E 1 Convergence analysis of MCM and

Nr = 10 000 Nr = 20 000 Nr = 40 000
SMFEM for test problem with lognormal
qx Sample mean 2.77182 2.77526 2.77055 distribution
Sample variance 0.30558 0.30873 0.30014
tCPU (sec) 6261 12 524 25 048
pu = 2 pu = 3 pu = 4
qx Mean 2.76780 2.76782 2.76782
Variance 0.29639 0.29726 0.29728
tCPU (sec) 44 388 2312

Abbreviations: MCM, Monte Carlo method; SMFEM, stochastic mixed finite element method.

Polynomials of order two are sufficient to achieve convergence to the fourth significant digit for the mean whereas
polynomials of order three are required to achieve the same level of accuracy for the variance. This is in agreement with
the convergence rate of the mean and variance solution for the potential recorded for the second order problem.
It is apparent from the data in Table 1 that the Monte Carlo mean for the x-component of the velocity field does not
converge for the sample sizes considered. This suggests that a larger sample is required to achieve the desired level of
accuracy. Equally, the variance does not converge for the maximum sample size herein considered.
Table 1 also includes solution timings for MCM and SMFEM. Although the data show that SMFEM is more efficient
than MCM, this conclusion cannot be generalized. In fact, the performance of preconditioned MINRES deteriorates sig-
nificantly for conductivity fields with large standard deviations when lognormal distributions are used. Evidence of this
is reported and discussed in Section 9.1.

8 ITERATIVE SOLVERS FO R S FEM

The algorithms used for the numerical experiments are structured so that the coefficient matrix is never fully assem-
bled. Only the non-zero entries of the polynomial chaos coefficients (appropriately indexed) and N matrices (associated
with the polynomial chaos discretization of the spatial random field) are stored. However, in order to guarantee the
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1639

T A B L E 2 Dimensions of P, N, and total number of

pu = 2 pu = 3 pu = 4
unknowns
d=4
P 15 35 70
N 70 210 495
1
h= 32
16 335 38 115 76 230
1
h= 64
63 375 147 875 295 750
1
h= 128
249 615 582 435 1 164 870
d=6
P 28 84 210
N 210 924 3,003
1
h= 32
30 492 91 476 228 690
1
h= 64
118 300 354 900 887 250
1
h= 128
465 948 1 397 844 3 494 610

well-posedness of A, N has to be very large. Hence, the memory requirements of SG for the stochastically nonlinear case
are significantly larger than the corresponding linear case.
In Part 1, it was shown that the version of the mean-based preconditioners, bdiag and mean , based on an incomplete
Choleski factorization of K0 was significantly less efficient than the AMG and UMFPACK versions. Hence, only the latter
two versions are considered. A symmetric Gauss-Seidel smoother is used for the AMG implementation. The simulations
have been performed in serial within the MATLAB environment installed on the SRIF-3 Cluster machine (Merlin) at
Cardiff University. Thus, the CPU timings reported in the following sections can be compared directly with those reported
in Part 1.

8.1 Block-diagonal preconditioner

8.1.1 Test problem 1—variable 𝝈

This test problem is described in Section 7. The conductivity coefficient  is a lognormal field, the spatial variability of
which is described in Section 7. The underlying Gaussian distribution has constant mean 𝜇 = 1 and four different values
of the standard deviation are considered. The discretization parameter is fixed at h = 1∕32.
The size of the stochastic space and the total number of Kronecker products are as those reported in Table 2 and the
total number of unknowns corresponds to those for h = 1∕32 in Table 2.
The number of conjugate gradient iterations, preconditioned by bdiag , Nit , and tCPU are reported in Table 3. Set-up
times depend only on the size of the problem (which in this case corresponds to h = 1∕32), and therefore, they are
approximately equal for all values of 𝜎.
The results presented in Table 3 can be summarized as follows:
1. The performance of the block-diagonal preconditioner deteriorates significantly for large values of standard deviation;
2. Nit and tCPU show exponential growth with respect to pu , for both values of d;
3. There is a slight deterioration in d-optimality for pu = 4 for both versions of the preconditioner;

8.1.2 Test problem 2—discontinuous-isotropic conductivity field

In this test problem, the domain D = [0, 1]2 is partitioned into four equal subdomains. Dirichlet conditions are imposed on
the vertical sides ΓD = {0, 1} × [0, 1]. Homogeneous Neumann boundary conditions are imposed on the horizontal bound-
aries. The conductivity coefficient  is a spatially discontinuous lognormal random field. Four cases are presented, three
of which have constant coefficient of variation 𝛿 and one with spatially variable 𝛿. The underlying Gaussian distributions
(one for each of the four sub-domains) are characterized by the following parameters:
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1640 TRAVERSO and PHILLIPS

T A B L E 3 CG iterations and solution

pu = 2 pu = 3 pu = 4
timings for bdiag —Test Problem 1
tCPU tCPU tCPU
𝝈 Nit (sec) Nit (sec) Nit (sec)

d=4
UMFPACK 0.3 11 0.7 14 4.73 17 28.66
0.5 16 0.96 23 7.83 30 50.05
0.7 23 1.38 34 11.52 48 80.09
0.9 29 1.74 51 17.38 80 134.01
AMG 0.3 13 1.32 16 5.76 19 32.56
0.5 18 1.23 25 8.98 32 54.67
0.7 24 1.69 37 13.3 52 88.93
0.9 31 2.1 54 19.33 83 142.59
d=6
UMFPACK 0.3 11 1.97 14 25.48 17 239.24
0.5 17 3 24 43.65 30 419.48
0.7 23 4.05 35 63.48 50 697.9
0.9 30 5.27 52 94.26 83 1158.83
AMG 0.3 13 2.51 16 29.93 19 268.52
0.5 18 3.5 25 46.61 33 468.54
0.7 24 4.6 38 70.83 52 735.08
0.9 31 5.92 55 102.78 87 1239.99

1. Case 1:

𝜇D1 = 1.0, 𝜎D1 = 0.5, 𝜇D2 = 0.1, 𝜎D2 = 0.05,

𝜇D3 = 0.01, 𝜎D3 = 0.005, 𝜇D4 = 0.0001, 𝜎D4 = 0.00005;

2. Case 2:
𝜇D1 = 1.0, 𝜎D1 = 0.7, 𝜇D2 = 0.1, 𝜎D2 = 0.07,

𝜇D3 = 0.01, 𝜎D3 = 0.007, 𝜇D4 = 0.0001, 𝜎D4 = 0.00007;

3. Case 3:
𝜇D1 = 1.0, 𝜎D1 = 1.0, 𝜇D2 = 0.1, 𝜎D2 = 0.1,

𝜇D3 = 0.01, 𝜎D3 = 0.01, 𝜇D4 = 0.0001, 𝜎D4 = 0.0001;

4. Case 4:
𝜇D1 = 1.0, 𝜎D1 = 1.0, 𝜇D2 = 0.1, 𝜎D2 = 0.07,

𝜇D3 = 0.01, 𝜎D3 = 0.005, 𝜇D4 = 0.0001, 𝜎D4 = 0.0001.

A KLE is performed for each subdomain and the number d of terms retained in the expansion is the same for each
subdomain. The case in which different values of d are used in each subdomain has not been considered and is a subject
for further study. The same spatial model with lx = ly = 0.5 is used for each subdomain. The discretization parameter is
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1641

T A B L E 4 CG iterations
pu = 2 pu = 3 pu = 4
and solution timings for
bdiag —Test Problem 2 tCPU tCPU tCPU
𝝈
𝜹= 𝝁
Nit (sec) Nit (sec) Nit (sec)

d=4
UMFPACK 0.5 16 1.02 22 7.47 27 45.3
0.7 22 1.33 32 10.92 43 72.27
1.0 33 1.99 56 19.21 84 141.2
1.0,0.7,0.5,1.0 33 1.99 55 18.73 84 140.98
AMG 0.5 17 1.21 23 8.28 28 48.07
0.7 23 1.57 34 12.19 45 77.22
1.0 34 2.32 57 20.43 86 147.55
1.0,0.7,0.5,1.0 34 2.31 57 20.5 86 147.41
d=6
UMFPACK 0.5 17 3 23 41.73 28 392.59
0.7 23 4.08 34 61.76 45 633.19
1.0 35 6.19 58 105.52 90 1265.67
1.0,0.7,0.5,1.0 35 6.16 58 105.88 90 1261.79
AMG 0.5 18 3.48 24 44.69 29 420.91
0.7 24 4.61 35 65.76 47 670.45
1.0 36 6.9 61 113.85 93 1313.96
1.0,0.7,0.5,1.0 36 6.88 60 111.57 93 1320.7

fixed, h = 1∕32, and the size of the problem is given in Table 2. Iteration count and timings for CG preconditioned with
the block-diagonal of A are given in Table 4. Also, for this test problem, the set-up times are approximately equal for all
values of 𝛿.
Similar observations are drawn for this test problem as the ones highlighted in Section 8.1.1. The exponential growth
of Nit and tCPU with increasing p is also observed for this test problem. In addition, it appears that the deterioration in
the performance of the preconditioner is exclusively due to the increase in the value of 𝜎 in each subdomain from case to
case. In fact, in all four cases, the mean values 𝜇D1 ,D2 ,D3 ,D4 are equal. This indicates that the preconditioned solver is robust
with respect to discontinuities in the mean value of the permeability.

8.2 Block symmetric Gauss-Seidel preconditioner

In this section, the same test problems presented in Section 8.1 are solved using a bSGS preconditioner for CG. The
algorithm used in the experiments is described in Section 5.2. A fixed number of iterations, maxitb, is used as stopping
criteria for bSGS , each iteration comprising a forward and backward sweep. The experiments reported in the following
sections are performed using maxitb = 1. The reason for this choice together with an in-depth analysis on the performance
of bSGS for several values of maxitb is given in Section 8.2.3.
The UMFPACK implementation of the bSGS preconditioner is straightforward and it is identical to the one used in
Part 1 for the linear case. In contrast, the AMG implementation requires additional pre-processing to be implemented.
In fact, in contrast to the linear case, the tensor products Gk ⊗ Kk , k = 1, … , N, have several contributions to the blocks
of the leading diagonal of the coefficient matrix, depending on the value of d and pu . So, for example, fixing d = 2 and
pu = 3, the contributions are shown in Table 5.
Note that the matrix G1 is diagonal in that the product G1 × K1 contains the mean information (this is omit-
ted from Table 5). In the nonlinear case the AMG grids have to be computed for each diagonal block entries of the
global system. Thus, the AMG pre-processing is implemented P times and the grids are stored before the iterative
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1642 TRAVERSO and PHILLIPS

T A B L E 5 Gk × Kk contributions to the blocks of

(i, j) Gk × Kk
the diagonal of A
(1, 1)
(2, 2) G4 (2, 2) × K4
(3, 3) G6 × K6
(4, 4) (G4 × K4 ) + (G11 × K11 )
(5, 5) (G4 × K4 ) + (G6 × K6 ) + (G13 × K13 )
(6, 6) (G6 × K6 ) + (G15 × K15 )
(7, 7) (G4 × K4 ) + (G11 × K11 ) + (G22 × K22 )
(8, 8) (G4 × K4 ) + (G6 × K6 ) + (G11 × K11 ) + (G13 × K13 ) + (G24 × K24 )
(9, 9) (G4 × K4 ) + (G6 × K6 ) + (G13 × K13 ) + (G15 × K15 )
(10, 10) (G6 × K6 ) + (G15 × K15 ) + (G26 × K26 ) + (G28 × K28 )

T A B L E 6 CG iterations and solution

pu = 2 pu = 3 pu = 4
timings for bSGS —Test Problem 1
tCPU Nit tCPU Nit tCPU
𝝈 Nit (sec) (sec) (sec)

d=4
UMFPACK 0.3 4 0.87 4 2.44 5 14.95
0.5 6 0.65 7 4.28 8 23.9
0.7 8 0.86 10 6.11 12 35.8
0.9 10 1.07 14 8.53 18 53.73
AMG 0.3 6 1.11 7 4.35 7 20.82
0.5 8 0.94 9 5.6 11 32.63
0.7 10 1.21 14 8.71 20 59.53
0.9 14 1.64 23 14.27 46 136.93
d=6
UMFPACK 0.3 4 1.27 4 13.11 5 123.46
0.5 6 1.91 7 22.9 8 198.15
0.7 8 2.55 10 32.62 12 297.21
0.9 11 3.51 14 45.85 19 473.32
AMG 0.3 6 2 7 22.94 7 171.93
0.5 8 2.68 10 32.71 12 297.72
0.7 11 3.66 15 49.01 22 546.12
0.9 15 5 25 81.93 52 1279.58

solution process begins. Clearly, the preconditioner set-up time contributes significantly to the overall CPU cost of
the solver.

8.2.1 Test problem 1—variable 𝝈

Table 6 reports the number of CG iterations and CPU times for test problem 2.
The results presented in Table 6 can be summarized as follows:
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1643

T A B L E 7 CG iterations
pu = 2 pu = 3 pu = 4
and solution timings for
bSGS —Test Problem 2 tCPU tCPU tCPU
𝝈
𝜹= 𝝁
Nit (sec) Nit (sec) Nit (sec)

d=4
UMFPACK 0.5 6 0.69 7 4.27 7 20.86
0.7 8 0.86 9 5.54 11 33.04
1.0 11 1.18 15 9.23 19 56.64
1.0,0.7,0.5,1.0 11 1.19 15 9.22 19 56.73
AMG 0.5 7 0.91 8 4.97 10 29.7
0.7 9 1.06 12 7.46 16 47.59
1.0 14 1.67 23 14.43 48 143.13
1.0,0.7,0.5,1.0 14 1.67 23 14.42 48 143.13
d=6
UMFPACK 0.5 6 1.91 7 22.82 8 198.34
0.7 8 2.55 9 29.49 11 273.95
1.0 11 3.5 15 49.18 21 520.41
1.0,0.7,0.5,1.0 11 3.51 15 49.08 21 523.93
AMG 0.5 8 2.68 9 29.45 10 245.42
0.7 10 3.38 13 42.49 18 440.4
1.0 16 5.41 27 89.2 55 1348.74
1.0,0.7,0.5,1.0 16 5.39 27 88.5 55 1345.02

1. The block symmetric Gauss-Seidel preconditioner displays a significant improvement in terms of number of CG
iterations. This improvement becomes more evident for large values of 𝜎;
2. The comparison of the data with those of Table 3 (block-diagonal preconditioner) reveals that the Gauss-Seidel pre-
conditioner is generally computationally cheaper and the improvement in performance increases with larger values
of 𝜎;
3. The difference in performance between the exact and approximate versions of the preconditioner increases
for larger 𝜎. In fact, for 𝜎 = 0.9, the AMG solution times are approximately three times larger than for
UMFPACK.

8.2.2 Test problem 2—discontinuous-isotropic conductivity field.

Table 7 reports the number of CG iterations count and CPU times for test problem 3 using a bSGS preconditioner.
As for test problem 2, a significant improvement in both Nit and tCPU is achieved. A large saving in computa-
tional cost was recorded for higher polynomial orders and large 𝛿. In fact, the CPU time is reduced by 60% com-
pared with results obtained using the bdiag preconditioner. Significant time reduction is equally achieved for lower
polynomial orders and coefficient of variation 𝛿. This is around 37% for pu = 2 and 𝛿 = 0.5, and around 53% for
pu = 3 and 𝛿 = 0.7.
As observed previously for the preconditioner bdiag , it appears that a discontinuous conductivity coefficient
(jumps in the mean conductivity value at the subdomain boundaries) does not worsen the preconditioner per-
formance. It is, in fact, the standard deviation that has a significant negative impact on the performance of
both bdiag and bSGS . Not even using the bSGS algorithm and therefore including the off-diagonal blocks of
A (which retain information on the fluctuations about the mean) can optimality of Nit with respect to 𝜎 be
achieved.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1644 TRAVERSO and PHILLIPS

T A B L E 8 CG iterations and
𝝈 = 0.3 𝝈 = 0.5 𝝈 = 0.7 𝝈 = 0.9
solution timings (sec) for bSGS for
d=4 maxitb Nit tCPU Nit tCPU Nit tCPU Nit tCPU various values of maxitb—Test
1 5 14.95 8 23.90 12 35.79 18 53.72 Problem 1

2 3 13.26 5 22.11 8 35.42 13 57.57

3 2 11.79 4 23.50 7 41.14 10 58.79
4 2 14.48 4 29.00 6 43.47 9 65.13
5 2 17.39 3 26.10 5 43.45 8 69.51
6 1 10.07 3 30.13 4 40.38 7 70.65
8 1 12.99 2 25.87 4 51.83 6 77.51
14 1 21.54 1 21.54 3 64.56 4 86.06
32 1 47.26 1 47.14 1 47.17 3 141.50
75 1 108.45 1 108.56 1 108.29 1 108.17
d=6
1 5 123.46 8 198.15 12 297.21 19 473.32
2 3 108.02 5 180.14 9 322.50 14 505.53
3 2 93.71 4 188.48 7 329.93 11 518.46
4 2 115.72 4 233.33 6 348.50 10 580.29
5 2 138.72 3 208.24 5 347.25 8 557.42
6 1 80.55 3 244.95 5 400.84 8 645.87
8 1 102.93 2 205.71 4 410.97 7 719.04
14 1 169.66 1 170.22 3 509.89 5 849.04
34 1 393.42 1 394.49 1 392.42 3 1173.16
80 1 899.37 1 906.18 1 905.44 1 905.81

8.2.3 Performance analysis

The numerical experiments presented show that CG equipped with a bSGS preconditioner is significantly more efficient
than traditional mean-based preconditioners. This conclusion depends on the stopping criterion chosen for the inner
preconditioned Gauss-Seidel algorithm. The results reported in the tables are obtained using a single iteration of the bSGS
algorithm.
The choice of maxitb can be optimized. Consider test problem 2 with fixed pu = 4. Simulations are performed increas-
ing the value of maxitb until only one CG iteration is required for convergence. CG iteration count and timings for these
experiments with d = 4 and d = 6 are reported in Table 8. Note that for this analysis, the UMFPACK version of the
preconditioner was used.
The results reported in Table 8 suggest that the best solution times are not given by the same stopping criterion for
all values of 𝜎. In fact, it appears that for large standard deviations 𝜎 = 0.7, 0.9, the best solution timings are obtained for
small maxitb. However, for 𝜎 = 0.3, 0.5, the best performance in terms of CPU time is obtained using moderate values of
maxitb.
The case in which maxitb is large and tCPU is low corresponds to the situation in which convergence is
obtained in one CG iteration. It is clear that, under these circumstances, the bulk of the computational work is
performed by the preconditioner (bSGS) and very little by the main solver (CG). Given that the preconditioner
should only serve as a means to improve the conditioning of the system matrix, the results showing just one
CG iteration are not considered in the following analysis. On the other hand, this aspect reveals that an inde-
pendent Gauss-Seidel (symmetric or not) solver could be a very efficient alternative to Krylov subspace iterative
schemes. In Section 8.3, results obtained using Gauss-Seidel solvers are reported for all test problems considered in
this paper.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1645

20 20
best fit line best fit line maxitb = 1
19 19
σ=0.3 maxitb = 1 σ=0.3
18 σ=0.5
18 σ=0.5
17 σ=0.7 17 σ=0.7
16 σ=0.9 16 σ=0.9

15 15
maxitb = 2
14 14
maxitb = 2
13 13

CG iterations, Nit
CG iterations, Nit

12 12
maxitb = 3
11 maxitb = 3 11
10 10 maxitb = 4
maxitb = 4
9 9
8 maxitb = 5 8
maxitb = 6
7 7
6 maxitb = 6 6
5 maxitb = 5
5
4 4
3 3
2 2
1 1

10 20 30 40 50 60 70 80 90 100 200 300 400 500 600 700 800

Solution time, t CP U (sec.) Solution time, t CP U (sec.)

(A) d = 4 (B) d = 6

F I G U R E 3 Performance analysis of CG preconditioned with bSGS for Test Problem 1. A, d = 4. B, d = 6 [Colour figure can be viewed
at wileyonlinelibrary.com]

Excluding the data associated with one CG iteration, Table 8 shows that, in general, a small number of internal
iterations for the bSGS preconditioner is sufficient to achieve the best performance for all values of standard deviation
considered for this test problem. However, it is only for 𝜎 = 0.9 (d = 4, 6) and 𝜎 = 0.7 (d = 6), that the best performance
is achieved using maxitb = 1. For 𝜎 = 0.5 (d = 4, 6) and 𝜎 = 0.7 (d = 4), the best performance is given by maxitb = 2, and
for 𝜎 = 0.3 (d = 4, 6), for maxitb = 3.
Figure 3A and B shows the number of CG iterations versus CPU times for maxitb = 1, 2, 3, 4, 5, 6 for d = 4 and d = 6,
respectively. The figures highlight that there is a clear linear relationship between the number of CG (preconditioned
with bSGS ) iterations, computational time, and the standard deviation of the spatial random field for all values of maxitb.
This figure clearly shows that the best convergence rate is given by maxitb = 1 and, for this reason, it was chosen as the
optimal stopping criterion for the bSGS preconditioner.

8.3 Gauss-Seidel solvers

The performance analysis carried out on test problem 2 in the previous section revealed that for small standard deviation
(𝜎 = 0.3 and 𝜎 = 0.5), the Gauss-Seidel algorithm used as a stand alone solver could be a valid alternative to Krylov sub-
space solvers for the solution of SFEM systems with lognormal permeability. The same observation was made for the linear
case in Part 1. Here results using the bSGS solver (bSGS) are presented. The circumstances under which Gauss-Seidel is
more efficient than Krylov subspace solvers are explained.
The symmetric Gauss-Seidel solver includes a forward and a backward sweep per iteration and the algorithm is essen-
tially the one used for bSGS . The nonsymmetric case only includes a forward sweep per iteration. In both cases, the
stopping criteria are determined by the error norm, satisfying a specific tolerance.
Reorderings of the block structure of A aimed at reducing the bandwidth of the coefficient matrix are irrelevant
for the lognormal case given that A is block dense. In our implementation, the structure as presented in Figure 1
is retained using the summation of progressive (i = 1, … , N) Kronecker terms. This ordering is the most natural as
it represents the summation of decreasing modes obtained from the polynomial chaos expansion of the permeability
(see Equation (14)).
The tolerance for the GS solvers is set to 10−8 . In each table, we list iteration count Nit and solution
times tCPU for both bSGS and bGS. Only experiments using UMFPACK to invert the diagonal blocks of A are
reported.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1646 TRAVERSO and PHILLIPS

T A B L E 9 bSGS and bGS iterations and

pu = 2 pu = 3 pu = 4
solution timings—Test Problem 2
tCPU tCPU tCPU
𝝈 Nit (sec) Nit (sec) Nit (sec)

d=4
bSGS 0.3 8 0.58 9 2.91 9 12.6
0.5 13 0.92 16 5.2 19 26.72
0.7 21 1.47 30 9.68 41 57.33
0.9 33 2.31 54 17.45 86 120.64
bGS 0.3 9 0.32 11 1.77 12 8.38
0.5 15 0.53 20 3.23 25 17.54
0.7 23 0.82 36 5.83 53 37.15
0.9 35 1.25 65 10.42 110 77.24
d=6
bSGS 0.3 8 1.48 9 14.06 9 99.5
0.5 13 2.4 17 26.6 20 220.36
0.7 22 4.06 31 48.47 43 470.07
0.9 35 6.45 59 92.13 96 1057.31
bGS 0.3 9 0.84 11 8.61 12 66.24
0.5 15 1.39 20 15.66 25 138.46
0.7 23 2.12 36 28.1 53 291.58
0.9 35 3.23 65 50.82 111 610.59

8.3.1 Test Problem 1—variable 𝝈

Table 9 lists iteration counts and timings for test problem 1. The findings of this table are summarized as follows:
1. GS solvers are not optimal with respect to 𝜎;
2. bSGS is computationally more efficient than CG preconditioned with bSGS only for small standard deviations;
3. Nonsymmetric Gauss-Seidel solver (bGS) is very efficient for small and moderate standard deviations. However, for
large values of 𝜎 it is outperformed by CG preconditioned with bSGS ;
4. As for the previous case, the bGS solver is consistently more efficient than the symmetric implementation.

8.3.2 Test problem 2—discontinuous-isotropic conductivity field

Table 10 lists iteration count and timings for test problem 2. Similar observations to the ones highlighted for test
problem 2 are found from the data presented in this table. Furthermore, the results show that a discontinuous per-
meability has no negative impact on the performance of Gauss-Seidel solvers. This becomes evident if we compare
Nit for 𝛿 = 0.5 for this problem with that of test problem 2 (continuous permeability) for 𝜎 = 0.5 (which corresponds
to 𝛿 = 0.5).
The CPU time for the block Gauss-Seidel preconditioners increases quadratically with respect to mesh size h. There-
fore, on a mesh with h = 1∕64, it is predicted that the CPU times in Table 10, which are for a mesh with h = 1∕32, would
increase by a factor of four. So for test problem 3, in which the conductivity field is discontinuous, the predicted CPU
times on a mesh with h = 1∕64 with pu = 4 are 240 and 2120 seconds, respectively, for d = 4 and d = 6. If these times
are compared with the CPU times for MCM on the much simpler test problem 1 given in Table 1 (tCPU = 25 000 for
Nr = 40 000), then we see that the SMFEM with block GS preconditioner is at least an order of magnitude faster in terms of
CPU time.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1647

T A B L E 10 bSGS and bGS

pu = 2 pu = 3 pu = 4
iterations and solution timings—Test
Problem 2 tCPU tCPU tCPU
𝝈
𝜹= 𝝁
Nit (sec) Nit (sec) Nit (sec)

d=4
bSGS 0.5 11 0.78 13 4.2 14 19.64
0.7 17 1.2 23 7.41 29 40.68
1.0 32 2.25 53 17.09 84 117.66
1.0,0.7,0.5,1.0 32 2.24 53 17.07 84 118.49
bGS 0.5 12 0.43 15 2.43 17 11.97
0.7 17 0.6 25 4.07 33 23.32
1.0 30 1.06 53 8.59 87 61.46
1.0,0.7,0.5,1.0 30 1.07 53 8.58 87 61.11
d=6
bSGS 0.5 11 2.04 13 20.29 15 178.55
0.7 18 3.33 24 37.48 31 343.28
1.0 35 6.47 58 90.76 95 1048.38
1.0,0.7,0.5,1.0 35 6.45 58 90.82 95 1049.12
bGS 0.5 12 1.12 15 11.73 17 94.08
0.7 17 1.58 25 19.58 34 187.85
1.0 30 2.79 53 41.42 96 529.96
1.0,0.7,0.5,1.0 30 2.79 53 41.48 96 533.87

8.4 Comparison and conclusions

In this section, a large number of methods have been tested to identify the most efficient solver for the stochastic formu-
lation of the diffusion problem. To identify the most efficient and robust methods with respect to h, 𝜎 and discontinuous
𝜇, the data presented in the previous tables are summarized in Figures 4 and 5. Only the case p = 4 is considered for
d = 4, 6. The methods included in the figures are listed below.

1. CG with bdiag (AMG);

2. CG with bdiag (UMFPACK);
3. CG with mean (AMG);
4. CG with mean (UMFPACK);
5. CG with bSGS (UMFPACK);
6. bSGS (UMFPACK);
7. bGS (UMFPACK).

Note that for the AMG case, the time required to construct the grids and smoother for the approximation is added to
the solution times. The UMFPACK case does not require any set-up time.
Figures 4 and 5 show that the conjugate gradient solver preconditioned with bSGS is the most efficient method for
problems with medium/large standard deviation and discontinuous permeability. Gauss-Seidel solvers also perform well
in these circumstances and for small 𝜎 they are in fact the best-performing methods.
Mean-based preconditioners are, in general, not robust and efficient for SFEM with lognormal distributions. There is
very little difference in terms of performance between the AMG and UMFPACK versions of the preconditioner.
The outcome of this analysis reveals that CG preconditioned with bSGS is robust and efficient, performing well in
all settings considered in this article and therefore should generally be used for the solution of SFEM with lognormal
distributions. Gauss-Seidel solvers represent a viable alternative to Krylov subspace iterative methods.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1648 TRAVERSO and PHILLIPS

10
σ=0.3 σ=0.3
σ=0.5 σ=0.5
7 9
σ=0.7 σ=0.7
σ=0.9 σ=0.9
6 8

) (sec.)
) (sec.)

5 7

CPU
CPU

Solution time, log(t

Solution time, log(t

4 6

3 5

2 4

1 3

0 2
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Methods Methods

(A) d = 4 (B) d = 6

F I G U R E 4 Comparison of methods for the solution of SFEM for test problem 1. A, d = 4. B, d = 6 [Colour figure can be viewed at
wileyonlinelibrary.com]

8
δ=0.5 δ=0.5
δ=0.7 δ=0.7
7 9
δ=1.0 δ=1.0
δ=1.0,0.7,0.5,1.0 δ=1.0,0.7,0.5,1.0
8
6
Solution time, log(tCPU) (sec.)

) (sec.)

7
CPU

5
Solution time, log(t

6
4
5

3
4

2
3

1 2
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Methods Methods

(A) d = 4 (B) d = 6

F I G U R E 5 Comparison of methods for the solution of SFEM for test problem 2. A, d = 4. B, d = 6 [Colour figure can be viewed at
wileyonlinelibrary.com]

9 SMFEM SOLVERS

9.1 Schur complement preconditioner

This section reports the performance of preconditioned MINRES for the system derived for SMFEM (cf. Section 7). The
preconditioner used is the one described in Section 6.2. The Schur complement is computed exactly (using, eg, UMFPACK)
or approximated using one V-cycle of AMG code.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1649

T A B L E 11 MINRES iterations
pu = 2 pu = 3 pu = 4
and solution timings for
scomp —Test Problem 2 tCPU tCPU tCPU
𝝈 Nit (sec) Nit (sec) Nit (sec)

d=4
UMFPACK 0.3 62 7.32 75 49.05 86 330.55
0.5 92 10.42 127 83.82 171 660.2
0.7 136 15.71 226 149.58 346 1345.68
0.9 206 23.47 397 262.34 698 2710.47
AMG 0.3 67 7.91 80 52.42 93 357.58
0.5 97 11.13 136 88.92 182 700.29
0.7 145 16.73 238 157.69 368 1422.33
0.9 218 25.08 420 276.29 733 2846.35
d=6
UMFPACK 0.3 63 25.94 76 617.75 89 12 009.58
0.5 95 39.68 134 1079.71 180 24 329.95
0.7 143 59.29 237 1902.71 370 51 069.64
0.9 216 90.11 424 3418.43 753 103 277.29
AMG 0.3 67 27.6 81 659.43 94 13 927.52
0.5 100 41.21 141 1140.65 190 26 151.63
0.7 150 61.97 249 2007.98 385 53 101.14
0.9 227 94.27 440 3577.61 783 108 174.47

9.1.1 Test problem 1—variable 𝝈

The performance of the Schur complement preconditioner for varying 𝜎 is reported in Table 11. As for the previous case,
the set-up time for the preconditioner is performed only once.
The results reported in Table 11 can be summarized as follows:

1. MINRES performance deteriorates significantly for increasing values of 𝜎. This is in line with all methods con-
sidered in this article. However, for the nonlinear case, the use of SMFEM becomes impractical. In fact, the
experiments show that for 𝜎 = 0.9, it takes more than 30 hours to solve the stochastic linear system for a very coarse
1
discretization (h = 32 );
2. The performance of the AMG and UMFPACK versions of the Schur complement preconditioner is
similar.

9.1.2 Test problem 2—discontinuous-isotropic conductivity field

As has already been shown for other methods, the performance of the solver and preconditioners are not affected by
spatial discontinuities in the permeability. In fact, the timings reported in Table 12 are comparable to those reported for
the continuous test problem in Table 11.
Table 12 shows that the solver performance depends on the largest value of 𝛿 included in the domain. So, for example,
for the case of variable 𝛿 (different coefficients of variation for the four sub-domains), MINRES performance is fully
governed by the largest value of 𝛿, that is, 𝛿 = 1. In fact, the timings are almost equivalent to the case of constant 𝛿 = 1
for all sub-domains.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1650 TRAVERSO and PHILLIPS

T A B L E 12 MINRES iterations and

pu = 2 pu = 3 pu = 4
solution timings for scomp —Test
tCPU tCPU tCPU Problem 2
𝝈
𝜹= 𝝁
Nit (sec) Nit (sec) Nit (sec)

d=4
UMFPACK 0.5 88 10.12 122 80.44 161 620.97
0.7 130 14.88 211 139.07 316 1226.72
1.0 235 27.15 474 314.05 864 3338.6
1.0,0.7,0.5,1.0 235 27.03 473 314.9 864 3360.14
AMG 0.5 92 10.44 128 83.65 170 651.08
0.7 136 15.55 222 145.62 333 1294.75
1.0 247 28.29 499 329.87 898 3468.6
1.0,0.7,0.5,1.0 247 28.39 498 327.16 898 3484.89
d=6
UMFPACK 0.5 91 38.03 126 1013.49 170 23 547.75
0.7 135 56.16 223 1814.07 341 47 604.68
1.0 250 104.13 513 4173.31 940 129 394.1
1.0,0.7,0.5,1.0 250 104.89 513 4154.09 939 131 601.3
AMG 0.5 94 38.97 132 1062.56 177 24 152.55
0.7 141 58.36 231 1870.85 355 49 131.72
1.0 259 108.41 533 4334.59 984 136 578.2
1.0,0.7,0.5,1.0 259 107.6 532 4287.98 984 136 140.52

9.2 Conclusions

While it was concluded that the performance of MINRES equipped with the Schur complement preconditioner described
in (40) is acceptable for the solution of the stochastic mixed formulation in the linear case (see Part 1), the same cannot
be concluded for the nonlinear case. The experiments reported in Tables 11 and 12 show that the CPU cost is too large
1
(30 hours to solve test problem 2 on a coarse mesh, h = 32 ) for this method to be used with lognormal random fields.
It becomes apparent that for the nonlinear case that it is crucial to include information contained in the off-diagonal
blocks of the coefficient matrix into the preconditioned system. The Kronecker product preconditioner of Ullmann4
offers this possibility. Very recently, Powell and Ullmann11 extended its implementation to the nonlinear case achieving
a significant improvement in MINRES CPU cost. The authors also proposed H(div) preconditioning using augment-
ing schemes which, although being dependent on the choice of the augmentation parameter, seem to achieve very
promising results.

10 CO NC LU SION S

The performance of iterative methods for solving the linear systems of equations arising from stochastic finite element
methods for single phase fluid flow in porous media is investigated. The uncertainty in the permeability is modeled using
lognormal random fields and is characterized by its mean and standard deviation, which can vary spatially. In addition
to being optimal with respect to the number of random variables in the KLE, the order of the polynomial chaos, and
the finite element discretization parameter, the iterative solver is required to be optimal with respect to 𝜇 and 𝜎. The
mean and standard deviation may vary discontinuously throughout the domain. When the permeability possesses large
standard deviation, it may fail to satisfy the condition (5). However, the condition can be satisfied by transforming the
Gaussian random field into a lognormal one by projecting the KLE onto the polynomial chaos of order at most p.6,26 The
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TRAVERSO and PHILLIPS 1651

coefficient matrix arising from the discretization of (1) and (6) using lognormal random fields becomes block dense and
ill-conditioned, which makes the linear systems very difficult and expensive to solve.
The conjugate gradient method was used to solve the linear systems of equations arising from stochastic finite ele-
ment discretization of the primal formulation. Block diagonal, mean-based, and block Gauss-Seidel preconditioners were
implemented to accelerate the convergence of the CG method. Two versions of the block Gauss-Seidel preconditioner were
implemented—one based on a sparse direct solver and the other on AMG. For small values of 𝜎, the preconditioner based
on block Gauss-Seidel was found to be optimal with respect to h, d, and p. This preconditioner also considerably reduced
the number of CG iterations in comparison to the block diagonal preconditioner. For 𝜎 ≥ 0.5, this optimality is lost. How-
ever, the block Gauss-Seidel preconditioner still outperforms the block diagonal preconditioner and the improvement is
greater as the mesh is refined. Furthermore, the growth in the number of CG iterations is more gradual as 𝜎 increases
compared with the block diagonal preconditioner. The introduction of spatially varying random fields does not deterio-
rate the performance of the preconditioners. In other words, the performance is no worse than when a constant random
field corresponding to the largest standard deviation used in the discontinuous case is used. The outcome of this anal-
ysis reveals that the AMG version of the block Gauss-Seidel preconditioner is very efficient and the most robust solver
considered for the solution of SFEM linear systems (linear case). However, although CG with a block Gauss-Seidel pre-
conditioner performs better than with block-diagonal and mean preconditioners, these methods are always outperformed
by stand-alone Gauss-Seidel solvers when solving these systems.
For the mixed formulation, MINRES was used to solve the linear systems of equations arising from the SMFEM. A pre-
conditioner based on the Schur complement was used. For small values of 𝜎, optimal convergence behavior was obtained
with respect to h and d. There was a weak dependence on p. However, the performance of the preconditioner deteriorated
significantly for moderate to large values of 𝜎. As for the primal formulation, the introduction of discontinuous conduc-
tivity coefficients had little impact on the performance of the preconditioner. However, SMFEM becomes more expensive
than SFEM for this class of problems.
Although computationally more expensive than SFEM, the efficient solution of SMFEM is largely dependent on the
preconditioner used with the chosen iterative solver. The numerical experiments showed that the Schur complement
preconditioner is h-optimal not only when the complement is inverted exactly but also when it is inverted approximately
using one V-cycle of AMG.
The experiments also showed that the preconditioner is not robust with respect to the permeability. The precondi-
tioner that only uses the diagonal of the coefficient matrix A is robust when the permeability possesses small standard
deviations but generally inadequate for large standard deviations. In the latter case, in fact, the off-diagonal blocks of
A become significantly more important and these are not included in this preconditioner. This lack of robustness was
also encountered when using the mean-based preconditioner for the solution of linear systems obtained by SFEM. The
off-diagonal blocks of the coefficient matrix were successfully incorporated into a preconditioner by means of a symmet-
ric block Gauss-Seidel algorithm. Unfortunately, due to the structure (and specifically the presence of a zero-block) of the
coefficient matrix of the SMFEM system, the same approach cannot be used for the Schur complement preconditioner.
The efficient solution of discrete linear systems obtained from stochastic mixed formulations is currently a very active
research area. The Kronecker product preconditioner proposed by Ullmann4 significantly reduces MINRES iteration
counts. However, this does not always translate into improvements in CPU performance. In fact, it is shown that the Schur
complement preconditioner performs better (in terms of CPU cost) than the Kronecker preconditioner for test problems
in which the permeability possesses a large standard deviation.4 More recently, Ullmann and Powell30 introduced an
approximation to the Schur complement as preconditioner that is robust with respect to the discretization parameters
and only slightly sensitive to the statistical properties of the permeability.

ORCID
Timothy Nigel Phillips https://orcid.org/0000-0001-6455-1205

REFERENCES
1. Graham G, Kuo F, Nuyens D, Scheichl R, Sloan IH. Quasi-Monte Carlo methods for for elliptic PDEs with random coefficients and
applications. J Comput Phys. 2011;230:3668-3694.
2. Cliffe KA, Giles MB, Scheichl R, Teckentrup AL. Multilevel Monte Carlo methods and applications to elliptic PDEs with random
coefficients. Comput Visualiz Sci. 2011;14:3-15.
3. Babuška I, Tempone R, Zouraris GE. Galerkin finite element approximations of stochastic elliptic partial differential equations. SIAM
J Numer Anal. 2004;42(2):800-825.
 10970363, 2020, 11, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/fld.4842 by Cochrane Chile, Wiley Online Library on [19/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1652 TRAVERSO and PHILLIPS

4. Ullmann E. Solution strategies for stochastic finite element discretizations (PhD Thesis). Technische Universität Bergakademie Freiberg;
2008.
5. Ghanem RG, Spanos PD. Stochastic Finite Elements: A Spectral Approach. New York, NY: Springer-Verlag; 1991.
6. Ghanem RG, Spanos PD. Stochastic Finite Elements. Mineola, NY: Dover Publications Inc; 2003.
7. Ghanem RG, Kruger RM. Numerical solution of spectral stochastic finite element systems. Comput Methods Appl Mech Eng.
1996;129:289-303.
8. Pellissetti MF, Ghanem RG. Iterative solution of systems of linear equations arising in the context of stochastic finite elements. Adv Eng
Softw. 2000;313:607-616.
9. Powell CE, Elman CH. Block-diagonal preconditioning for spectral stochastic finite-element systems. IMA J Numer Anal.
2009;29(2):350-375.
10. Ernst OG, Powell CE, Silvester D, Ullmann E. Efficient solvers for a linear stochastic Galerkin mixed formulation of diffusion problems
with random data. SIAM J Sci Comput. 2009;31(2):1424-1447.
11. Powell C, Ullmann E. Preconditioning stochastic Galerkin saddle point systems. SIAM J Matrix Anal A. 2010;31:2813-2840.
12. Rossell E, Vandewalle S. Iterative solvers for the stochastic finite element method. SIAM J Sci Comp. 2010;32(1):372-397.
13. Brezzi F, Fortin M. Mixed and Hybrid Finite Element Methods. New York, NY: Springer-Verlag; 1991.
14. Traverso L, Phillips TN, Yang Y. Efficient stochastic finite element methods for flow in heterogeneous porous media. Part 1: Gaussian
conductivity coefficients. Int J Numer Meth Fluids. 2013;74:359-385.
15. Deb MK, Babuška I, Oden JT. Solution of stochastic partial differential equations using Galerkin finite element techniques. Comput
Methods Appl Mech Eng. 2001;190(48):6359-6372.
16. Matthies HG, Keese A. Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations. Comput Methods Appl
Mech Eng. 2005;194:1295-1331.
17. Lu Z, Zhang D. Stochastic simulations for flow in nonstationary randomly heterogeneous porous media using a KL-based
moment-equation approach. SIAM Multiscale Model Simul. 2007;6(1):228-245.
18. Xiu D, Karniadakis GE. Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Comput Methods Appl
Mech Eng. 2002;191:4927-4948.
19. Xiu D, Karniadakis GE. The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM J Sci Comput. 2002;24(2):619-644.
20. Constantine PG. Spectral methods for parameterized matrix equations (PhD Thesis). Stanford University; 2009.
21. Gelhar LW. Stochastic Subsurface Hydrology. Upper Saddle River, NJ: Prentice-Hall; 1983.
22. Rubin Y. Applied Stochastic Hydrogeology. Oxford, UK: Oxford University Press; 2003.
23. Freeze RA. A stochastic-conceptual analysis of one-dimensional groundwater flow in nonuniform homogeneous media. Water Resour Res.
1975;11(5):725-741.
24. Ghanem R. The nonlinear Gaussian spectrum of lognormal stochastic processes and variables. ASME J Appl Mech. 1999;66(4):964-973.
25. Ghanem R. Stochastic finite elements for heterogeneous media with multiple random non-gaussian properties. J Appl Mech T ASME.
1999;125(1):26-40.
26. Sudret B, Kiureghian AD. Stochastic finite element methods and reliability, a state-of-the-art-report. Technical Report
UCB/SEMM-2000/08, University of California; 2000.
27. Keese A. Numerical solution of systems with stochastic uncertainties a general purpose framework for stochastic finite elements (PhD
thesis). Technische Universität Braunschweig; 2004.
28. Furnival DG. Iterative methods for the stochastic diffusion problem (PhD Thesis). University of Maryland; 2008.
29. Elman HE, Furnival DG, Powell CE. H(div) preconditioning for a mixed finite element formulation of the diffusion problem with random
data. Math Comput. 2010;79(270):733-760.
30. Ullmann E, Powell CE. Solving log-transformed random diffusion problems by stochastic Galerkin mixed finite element methods.
SIAM/ASA J Uncert Quantif . 2015;3(1):509-534.

SUPPORTING INFORMATION
Additional supporting information may be found online in the Supporting Information section at the end of this article.

How to cite this article: Traverso L, Phillips TN. Efficient stochastic finite element methods for flow in
heterogeneous porous media. Part 2: Random lognormal permeability. Int J Numer Meth Fluids.
2020;92:1626–1652. https://doi.org/10.1002/fld.4842