A new numerical scheme for simulating non-gaussian

and non-stationary stochastic processes

arXiv:2008.04240v1 [physics.comp-ph] 10 Aug 2020

Zhibao Zhenga,b,∗, Hongzhe Daia,b , Yuyin Wanga,b , Wei Wanga,b

a
Key Lab of Structures Dynamic Behavior and Control, Harbin Institute of Technology,
Ministry of Education, Harbin 150090, China
b
School of Civil Engineering, Harbin Institute of Technology, Harbin 150090, China

Abstract

This paper presents a new numerical scheme for simulating stochastic pro-
cesses specified by their marginal distribution functions and covariance func-
tions. Stochastic samples are firstly generated to automatically satisfy target
marginal distribution functions. An iterative algorithm is proposed to match
the simulated covariance function of stochastic samples to the target covari-
ance function, and only a few times iterations can converge to a required
accuracy. Several explicit representations, based on Karhunen-Loève expan-
sion and Polynomial Chaos expansion, are further developed to represent
the obtained stochastic samples in series forms. Proposed methods can be
applied to non-gaussian and non-stationary stochastic processes, and three
examples illustrate their accuracies and efficiencies.
Keywords: Stochastic samples, Non-gaussian, Non-stationary,
Karhunen-Loève expansion, Polynomial Chaos expansion

∗
Corresponding author.
Email address: [email protected] (Zhibao Zheng)

Preprint submitted to Journal August 11, 2020

1. Introduction

With widely developments of uncertainty quantification theories and meth-

ods, stochastic problems involving uncertainties commonly arise in various
fields of engineering, such as computational mechanics [1], financial anal-
ysis [2] and biomedical science [3]. A large number of these problems in-
volves uncertain quantities which should be modeled as random processes or
fields. On the one hand, assumptions regarding probabilistic distributions
are made due to the incomplete experimental data [4]. On the other hand,
stochastic simulations are provided for sufficient observation data. Thus,
applications of stochastic process and field theories to engineering problems
have gained considerable interests. In general, stochastic processes are as-
sumed to be gaussian because of simplicity and the Central Limit Theorem.
Since the Gaussian stochastic processes can be completely described by their
second-order statistics, [5], methods for simulating gaussian stochastic pro-
cesses [6, 7, 8] have been quite well established. However, the gaussian as-
sumption does not work due to the fact that some physical phenomena are
obviously non-gaussian in some cases [9]. The difficulty for simulating non-
gaussian stochastic processes is that all the joint distribution functions are
needed to completely characterize the non-gaussian properties. The problem
is even further complicated when the process is also non-stationary since the
marginal distributions depend on time or space. With these motivations, the
efficient simulation of non-gaussian stochastic processes are urgent because
of practical and theoretical importance.
Spectral representation is the first widely developed method for the simu-
lation of non-gaussian stochastic processes [10]. This method is implemented

2
in frequency domain and is initially developed for gaussian stochastic pro-
cesses [11]. It has been extended to non-gaussian stochastic processes by com-
bining the spectral representation method with non-linear transformations
[10], i.e., tranforming gaussian stochastic samples generated by the spectral
representation method into the non-gaussian stochastic process and match-
ing the target power spectral density function and non-gaussian marginal dis-
tribution function. Extensive studies based on this method can be found in
[12, 13, 14, 15]. Different from the spectral representation method, Karhunen-
Loève (KL) expansion [1, 16] is implemented in time or space domains, which
is usually used in the simulation of stationary and non-stationary gaussian
processes [17, 18, 19, 20]. Iterative algorithms for updating the non-gaussian
expanded random variables are proposed in [21, 22] for the simulation of non-
gaussian stochastic processes. The method can be applied to highly skewed
non-gaussian marginal distribution functions. Hence, KL expansion provides
a unified and powerful framework for the simulation of stochastic processes,
which is potentially capable of providing a better fit to non-gaussian and
non-translational data [23]. Another important technique, Polynomial Chaos
(PC) expansion, has also been developed for simulation of non-gaussian and
non-stationary stochastic processes and fields in [24, 25]. The method rep-
resents the target stochastic process and field as multidimensional Hermite
polynomial chaos in a set of normalized gaussian random variables. The
accuracy and efficiency of this method were further examined in [26, 27].
In this paper, we present numerical schemes for simulating non-gaussian
and non-stationary stochastic processes that have been specified by their co-
variance functions and non-gaussian marginal distribution functions. The

3
basic idea is to firstly generate stochastic samples that satisfying target
marginal distribution functions, and then match target covariance functions
by developing delicate iterative algorithm. In this way, the simulation of
both gaussian and non-gaussian stochastic processes can be implemented
in an unified framework since marginal distribution functions are automati-
cally satisfied by generated samples, and the accuracy and efficiency of the
simulation are only dependent on matching the target covariance functions.
Another advantage is that there are no differences in the application of the
proposeed iterative algorithm to stationary and non-stationary stochastic
processes. Thus, the proposed method can be considered as a unified nu-
merical scheme for simulating samples of stochastic processes. Further, it’s
usually not convenient to apply stochastic samples in practical problems. In
this paper, we exploit KL expansion for expanding the obtained stochastic
samples since KL expansion is optimal among series expansion methods in
the global mean square error with respect to the number of random variables
in the representation. Thus, the proposed strategy is capable of representing
stochastic processes with sufficient accuracy with as few random variables as
possible. In order to meet the requirements of different practical problems, we
also exploit PC expansion and KL-PC expansion (combination of KL expan-
sion and PC expansion) to represent the obtained stochastic samples, whose
methodology are similar to KL expansion but based on different expansions.
The accuracies and efficiencies are demonstrated by several numerical ex-
amples. Proposed methods can be readily generalized to multi-dimensional
random fields [25, 28, 29, 30, 31], but it’s beyond the scope of this article and
will be studied in subsequent papers.

4
 The paper is organized as follows: a new algorithm for simulating stochas-
tic samples is presented in Section 2, Section 3 develops several numerical
algorithms for representing the obtained stochastic samples and three illus-
trative examples are finally given in Section 4 to demonstrate the proposed
algorithms.

2. Simulation of stochastic samples

Consider a stochastic process ω (x, θ), x ∈ D specified by its covari-

ance function C (x1 , x2 ) and marginal distribution function F (y; x). In or-
der to obtain stochastic samples that satisfying the target covariance func-
tion and marginal distribution function, we discretize spatial domain as
n on
x = {x1 , · · · , xn } and generate random variables samples {ηi (θk )}N
k=1
i=1
according to the target marginal distribution function F (y; x), where N is
the number of random variables and ηi (θ) = ω (xi , θ) , i = 1, · · · , n. Note
that random variables {ηi (θ)}ni=1 automatically satisfy the target marginal
distribution function, i.e., ηi (θ) ∼ F (y; xi ). However, the generated sam-
ples of random variables {ηi (θ)}ni=1 don’t match the target covariance func-
tions C (x1 , x2 ) since statistical correlations between samples {ηi (θk )}N
k=1 and

{ηj (θk )}N

k=1 are no constraints. Thus, delicate algorithm is required to be

developed to match the target covariance function C (x1 , x2 ).

Statistical correlations between random variable ηi (θ) and ηj (θ) are given
as
N
1 X
Tij = [ηi (θk ) − η̄i ] [ηj (θk ) − η̄j ] (1)
N − 1 k=1
where η̄i and η̄j are the mean of random variables ηi (θ) and ηj (θ), respec-

5
tively. Expanding Eq.(1) yields,
N
1 X
Tij = (ηi (θk ) ηj (θk ) − ηi (θk ) η̄j − η̄i ηj (θk ) + η̄i η̄j )
N − 1 k=1
" N N
! N
!
1 X X 1 X
= ηi (θk ) ηj (θk ) − ηi (θk ) ηj (θk )
N − 1 k=1 k=1
N k=1
N
! N ! N
! N
!#
1 X X 1 X 1 X
− ηi (θk ) ηj (θk ) + N ηi (θk ) ηj (θk )
N k=1 k=1
N k=1
N k=1
" N N
! N !#
1 X 1 X X
= ηi (θk ) ηj (θk ) − ηi (θk ) ηj (θk )
N − 1 k=1 N k=1 k=1
N N
! N !
1 X 1 X X
= ηi (θk ) ηj (θk ) − ηi (θk ) ηj (θk )
N − 1 k=1 N (N − 1) k=1 k=1

(2)

By introducing matrix Y and assembling random samples, we have

 
η (θ ) · · · ηn (θ1 )
i  1 1
.. ..
h 
Y = {η1 (θk )}N , · · · , {η (θ )} N
= (3)
 
k=1 n k k=1  . . 
 
η1 (θN ) · · · ηn (θN )

Then Eq.(2) can be rewritten in matrix

Y TY Y T UUT Y
T = − (4)
N − 1 N (N − 1)
where U = [1]N ×1 , and T is the simulated covariance matrix of random
n on
variables samples {ηi (θk )}N
k=1 .
i=1
Both target covariance matrix C and simulated covariance matrix T are
symmetric and positive definite, thus Cholesky decompositions can be used
to performed on matrix C and T as

C = PTP (5)

6
and
T = QT Q (6)

where Q and P are upper triangular matrices.

In order to match the target covariance matrix C, a new random samples
matrix Y 0 is introduced as

Y 0 = Y Q−1 P (7)

By the transformation in Eq.(7), simulated covariance matrix T 0 can match

the target covariance matrix C and proof as follows,

Proof. Simulated covariance matrix T 0 of the new random samples Y 0 is

0 Y 0T Y 0 Y 0T U U T Y 0
T = −
N −1 N (N − 1)
P Q Y Y Q−1 P
T −T T
P T Q−T Y T U U T Y Q−1 P
= −
N −1 N (N − 1)
 T T T

T −T Y Y Y UU Y
=P Q − Q−1 P
N − 1 N (N − 1)
= P T Q−T T Q−1 P

=C

It has to be noted that sample realizations {ηi (θk )}N

k=1 in each column

of Y 0 in Eq.(7) are different from those in each column of Y in Eq.(3) due

to the factor matrix Q−1 P . The realizations in each column of Y 0 change
with updated factor matrix Q−1 P and do not match the marginal distribu-
tion function F (y; xi ). A fact that re-ordering the sample realizations will

7
not change distributions of random variables but change the statistical cor-
relations, i.e., simulated covariance matrix, is enlightened. Hence, we use
the strategy that re-ordering the sample realizations {ηi (θk )}N
k=1 in each col-

umn of Y in Eq.(3) follows the ranking of the realizations in each column of

Y 0 in Eq.(7). The target covariance matrix C is matched by repeating the
procedure of Eq.(4), Eq.(6), Eq.(7) and re-ordering stochastic samples.
The resulting procedure for simulating stochastic process samples is sum-
marized in Algorithm 1 as follows,

Algorithm 1 Algorithm for simulating stochastic process samples

1: Discretize spatial domain x = {x1 , · · ·, xi , · · ·, xn } and generate random
variables samples {ηi (θk )}N
k=1 ∼ F (y; xi ) , i = 1, · · · , n.

2: Compute the upper triangular matrix P by use of a Cholesky decompo-

sition in Eq.(5).
3: repeat
4: Compute upper triangular matrix Q(k−1) by Eq.(6) based on simu-
lated covariance matrix T (k−1) obtained by Eq.(4).
5: Compute Y 0(k−1) through Eq.(7) and re-order samples in each column
of Y following the ranking of the realizations in each column of Y 0 .
6: Compute simulated covariance matrix T (k) by Eq.(4).

7: until T (k) − C kCk < ε

Algorithm 1 provides a simple and efficient framework to simulate stochas-

tic processes samples. Spatial discrete points and initial random variables
samples are generated in Step 1. The number of random variables is equal
to the that of spatial points and it’s not necessary to discretize excessive

8
spatial points. A Cholesky decomposition of target covariance matrix C is
performed in Step 2. The computational cost can be neglected since only one
time decomposition needs to be computed for matrix C. The Step 3 to Step
7 includes a loop iteration procedure to match the target covariance matrix
C, and the computational cost in these steps is low since only Cholesky de-
compositions and re-ordering samples are involved. The convergence error in
Step 7 can adopt 2-norm or infinite-norm (here same to 1-norm) and we adopt
2-norm in this paper. Note that, Algorithm 1 can be applied to non-gaussian
and non-stationary stochastic processes and can be readily generalized to
high-dimensional random fields.

3. Expansions of stochastic processes

Algorithm 1 provides an efficient procedure to simulate samples of stochas-

tic processes. However, sample-based descriptions of stochastic processes are
not suitable for subsequent applications in some cases [1, 4] and it’s necessary
to develop methods to represent the obtained samples of sochastic process.
In general, a stochastic process ω (x, θ) can be expressed as
∞
X
ω (x, θ) = ξi (θ) fi (x) (8)
i=0

where {ξi (θ)}∞ ∞

i=0 and {fi (x)}i=0 are a set of random variables and deter-

ministic functions, respectively. In practical, Eq.(8) can be truncated at the

term M as
M
X
ω (x, θ) = ξi (θ) fi (x) (9)
i=0

There exist three unknown ‘variables’ in Eq.(9), i.e., stochastic process

ω (x, θ), random variables {ξi (θ)}M M
i=0 and deterministic functions {fi (x)}i=0 .

9
Eq.(9) can be determined if any two variables are available. In this con-
text, only a set of random variables {ξi (θ)}M
i=0 or deterministic functions

{fi (x)}M
i=0 are required to be determined since samples of the stochastic

process ω (x, θ) have been obtained by Algorithm 1. Three stratigies are

developed to simulate stochastic processes through Eq.(9): (i). select a set
of deterministic basis {fi (x)}M M
i=0 , then compute random variables {ξi (θ)}i=0 ;

(ii). select a set of random variables {ξi (θ)}M

i=0 , then compute deterministic

functions {fi (x)}M M

i=0 ; (iii). select a set of deterministic basis {fi (x)}i=0 and

random variables {ξi (θ)}M

i=0 , then compute unknown projection coefficients.

We’ll develop corresponding algorithms based on above strategies in Section

3.1, Section 3.2 and Section 3.3, respectively.

3.1. Algorithm based on stochastic samples and KL expansion

In order to obtain deterministic basis {fi (x)}M

i=0 in Strategy (i), we adopt

Karhunen-Loève (KL) expansion due to its minimum mean-square error

property. The KL expansion is a special case of Eq.(8) and is based on a
spectral decomposition of the covariance function C (x1 , x2 ) of the stochastic
process ω (x, θ) with the form
M
X p
ω (x, θ) = ω̄ (x) + ξi (θ) λi fi (x) (10)
i=1

where ω̄ (x) is the mean function of the stochastic process ω (x, θ), M is the
number of terms of KL expansion and {ξi (θ)}M
i=1 is a set of uncorrelated

random variables with zero mean and unit variance, i.e.,

E {ξi (θ)} = 0, E {ξi (θ) ξj (θ)} = δij (11)

10
and given by
Z
1
ξi (θ) = √ [ω (x, θ) − ω̄ (x)] fi (x) dx (12)
λi D

where {λi } and {fi (x)} are the eigenvalues and eigenfunctions of the covari-
ance function C (x1 , x2 ), obtained from solving the following homogeneous
Fredholm integral equation of the second kind
Z
C (x1 , x2 ) fi (x1 ) dx1 = λi fi (x2 ) (13)
D

which satisfies Z
fi (x) fj (x) dx = δij (14)
D

The solution of Eq.(13) can be determined numerically for problems of

practical interests. It is known that, for fixed M , the resulting random pro-
cess approximation ω (x, θ) is optimal among series expansion methods with
respect to the global mean square error [1]. If the stochastic process ω (x, θ)
is gaussian, then {ξi (θ)}M
i=1 are independent standard gaussian random vari-

ables. But for non-gaussian stochastic processes, {ξi (θ)}M

i=1 are generally

non-gaussian and there are no general methods for determining them. In

order to determine their distributions, Eq.(12) has to be solved. Here we
adopt a sample-based method to compute the samples of random variables
{ξi (θ)}M
i=1 as
Z
1
ξi (θk ) = √ [ω (x, θk ) − ω̄ (x)] fi (x) dx, k = 1, · · · , N (15)
λi D

which needs N times deterministic integral and has very low computational
costs. The above method is summarized in Algorithm 2 as

11
Algorithm 2 Algorithm based on stochastic samples and KL expansion
1: Generate samples of the stochastic process by use of Algorithm 1.

2: Solve a set of deterministic basis {fi (x)}M by Eq.(13).

n i=1 oM
N
3: Compute random variables’ samples {ξi (θk )}k=1 by Eq.(15).
i=1

Algorithm 2 provides an efficient method to expand stochastic samples

ontained from Algorithm 1. Deterministic basis {fi (x)}M
i=1 are solved by

use of KL eapansion, thus guarenting the minimum numbers of expansion

items M . Random variables {ξi (θ)}M
i=1 are described by samples and can be

computed with very low computational costs.

3.2. Algorithm based on stochastic samples and PC expansion

For the Strategy (ii), it’s not easy to select random variables {ξi (θ)}M
i=1 in

Eq.(9) directly, here we utilize Polynomial Chaos (PC) expansion to repre-

sent the random variables {ξi (θ)}M
i=1 and then compute deterministic coeffi-

cient functions {fi (x)}M

i=1 . A general representation of the stochastic process

ω (x, θ) in PC expansion has the following form

P
X
ω (x, θ) = ω̄ (x) + Γi (θ) fi (x) (16)
i=1

where ω̄ (x) is the mean function of the stochastic proces ω (x, θ), {Γi (θ)}Pi=1
are Polynomial Chaos basis and can be generated by Rodriguez formula [32].
The deterministic coefficient functions {fi (x)}M
i=1 can be obtained by

E {[ω (x, θ) − ω̄ (x)] Γi (θ)}

fi (x) = (17)
E {Γ2i (θ)}
where E {·} is the expectation operator and the above method is summarized
in Algorithm 3 as

12
Algorithm 3 Algorithm based on stochastic samples and PC expansion
1: Generate samples of the stochastic process by use of Algorithm 1.
M
2: Choose standard random variables {γi (θ)}i=1γ .
M
3: Generate PC basis {Γi (θ)}Pi=1 of random variables {γi (θ)}i=1γ .
4: Compute {fi (x)}Pi=1 by Eq.(17).

There are various choices for specifying random variables γi (θ) in Step
2, such as gaussian random variables and uniform random variables, and the
corrsponding PC basis should be adopted as Hermite Polynomial Chaos basis
[1] and Generalized Polynomial Chaos basis [1, 32], respectively. Further,
Algorithm 3 provides a suitable method to simulate stochastic processes if
Eq.(13) is not easy to solve. However the computational efficiency decreases
if the number Mγ of random variables γi (θ) or the order of PC basis is too
large.

3.3. Algorithm based on stochastic samples and KL-PC expansion

A natural idea is to combine methods in Section 3.1 and Section 3.2

for the Strategy (iii). Hence, we choose deterministic functions {fi (x)}M
i=1

obtained by Eq.(13) and express random variable ξi (θ) in Eq.(10) by use of

PC expansion as
P
X
ξi (θ) = cij Γj (θ) (18)
j=1

Substituting Eq.(18) into Eq.(10) yields

M X
X P p
ω (x, θ) = ω̄ (x) + cij Γj (θ) λi fi (x) (19)
i=1 j=1

13
Hence, unknown projection coefficients cij can be computed by
Z
1
cij = √  E {[ω (x, θ) − ω̄ (x)] Γj (θ)} fi (x) dx (20)
λi E Γ2j (θ) D

The above method is summarized in Algorithm 4 as

Algorithm 4 Algorithm based on stochastic samples and KL-PC expansion

1: Generate samples of the stochastic process by use of Algorithm 1.

2: Solve a set of deterministic basis {fi (x)}M

i=1 by Eq.(13).
M
3: Choose standard random variables {γi (θ)}i=1γ .
M
4: Generate PC basis {Γi (θ)}Pi=1 of random variables {γi (θ)}i=1γ .
5: Compute {cij }i,j=1 by Eq.(20).

Algorithm 4 combines KL expansion and PC expansion since PC ex-

pansion provides an explicit representation for random variables {ξi (θ)}M
i=1

obtained in Algorithm 2. Algorithm 4 is a good choice if sample-based de-

scriptions are not convenient for some practical problems. Further, stochastic
processes may be described directly by real samples instead of being speci-
fied by their covariance functions and marginal probability distributions. We
can use real samples instead of stochastic samples in Step 1 of Algorithm 2,
Algorithm 3 and Algorithm 4, thus Algorithm 1 is no longer necessary.
This section provides three numerical algorithms to expand the stochas-
tic samples obtained by Algorithm 1. Different algorithms can be chosen
according to properties of practical problems and the performances of pro-
posed algorithms are shown in next section.

14
4. Numerical examples

Three numerical examples, including a non-gaussian and stationary stochas-

tic process, a non-gaussian and non-stationary stochastic process and a strongly
non-gaussian and non-stationary stochastic process, are used to demonstrate
the accuracies and efficiencies of the proposed algorithms. The performances
of Algorithm 2, Algorithm 3 and Algorithm 4 are indicated in Example 4.1,
and only Algorithm 2 is used to represent stochastic samples in Example 4.2
and Example 4.3. Since the simulated marginal distribution function auto-
matically matches the target one, the accuracies of the proposed methods
are validated by comparing the simulated covariance function with the exact
one. For all examples, spatial domains are discretized as 50 points, conver-
gence errors are set as 5 × 10−3 and 1 × 104 initial samples for each random
variable are generated.

4.1. Example 1: non-gaussian and stationary stochastic process

Consider a stochastic process with beat marginal distribution function

[21] Z u
Γ (p + q)
F (y; p, q) = z p−1 (1 − z)q−1 dz (21)
Γ (p) Γ (q) 0

and squared exponential covariance function

2
C (x1 , x2 ) = e−(x1 −x2 ) (22)

In Eq.(21), Γ (·) is the gamma function and parameters are given by

y − ymin
u= , p = 4, q = 2 (23)
ymax − ymin

15
 The expectation function and variance function of the beat distribution
in Eq.(21) are

 µF (x) = (ymax − ymin ) p + ymin
p+q
(24)
2
 σ (x) = (ymax − ymin ) 2 pq
F 2
(p+q) (p+q+1)

According to Eq.(22), we can obtain σF2 (x) = 1. Letting µF (x) = 0 and

√ √
solving Eq.(24) yield ymin = − 14 ≈ −3.74, ymax = 3.5 ≈ 1.87.
 5
Fig.1 shows iterations of sample covariance functions T (k) k=0 , where
T (0) is the covariance function of the initial stochastic samples. It indicates
that a rough approximation of the target covariance function can be obtained
with only one round of iteration. The convergence error in each iteration is
shown in Fig.2, which demonstrates Algorithm 1 has a good convergence and
a high accuracy.

 5
Figure 1: Iterations of simulated covariance functions T (k) k=0
.

16
 1

0.8

0.6
Errors

0.4

0.2

0
1 2 3 4 5

Figure 2: Iterative errors.

In order to verify the performances of the porposed methods, we utilize

Algorithm 2, Algorithm 3 and Algorithm 4 to represent obtained stochastic
samples, respectively. For Algorithm 3, Fig.3 shows first four eigenfunctions
and eigenvalues of C (x1 , x2 ) obtained by Eq.(13).

3
0.8
2
0.7

1 0.6

0.5
fn (x)

λn

0
0.4

-1 0.3

0.2
-2
0.1

-3
0 0.2 0.4 0.6 0.8 1 1 2 3 4
x Index n

Figure 3: Eigenfunctions (left) and eigenvalues (right) of covariance C (x1 , x2 ).

Fig.4 shows cumulative distribution functions (CDFs) of random vari-

17
ables {ξi (θ)}4i=1 computed by Eq.(15) and Table.1 indicates that they are
uncorrelated random variables, which is consistent with the theory of KL
expansion.

0.8

0.6
CDFs

0.4

0.2

0
-4 -2 0 2 4

4
Figure 4: CDFs of random variables {ξi (θ)}i=1 .

Table 1: Statistical correlations between ξi (θ) and ξj (θ)

E {ξi ξj } ξ1 (θ) ξ2 (θ) ξ3 (θ) ξ4 (θ)

ξ1 (θ) 1.0005
ξ2 (θ) 0.0004 0.9995 sym.
ξ3 (θ) 0.0042 0.0016 0.9927
ξ4 (θ) 0.0272 0.0087 −0.0082 1.0248

Fig.5 shows comparisons between target (top left), sample (top mid) and
KL-simulated (top right) covariance. The relative errors between target and

18
sample covariance (bottom left), target and KL-simulated covariance (bot-
tom mid) and sample and KL-simulated covariance (bottom right) demon-
strate the high accuracy of the proposed Algorithm 2.

Figure 5: Comparisons between target, sample and KL-simulated covariance.

For Algorithm 3, we adopt 2-order Hermite Polynomial Chaos basis, the

number of total terms M is 14. Fig.6 shows {fi (x)}Pi=1 obtained by Eq.(17).

19
 1.5

0.5
fi (x)

-0.5

-1

-1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

14
Figure 6: Coefficient functions {fi (x)}i= of PC expansion.

Comparisons between target, sample and PC-simulated covariance (the

corresponding legends of relative errors are the same as Fig.5) are shown in
Fig.7, which demonstrates the high accuracy of the proposed Algorithm 3.

Figure 7: Comparisons between target, sample and PC-simulated covariance.

20
 For Algorithm 4, Table.2 shows projection coefficients cij obtained by
Eq.(20) and comparisons between target, sample and KL-PC-simulated co-
variance (the corresponding legends of relative errors are the same as Fig.5)
are shown as Fig.8, which demonstrates the high accuracy of the proposed
Algorithm 4.

Table 2: Coefficients cij of the KL-PC expansion

cij 1 2 3 4 5 6 7

1 −0.2273 −0.3286 −0.6603 −0.4586 0.0511 0.5482 0.3417

2 −0.3109 0.8586 0.2774 0.5514 −0.9518 −0.6500 −0.6917
3 0.3333 −0.0232 1.1992 −0.1781 0.3611 0.0587 −0.3919
4 −0.3514 −1.0478 0.7115 −0.1398 1.4399 1.0510 0.5405

cij 8 9 10 11 12 13 14

1 0.2638 −0.0346 −0.0014 0.3181 −0.3325 −0.5259 0.7197

2 0.4569 0.0296 0.7990 −0.3317 −0.2799 −0.0816 0.2825
3 −1.0618 0.7456 0.3166 −0.6234 0.2455 −0.1098 −0.4125
4 −1.3680 −0.2531 −0.2757 −0.3412 0.0206 −0.1233 −0.9378

21
 Figure 8: Comparison between target, sample and KL-PC-simulated covariance.

4.2. Example 2: non-gaussian and non-stationary stochastic process

Consider the stochastic process with beat marginal distribution function

in Eq.(21) and Brown-Bridge covariance function [21]

C (x1 , x2 ) = min (x1 , x2 ) − x1 x2 (25)

According to Eq.(25), we can obtain σF2 (x) = C (x, x) = x − x2 . Letting

µF (x) = 0 and solving Eq.(24) yields,
p p
ymin = − 14 (x − x2 ), ymax = 3.5 (x − x2 ) (26)

 4
Fig.9 shows iterations of sample covariance functions T (k) k=0
and Fig.10
shows convergence error in each iteration, which once again demonstrate the
good convergence and the high accuracy of Algorithm 1.

22
  4
Figure 9: Iterations of sample covariance functions T (k) k=0
.

0.9
0.8
0.7
0.6
Errors

0.5
0.4
0.3
0.2
0.1

1 2 3 4

Figure 10: Iterative errors.

Example 4.1 validate Algorithm 2, Algorithm 3 and Algorithm 4, here we

only consider using Algorithm 2 expand the obtained stochastic samples from
Algorithm 1. According to Eq.(13), analytical eigenfunctions and eigenvalues
of C (x1 , x2 ) can be obtained as
√ 1
fk (x) = 2 sin kπx, λk = , k = 1, 2, · · · (27)
k2π2
First six eigenfunctions and eigenvalues are shown in Fig.11.

23
 1.5 0.1

1
0.08

0.5
0.06
fn (x)

λn
0

0.04
-0.5

-1 0.02

-1.5
0 0.2 0.4 0.6 0.8 1 1 2 3 4 5 6
x Index n

Figure 11: Eigenfunctions (left) and eigenvalues (right) of covariance C (x1 , x2 ).

Fig.12 shows CDFs of random variables {ξi (θ)}6i=1 and they are uncorre-
lated as shown as Table.3, which is once again consistent with the theory of
KL expansion. Fig.13 shows comparisons between target, sample and KL-
simulated covariance (the corresponding legends of relative errors are the
same as Fig.5), which verify the applicability of the proposed Algorithm 2 to
non-gaussian and non-stationary stochastic processes.

24
 1

0.8

0.6
CDFs

0.4

0.2

0
-4 -2 0 2 4

6
Figure 12: CDFs of random variables {ξi (θ)}i=1 .

Table 3: Statistical correlations between ξi (θ) and ξj (θ)

E {ξi ξj } ξ1 (θ) ξ2 (θ) ξ3 (θ) ξ4 (θ) ξ5 (θ) ξ6 (θ)

ξ1 (θ) 0.9991
ξ2 (θ) −0.0010 1.0006
ξ3 (θ) −0.0025 0.0015 1.0005 sym.
ξ4 (θ) −0.0011 0.0017 0.0004 1.0001
ξ5 (θ) −0.0004 0.0006 −0.0010 −0.0006 1.0000
ξ6 (θ) 0.0026 −0.0009 −0.0003 0.0009 0.0005 0.9999

25
 Figure 13: Comparisons between target, sample and KL-simulated covariance.

4.3. Example 3: strongly non-gaussian and non-stationary stochastic process

Consider a stochastic process with shifted lognormal marginal distribu-
tion proposed in [21]
 
ln (y − δ (x)) − µ (x)
F (y; µ, δ, σ) = Φ (28)
σ
and covariance function

C (x1 , x2 ) = e−(x1 +x2 )−|x1 −x2 | (29)

The expectation function and variance function of the shifted lognormal

distribution in Eq.(28) are

 µF (x) = δ (x) + eµ(x)+ 21 σ2
  (30)
 σ 2 (x) = eσ2 − 1 e2µ(x)+σ2
F

According to Eq.(29), the variance function is σF2 (x) = C (x, x) = e−2x . Let-
p
ting µF (x) = 0, σ = 1 and solving Eq.(30) yield µ (x) = −x−ln e (e − 1) ≈
−x
−x − 0.7707 and δ (x) = − √ee−1 ≈ −0.7629e−x .

26
  6
Fig.14 shows iterations of sample covariance functions T (k) k=0
and
the corresponding convergence error of each iteration is shown as Fig.15.
The good convergence of Algorithm 2 for non-gaussian and non-stationary
stochastic processes is demonstrated.

 6
Figure 14: Iterations of sample covariance functions T (k) k=0
.

27
 1

0.8

0.6
Errors

0.4

0.2

0
1 2 3 4 5 6

Figure 15: Iterative errors.

According to Eq.(13), first six eigenfunctions and eigenvalues of C (x1 , x2 )

are shown in Fig.16.

1.5 0.3

1 0.25

0.5
0.2
fn (x)

λn

0
0.15
-0.5
0.1
-1

-1.5 0.05

-2
0 0.2 0.4 0.6 0.8 1 1 2 3 4 5 6
x Index n

Figure 16: Eigenfunctions (left) and eigenvalues (right) of covariance C (x1 , x2 ).

Fig.17 shows CDFs of random variables {ξi (θ)}6i=1 and their uncorre-
lated properties are shown as Table.4. Fig.18 shows comparisons between

28
target, sample and KL-simulated covariance (the corresponding legends of
relative errors are the same as Fig.5), which verify the applicability of the
proposed Algorithm 2 to strongly non-gaussian and non-stationary stochastic
processes.

0.8

0.6
CDFs

0.4

0.2

0
-4 -2 0 2 4

6
Figure 17: CDFs of random variables {ξi (θ)}i=1 .

Table 4: Statistical correlations between ξi (θ) and ξj (θ)

E {ξi ξj } ξ1 (θ) ξ2 (θ) ξ3 (θ) ξ4 (θ) ξ5 (θ) ξ6 (θ)

ξ1 (θ) 1.0051
ξ2 (θ) −0.0015 0.9979
ξ3 (θ) −0.0148 0.0014 1.0049 sym.
ξ4 (θ) 0.0141 0.0004 −0.0028 0.9988
ξ5 (θ) −0.0050 0.0066 0.0077 0.0038 0.9997
ξ6 (θ) −0.0093 0.0069 0.0029 −0.0076 0.0061 1.0042

29
 Figure 18: Comparisons between target, sample and KL-simulated covariance.

5. Conclusion

In this paper, efficient numerical schemes have been presented for simu-
lating non-gaussian and non-stationary stochastic processes specified by co-
variance functions and marginal distribution functions. In order to simulate
samples of the target stochastic process, stochastic samples automatically
matching the target marginal distribution function are firstly generated, and
an iterative algorithm is proposed to match the target covariance function
by transform the order of initial stochastic samples. Three numerical exam-
ples demonstrate the fast convergence and the high accuracy of the proposed
algorithm. In order to overcome the difficulty that sample-descriptions are
not convenient to applied to subsequent stochastic analysis, three numerical
algorithms are developed to represent the obtained stochastic samples based
on KL expansion and PC expansion. Different algorithms can be used for dif-
ferent problems of practical interests and the performances of the developed

30
algorithms are indicated by numerical examples. All proposed algorithms can
be readily extended to multi-dimensional random fields and will be shown in
subsequent researches.

Acknowledgments

This research was supported by the National Natural Science Foundation

of China (Project 11972009). This support is gratefully acknowledged.

References

[1] R. G. Ghanem, P. D. Spanos, Stochastic finite elements: a spectral

approach, Courier Corporation, 2003.

[2] P. Tankov, Financial modelling with jump processes, Chapman and

Hall/CRC, 2003.

[3] M. Aboy, O. W. Márquez, J. McNames, R. Hornero, T. Trong, B. Gold-

stein, Adaptive modeling and spectral estimation of nonstationary
biomedical signals based on kalman filtering, IEEE Transactions on
Biomedical Engineering 52 (2005) 1485–1489.

[4] G. Stefanou, The stochastic finite element method: past, present and
future, Computer methods in applied mechanics and engineering 198
(2009) 1031–1051.

[5] A. Papoulis, S. U. Pillai, Probability, random variables, and stochastic

processes, Tata McGraw-Hill Education, 2002.

31
 [6] M. Grigoriu, Evaluation of karhunen–loève, spectral, and sampling rep-
resentations for stochastic processes, Journal of engineering mechanics
132 (2006) 179–189.

[7] C. E. Rasmussen, Gaussian processes in machine learning, in: Summer

School on Machine Learning, Springer, 2003, pp. 63–71.

[8] K. Phoon, H. Huang, S. Quek, Comparison between karhunen–loeve

and wavelet expansions for simulation of gaussian processes, Computers
& structures 82 (2004) 985–991.

[9] G. Deodatis, R. C. Micaletti, Simulation of highly skewed non-gaussian

stochastic processes, Journal of engineering mechanics 127 (2001) 1284–
1295.

[10] F. Yamazaki, M. Shinozuka, Digital generation of non-Gaussian stochas-

tic fields, Journal of Engineering Mechanics 114 (1988) 1183–1197.

[11] M. Shinozuka, G. Deodatis, Simulation of stochastic processes by spec-

tral representation, Applied Mechanics Reviews 44 (1991) 191–204.

[12] I. Elishakoff, Y. J. Ren, M. Shinozuka, Conditional simulation of non-

gaussian random fields, Engineering Structures 16 (1994) 558–563.

[13] R. Popescu, G. Deodatis, J. H. Prevost, Simulation of homogeneous

nongaussian stochastic vector fields, Probabilistic Engineering Mechan-
ics 13 (1998) 1–13.

[14] M. Grigoriu, Simulation of stationary non-Gaussian translation pro-

cesses, Journal of engineering mechanics 124 (1998) 121–126.

32
[15] Z. Liu, W. Liu, Y. Peng, Random function based spectral representa-
tion of stationary and non-stationary stochastic processes, Probabilistic
Engineering Mechanics 45 (2016) 115–126.

[16] S. Huang, S. Quek, K. Phoon, Convergence study of the truncated

karhunen–loeve expansion for simulation of stochastic processes, Inter-
national journal for numerical methods in engineering 52 (2001) 1029–
1043.

[17] B. Sudret, A. Der Kiureghian, Stochastic finite elements and reliability:

a state-of-the-art report, University of California, Berkeley (2000) 114–
120.

[18] F. Poirion, I. Zentner, Non-gaussian non-stationary models for natural

hazard modeling, Applied Mathematical Modelling 37 (2013) 5938–
5950.

[19] H. Kim, M. D. Shields, Modeling strongly non-gaussian non-stationary

stochastic processes using the Iterative Translation Approximation
Method and Karhunen–Loève expansion, Computers & Structures 161
(2015) 31–42.

[20] Z. Liu, Z. Liu, Y. Peng, Dimension reduction of Karhunen-Loeve ex-

pansion for simulation of stochastic processes, Journal of Sound and
Vibration 408 (2017) 168–189.

[21] K. Phoon, S. Huang, S. Quek, Simulation of second-order processes

using Karhunen–Loeve expansion, Computers & structures 80 (2002)
1049–1060.

33
[22] K. Phoon, H. Huang, S. Quek, Simulation of strongly non-Gaussian
processes using Karhunen–Loeve expansion, Probabilistic Engineering
Mechanics 20 (2005) 188–198.

[23] H. Dai, Z. Zheng, H. Ma, An explicit method for simulating non-gaussian

and non-stationary stochastic processes by Karhunen–Loève and poly-
nomial chaos expansion, Mechanical Systems and Signal Processing 115
(2019) 1–13.

[24] S. Sakamoto, R. Ghanem, Polynomial chaos decomposition for the sim-

ulation of non-Gaussian nonstationary stochastic processes, Journal of
engineering mechanics 128 (2002) 190–201.

[25] S. Sakamoto, R. Ghanem, Simulation of multi-dimensional non-gaussian

non-stationary random fields, Probabilistic Engineering Mechanics 17
(2002) 167–176.

[26] B. Puig, F. Poirion, C. Soize, Non-Gaussian simulation using hermite

polynomial expansion: convergences and algorithms, Probabilistic En-
gineering Mechanics 17 (2002) 253–264.

[27] R. Field Jr, M. Grigoriu, On the accuracy of the polynomial chaos

approximation, Probabilistic Engineering Mechanics 19 (2004) 65–80.

[28] N. D. Lagaros, G. Stefanou, M. Papadrakakis, An enhanced hybrid

method for the simulation of highly skewed non-gaussian stochastic
fields, Computer Methods in Applied Mechanics and Engineering 194
(2005) 4824–4844.

34
[29] Z. Zheng, H. Dai, Simulation of multi-dimensional random fields by
Karhunen–Loève expansion, Computer Methods in Applied Mechanics
and Engineering 324 (2017) 221–247.

[30] W. Betz, I. Papaioannou, D. Straub, Numerical methods for the dis-

cretization of random fields by means of the Karhunen–Loève expansion,
Computer Methods in Applied Mechanics and Engineering 271 (2014)
109–129.

[31] V. Christou, P. Bocchini, M. J. Miranda, Optimal representation of

multi-dimensional random fields with a moderate number of samples:
Application to stochastic mechanics, Probabilistic Engineering Mechan-
ics 44 (2016) 53–65.

[32] D. Xiu, Numerical methods for stochastic computations: a spectral

method approach, Princeton university press, 2010.

35