Research Article

Received 27 November 2015, Accepted 27 March 2017 Published online in Wiley Online Library

(wileyonlinelibrary.com) DOI: 10.1002/sim.7313

Bayesian bivariate meta-analysis of

diagnostic test studies with
interpretable priors
Jingyi Guo,*† Andrea Riebler and Håvard Rue
In a bivariate meta-analysis, the number of diagnostic studies involved is often very low so that frequentist meth-
ods may result in problems. Using Bayesian inference is particularly attractive as informative priors that add a
small amount of information can stabilise the analysis without overwhelming the data. However, Bayesian anal-
ysis is often computationally demanding and the selection of the prior for the covariance matrix of the bivariate
structure is crucial with little data. The integrated nested Laplace approximations method provides an efficient
solution to the computational issues by avoiding any sampling, but the important question of priors remain. We
explore the penalised complexity (PC) prior framework for specifying informative priors for the variance param-
eters and the correlation parameter. PC priors facilitate model interpretation and hyperparameter specification
as expert knowledge can be incorporated intuitively. We conduct a simulation study to compare the properties
and behaviour of differently defined PC priors to currently used priors in the field. The simulation study shows
that the PC prior seems beneficial for the variance parameters. The use of PC priors for the correlation parame-
ter results in more precise estimates when specified in a sensible neighbourhood around the truth. To investigate
the usage of PC priors in practice, we reanalyse a meta-analysis using the telomerase marker for the diagnosis
of bladder cancer and compare the results with those obtained by other commonly used modelling approaches.
Copyright © 2017 John Wiley & Sons, Ltd.

Keywords: bivariate random effects model; diagnostic meta-analysis; integrated nested Laplace approximation
(INLA); penalised complexity priors; prior distributions

1. Introduction

A meta-analysis combines the measures of performance across multiple studies and finds application in
several scientific areas, such as medicine, pharmacology, epidemiology, education, psychology, crimi-
nology and ecology [1]. Within epidemiology, the application to diagnostic test studies is very common
[2, 3]. The most common way to present the performance of one diagnostic test study is the two-by-two
table, that is, a cross-tabulation with four entries: the number of patients tested positive that are diseased
(according to some perfect reference test), those tested positive that are not diseased, those tested nega-
tive that are however diseased and finally those tested negative that are not diseased. Those table entries
are commonly referred to as true positives (TP), false positives (FP), false negatives (FN) and true neg-
atives (TN), respectively, and can be used to derive estimates of sensitivity and specificity. Sensitivity
measures the probability of correctly diagnosing the presence of a disease, while specificity measures the
probability of correctly diagnosing the absence of a disease. The main goal of a bivariate meta-analysis
is to derive summary estimates of sensitivity and specificity from several studies. Pairs of sensitivity and
specificity are jointly analysed, and the correlation between them is incorporated using a bivariate ran-
dom effects approach [2, 3]. The accurate estimation of correlation and variance parameters becomes
important when interest lies in the joint distribution of sensitivity and specificity, which is, for example,
needed to predict a new outcome.

Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim PO 7491, Norway
*Correspondence to: Jingyi Guo, Department of Mathematical Sciences, Norwegian University of Science and Technology,
Trondheim PO 7491, Norway.
† E-mail: [email protected]

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

Bivariate meta-analysis is an active area of methodological research [3–8]. One reason is that many
diagnostic meta-analyses combine results from only a small number of studies and the data involved in a
study may be sparse. This makes frequentist methods challenging because maximum likelihood estima-
tion may have problems [5, 9]. Recently, Bayesian hierarchical models became more and more popular.
One reason might be that the use of priors can stabilise the analysis whereby the data will still dominate if
enough information is available. In this way, researchers can incorporate the expected degree of between-
study heterogeneity and information obtained from comparable studies [10]. Markov Chain Monte Carlo
(MCMC) methods are frequently used to analyse Bayesian hierarchical models [6]. However, MCMC
often is computationally intense and not practical, such as in a simulation study where a range of methods
should be compared on a large number of data sets. The selection of the prior for the covariance matrix
of the bivariate structure adds another difficulty, as the obtained estimates are likely to be sensitive when
the data set is small. In practice, parameters of a prior are often taken from previous analyses, whereby
their suitability for the study of interest is not always obvious.
Paul et al. [5] proposed to use integrated nested Laplace approximations (INLA) to perform full
Bayesian inference for bivariate meta-analysis [11]. The new R-package meta4diag provides a spe-
cialised and easy-to-use interface to the functionality implemented in the very general R-package INLA
[12]. Harbord [13] noted that INLA has considerable promise for use in these kind of models, but specify-
ing suitable prior distributions remains a difficulty. Traditionally, non-informative or weakly informative
priors have been used. Lambert et al. [14] compared the performance of 13 different vague priors for
the variance parameters using simulated data from bivariate meta-analysis and concluded that the results
are very sensitive across different priors when the sample size is small. Thorlund et al. [15] showed that
in multiple treatment comparison meta-analyses, moderately informative priors for the variance param-
eters perform better in terms of model fit and credible interval length compared with non-informative
or weakly informative priors. The influence of different priors for the correlation parameter is less clear.
In bivariate meta-analysis of diagnostic test studies, Paul et al. [5] used a normal prior on the Fisher’s
z-transformed correlation parameter, and concluded that a U-shaped prior performs better than an approx-
imate uniform prior on the correlation scale. Most priors that are used for the correlation parameter are
centred at a reference value equal to zero, and the density mass around zero is symmetrically distributed.
However, because sensitivity and the specificity in a diagnostic test of meta-analysis study are commonly
assumed to be negatively correlated [2], Harbord [13] suggested to use a stronger prior for the correlation
parameter that is not symmetric around zero but defined around a (negative) reference value.
Recently, Simpson et al. [16] proposed the new framework of penalised complexity (PC) priors, which
allows such a specification intuitively. Here, the bivariate model component is seen as a flexible extension
of a base model, that is, a simpler model or, in case of the correlation, a model in which we believe a
priori, to which it will shrink to if not otherwise indicated by the data. Unlike the prior by Paul et al. [5],
the amount of shrinkage and the reference value in this prior can be controlled in an interpretable way
using expert knowledge.
In this paper, we use the PC prior framework to motivate prior distributions for the variances and the
correlation parameter, and compare their performance to commonly used priors through an extensive
simulation study. Section 2 introduces the bivariate meta-analysis model as used in the paper. Section
3 discusses PC priors for the correlation parameters and proposes different ways to incorporate prior
knowledge. Further, PC priors for the variance parameters are presented. Section 4 presents an extensive
simulation study to assess the performance of the PC prior under default settings, sparse data and a
misspecification setting. A meta-analysis investigating the use of a telomerase marker for bladder cancer
is presented in Section 5, where we compare the results to other commonly used approaches. In Section
6, we discuss the potential to extend the PC priors for the trivariate model or the absence of a perfect
reference test. We end the paper with a discussion and concluding remarks in Section 7.

2. The Bayesian bivariate model

Chu and Cole [3] proposed a hierarchical model that analyses the two-by-two tables of multiple diagnostic
test studies together. The model has two levels. At the data level, the number of true positives and the
number of true negatives are assumed to be separately binomial distributed. These two models are linked
at the latent level through a bivariate random effect that accounts for between-study heterogeneity.
Let i = 1, … , I be the study index, and TP, FP, TN and FN denote the number of true positives, false
positives, true negatives and false negatives, respectively. Further, let Se and Sp denote sensitivity and

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

specificity, respectively. Then the model is as follows:

TPi |Sei ∼ Binomial(TPi + FNi , Sei ), logit(Sei ) = 𝜇 + 𝐔i 𝜶 + 𝜙i ,

TNi |Spi ∼ Binomial(TNi + FPi , Spi ), logit(Spi ) = 𝜈 + 𝐕i 𝜷 + 𝜓i ,
( ) [( ) ( )] (1)
𝜙i 0 𝜎𝜙2 𝜌𝜎𝜙 𝜎𝜓
∼N , ,
𝜓i 0 𝜌𝜎𝜙 𝜎𝜓 𝜎𝜓2

where 𝜇, 𝜈 are intercepts for logit(Sei ) and logit(Spi ), respectively, and 𝐔i , 𝐕i are covariate vectors,
if available, and 𝛼 and 𝛽 denote the corresponding regression coefficients. The covariance matrix of
the random effects parameters 𝜙i and 𝜓i is parameterised using between-study variances 𝜎𝜙2 , 𝜎𝜓2 , and
between-study correlation 𝜌.
Another level is added to specify hyperpriors for all parameters. For the intercepts 𝜇 and 𝜈, as well as for
potential regression parameters, it is common to use normal priors with mean zero and variance fixed to
a large value, for example, 100. Specifying prior distributions for the between-study variances 𝜎𝜙2 and 𝜎𝜓2
and the between-study correlation 𝜌 is challenging. In some applications, an inverse Wishart distribution
is used for the entire covariance matrix because of its conjugacy for the normal model [4,6,17]. However,
using an inverse Wishart prior the correlation estimates might depend on the value of the variances, which
may lead to overestimated correlation estimates (see [17, 18] for a discussion).
Here, we follow Wei and Higgins [19] and Burke et al. [20] and specify priors for the variance com-
ponents and the correlation component separately. Barnard et al. [21] proposed a separation strategy to
separate the components in a covariance matrix as
( ) ( )( )( )
𝜎𝜙2 𝜌𝜎𝜙 𝜎𝜓 𝜎𝜙 0 1𝜌 𝜎𝜙 0
𝚺= = 𝐕1∕2 𝐑𝐕1∕2 = , (2)
𝜌𝜎𝜙 𝜎𝜓 𝜎𝜓2 0 𝜎𝜓 𝜌1 0 𝜎𝜓

where 𝐕1∕2 is a diagonal matrix with the standard deviations as elements and 𝐑 is the 2 × 2 matrix of
correlations. With this separation, we can define priors for the variances and the correlation separately.
Mainly vague or mildly informative priors for 𝜎𝜙2 , 𝜎𝜓2 , and 𝜌 are used [4,5,22]. For the variance parameters
𝜎𝜙2 and 𝜎𝜓2 , inverse gamma priors are commonly chosen, whereby the value of the hyperparameters are
often copied from the literature. Thorlund et al. [15] found in the context of multiple treatment comparison
meta-analyses that the use of mildly informative priors for the variance components yielded a better model
fit and narrower credible intervals compared with weakly informative and non-informative priors. Paul
et al. [5] reached a similar conclusion for the correlation parameter, when they compared three differently
shaped priors. They proposed to use a U-shaped prior for the correlation parameters instead of a roughly
uniform prior to obtain unbiased estimates. It is common to use a normal distribution on the Fisher’s
z-transformed correlation 𝜃 = logit((𝜌 + 1)∕2) that is centred at 0 as prior for the correlation parameter.
However, in a bivariate meta-analysis of diagnostic test studies, we expect sensitivity and specificity to
be negatively correlated [2]. For this reason, Harbord [13] proposed to use a stronger prior for 𝜌 that is
not symmetric around zero, but defined around a (negative) constant 𝜌0 . Incorporating this prior belief
may additionally stabilise the analysis if few studies are available. Here, we use the framework of PC
priors that will allow us to address Harbords suggestion in an interpretable way.

3. Penalised complexity priors

The construction of PC priors is based on four principles: (i) Occam’s razor – a simpler model, called
base model, should be preferred until there is enough information to support a more complex model.
(ii) Measure complexity – the Kullback–Leibler divergence is used to measure model complexity. (iii)
Constant-rate penalisation – the deviation from the base model is penalised using a constant decay rate.
(iv) User-defined scaling – the user has some idea of the range in which the parameter of interest lies.

3.1. Penalised complexity priors for the correlation parameter

Often a normal distribution is placed on the Fisher’s z-transformed correlation, that is, 𝜃 ∼ N(𝜇, 𝜎 2 ), and
suitable values for 𝜇 and 𝜎 2 have to be chosen. Paul et al. [5] proposed to use 𝜇 = 0 and 𝜎 2 = 5 (from
here on referred to as Paul et al. prior). The prior mean 𝜇 could be easily changed to a negative value,
say, however, the density distribution weight on both sides of this value will still be the same. It seems

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

more natural to define a prior that satisfies P(𝜌 ⩽ 𝜌0 ) = 𝜔, where 𝜌0 is a (negative) value and 𝜔 denotes
the prior probability that 𝜌 is smaller than 𝜌0 .

3.1.1. Equal probability density on both sides of 𝜌0 . Consider the bivariate random effects model in
Equation (1), where the bivariate random effect is assumed to be normally distributed with zero-mean
and covariance matrix 𝚺. We follow the separation strategy given in Equation (2) and consider only
the correlation matrix 𝐑 in this section. Let N(𝟎, 𝐑b ) and N(𝟎, 𝐑f ) denote the base and flexible model,
respectively, where
( ) ( )
1 𝜌0 1𝜌
𝐑b = and 𝐑f = ,
𝜌0 1 𝜌1

Here, 𝜌0 is fixed to a (negative) value in which one believes with respect to the application at hand,
while 𝜌 is a random variable. Thus, 𝐑b is a special case of 𝐑f , to which we would like to shrink if
not otherwise indicated by the data. The increased complexity introduced by N(𝟎, 𝐑f ) compared with
N(𝟎, 𝐑b ) is measured by the Kullback–Leibler divergence (KLD) [23]. The KLD is a measure of the
‘information’ lost when the base model N(𝟎, 𝐑b ) is used to approximate the more flexible model N(𝟎, 𝐑f ),
and is a function of 𝜌:
{ ( )}
( ) 1 ( −1 ) |𝐑f |
KLD N(𝟎, 𝐑f )||N(𝟎, 𝐑b ) = KLD(𝜌) = tr 𝐑b 𝐑f − 2 − ln
2 |𝐑b |
1 − 𝜌0 𝜌 1 ( ) 1 ( )
= − 1 − ln 1 − 𝜌2
+ ln 1 − 𝜌2
0 ,
1 − 𝜌20 2 2

√
and d(𝜌) = 2KLD(𝜌) is defined as the ‘distance’ between the base model and the flexible model [16].
A distance equal to zero means the flexible model shrinks to the base model, whereas increasing distance
indicates that the flexible model moves further away from the base model. The base model is preferred
unless there is enough supporting evidence from the data. Therefore a prior distribution on the distance
scale should have mode at 0 and the density mass should decrease with increasing distance. Any dis-
tribution that fulfils those features can be used as prior distribution on the distance. Simpson et al. [16]
proposed a constant rate penalisation for the distance d. This leads to an exponential prior with parame-
ter 𝜆 that controls the decay rate of the deviation from the base model. Using a change of variables, we
derive the prior for the correlation as 𝜋(𝜌) = 𝜆 exp(−𝜆d(𝜌))|𝜕d(𝜌)∕𝜕𝜌|. It is worth noting that while the
distance goes from zero to infinity, we can deviate from the base model in two directions, either from 𝜌0
to 1 or from 𝜌0 to −1. The exponential prior thus is a two-sided prior that shares the same parameter 𝜆
and has identical behaviour on the distance scale for 𝜌 ∈ [−1, 𝜌0 ] and 𝜌 ∈ [𝜌0 , 1], leading to the same
probability weights on both sides of 𝜌0 in the correlation scale.

3.1.2. Unequal probability density on both sides of 𝜌0 . In order to distribute the density unevenly to
each side of 𝜌0 but keeping the assumption of a constant rate penalisation, we use two rate parameters
𝜆1 and 𝜆2 that indicate the decay rate of the deviation from the base model when 𝜌 ⩽ 𝜌0 and 𝜌 > 𝜌0 ,
respectively. In order to merge these two exponential priors 𝜋1 (d; 𝜆1 ) and 𝜋2 (d; 𝜆2 ) in the distance scale
into one continuous prior distribution 𝜋(𝜌; 𝜆1 , 𝜆2 ) in the correlation scale, we define
{ | |
𝜔1 𝜋1 (d(𝜌); 𝜆1 ) | 𝜕d(𝜌) |, 𝜌 ⩽ 𝜌0
𝜋(𝜌; 𝜆1 , 𝜆2 ) = | 𝜕𝜌 |
| 𝜕d(𝜌) |
𝜔2 𝜋2 (d(𝜌); 𝜆2 ) | 𝜕𝜌 | , 𝜌 > 𝜌0
| |

𝜌 1
where 𝜔1 = ∫−10 𝜋(𝜌; 𝜆1 , 𝜆2 ) d𝜌 and 𝜔2 = ∫𝜌 𝜋(𝜌; 𝜆1 , 𝜆2 ) d𝜌 ∈ [0, 1] denote the probabilities of 𝜌 ⩽ 𝜌0
0
and 𝜌 > 𝜌0 , respectively. To obtain a proper probability density, 𝜔1 and 𝜔2 must satisfy 𝜔1 + 𝜔2 = 1.
Further, to get a continuous probability density for d = 0, 𝜔1 and 𝜔2 must satisfy 𝜔1 𝜆1 = 𝜔2 𝜆2 .
The value of the hyperparameters 𝜆1 and 𝜆2 can be specified by defining the density behaviour around
𝜌0 . Consider, for example, the two conditions P(𝜌 ⩽ umin | − 1 < umin < 𝜌0 ) = 𝛼1 , where umin is a
‘left tail event’ and 𝛼1 is a small probability of this event, and P(𝜌 ⩽ 𝜌0 ) = 𝜔1 , which is equivalent to

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

Table I. Three different strategies to specify a penalised complexity prior for the correlation
parameter 𝜌.
Condition and parameters
Strategy P(𝜌 ⩽ umin | − 1 < umin < 𝜌0 ) = 𝛼1 P(𝜌 ⩾ umax |𝜌0 < umax < 1) = 𝛼2 P(𝜌 ⩽ 𝜌0 ) = 𝜔1
1 ✓ ✓
2 ✓ ✓
3 ✓ ✓
Shown are the corresponding probability contrasts that need to be provided to determine the parameters
of the prior distribution.

𝜌
P(𝜌 > 𝜌0 ) = 𝜔2 . Solving ∫u 0 𝜋(𝜌; 𝜆1 , 𝜆2 )d𝜌 = 𝜔1 − 𝛼1 and 𝜔1 𝜆1 = (1 − 𝜔1 )𝜆2 , we find
min

log(𝜔1 ) − log(𝛼1 ) 𝜔1 𝜆1
𝜆1 = and 𝜆2 =
d(umin ) 1 − 𝜔1

(Appendix A). Instead of the two conditions presented, alternative conditions could be specified. Here, we
provide three different strategies to choose 𝜆1 and 𝜆2 based on prior knowledge (Table I). For all strategies,
we first need to set 𝜌0 , which represents a reference value. A priori we believe that the true correlation 𝜌
is likely to be close to 𝜌0 . For most bivariate meta-analyses, 𝜌0 would be set negative; however, it could
also be set equal to zero or even positive if sensible for the application considered. The details to find 𝜆1
and 𝜆2 under the different strategies are given in Appendix A. Using the R-package meta4diag, the
parameters 𝜆1 and 𝜆2 are derived automatically from the user-specified probability conditions [12].

3.1.3. Choice of hyperparameters. The choice of the parameters for the different strategies is still a del-
icate issue, in particular if the meta-analysis consists of few studies and the number of patients is low.
Ideally, there is expert knowledge available to set up sensible probability contrasts. Depending on how
concrete the expert knowledge is, the informativeness of the prior distribution can be controlled. Alter-
natively, we could motivate the probability contrasts based on a collection of existing meta-analyses.
Menke [24] studied 50 independent bivariate meta-analyses that were selected randomly from the liter-
ature within a Bayesian setting, whereas Diaz [25] reported frequentist estimates based on a literature
review of 61 bivariate meta-analyses of diagnostic accuracy published in 2010. Based on these two pub-
lications, the distribution of the correlation seems asymmetric around zero. We find that around half of
the correlation point estimates are negative, with a mode around −0.2. Only a small proportion are larger
than 0.4 and values larger than 0.8 are rare. Based on these findings, we choose three differently behaved
PC priors for our simulation study in Section 4 that are all defined around 𝜌0 = −0.2:

PC 1 ∶ 𝜌 ∼ PC(𝜌0 = −0.2, 𝜔1 = 0.4, umin = −0.95, 𝛼1 = 0.05)

PC 2 ∶ 𝜌 ∼ PC(𝜌0 = −0.2, umin = −0.9, 𝛼1 = 0.05, umax = 0.8, 𝛼2 = 0.05)
PC 3 ∶ 𝜌 ∼ PC(𝜌0 = −0.2, 𝜔1 = 0.6, umax = 0.4, 𝛼2 = 0.05)

Figure 1 shows all priors in the original scale and the distance scale. PC 1 is the least informative prior,
while PC 3 is the most informative prior with almost no support for correlation values close to one (panel
a). We see that all priors go to infinity at the boundary values −1 and 1. This is a consequence of the
bounded parameterisation of a correlation parameter. Looking at the distance scale, we see a mode at
the base model, that is, at distance equal to zero. The density support in the distance scale is positive;
however, we can deviate from 𝜌0 in two different directions at possibly different decay rates. The right
side of panel b) shows the prior when deviating towards 𝜌 = 1 with rate 𝜆2 , while the left side shows
the resulting prior when deviating towards 𝜌 = −1 with rate 𝜆1 . The density decreases towards zero the
further we deviate from the base model, that is, 𝜌0 = −0.2, meaning the larger the distance to 𝜌0 gets.
We compare the performance of PC 1, PC 2 and PC 3 to the Paul et al. prior and a less informative PC
prior which, similar to the Paul et al. prior, also fulfils P(𝜌 < 0) = 0.5 and P(𝜌 < −0.9) ≈ 0.1:

PC 0 ∶ 𝜌 ∼ PC(𝜌0 = 0, 𝜔1 = 0.5, umin = −0.9, 𝛼1 = 0.1)

Paul et al. ∶ 𝜃 ∼ N(𝜇 = 0, 𝜎 2 = 5)

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

Figure 1. Shown are the PC 1, PC 2 and PC 3 prior in solid, dashed and dotted lines, respectively. Panel (a) shows
the prior distributions in the correlation scale, and panel (b) shows the prior distributions in the distance scale. All
penalised complexity priors have reference value at 𝜌0 = −0.2.

Figure 2. Shown are the Paul et al. prior and PC 0 prior for the correlation parameter in grey and black lines,
respectively. Panel (a) shows the prior distributions in the correlation scale; panel (b) shows the prior distributions
in the Fisher’s z-transformed correlation scale; and panel (c) in the distance scale.

Figure 2 shows both priors. In addition to the original and the distance scale, we also present the priors on
the Fisher’s z-transformed correlation scale. The PC 0 prior is less informative compared with the priors
shown in Figure 1; however, compared with the Paul et al. prior, it is more informative, which becomes
apparent in all three scales by a more concentrated shape around the base model. The flatness of the Paul
et al. prior in the distance scale implies that in an area around the base model, it behaves almost like a
uniform prior, which means the base model is not necessarily preferred. In the correlation scale, we see
that the PC 0 prior goes to infinity at the boundaries whereas the Paul et al. prior goes to zero. This is the
result of a different behaviour when doing the change from variables from an infinite to a finite support.

3.2. Penalised complexity priors for the variance parameters

Here, we illustrate the PC priors for the variance components 𝜎𝜙2 or 𝜎𝜓2 in model (1). In section 3.2 of
[16], Simpson et al. derived the PC prior for a precision parameter, which is the inverse of the variance
parameter. It was found to be type-2 Gumbel distribution that leads to a Weibull distribution with shape
parameter equal to 1∕2 for the variance parameter. The second parameter depends on the rate of the decay
𝜆. A simple choice to set 𝜆 is to provide (u, a) such that P(𝜎 > u) = a, where a ∈ [0, 1] and u > 0, leading
to an explicit relationship between the distribution parameter and density behaviour 𝜆 = − log(u)∕a. The
magnitude of the penalty for deviating from the base model 𝜎 2 → 0 can be controlled through setting the
value of the parameter 𝜆, and it is thus controlled by u and a.
Figure 3 shows an example of a PC prior for a variance parameter defined under the condition P(𝜎 >
3) = 0.05. This choice corresponds to the belief that the sensitivities or specificities lie in the interval
[0.5, 0.95] with probability 0.95. We compare this prior with an inverse gamma prior with shape parameter

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

Figure 3. Illustration of the penalised complexity prior for a variance parameter calibrated such that P(𝜎 > 3) =
0.05 (dashed) compared with an Γ−1 (shape = 0.25, rate = 0.025) (solid). Panel (a) shows both densities in
variance scale while panel (b) shows them on the distance scale.

equal to 0.25 and rate parameter equal to 0.025, that is, Γ−1 (0.25, 0.025), as proposed by Paul et al. [5].
In the distance scale, it can be seen that the inverse gamma prior has zero density at distance zero, which
means that it is not possible to shrink to the base model 𝜎 2 → 0 that corresponds to an exclusion of
the random effects. Hence, the inverse gamma prior supports overfitting in the sense that a too complex
model is favoured even if not supported by the data.

4. Simulation study

In this section, we conduct an extensive simulation study to compare the effects on the posterior marginals
when using different priors. We consider three different simulation settings, which are described in the
following.

4.1. Default data simulation setting

We simulate data from model (1) as described in Paul et al. [5] with the following specifications:
(1) The number of individuals per study ni (composed of diseased and non-diseased individuals) are
sampled from a shifted gamma distribution and rounded off to the nearest integer, see panel a) of
Figure 4.
(2) The number of studies are set to 10, 25 or 50 representing small, medium and large meta-analyses.
(3) The true mean values of sensitivity and specificity are 80%∕70%, 90%∕90%, or 95%∕30%.
(4) Correlation between logit-transformed sensitivity and logit-transformed specificity across studies:
−0.95, −0.8, −0.6, −0.4, −0.2, 0, 0.2, 0.4, 0.6;
(5) The true variances of logit-transformed sensitivity and logit-transformed specificity are 1 for all the
scenarios.
This leads to a total of 81 different scenarios, and we refer to these scenarios as the ‘default simulation
setting’. For each data scenario, 1000 data sets are generated. The detailed specifications for the default
simulation setting are shown in Table II.

4.2. Sparse data simulation setting

To not only assess the situation when a meta-analysis comprises a few number of studies, as in the default
data simulation setting, but also when the data in these studies are sparse, we restrict the gamma dis-
tribution used in point (1) of Section 4.1 to get less individuals per study. We simulate the number of
individuals per study ni from a truncated shifted gamma distribution, that is, a gamma distribution that
is truncated at 140 and shifted with lag 30. This increases the proportion of zero cells in the generated
two-by-two tables. Figure 4 shows the distribution of the study sample size ni used in the default data
simulation setting (panel a) and the sparse data setting (panel b). The maximum study sample size under
the default situation is 1262, and the minimum study sample size is 30. Among all the scenarios, the

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

Figure 4. Shown is the range of the number of study participants ni in the default (panel a) and sparse (panel b)
data simulation setting. Panel (a) shows the shifted gamma distribution with shape = 1.2, rate= 0.01 and lag = 30.
Panel (b) shows the truncated shifted gamma distribution with shape = 1.2, rate= 0.01, truncated at 140 and then
lagged by 30, so that ni is limited to 170. Median and quartiles are indicated vertical dashed lines.

Table II. The 81 different scenarios in the ‘default

data simulation setting’.
Scenario logit−1 (𝜇) logit−1 (𝜈) 𝜎𝜙2 𝜎𝜓2 I

1-9 0.8 0.7 1 1 10

10-18 0.8 0.7 1 1 25
19-27 0.8 0.7 1 1 50
28-36 0.9 0.9 1 1 10
37-45 0.9 0.9 1 1 25
46-54 0.9 0.9 1 1 50
55-63 0.95 0.3 1 1 10
64-72 0.95 0.3 1 1 25
73-81 0.95 0.3 1 1 50
Each subset of nine scenarios corresponds to the
correlation values 𝜌 = −0.95, −0.8, −0.6, −0.4,
−0.2, 0, 0.2, 0.4, 0.6.

highest frequency of data sets containing at least one zero cell is 59.7%. In contrast, the maximum value
of the study sample size under the sparse data situation is 169 and the minimum value is 30. The highest
frequency of data sets containing at least one zero cell is 74.2%. The remaining simulation settings are
kept as in Section 4.1, which leads to additional 81 different scenarios.

4.3. Misspecification data simulation setting

Finally, we consider as misspecification data simulation setting, where the correlation between logit-
transformed sensitivity and logit-transformed specificity is misspecified. For this purpose, let w ∈ [0, 1].
We simulate w ⋅ 100% of the studies in one meta-analysis using a correlation value 𝜌1 and the remaining
studies using a correlation value 𝜌2 . We consider thereby four different combinations of 𝜌1 and 𝜌2 and
three different values of w. All other simulation specifications are kept as in the default data scenario
leading to a total of 108 additional simulation scenarios. The detailed specifications are shown in Table III.

4.4. Measures of performance

For each scenario, 1000 data sets are generated, and for each data set, posterior summary estimates based
on different hyperpriors are computed using the R-package meta4diag. We compare the performance
using errors (the difference between the posterior median and the true value), bias (mean errors), MSE
(sum of squared bias and squared standard deviation of the posterior medians) and 95% coverage prob-
abilities (frequency in which the true value is within the 95% credible interval) for all the parameters in
the default and sparse data scenario. In the misspecification setting, we use the same measures of perfor-
mance for sensitivity, specificity and the log standard deviation parameters. For the correlation parameter,

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

Table III. The 108 different scenarios in the ‘misspecification

data simulation setting’.
Scenario 𝜌1 𝜌2 logit−1 (𝜇) logit−1 (𝜈) 𝜎𝜙2 𝜎𝜓2

1–9 −0.7 −0.2 0.8 0.7 1 1

10–18 −0.7 0.2 0.8 0.7 1 1
19–27 −0.7 0.7 0.8 0.7 1 1
28–36 −0.2 0.2 0.8 0.7 1 1
37–45 −0.7 −0.2 0.9 0.9 1 1
46–54 −0.7 0.2 0.9 0.9 1 1
55–63 −0.7 0.7 0.9 0.9 1 1
64–72 −0.2 0.2 0.9 0.9 1 1
73–81 −0.7 −0.2 0.95 0.3 1 1
82–90 −0.7 0.2 0.95 0.3 1 1
91–99 −0.7 0.7 0.95 0.3 1 1
100–108 −0.2 0.2 0.95 0.3 1 1
Each subset of nine scenarios corresponds to the combination of three
different number of studies I = 10, 25, 50 and three different values for
w = 0.5, 0.7, 0.3. For example, when w = 0.5, half of the studies in one
meta-analysis are simulated using 𝜌1 and the other half are simulated
using 𝜌2 .

there is no true value available, so we report the 0.025-, 0.5- and 0.975-quantile estimates obtained from
an equal-weight mixture of posterior distributions of 𝜌 over 1000 data sets for each scenario instead.

4.5. Simulation result

4.5.1. Comparison of different priors for the variance parameters. To study the effect of changing the
prior for the variance parameters, we consider the default data simulation setting and fix the prior for
the correlation parameter to the Paul et al. prior. We compare the result from a PC prior with u = 3
and 𝛼 = 0.05 with an inverse gamma prior Γ−1 (0.25, 0.025). Detailed results are given in the Supporting
Information (Part B). We find that changing the priors for the variance parameters does almost not affect
the posterior marginal of the correlation parameter (Tables S13–S15). Both priors result in similar bias and
MSE for sensitivity and specificity; however, using PC priors leads to better 95% coverage probabilities
for sensitivity and specificity (Tables S1–S6). Figure 5 shows errors, MSE and 95% coverage probabilities
for the posterior marginal of the two estimated log standard deviations for both prior settings when the
number of studies is small (i.e. I = 10). From the error plot, we can see that the errors obtained when
using the PC prior have less spread than when using the inverse gamma prior. Furthermore, the bias,
indicated as circle in the error plot, is closer to zero for the PC prior than the inverse gamma prior. We
also observe a smaller MSE and marginally better coverage probabilities when using the PC prior for
the variance parameters. Increasing the number of studies, we observe similar results but the differences
decrease slightly (Tables S7 and S10). Similar results can be found also in the remaining scenarios (Tables
S8, S9, S11 and S12). In the following, we will use the PC prior with u = 3 and 𝛼 = 0.05 for both
variance parameters.

4.5.2. Comparison of different priors for the correlation parameter. Next, we study the effect of chang-
ing the prior for the correlation parameter. Arguing that a central innovation is to move the reference
value from zero to a negative value, we may just centre the normal distribution for 𝜃 around a negative
value, for example, logit(−0.2), instead of zero. Figure 6 shows the normal prior proposed by Paul et al.
[5] and the shifted normal prior both with variance equal to 5 in the correlation, transformed correlation
and distance scale. There is almost no difference in the distance scale. However, in the correlation scale,
the shifted normal distribution leads to more density mass in the left tail and less in the right tail, while
keeping the density in the central region approximately uniform distributed. For this reason, posterior
marginals for all parameters stay almost unchanged using these two prior. In the following, we compare
the Paul et al. prior with differently chosen PC priors.
Less-informative priors: PC 0 prior versus Paul et al. prior We start by comparing PC 0 with the
Paul et al. prior (see Figure 2 for an illustration of both priors). Both priors provide a small amount of
information at the boundaries. We find that both priors result in almost the same bias, MSE and coverage

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

Figure 5. Errors, bias, MSE and 95% coverage probabilities for log(𝜎𝜙 ) (top row) and log(𝜎𝜓 ) (bottom row).
The black bars (shown on the left) in each plot indicate the results using a Γ−1 (0.25, 0.025) prior whereas the
light grey bars (shown on the right) are for the PC(u = 3, 𝛼 = 0.05). The dot within the bars in the error plot
indicates the respective bias. The dashed line in the coverage plot corresponds to the nominal level of 95%. The
prior for the Fisher’s z-transformed correlation parameter is always 𝜃 ∼ N(0, 5). The results are obtained using
integrated nested Laplace approximation for data simulated with I = 10, 𝜇 = 0.8, 𝜈 = 0.7, 𝜎𝜙2 = 𝜎𝜙2 = 1 and
𝜌 = (−0.95, −0.8, −0.6, −0.4, −0.2, 0, 0.2, 0.4, 0.6)

Figure 6. Shown are the Paul et al. prior (solid) and the shifted normal prior (dashed) for the Fisher’s
z-transformed correlation parameter 𝜃 in the correlation, transformed correlation and distance scale.

probabilities for the sensitivity and specificity for all the default data scenarios (Tables S16–S21). For
the log standard deviation, performance measures are very similar for sensitivity/specificity pairs equal
to 0.95∕0.3 and 0.8∕0.7 (Tables S22, S24, S25 and S27). However, worse coverage probabilities can be
found for the PC 0 prior when the true sensitivity and specificity values are close to 1, here both equal to
0.9, and the number of studies is small, that is, I = 10. Differences disappear rapidly when increasing
the study size to I = 25 and I = 50 (Tables S23 and S26). Bias and MSE are almost the same for the log
standard deviation for all scenarios. The resulting effects on the correlation parameter are displayed in
Figure 7 for default data simulation scenarios 1–18. We find that the posterior results are approximately
point-symmetric around 0 because of the symmetry of the prior. The black circles in the left column
indicate that using the Paul et al. prior leads to less biased estimates; however, the error bars of PC 0
(grey) indicate a smaller spread when the true correlation is close to the assumed base model, that is, 0,
also with almost zero bias. From the MSE and 95% coverage probability plot, it is obvious that using PC
0, we obtain a smaller MSE and better coverage probabilities when the true correlation is in a sensible
range around the base model. The spread of errors decreases for both priors when the absolute correlation
increases. This might be because the correlation estimates are then bounded on one side, whereas the
estimates of the non-extreme correlation have two sides to vary. The described results apply to both I = 10
(top row) and I = 25 (bottom row) but become less extreme as the number of studies increases. Detailed
results for the correlation parameter in all scenarios can be found in Tables S28–S30.

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

Figure 7. Errors, bias, MSE and 95% coverage probabilities for the correlation parameter. The black bars (shown
on the left) in each plot indicate the results using the Paul et al. prior whereas the light grey bars (shown on the
right) are for PC 0. The dot within the bars in the error plot indicates the respective bias. The dashed line in the
coverage plot corresponds to the nominal level of 95%. The small star under zero indicates the base model 𝜌0 = 0.
The prior for the variance parameter is always PC(u = 3, 𝛼 = 0.05). The results are obtained using integrated
nested Laplace approximation from data simulated with I = 10 (top row), I = 25 (bottom row) and 𝜇 = 0.8,
𝜈 = 0.7, 𝜎𝜙2 = 𝜎𝜙2 = 1 and 𝜌 = (−0.95, −0.8, −0.6, −0.4, −0.2, 0, 0.2, 0.4, 0.6) under the default data scenario.

Informative priors versus less-informative priors in all three data simulation settings Next, we
consider PC priors specified at a negative reference value 𝜌0 = −0.2 and having different behaviour on
both sides of 𝜌0 . Choosing 𝜌0 means that we change the base model assuming a negative correlation as
natural. We compare PC 1, PC 2 and PC 3 as specified in Section 3 with the Paul et al. prior for the
transformed correlation under all three data simulation settings. Detailed results are given in Supporting
Information D–F. A comparison to PC 0 is available when comparing with Supporting Information C.
The estimates of sensitivity and specificity are not sensitive to the changes in the prior distributions
and stay almost unchanged in all three simulation scenarios. If more studies are available, estimates
of sensitivity and specificity are more precise indicated by a smaller absolute bias and lower MSE for
all priors for all scenarios. For sensitivity/specificity pairs equal to 0.95∕0.3 and 0.9∕0.9 in the default
data simulation setting, we observe worse performance measures for log(𝜎𝜙 ) using the PC 1 and PC 3
priors compared with the Paul et al. prior and the PC 0 prior if I = 10. A similar effect is not seen for
log(𝜎𝜓 ) (Tables S37–S42). Differences for log(𝜎𝜙 ) disappear however already with I = 25. In all other
simulation scenarios, we observe very similar bias, MSE and 95% coverage probabilities for both log
standard deviations under all priors.
Figures 8 and 9 display the results for the correlation parameter for scenarios 1–18 under the default
and the sparse data simulation setting, respectively. The results under both simulation setting are almost
identical. The error estimates obtained by the PC priors are no longer symmetric around the base model
as the priors behave differently on both sides of 𝜌0 . Going from PC 1 to PC 3, the priors get more concen-
trated around the base model, that is, they are more informative. This results in improved performance
measures when the true correlation is close to the base model, that is, 𝜌0 = −0.2. If we move further away
from the base model, the performance in particular the bias decreases. However, the true correlation must
be quite far away to obtain worse measures compared with the Paul et al. prior. In contrast to the PC pri-
ors, the Paul et al. prior behaves constantly well in terms of coverage probabilities but performs clearly
worse in terms of MSE around the base model. Results for the other simulation settings are comparable
(Tables S44, S45, S59 and S60).
In the misspecification data simulation setting, we see almost no effect on the performance measures for
sensitivity, specificity and the log standard deviations when changing the hyperpriors. Here, no hyperprior
seems clearly beneficial (Tables S61–S96). Because the data within one meta-analysis are simulated
using different true correlation values, it is no longer possible to assess the accuracy of the estimated
correlation parameter for the different priors. We formed an equal-weight mixture over all the 1000 data
sets in each scenario to approximate the expected behaviour of the posterior distribution over different
realisations of the data. Tables S97–S105 show the 0.025-, 0.5- and 0.975-quantiles of this mixture of
posterior distributions for the different priors and different scenarios. Using PC priors, the median tends

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

Figure 8. Errors, bias, MSE and 95% coverage probabilities for the correlation parameter under the default data
simulation setting. The dark grey bars in each plot indicate the results using the Paul et al. prior, whereas the
lighter grey bars are for PC 1, PC 2 and PC 3. The dot within the bars in the error plot indicates the respective bias.
The dashed line in the coverage plot corresponds to the nominal level of 95%. The small star under −0.2 indicates
the base model 𝜌0 = −0.2. The prior for the variance parameter is always PC(u = 3, 𝛼 = 0.05). The results are
obtained using integrated nested Laplace approximation for data simulated with I = 10 (top row), I = 25 (bottom
row), 𝜇 = 0.8, 𝜈 = 0.7, 𝜎𝜙2 = 𝜎𝜙2 = 1 and 𝜌 = (−0.95, −0.8, −0.6, −0.4, −0.2, 0, 0.2, 0.4, 0.6).

Figure 9. Errors, bias, MSE and 95% coverage probabilities for the correlation parameter under the sparse data
simulation setting. The dark grey bars in each plot indicate the results using the Paul et al. prior whereas the lighter
grey bars are for PC 1, PC 2 and PC 3. The dot within the bars in the error plot indicates the respective bias. The
dashed line in the coverage plot corresponds to the nominal level of 95%. The small star under −0.2 indicates
the base model 𝜌0 = −0.2. The prior for the variance parameter is always PC(u = 3, 𝛼 = 0.05). The results are
obtained using integrated nested Laplace approximation for data simulated with I = 10 (top row), I = 25 (bottom
row), 𝜇 = 0.8, 𝜈 = 0.7, 𝜎𝜙2 = 𝜎𝜙2 = 1 and 𝜌 = (−0.95, −0.8, −0.6, −0.4, −0.2, 0, 0.2, 0.4, 0.6).

to be naturally closer to the specified base model of −0.2 compared with using the Paul et al. prior.
Besides, the length of 95% credible intervals when using PC priors is usually smaller than when using the
Paul et al. prior. We note that there is no prior that always performs best. In practice, the performance of
informative priors will depend on many factors, such as incorporated expert knowledge or sample size.
Overall, we conclude that both the PC prior and the Paul et al. prior are fairly robust under the chosen
misspecification setting.

5. Diagnosis of bladder cancer using the telomerase marker

In this section, we apply the methodology to a data set presented and analysed by Glas et al. [26]. The
data set contains a meta-analysis of 10 case control studies in which the telomerase marker is used for
the diagnosis of bladder cancer (Table IV).

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

Table IV. Meta-analysis in which the telomerase

marker is used to diagnose bladder cancer taken
from Glas et al. [26].
Study TP FN TN FP
Ito et al. (1998) 25 8 25 1
Rahat et al. (1999) 17 4 11 3
Kavaler et al. (1998) 88 16 31 16
Yoshida et al. (1997) 16 10 80 3
Ramakumar et al. (1999) 40 17 137 1
Landman et al. (1998) 38 9 24 6
Kinoshita et al. (1997) 23 19 12 0
Gelmini et al. (2000) 27 6 18 2
Cheng et al. (2000) 14 3 29 3
Cassel et al. (2001) 37 7 7 22
TP, true positives; FN, false negatives; TN, true nega-
tives; FP, false positives.

This data set has been analysed in various publications, and estimation problems were found when
using frequentist approaches [5, 9]. Paul et al. [5] and Riley et al. [9] used SAS PROC NLMIXED and
tried different starting values resulting in an estimated between-study correlation of −1 and 95% confi-
dence intervals from −1 to 1. The small number of studies and the partly rare data lead to instabilities in
the estimation procedure. There are frequentist approaches that lead to more stable estimates. For exam-
ple, Kuss et al. [27] proposed to use beta-binomial distributions for the marginal numbers of TP and TN
and links these margins by a bivariate copula distribution. Chen et al. [28] proposed a composite like-
lihood approach, which resolves the non-convergence problem because there is no longer a correlation
parameter involved in the model.
We apply a Bayesian approach, where we use PC priors to incorporate intuitively prior information into
the analysis. For this application, we keep the PC(3, 0.05) prior for the variance parameters as motivated
in Section 3, as we believe that both sensitivity and specificity lie with 95% probability in the range
[0.5, 0.95]. Motivating the prior for the correlation is more challenging. Here, knowledge from a clinical
epidemiologist could be directly incorporated. We believe that a negative base model is sensible because
of potential threshold variations between studies and keep 𝜌0 = −0.2 [26]. At the same time, we have no
strong knowledge regarding the variation around this value and assign the PC 1 prior for the correlation
parameter. This prior is more informative than PC 0 but less than PC 2 and PC 3. The model and all prior
settings are specified using the R-package meta4diag that uses INLA for full Bayesian inference [12].
The R-code for implementing the model is shown and explained in the Supporting Information (Part A).
Alternatively, the graphical user interface can be used avoiding any R programming [12].
Figure 10 shows the posterior marginal distributions for all parameters compared to MCMC samples
obtained by JAGS [29]. The MCMC approach needed at least 800 000 samples to converge and about
3.5 h for 1000 000 samples while INLA needed 2 s on the same machine. It can be seen that INLA and
MCMC coincide very well.
The posterior summary estimates for sensitivity and specificity obtained by INLA are 0.77 (95% CI:
[0.70, 0.82]) and 0.91 (95% CI: [0.77, 0.97]), respectively, and are shown as summary point (black filled
point) in Figure 11. The posterior correlation was estimated to −0.87 (95% CI: [−0.99, −0.20]) and is
slightly lower than −0.89, which was obtained by Paul et al. [5]. However, the credible interval is slightly
smaller. The strong negative correlation indicates a threshold effect that agrees with the findings of Glas
et al. [26]. As there are no covariates involved, the parameter estimates can be transformed to the output
of an SROC analysis [30]. The resulting SROC curve together with a joint 95% credible and prediction
region is shown in Figure 11.
We also compare the estimates to those obtained by other existing model approaches (Table V). Ignor-
ing the correlation in the model using the composite likelihood approach [28] leads to more concentrate
estimates of the variance parameters compared with taking the correlation into account. Estimates for
sensitivity and specificity remain similar. The beta-binomial model assuming a Plackett copula results in
specificity estimates that are closer to the results of the original analysis than those obtained by the other
methods, while correlation estimates are closer when assuming a Gaussian copula. It is worth to note
that the original analysis, as well as the beta-binomial model and the composite likelihood approach, are

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

Figure 10. Posterior marginals obtained by integrated nested Laplace approximation (solid lines) and correspond-
ing histograms of 1 000 000 samples obtained from a Markov Chain Monte Carlo run with 5000 burn-in iterations
in JAGS. As prior distributions, the PC 1 prior for the correlation parameter and the PC prior with hyperparameter
u = 3 and 𝛼 = 0.05 for the variance parameters were used.

Figure 11. SROC plot for telomerase data obtained from meta4diag using the PC 1 prior for the correlation
parameter and the PC(3, 0.05) prior for the variance parameters. Shown are observed study-specific sensitivity
and specificity values (black cross: study sizes are indicated by the grey bubble), SROC line (black solid line),
overall sensitivity and specificity estimates from the bivariate model (black dot), credible region (black dashed
line) and prediction region (grey dotted line).

frequentist approaches, and so is metandi, which implements the bivariate meta-analysis using STATA
[31]. Similar to using SAS PROC NLMIXED also metandi seems to have convergence issues resulting
in an estimated between-study correlation of −1 and 95% confidence intervals from −1 to 1. Implement-
ing the bivariate model in a Bayesian setting using either Paul et al. priors or PC priors results in similar
estimates for sensitivity and specificity. From PC 1 to PC 3, the estimates of the correlation parameter are

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

Table V. Mean within 95% confidence intervals for the frequentist approaches (lines 1–5) and posterior
median within 95% credible intervals for the Bayesian approaches (lines 6–9).
Method Sensitivity Specificity 𝜎𝜙2 𝜎𝜓2 𝜌

Original analysis 0.71 0.750.79 0.71 0.860.94 — — − − 0.73−

Beta-binomial copula (Gauss) [27] 0.75 0.790.83 0.66 0.820.91 — — −0.88 − 0.71−0.38
Beta-binomial copula (Plackett) [27] 0.72 0.780.82 0.70 0.870.95 — — −0.96 − 0.85−0.56
Composite likelihood [28] 0.70 0.770.82 0.86 0.900.93 0.04 0.150.47 1.85 2.753.64 —
Bivariate model (Metandi) 0.70 0.770.82 0.75 0.920.97 0.05 0.180.73 1.09 3.3110.1 −1.00 − 1.00+1.00
Bivariate model (Paul) [5] 0.69 0.770.83 0.74 0.910.98 0.04 0.170.63 1.10 3.109.49 −0.99 − 0.89−0.18
Bivariate model (PC 1) 0.70 0.770.82 0.77 0.910.97 0.05 0.190.71 1.13 3.008.71 −0.99 − 0.87−0.20
Bivariate model (PC 2) 0.70 0.770.83 0.77 0.910.97 0.04 0.180.70 1.12 2.918.30 −0.99 − 0.80−0.15
Bivariate model (PC 3) 0.69 0.770.83 0.77 0.910.97 0.04 0.180.70 1.11 2.868.13 −0.98 − 0.76−0.16
Shown are the results for all parameters for the telomerase data from Glas et al. [26].

shrinked more to the base model −0.2, as the priors become more informative. In contrast, the estimates
of 𝜎𝜓2 become smaller.

6. Applicability of PC priors to extended model formulations

The PC priors presented are particularly useful for the bivariate model where available prior information
on variance components and about a commonly negative correlation between sensitivity and specificity
can be directly incorporated into the corresponding hyperprior to stabilise inference. Considering cohort
studies the application of penalised complexity priors in a trivariate model, where prevalence, sensitivity
and specificity are modelled jointly [32], is of interest. However, this is not straightforward as the rela-
tionship between prevalence and the accuracy parameters is not clear. Furthermore, care must be taken to
ensure that the resulting covariance matrix is positive definite. Specifying separate priors for each vari-
ance and each correlation component is therefore no longer directly possible. However, as discussed in
general, for example, by Barnard et al. [21], and in the bivariate random-effects meta-analysis setting by
Burke et al. [20], we still think that separate priors should be used for the between-study variances and
the correlation matrix instead of an inverse Wishart prior for the entire covariance matrix. Thus, simi-
lar PC priors as used here in the bivariate setting could be used for between-study variances. Regarding
the correlation parameters, we might formulate univariate PC priors step-wise for the three correlation
parameters in a conditional fashion. Alternatively, we propose to follow Simpson et al. [16, Section 7]
and formulate a multivariate PC prior for the correlation matrix. Here, the default base model would cor-
respond to using the identity matrix as correlation matrix. This corresponds to a base model where the
correlation between sensitivity and specificity is zero. In practice, we might want to extend this procedure
to a base model where the correlation between sensitivity and specificity is equal to −0.2, for example.
Another obvious and feasible extension is to account for differing and imperfect reference standards
(e.g. [33–36]). Considering the bivariate model the number of diseased and non-diseased patients, and
therefore TP, TN, FP and FN are unknown and can thus not be directly modelled using Equation (1).
Menten et al. [35] proposed to extend the bivariate models used here by adding a latent class model
with informative priors to model the reference test accuracy. Consequently, a separate model is needed
to model sensitivity and specificity of the reference test that can be done similar as for the index test.
Three different approaches have been proposed to incorporate prior information, complete pooling, no-
pooling and partial pooling. The use of PC priors might be useful again to translate the opinions from
experts into sound prior distributions. Furthermore, the complete pooling would be directly included in
the partial pooling approach as a PC prior for the standard deviation would allow shrinkage to zero if not
indicated otherwise by the data.

7. Discussion

One of the main challenges of bivariate meta-analysis is that often only few studies are available and/or
data within these studies are sparse. Frequentist approaches may encounter problems leading to unreliable
parameter estimates such as correlation estimates close to the boundary. There exist different approaches
to address this issue. Chen et al. [28] propose a composite likelihood approach that avoids taking the
correlation between sensitivity and specificity into account. Kuss et al. [27] found that a beta-binomial

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

model linked with a copula approach is superior to the bivariate meta-regression model when estimated in
a frequentist setting. However, their results are sensitive with respect to the copula class used and expert
knowledge is hard to incorporate. Zapf et al. [8] propose a nonparametric frequentist model that shows
good convergence properties but does not yet allow the incorporation of covariate or expert knowledge.
Chen et al. [37] propose a marginal beta-binomial model based on the composite likelihood approach
and find a robust performance when the model is misspecified.
In this paper, we use a Bayesian approach where additional information is incorporated into the model
by the means of prior distributions. Thorlund et al. [15] propose to use moderately informative priors
for the variance parameters in the bivariate model instead of uninformative priors when the number of
studies is small. This finding was supported by Paul et al. [5]. However, assuming inverse gamma priors
for the variance parameters and a normal distribution for a transformation of the correlation parameter,
it is not clear how to specify the corresponding prior parameters intuitively or how to link them to avail-
able expert knowledge. Here, we investigated the use of PC priors [16], which do not directly rely on a
parametric distribution for the parameter of interest, but follow the principle of shrinkage to a so-called
base model. We specified the priors separately for the variance and the correlation parameters. Thinking
about the correlation parameter in most applications, we may shrink to zero, which means no correlation
and corresponds to the simplest model. In bivariate meta-analysis, we generally may want to shrink to
a negative correlation 𝜌0 as a base model, which is motivated by a potential threshold variation between
studies [26]. Defining the base model 𝜌0 , we further expressed our beliefs about deviating from this model
in terms of probability contrasts such as P(𝜌 < 𝜌0 ) = 𝛼. Adding one piece of further information such
as a belief about the tail behaviour fully specified the PC prior. For the variance parameters, zero was
used as the intuitive base model, which means no random effect, and the hyperparameter controlling the
deviation from zero were specified by using a priori range in which we believed sensitivity or specificity
should lay.
We used an extensive simulation study to compare the performance of differently defined PC priors
to commonly used prior distributions. We found that bias and MSE of overall sensitivity and specificity
are not very sensitive to the choice of different priors for the variance and correlation parameters. How-
ever, we observed improved coverage probabilities when using PC priors for the variance components.
Furthermore, using PC priors lead to a lower MSE and better coverage probabilities for the variance
parameters compared with an inverse gamma distribution.
When there are only a few studies, changing the prior distribution for the correlation parameter will
have an effect on the posterior marginals of the correlation. Our findings indicate that no prior is supe-
rior in all settings. Sensitivity and specificity estimates are not sensitive to the choice of prior for the
correlation parameter. Also, results for the log standard deviation are very similar. Only when the true
values for both sensitivity and specificity are 0.9 and we only have 10 studies in a meta-analysis may
we observe worse coverage probabilities for the PC prior, while bias and MSE are almost not affected.
If the true correlation is in the neighbourhood of the base model, more informative PC priors perform
better, but their performance deteriorates if the true correlation is further apart. While the PC prior per-
forms slightly worse when the true correlation is strongly positive or close to −1, it is beneficial for all
other true correlation values. That means for the values we regard sensible if not indicated otherwise by
the data. These results were also confirmed in a sparse data simulation setting, where the proportion of
two-by-two tables with at least one entry equal to zero is high.
One may wonder why the choice of hyperpriors is important if the main parameters of interest, namely,
overall sensitivity and specificity, are almost not affected when changing them. However, variance and
correlation parameters become important when interest lies in the joint distribution of sensitivity and
specificity in order to predict a new outcome, say. Furthermore, accurate estimates are needed to compute
the SROC curve. Thus, we feel it is important to get good estimates also for the correlation and variance
parameters, especially when the sample size is small.
We also considered a misspecification simulation scenario where a certain percentage of the studies
were simulated under one correlation parameter and the remaining part under another one. Here, we
found that performance measures for all parameters, except the correlation parameter, are very similar for
all prior distributions. Because there is no longer a true value for the correlation parameter, performance
measure could not be computed.
For inference, we used INLA, which replaces MCMC sampling by approximating the posterior
marginals directly [5, 11]. This is particularly attractive when many data sets, for example, in a sim-
ulation study with different scenarios, should be analysed as MCMC would be very time consuming.
The recent R-package meta4diag [12] implements Bayesian bivariate meta-analysis based on INLA

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

and offers a purpose-built user interface to make the full-functionality available without requiring any R
knowledge [12].
Care should be taken when the number of studies involved is really small. The bivariate model is a
complex model and data are needed to estimate all present parameters. Agreeing with Takwoingi et al.
[38], we propose to use a simplified model for meta-analyses including few studies, say less than 10. One
potential simplification is to specify separate models for sensitivity and specificity that corresponds to
assuming that the correlation parameter for the bivariate random effects is equal to zero. Alternatively,
we may fix the correlation to a suitable (negative) value. This feature is also supported in meta4diag.
In future work, we would like to analyse sensitivity of PC priors more systematically based on the work
of Roos and Held [39] and Roos et al. [40].

Appendix A: Derivation of the penalised complexity prior for the correlation

parameter
In this appendix, we derive the penalised complexity prior for the correlation parameter in the bivariate
model. Consider the bivariate random effects model in Equation (1), where the bivariate random effect
is assumed to be normally distributed with zero-mean and covariance matrix 𝚺. We follow the separation
strategy given in Equation (2) and consider only the correlation matrix 𝐑 in this section. Let N(𝟎, 𝐑b )
and N(𝟎, 𝐑f ) denote the base and flexible model, respectively
( ) ( )
1 𝜌0 1𝜌
𝐑b = and 𝐑f = , (A.1)
𝜌0 1 𝜌1

where 𝜌0 is fixed to a (negative) value in which one believes with respect to the application, while 𝜌 is a
random variable.
The increased complexity between N(𝟎, 𝐑f ) and N(𝟎, 𝐑b ) is measured by the KLD that is
{ ( )}
( ) 1 ( ) |𝐑f |
KLD N(𝟎, 𝐑f )||N(𝟎, 𝐑b ) = tr 𝐑−1
b 𝐑f − 2 − ln .
2 |𝐑b |

Substituting 𝐑b and 𝐑f from Equation (A.1) into the KLD, we get

( ) 1 − 𝜌0 𝜌 1 ( ) 1 ( )
KLD N(𝟎, 𝐑f )||N(𝟎, 𝐑b ) = − 1 − ln 1 − 𝜌2
+ ln 1 − 𝜌2
0 ,
1 − 𝜌20 2 2
√
and d(𝜌) = 2KLD(𝜌) is the distance between the base model and flexible model. In order to specify a
suitable distribution on the distance, we follow the rules according to Simpson et al. [16] and assume a
constant penalisation, such that the prior satisfies

𝜋d (d + 𝛿)
= r𝛿 , d, 𝛿 ⩾ 0
𝜋d (d)

for some constant decay-rate 0 < r < 1. This leads to an exponential prior on the distance scale, 𝜋(d) =
𝜆 exp(−𝜆d), for r = exp(−𝜆). With a transformation of parameter, we obtain the prior density for 𝜌,
| 𝜕d(𝜌) |
𝜋(𝜌) = 𝜆 exp(−𝜆d(𝜌)) || |,
|
| 𝜕𝜌 |
where ( )
𝜕d(𝜌) 𝜌 𝜌0 /√
= − 2KLD .
𝜕𝜌 1−𝜌2
1 − 𝜌20
This density implies identical behaviour on both sides of 𝜌0 . In order to distribute the density unevenly
around 𝜌0 but keeping the assumption of a constant rate penalisation, we use two one-sided exponential
priors 𝜋1 (d; 𝜆1 ) and 𝜋2 (d; 𝜆2 ) on the distance with rate parameters 𝜆1 and 𝜆2 . The functions 𝜋1 and 𝜋2 are
used to construct the prior for the correlation when 𝜌 ⩽ 𝜌0 and 𝜌 > 𝜌0 , respectively. The parameters 𝜆1

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

and 𝜆2 indicate the decay rate of the deviation from the base model when 𝜌 ⩽ 𝜌0 and 𝜌 > 𝜌0 , respectively.
In order to merge these two exponential priors in the distance scale into one prior distribution 𝜋(𝜌; 𝜆1 , 𝜆2 )
in the correlation scale, we define

⎧ | 𝜕d(𝜌) |
⎪ 𝜔1 𝜋1 (d(𝜌); 𝜆1 ) || 𝜕𝜌 || , 𝜌 ⩽ 𝜌0
𝜋(𝜌; 𝜆1 , 𝜆2 ) = ⎨ | 𝜕d(𝜌) |
⎪ 𝜔2 𝜋2 (d(𝜌); 𝜆2 ) || 𝜕𝜌 || , 𝜌 > 𝜌0
⎩

𝜌 1
where 𝜔1 = ∫−10 𝜋(𝜌; 𝜆1 , 𝜆2 )d𝜌 and 𝜔2 = ∫𝜌 𝜋(𝜌; 𝜆1 , 𝜆2 )d𝜌 ∈ [0, 1], are the probability when 𝜌 ⩽ 𝜌0 and
0
𝜌 > 𝜌0 , and satisfy 𝜔1 + 𝜔2 = 1 and 𝜔1 𝜆1 = 𝜔2 𝜆2 . Thus we have

𝜆2 𝜆1
𝜔1 = and 𝜔2 = .
𝜆1 + 𝜆2 𝜆 1 + 𝜆2

To specify the hyperparameters 𝜆1 and 𝜆2 , we have to specify a density behaviour on each side of 𝜌0 .
The idea behind is to control the prior mass in the tail(s) and(or) the probability that 𝜌 < 𝜌0 . We define 3
strategies to obtained the value of 𝜆1 and 𝜆2 .
The first strategy is defined under condition P(𝜌 ⩽ umin | − 1 < umin < 𝜌0 ) = 𝛼1 where umin is a ‘left
tail event’ and 𝛼1 is the weight on this event. From the condition, we obtain

P(𝜌 ⩽ umin | − 1 < umin < 𝜌0 ) = 𝛼1

𝜌0 | 𝜕d(𝜌) |
=⇒ 𝜔1 𝜋1 (d(𝜌); 𝜆1 ) || | d𝜌 = 𝜔1 − 𝛼1
|
∫umin | 𝜕𝜌 |
d(umin )
=⇒ 𝜔1 𝜆1 exp(−𝜆1 d)dd = 𝜔1 − 𝛼1
∫0
=⇒𝜔1 − 𝜔1 exp(𝜆1 d(umin )) = 𝜔1 − 𝛼1
log(𝜔1 ) − log(𝛼1 )
=⇒𝜆1 = .
d(umin )

The third strategy is defined under condition P(𝜌 ⩾ umax |𝜌0 < umax < 1) = 𝛼2 where umax is a ‘right tail
event’ and 𝛼2 is the weight on this event. From the condition, we obtain

P(𝜌 ⩾ umax |𝜌0 < umax < 1) = 𝛼2

umax | 𝜕d(𝜌) |
=⇒ 𝜔2 𝜋2 (d(𝜌); 𝜆2 ) || | d𝜌 = 𝜔2 − 𝛼2
|
∫𝜌0 | 𝜕𝜌 |
d(umax )
=⇒ 𝜔2 𝜆2 exp(−𝜆2 d)dd = 𝜔2 − 𝛼2
∫0
=⇒𝜔2 − 𝜔2 exp(𝜆2 d(umax )) = 𝜔2 − 𝛼2
log(1 − 𝜔1 ) − log(𝛼2 )
=⇒𝜆2 = .
d(umax )

The second strategy is defined under conditions P(𝜌 ⩽ umin | − 1 < umin < 𝜌0 ) = 𝛼1 and P(𝜌 ⩾
umax |𝜌0 < umax < 1) = 𝛼2 without giving any information about 𝜔. In order to obtain the value for 𝜆1
and 𝜆2 , we solve the following equations numerically

⎧ 𝜆1 = log(𝜔1 )−log(𝛼1 )
⎪ d(umin )
log(1−𝜔1 )−log(𝛼2 )
⎨ 𝜆2 =
⎪ d(umax )
⎩ 𝜔1 𝜆1 = (1 − 𝜔1 )𝜆2 .

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
J. GUO, A. RIEBLER AND H. RUE

References
1. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. John Wiley & Sons: Chichester,
UK, 2009.
2. Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity
and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology 2005;
58(10):982–990.
3. Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model
approach. Journal of Clinical Epidemiology 2006; 59(12):1331–1332.
4. Verde PE. Meta-analysis of diagnostic test data: a bivariate Bayesian modeling approach. Statistics in Medicine 2010;
29(30):3088–3102.
5. Paul M, Riebler A, Bachmann LM, Rue H, Held L. Bayesian bivariate meta-analysis of diagnostic test studies using
integrated nested Laplace approximations. Statistics in Medicine 2010; 29(12):1325–1339.
6. Menke J. Bivariate random-effects meta-analysis of sensitivity and specificity with the Bayesian SAS PROC MCMC:
Methodology and empirical evaluation in 50 meta-analyses. Medical Decision Making 2013; 33(5):692–701.
7. Eusebi P, Reitsma JB, Vermunt JK. Latent class bivariate model for the meta-analysis of diagnostic test accuracy studies.
BMC Medical Research Methodology 2014; 14(88):9.
8. Zapf A, Hoyer A, Kramer K, Kuss O. Nonparametric meta-analysis for diagnostic accuracy studies. Statistics in Medicine
2015; 34(29):3831–3841.
9. Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation
of between-study correlation. BMC Medical Research Methodology 2007; 7(1):15.
10. Turner RM, Jackson D, Wei Y, Thompson SG, Higgins J. Predictive distributions for between-study heterogeneity and
simple methods for their application in Bayesian meta-analysis. Statistics in Medicine 2015; 34(6):984–998.
11. Rue H, Martino S, Chopin N. Approximate Bayesian inference for latent Gaussian models by using integrated nested
Laplace approximations. Journal of The Royal Statistical Society: Series B (Statistical Methodology) 2009; 71(2):319–392.
12. Guo J, Riebler A. meta4diag: Bayesian bivariate meta-analysis of diagnostic test studies for routine practice. Technical
Report. arXiv preprint arXiv:1512.06220, 2015.
13. Harbord RM. Commentary on ‘multivariate meta-analysis: potential and promise’. Statistics in Medicine 2011;
30(20):2507–2508.
14. Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the
use of vague prior distributions in MCMC using WinBUGS. Statistics in Medicine 2005; 24(15):2401–2428.
15. Thorlund K, Thabane L, Mills EJ. Modelling heterogeneity variances in multiple treatment comparison meta-analysis–are
informative priors the better solution? BMC Medical Research Methodology 2013; 13(1):14.
16. Simpson DP, Rue H, Martins TG, Riebler A, Sørbye SH. Penalising model component complexity: A principled, practical
approach to constructing priors. arXiv preprint arXiv:1403.4630 2014.
17. Alvarez I, Niemi J, Simpson M. Bayesian inference for a covariance matrix. Proceedings of Annual Conference on Applied
Statistics in Agriculture, 2014, 71–82.
18. Barthelmé S. Why an inverse-Wishart prior may not be such a good idea, 2012. https://dahtah.wordpress.com/2012/03/07/
why-an-inverse-wishart-prior-may-not-be-such-a-good-idea/#comment-72, last accessed: 16.08.2016.
19. Wei Y, Higgins J. Bayesian multivariate meta-analysis with multiple outcomes. Statistics in Medicine 2013; 32(17):
2911–2934.
20. Burke DL, Bujkiewicz S, Riley RD. Bayesian bivariate meta-analysis of correlated effects: impact of the prior distributions
on the between-study correlation, borrowing of strength, and joint inferences. Statistical Methods in Medical Research
2016. https://doi.org/10.1177/0962280216631361.
21. Barnard J, McCulloch R, Meng X-L. Modeling covariance matrices in terms of standard deviations and correlations, with
application to shrinkage. Statistica Sinica 2000; 10(4):1281–1311.
22. Jackson D, Riley R, White IR. Multivariate meta-analysis: potential and promise. Statistics in Medicine 2011; 30(20):
2481–2498.
23. Kullback S, Leibler RA. On information and sufficiency. The Annals of Mathematical Statistics 1951; 22(1):79–86.
24. Menke J. Bivariate random-effects meta-analysis of sensitivity and specificity with SAS PROC GLIMMIX. Methods of
Information in Medicine 2010; 49(1):54–64.
25. Diaz M. Performance measures of the bivariate random effects model for meta-analyses of diagnostic accuracy. Computa-
tional Statistics & Data Analysis 2015; 83:82–90.
26. Glas AS, Roos D, Deutekom M, Zwinderman AH, Bossuyt PMM, Kurth KH. Tumor markers in the diagnosis of primary
bladder cancer. A systematic review. The Journal of Urology 2003; 169(6):1975–1982.
27. Kuss O, Hoyer A, Solms A. Meta-analysis for diagnostic accuracy studies: a new statistical model using beta-binomial
distributions and bivariate copulas. Statistics in Medicine 2014; 33(1):17–30.
28. Chen Y, Liu Y, Ning J, Nie L, Zhu H, Chu H. A composite likelihood method for bivariate meta-analysis in diagnostic
systematic reviews. Statistical Methods in Medical Research 2014; 26(2):914–930.
29. Plummer M. JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Proceedings of the 3rd
International Workshop on Distributed Statistical Computing, Vol. 124, Vienna, 2003.
30. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations.
Statistics in Medicine 2001; 20(19):2865–2884.
31. Harbord RM, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata Journal
2009; 9(2):211–229.
32. Chu H, Nie L, Cole SR, Poole C. Meta-analysis of diagnostic accuracy studies accounting for disease prevalence: alternative
parameterizations and model selection. Statistics in Medicine 2009; 28(18):2384–2399.

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017
 J. GUO, A. RIEBLER AND H. RUE

33. Walter SD, Irwig L, Glasziou PP. Meta-analysis of diagnostic tests with imperfect reference standards. Journal of Clinical
Epidemiology 1999; 52(10):943–951.
34. Chu H, Chen S, Louis TA. Random effects models in a meta-analysis of the accuracy of two diagnostic tests without a gold
standard. Journal of the American Statistical Association 2009; 104(486):512–523.
35. Menten J, Boelaert M, Lesaffre E. Bayesian meta-analysis of diagnostic tests allowing for imperfect reference standards.
Statistics in Medicine 2013; 32(30):5398–5413.
36. Held U, Brunner F, Steurer J, Wertli MM. Bayesian meta-analysis of test accuracy in the absence of a perfect reference
test applied to bone scintigraphy for the diagnosis of complex regional pain syndrome. Biometrical Journal 2015; 57(6):
1020–1037.
37. Chen Y, Hong C, Ning Y, Su X. Meta-analysis of studies with bivariate binary outcomes: a marginal beta-binomial model
approach. Statistics in Medicine 2016; 35(1):21–40.
38. Takwoingi Y, Guo B, Riley RD, Deeks JJ. Performance of methods for meta-analysis of diagnostic test accuracy with few
studies or sparse data. Statistical Methods in Medical Research 2015. https://doi.org/10.1177/0962280215592269.
39. Roos M, Held L. Sensitivity analysis in Bayesian generalized linear mixed models for binary data. Bayesian Analysis 2011;
6(2):259–278.
40. Roos M, Martins TG, Held L, Rue H. Sensitivity analysis for Bayesian hierarchical models. Bayesian Analysis 2015;
10(2):321–349.

Supporting information

Additional supporting information may be found online in the supporting information tab for this article.

Copyright © 2017 John Wiley & Sons, Ltd. Statist. Med. 2017