See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/351358356

Meta-analysis of virtual reality exposure therapy for social anxiety disorder

Article  in  Psychological Medicine · January 2022

DOI: 10.1017/S0033291721001690

CITATIONS READS

3 310

5 authors, including:

Nexhmedin Morina Isabel L Kampmann

University of Münster University of Münster
139 PUBLICATIONS   4,791 CITATIONS    19 PUBLICATIONS   470 CITATIONS   

SEE PROFILE SEE PROFILE

Paul Emmelkamp Thole Hoppen

University of Amsterdam University of Münster
619 PUBLICATIONS   23,796 CITATIONS    17 PUBLICATIONS   146 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

CONNECT View project

Do the blues make me worse off than you? A systematic review on the association of social comparison with depression and anxiety View project

All content following this page was uploaded by Nexhmedin Morina on 06 May 2021.

The user has requested enhancement of the downloaded file.

 Copy Edited by: S.N.S.

1
Psychological Medicine Meta-analysis of virtual reality exposure 2

cambridge.org/psm
therapy for social anxiety disorder 3
4
5
6
Nexhmedin Morina1 , Isabel Kampmann1, Paul Emmelkamp2, Corrado Barbui3
7
Correspondence and Thole H. Hoppen1 8
Cite this article: Morina N, Kampmann I,
Q1 9
1
Emmelkamp P, Barbui C, Hoppen TH (2021). Institute of Psychology, University of Münster, Münster, Germany; 2Department of Clinical Psychology, University 10
Meta-analysis of virtual reality exposure of Amsterdam, Amsterdam, the Netherlands and 3WHO Collaborating Centre for Research and Training in Mental 11
therapy for social anxiety disorder. Health and Service Evaluation, Department of Neuroscience, Biomedicine and Movement Sciences, Section of
12
Psychological Medicine 1–3. https://doi.org/ Psychiatry, University of Verona, Verona, Italy
10.1017/S0033291721001690 13
14
Received: 17 March 2021 15
Revised: 16 April 2021 Virtual reality exposure therapy (VRET) for social anxiety disorder (SAD) is a new
16
Accepted: 18 April 2021 technology-based form of exposure therapy that uses computer-generated virtual social envir-
17
onments as a means of systematically exposing patients to feared stimuli. An advantage of
Author for correspondence: 18
Nexhmedin Morina, VRET relative to exposure in vivo is that virtual exposure takes place in a highly controllable
19
E-mail: [email protected] environment. Recently, Horigome et al. (2020) published a meta-analysis on the efficacy of
20
VRET for SAD and reported that 22 studies [11 randomized controlled trials (RCTs) and
21
11 non-randomized studies] fulfilled their inclusion criteria. We have previously systematically
22
reviewed the literature on the efficacy of VRET for SAD (Carl et al., 2019; Emmelkamp,
23
Meyerbröker, & Morina, 2020; Kampmann, Emmelkamp, & Morina, 2016b) and were sur-
24
prised to read that a total 22 trials have investigated the efficacy of VRET for SAD.
25
Therefore, we investigated with great interest potential reasons for the discrepancy in the num-
26
ber of RCTs in the review by Horigome et al. and our reviews.
27
Horigome and colleagues reported the following inclusion criteria for their meta-analyses:
28
(1) a diagnosis of SAD, fear of public speaking, or public speaking anxiety; (2) VRET consisted
29
of at least three sessions; and (3) study participants numbered a minimum of 10 patients. We
30
critically evaluated the studies in the meta-analysis by Horigome et al. by applying their inclu-
31
sion criteria and evaluating whether all of the treatments labeled as VRET did indeed consist
32
of at least 50% of virtual reality components. As a result, we argue that the publication by
33
Horigome and colleagues misleadingly implies that there is a larger body of literature on
34
the efficacy of VRET for SAD than there actually is.
35
First, a clarification regarding the total number of included studies is needed. In both the
36
abstract and the results section the authors reported that a total of 22 studies were included in
37
their analyses. However, this number relates to the number of publications that the authors
38
used for data extraction. A close look at Table 1 in Horigome et al. reveals that the authors
39
counted the trial by Anderson et al. (2013) as well as the one by Safir, Wallach, and
40
Bar-Zvi (2012) twice, respectively. Accordingly, they rather included a total of 20 studies
41
described in 22 publications. Second, we argue that two of the included studies were errone-
42
ously labeled as VRET. First, the study by Lister et al. (2010†1) used virtual reality for assess-
43
ment purposes only and their treatment did not comprise VRET. Second, the study by Yuen
44
et al. (20191) comprised of in vivo exposure in addition to VRET. More specifically, during
45
sessions, participants were asked to engage in exposure to a real audience using video-
46
conferencing. For homework, they received VRET while applying techniques of acceptance
47
and commitment therapy and were encouraged to engage in at least three in vivo exposure
48
exercises in-between sessions. Accordingly, <50% of treatment consisted of VRET and there-
49
fore any potential treatment efficacy might have resulted from other treatment elements than
50
VRET (Morina, Ijntema, Meyerbröker, & Emmelkamp, 2015). Finally, six studies (Denizci
51
Nazligul et al., 2019; Grillon et al., 2006; Harris et al., 2002; Kovar, 2018; North et al., 1998;
52
Roy et al., 20031) included <10 participants in VRET. Horigome et al. reported that ‘study par-
53
ticipants numbered 10 or more people’, without specifying whether this number referred to
54
the number of participants per condition or per trial. Research, however, strongly suggests
© The Author(s), 2021. Published by 55
Cambridge University Press. This is an Open that trials with small samples (i.e. low statistical power) reduce the likelihood that a statistically
56
Access article, distributed under the terms of significant result reflects a true effect (Ioannidis, 2005). In fact, treatment effect estimates are
57
the Creative Commons Attribution licence often significantly larger in trails with small samples (Dechartres, Trinquart, Boutron, &
(http://creativecommons.org/licenses/by/4.0), 58
Ravaud, 2013; Kjaergard, Villumsen, & Gluud, 2001; Turner, Bird, & Higgins, 2013). A min-
which permits unrestricted re- use, 59
distribution and reproduction, provided the
imum of 10 participants per condition was also applied in other related meta-analyses (e.g.
60
original article is properly cited. Kampmann et al., 2016b; Morina et al., 2015). Consequently, we excluded the six trials
61
with <10 patients per condition and argue that only 12 studies need to be included in the
62
meta-analysis. In addition, Horigome et al. compared the efficacy of VRET to waitlist
63
64
†
The notes appear after the main text. 65
2 Nexhmedin Morina et al

Table 1. Within and between group comparison results 66

a 2 67
Within-group ES k n ES 95% CI p I Q p PI
68
Baseline v. 69
70
Post-treatment 12 228 1.20 0.91–1.50 <0.001 53.02 23.68 0.014 0.40–2.01
71
3 months FU 3 61 1.17 0.69–1.66 <0.001 5.42 3.03 0.220 0.47–1.87 72
6 months FU 2 38 1.19 0.70–1.68 <0.001 0.00 0.03 0.867 0.70–1.68 73
74
12 months FU 2 58 1.06 0.30–1.82 <0.001 73.31 3.75 0.053 −0.13 to 2.25
75
72 months FU 1 Number of trials too small 76
Longest available FU b
5 110 1.06 0.71–1.41 <0.001 33.58 5.87 0.291 0.49–1.63 77
78
VRET v. waitlist
79
At post-treatment 5 222 0.88 0.43–1.34 <0.001 61.34 10.16 0.038 0.03–1.79 80
VRET v. exposure in vivo 81
82
at post 7 272 0.07 −0.32 to 0.17 0.562 10.65 6.57 0.362 −0.40 to 0.25
83
3 months FU 2 109 −0.64 −1.68 to 0.40 0.229 84.28 6.36 0.012 −2.34 to 1.07 84

6 months FU 1 Number of trials too small 85

86
12 months FU 2 127 −0.02 −0.37 to 0.33 0.889 0.00 0.39 0.533 −0.37; 0.22
87
72 months FU 1 Number of trials too small 88

Longest available FU b
4 206 −0.24 −0.83 to 0.35 0.426 76.70 11.21 0.011 −1.42 to 0.94 89
90
k, number of treatment arms; n, total number of participants in all relevant trials; PI, prediction interval.
91
Note: Effect sizes (ES) are reported as Hedges’ g; the I 2-statistic and Q-statistic are indicators of heterogeneity in percentages, with higher percentages indicating high heterogeneity.
a 92
Number of treatment arms (i.e. k) is exactly the same as number of studies.
b
Only the longest available follow-up data per trial was retained, except for Anderson et al. (2013) were the 12 months follow-up was retained rather than the exceptionally long 72 months 93
follow-up.
94
95
96
conditions and treatment as usual (TAU) combined. However, Effect sizes were further large when pre-assessment scores of 97
only one trial that used cognitive therapy as a comparison condi- social anxiety were compared to follow-up assessment scores at 98
tion (Wallach, Safir, & Bar-Zvi, 2011) was defined as TAU and the different time points. These findings indicate that VRET can sig- 99
remaining studies comprised waitlist conditions. Although it is nificantly reduce symptoms of social anxiety. Note, however, that 100
questionable to categorize cognitive therapy as TAU, given their uncontrolled effect sizes do not account for the impact of time on 101
definition of cognitive behavioral therapy involving exposure in symptoms, therefore controlled effect sizes need to be viewed as 102
vivo as the gold standard SAD treatment, we also argue that it more reliable when it comes to assessing treatment efficacy. 103
is clinically more useful to report on the efficacy of VRET as com- These analyses revealed that the pooled between-group effect 104
pared to waitlist only. size comparing VRET to waitlist at post-treatment was also 105
In light of these limitations, we deemed it important to reana- large. Finally, the comparison of VRET to exposure in vivo yielded 106
lyze the data on the efficacy of VRET for SAD. To this end, we no significant differences between the two treatments, neither at 107
included 12 studies in our meta-analysis, six of which were RCTs post-assessment nor at follow-up. 108
(Anderson et al., 2013; Bouchard et al., 2017; Kampmann et al., When conducting our analyses on treatment efficacy at 109
2016a; Klinger et al., 2005; Safir et al., 2012; Wallach et al., 2011) follow-up, we deviated from Horigome et al. to some extent. 110
and the remaining six were non-controlled studies (Anderson, First, we did not conduct a meta-analysis with one trial only 111
Zimand, Schmertz, & Ferrer, 2007; Gebara, de Barros-Neto, given the fact that a meta-analysis is per definition a technique 112
Gertsenchtein, & Lotufo-Neto, 2016; Geraets et al., 2019; Kim that combines the results of multiple studies. In fact, in previous 113
et al., 2017; Robillard, Bouchard, Dumoulin, Guitard, & Klinger, meta-analyses we have argued that a minimum of four trials is 114
2010; Stupar-Rutenfrans, Ketelaars, & van Gisbergen, 2017). needed for a meaningful meta-analysis (Morina et al., 2015; 115
To calculate uncontrolled effect sizes, the posttreatment or Morina, Koerssen, & Pollet, 2016). Second and related to this, 116
follow-up mean was subtracted from the pretreatment mean we decided to conduct an additional follow-up analysis with a lar- 117
and for controlled effect sizes the control group mean was sub- ger amount of trials by including only the longest available 118
tracted from the treatment group mean at posttreatment or follow-up data per trial [with the exception of Anderson et al. 119
follow-up, respectively, and divided by the pooled standard devi- (2013) were the 12 months follow-up was retained rather than 120
ation. Subsequently, the outcome was multiplied by a sample size the exceptionally long 72 months follow-up]. As can be seen in 121
correction factor J = 1–(3/(4df − 1)) to obtain the effect size Table 1, change from pretreatment to follow-up resulted in a 122
Hedges’ g. Analyses were completed with the metafor package large effect size across VRET treatments. Furthermore, there 123
(v.1.9.8) in R 3.5 using random-effects models given the hetero- was no significant difference between VRET and exposure in 124
geneity of the studies (R Core Team, 2015; Viechtbauer, 2010). vivo at follow-up. 125
Detailed results of our analyses are presented in Table 1. Our within-group comparison findings are mostly in line with 126
Across all interventions, a large pre–post effect size was found. those reported by Horigome et al. Nonetheless, our findings show 127
Correspondence 3

a stronger treatment efficacy of VRET from pre-treatment to post- Horigome, T., Kurokawa, S., Sawada, K., Kudo, S., Shiga, K., Mimura, M., & 128
treatment and to follow-up. With respect to between-group com- Kishimoto, T. (2020). Virtual reality exposure therapy for social anxiety dis- 129
parisons, our results vary from those reported by Horigome et al. order: A systematic review and meta-analysis. Psychological Medicine, 50 130
(15), 2487–2497. https://doi.org/10.1017/S0033291720003785.
to some greater extent than the within-group findings. First, the 131
Ioannidis, J. P. A. (2005). Why most published research findings are false.
comparison of VRET to wait-list is smaller in our meta-analysis, 132
PLoS Medicine, 2(8), e124.
albeit still in the range of a large effect. Second, our analyses of the Kampmann, I. L., Emmelkamp, P. M., Hartanto, D., Brinkman, W-P, Zijlstra,
133
efficacy of VRET relative to exposure in vivo produced insignifi- B. J., & Morina, N. (2016a). Exposure to virtual social interactions in the 134
cant results only, whereas Horigome et al. report significant treatment of social anxiety disorder: A randomized controlled trial. 135
results between VRET and exposure in vivo at follow-up that go Behaviour Research and Therapy, 77, 147–156. https://doi.org/10.1016/j. 136
in both directions. Yet, we need to interpret these results with brat.2015.12.016. 137
caution as these are based on a limited number of RCTs. Kampmann, I. L., Emmelkamp, P. M., & Morina, N. (2016b). Meta-analysis of 138
technology-assisted interventions for social anxiety disorder. Journal of 139
Financial support. We have received no financial support for this paper. Anxiety Disorders, 42, 71–84. https://doi.org/10.1016/j.janxdis.2016.06.007. 140
Kim, H. E., Hong, Y-J, Kim, M-K, Jung, Y. H., Kyeong, S., & Kim, J-J (2017). 141
Effectiveness of self-training using the mobile-based virtual reality program
142
Notes in patients with social anxiety disorder. Computers in Human Behavior, 73,
143
614–619. https://doi.org/10.1016/j.chb.2017.04.017.
1 Due to the word limit, we refer here to the reference list in Horigome at al. 144
Kjaergard, L. L., Villumsen, J., & Gluud, C. (2001). Reported methodologic
(2020). 145
quality and discrepancies between large and small randomized trials in
meta-analyses. Annals of Internal Medicine, 135(11), 982–989. 146
Klinger, E., Bouchard, S., Légeron, P., Roy, S., Lauer, F., Chemin, I., & Nugues, 147
References
P. (2005). Virtual reality therapy versus cognitive behavior therapy for social 148
Anderson, P. L., Price, M., Edwards, S. M., Obasaju, M. A., Schmertz, S. K., phobia: A preliminary controlled study. Cyberpsychology & Behavior, 8(1), 149
Zimand, E., & Calamaras, M. R. (2013). Virtual reality exposure therapy 76–88. 150
for social anxiety disorder: A randomized controlled trial. Journal of Morina, N., Ijntema, H., Meyerbröker, K., & Emmelkamp, P. M. (2015). Can 151
Consulting and Clinical Psychology, 81(5), 751–760. https://doi.org/10. virtual reality exposure therapy gains be generalized to real-life? A
152
1037/a0033559. meta-analysis of studies applying behavioral assessments. Behaviour
153
Anderson, P. L., Zimand, E., Schmertz, S. K., & Ferrer, M. (2007). Usability Research and Therapy, 74, 18–24. https://doi.org/10.1016/j.brat.2015.08.010.
and utility of a computerized cognitive-behavioral self-help program for Morina, N., Koerssen, R., & Pollet, T. V. (2016). Interventions for children and 154
public speaking anxiety. Cognitive and Behavioral Practice, 14(2), 198– adolescents with posttraumatic stress disorder: A meta-analysis of compara- 155
207. https://doi.org/10.1016/j.cbpra.2006.02.006. tive outcome studies. Clinical Psychology Review, 47, 41–54. https://doi.org/ 156
Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, E., Forget, H., 10.1016/j.cpr.2016.05.006. 157
… Roucaut, F. X. (2017). Virtual reality compared with in vivo exposure in R Core Team. (2015). R: A language and environment for statistical comput- 158
the treatment of social anxiety disorder: A three-arm randomised controlled ing. Retrieved from http://www.R-project.org/. 159
trial. The British Journal of Psychiatry, 210(4), 276–283. Robillard, G., Bouchard, S., Dumoulin, S., Guitard, T., & Klinger, E. (2010). 160
Carl, E., Stein, A. T., Levihn-Coon, A., Pogue, J. R., Rothbaum, B., Using virtual humans to alleviate social anxiety: Preliminary report from
161
Emmelkamp, P., … Powers, M. B. (2019). Virtual reality exposure therapy a comparative outcome study. Studies in Health Technology and
162
for anxiety and related disorders: A meta-analysis of randomized controlled Informatics, 154, 57–60.
163
trials. Journal of Anxiety Disorders, 61, 27–36. Safir, M. P., Wallach, H. S., & Bar-Zvi, M. (2012). Virtual reality cognitive-
Dechartres, A., Trinquart, L., Boutron, I., & Ravaud, P. (2013). Influence of behavior therapy for public speaking anxiety: One-year follow-up. Behavior 164
trial sample size on treatment effect estimates: Meta-epidemiological Modification, 36(2), 235–246. https://doi.org/10.1177/0145445511429999. 165
study. BMJ (Clinical Research Ed.), 346, f2304. https://doi.org/10.1136/ Stupar-Rutenfrans, S., Ketelaars, L. E. H., & van Gisbergen, M. S. (2017). Beat 166
bmj.f2304. the fear of public speaking: Mobile 360 video virtual reality exposure train- 167
Emmelkamp, P. M. G., Meyerbröker, K., & Morina, N. (2020). Virtual reality ing in home environment reduces public speaking anxiety. Cyberpsychology, 168
therapy in social anxiety disorder. Current Psychiatry Reports, 22(7), 32. Behavior, and Social Networking, 20(10), 624–633. 169
https://doi.org/10.1007/s11920-020-01156-1. Turner, R. M., Bird, S. M., & Higgins, J. P. T. (2013). The impact of study size on 170
Gebara, C. M., de Barros-Neto, T. P., Gertsenchtein, L., & Lotufo-Neto, F. meta-analyses: Examination of underpowered studies in Cochrane reviews. 171
(2016). Virtual reality exposure using three-dimensional images for the PLoS ONE, 8(3), e59202. https://doi.org/10.1371/journal.pone.0059202.
172
treatment of social phobia. Brazilian Journal of Psychiatry, 38(1), 24–29. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor
173
Geraets, C. N. W., Veling, W., Witlox, M., Staring, A. B. P., Matthijssen, package. Journal of Statistical Software, 36(3). https://doi.org/10.18637/jss.
S. J. M. A., & Cath, D. (2019). Virtual reality-based cognitive behavioural v036.i03. Q2 174
therapy for patients with generalized social anxiety disorder: A pilot Wallach, H. S., Safir, M. P., & Bar-Zvi, M. (2011). Virtual reality exposure ver- 175
study. Behavioural and Cognitive Psychotherapy, 47(6), 745–750. https:// sus cognitive restructuring for treatment of public speaking anxiety: A pilot 176
doi.org/10.1017/S1352465819000225. study. The Israel Journal of Psychiatry and Related Sciences, 48(2), 91. 177
178
179
180
181
182
183
184
185
186
187
188
189

View publication stats