European Psychiatry 52 (2018) 47–53

Contents lists available at ScienceDirect

European Psychiatry
journal homepage: http://www.europsy-journal.com

Use of risk assessment instruments to predict violence in forensic

psychiatric hospitals: a systematic review and meta-analysis
Taanvi Ramesha , Artemis Igoumenoub , Maria Vazquez Montesc, Seena Fazela,*
a
Department of Psychiatry, University of Oxford, Oxford, UK
b
Consultant Forensic Psychiatrist, Barnet Enfield and Haringey Mental Health NHS Trust, UK
c
Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

A R T I C L E I N F O A B S T R A C T

Article history: Background and Aims: Violent behaviour by forensic psychiatric inpatients is common. We aimed to
Received 14 November 2017 systematically review the performance of structured risk assessment tools for violence in these settings.
Received in revised form 24 February 2018 Methods: The nine most commonly used violence risk assessment instruments used in psychiatric
Accepted 28 February 2018
hospitals were examined. A systematic search of five databases (CINAHL, Embase, Global Health, PsycINFO
Available online xxx
and PubMed) was conducted to identify studies examining the predictive accuracy of these tools in forensic
psychiatric inpatient settings. Risk assessment instruments were separated into those designed for
imminent (within 24 hours) violence prediction and those designed for longer-term prediction. A range of
accuracy measures and descriptive variables were extracted. A quality assessment was performed for each
eligible study using the QUADAS-2. Summary performance measures (sensitivity, specificity, positive and
negative predictive values, diagnostic odds ratio, and area under the curve value) and HSROC curves were
produced. In addition, meta-regression analyses investigated study and sample effects on tool
performance.
Results: Fifty-two eligible publications were identified, of which 43 provided information on tool
accuracy in the form of AUC statistics. These provided data on 78 individual samples, with information
on 6,840 patients. Of these, 35 samples (3,306 patients from 19 publications) provided data on all
performance measures. The median AUC value for the wider group of 78 samples was higher for
imminent tools (AUC 0.83; IQR: 0.71–0.85) compared with longer-term tools (AUC 0.68; IQR: 0.62-
0.75). Other performance measures indicated variable accuracy for imminent and longer-term tools.
Meta-regression indicated that no study or sample-related characteristics were associated with
between-study differences in AUCs.
Interpretation: The performance of current tools in predicting risk of violence beyond the first few days is
variable, and the selection of which tool to use in clinical practice should consider accuracy estimates.
For more imminent violence, however, there is evidence in support of brief scalable assessment tools.
© 2018 The Author(s). Published by Elsevier Masson SAS. This is an open access article under the CC BY-
NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. INTRODUCTION psychiatric patients from 122 studies in high income countries

found that 48% of patients on forensic wards were violent over a
Violence in inpatient psychiatric wards is a major problem for mean follow-up of 31 months, which was almost double that for
health services, with effects on patient and staff psychiatric acute psychiatric wards (26%, mean time period: 19 months) and
morbidity [1], wider implications on stigma for patients and over two-fold that for other less acute psychiatric inpatient
recruitment in psychiatric hospitals, alongside costs associated settings (22%, mean time period: 16 months) [2].
with injury, staff sickness, and potential litigation by victims. There Despite its importance, few instruments have been designed for
are higher reported rates of violence on forensic psychiatric wards the prediction of violence specifically for inpatient populations.
compared to general psychiatry; a review of nearly 70,000 Current guidelines from the National Institute for Health and Care
Excellence (NICE) [3] in England recommend the use of the Brøset
Violence Checklist (BVC) [4,5] or the Dynamic Appraisal of Situational
Aggression (DASA) [6] for the prediction of inpatient violence,
* Corresponding author at: Department of Psychiatry, University of Oxford,
although US and Australasian guidelines do not appear to recommend
Warneford Hospital, Oxford OX3 7JX, UK.
E-mail address: [email protected] (S. Fazel). any such tools for acute management of schizophrenia inpatients [7,8].

http://dx.doi.org/10.1016/j.eurpsy.2018.02.007
0924-9338/© 2018 The Author(s). Published by Elsevier Masson SAS. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
48 T. Ramesh et al. / European Psychiatry 52 (2018) 47–53

Previous work has typically combined forensic psychiatric Revised (LSI-R) [20], the Psychopathy Checklist Revised (PCL-R)
patients with other psychiatric populations and prisoners when [21], the Psychopathy Checklist Screening Version (PCL:SV) [22],
assessing the predictive accuracy of risk assessment instruments the Violence Risk Appraisal Guide (VRAG) [23,24] and the Violence
[9–12]. A meta-review of violence risk assessment systematic Risk Scale (VRS) [25]. Structured professional judgement (SPJ) tools
reviews and meta-analyses found that 90% of reviews published included the Historical Clinical Risk Management-20 (HCR-20)
before 2010 included mixed samples of different populations, and [26,27], the Short-Term Assessment of Risk and Treatability
thus the overall findings may not be informative to specific patient (START) [28,29] and the Violence Risk Screening-10 (V-RISK-10)
groups [13]. In addition, inpatient or institutional violence is often [30,31]. Tools developed specifically for sexual violence were not
grouped together with community or offending outcomes in included in this review as they are very rarely used in inpatients.
reviews [10,11,12]. As violence base rates and possible interven- Our systematic search returned no eligible studies focusing on the
tions, and also the strength of risk factors, are different between LSI-R or the V-RISK-10. Further information on each of the 9
inpatients and community-dwelling individuals, there is a need for included instruments can be found in Table 1.
a review specifically on inpatient violence.
Thus, we have aimed to systematically review and meta-analyse 2.3. Systematic search
the performance of structured risk assessment instruments used to
predict inpatient violence in forensic psychiatric samples. In A systematic search was conducted to identify studies that
addition, we have investigated sources of variation between measured the predictive validity of the nine instruments in
individual studies using meta-regression analyses. forensic psychiatric settings for the outcome of inpatient violence.
We searched five databases (CINAHL, Embase, Global Health,
2. METHODS PsycINFO and PubMed) from the earliest available start date up to
January 2017, using a keyword search of titles and abstracts with
2.1. Review protocol the following search terms: (PCL-R OR Psychopathy Checklist
Revised OR HCR-20 OR Historical Clinical Risk Management OR
This review followed the Preferred Reporting Items for PCL:SV OR Psychopathy Checklist Screening OR VRAG OR Violence Risk
Systematic reviews and Meta-Analyses (PRISMA) statement [14]. Appraisal Guide OR COVR OR Classification of Violence Risk OR LSI-R
A review protocol was published on PROSPERO on 23/11/16: OR Level Service Inventory OR VRS OR Violence Risk Scale OR START OR
(https://www.crd.york.ac.uk/PROSPERO/display_record.asp? Short Term Assessment Risk Treatability OR BVC OR Br?set Violence
ID=CRD42016049789). Checklist OR DASA OR Dynamic Appraisal of Situational Aggression OR
V-RISK-10 OR Violence Risk Screening 10 OR risk assess*) AND
2.2. Risk assessment tools inpatient* AND violen* AND risk AND (predict* OR valid*).
Additional studies were identified through hand-searching
Based on recent reviews and questionnaire surveys [15–17], the references of the identified studies, using the Google Scholar “cited
11 most commonly used instruments for forensic inpatient by” function, scanning the annotated bibliographies for each
violence risk prediction were identified. Actuarial instruments instrument, and corresponding with researchers in the field.
included the Brøset Violence Checklist (BVC) [4,5], the Classifica- Studies in all languages and those that were unpublished were
tion of Violence Risk (COVR) [18,19], the Dynamic Appraisal of considered for inclusion. Studies were excluded if: (1) they
Situational Aggression (DASA) [6], the Level of Service Inventory- measured the predictive validity of selected scales of a tool, as

Table 1
Characteristics of the nine included violence risk assessment instruments.

Instrument type and name No. of items Static or Dynamic Items Cut-off scoresa
Actuarial
BVC 6 All dynamic High 3
Low <3
COVR -b Mainly static High 26
Moderate 8 26
Low <8
DASA 7 All dynamic High 4
Low <4
PCL-R 20 Mainly static High 25
Moderate 15 24
Low <15
PCL:SV 12 Mainly static High 15
Low <15
VRAG 12 All static High 14
Moderate 7–13
Low < 8
VRS 26 Both High 42
Low <42
Structured professional judgement
HCR-20 20 Both High 30
Moderate 20-29
Low <20
START 44 Both -c
a
Information on cut-off scores relates only to those samples who reported a cut-off score; in some cases cut-off scores were unknown or a clinical risk judgement may have
been used instead.
b
COVR has a varying number of items depending on answers given to previous items.
c
No cut-off score was used for START classifications, as the low, moderate and high risk categorisation was given from the violence risk estimate section.
 T. Ramesh et al. / European Psychiatry 52 (2018) 47–53 49

the aim was to test the accuracy of the tool as a whole; (2) they operating point (showing summary sensitivity and specificity
focused on a specific subgroup of the forensic population (e.g., values), a summary AUC value, 95% confidence region and 95%
those with a diagnosis of learning disability), as our aim was to prediction region. We obtained summary accuracy estimates for
focus on the most common forensic psychiatric populations; (3) the sensitivity, specificity, positive predictive value (PPV; the
instruments were coded retrospectively without blinding to proportion of patients classified as higher risk who went on to be
outcomes, to avoid any possible observer biases in evaluating violent), negative predictive value (NPV; the proportion of patients
outcomes; (4) they were calibration studies for the actuarial tools, classified as low risk who went on to not be violent), diagnostic
as such development samples will provide inflated accuracy. odds ratio (DOR; the ratio of the odds of violent patients having
Where studies used overlapping samples, the sample with the been classified as higher risk relative to the odds of non-violent
larger number of participants was used in order to avoid double- patients having been classified as low risk) and the area under the
counting. Using this search strategy, we identified 52 studies curve (AUC) value.
eligible for inclusion.
To be included in the full meta-analysis, studies were required 2.5.2. Heterogeneity
to report numbers of true positives, false positives, true negatives, Heterogeneity is expected in meta-analyses of diagnostic or
and false negatives at a given tool-specific cut-off score for the prognostic test accuracy due to the bivariate nature of the analysis
outcome of inpatient violence over a defined time period. We and variation in cut-off scores; therefore, the standard Q and I2
contacted study authors if this information was unavailable in the statistics are not recommended [35–39], but with no consensus on
manuscript and they were asked to fill in a standardised form. The what to use [40]. Thus it is recommended that visual evaluation of
desired full range of outcome data were available in the manu- the scatter of points from the SROC curve and the size of the ellipse
scripts of 11 eligible studies (13 samples). Further data was of the prediction regions be used to assess heterogeneity. A greater
requested from the authors of the other 41 manuscripts and data scatter of points from the SROC curve and a larger prediction region
was obtained for an additional 8 studies (22 samples). Of the 52 are indicative of greater levels of heterogeneity [32].
eligible studies, 43 (78 samples) gave an overall performance
measure (the area under the curve value; AUC) and thus were 2.5.3. Meta-regression and subgroup analyses
included for calculating the median summary AUC value for a Meta-regression analyses were conducted to investigate the
wider sample. The final number of studies included in the meta- relationship between an overall accuracy estimate (the AUC value)
analysis of other performance measures (i.e. true and false and pre-specified study and sample characteristics, to test whether
positives/negatives with AUCs) was 19 (amounting to 35 samples). any had a moderating effect on the AUC. Sample-related variables
included sample size, gender, mean age of participants, and
2.4. Quality assessment proportion of patients with psychotic disorder, personality
disorder, or violent index offence. Study-related variables included
The QUADAS-2 tool, designed to assess methodological quality temporal design of the study (prospective vs. retrospective), type
for systematic reviews of studies investigating diagnostic or of instrument (actuarial vs. structured professional judgement),
prognostic accuracy, provided a risk of bias for each study, with low follow-up period post-assessment, and definition of violent
or high risk of bias categorisations. All included studies showed a outcome used (interpersonal violence vs. interpersonal violence
low risk of bias. and verbal aggression). Meta-regression analysis was performed
for studies included in the meta-analysis. We planned to
2.5. Data analysis investigate any significant findings on meta-regression using
subgroup analyses. We also performed an additional analysis of the
Risk assessment instruments were divided into two groups: alternative binning strategy (low/medium vs. high) for the longer-
those designed for the prediction of imminent violence over a 24- term tools.
hour period following the assessment (BVC and DASA) and those All analysis was conducted on Stata [41], using the midas
designed for the prediction of violence over a longer period (COVR, command to generate summary statistics and a SROC curve and the
HCR-20, PCL-R, PCL:SV, START, VRAG and VRS). Given that metareg command for meta-regression analyses. Summary PPVs
instruments used for violence risk assessment in a clinical setting and NPVs were not produced by the midas command and were
are primarily used to identify higher risk individuals that may need therefore calculated as medians. Summary AUC values for the
monitoring, we combined subjects who were classified as wider group of eligible samples were also calculated as medians.
moderate risk with those classified as high risk, and compared
these two categories to low risk patients. 3. RESULTS

2.5.1. Meta-analytic model 3.1. Descriptive characteristics

We followed guidelines in the Cochrane collaboration for
systematic reviews of diagnostic and prognostic test accuracy [32]. For the wider sample of studies that reported on AUC values,
We examined two central measures of accuracy: sensitivity (the information was collected for 6,840 participants in 78 samples
proportion of violent patients that a risk assessment tool predicted from 43 independent publications. There were 5,680 (83%) male
to be higher risk) and specificity (the proportion of non-violent patients and 1,150 female patients. In the meta-analysis of all
patients that an instrument predicted to be low risk). We then performance measures (with additional information on sensitivity
developed a bivariate random-effects model that jointly analyzed and specificity), information was collected for 3,306 participants in
pairs of sensitivities and specificities, taking into account their 35 samples from 19 independent publications (Table 2). Stand-
correlation with one another [33]. Without covariates, this model ardised outcome information on numbers of true and false
is a different parameterisation of the hierarchical summary positives and negatives for 24 samples was obtained directly
receiver operating characteristic (HSROC) model [34]. We then from study authors. When investigating all performance measures,
used summary receiver operating characteristic (SROC) plots to there were 2,645 (80%) male patients and 661 female patients and
present the results of each study in receiver operating character- the overall mean age of patients was 36.6 years (standard deviation
istic (ROC) space, with each study plotted as a single sensitivity- [SD] = 3.5). There was some variation in both sample size
specificity point. This produced a SROC curve, with a summary (mean = 94.5; SD = 120.4) and rate of violence over the study
50 T. Ramesh et al. / European Psychiatry 52 (2018) 47–53

Table 2
Descriptive and demographic characteristics of samples for imminent and longer-term instruments included in the full meta-analysis (k = 35).

Category and group Imminent (k = 6) Longer-Term (k = 29)

Tool Information
Type of tool
Actuarial 6 (100) 13 (45)
Structured professional judgement 0 (0) 16 (55)
Tool used
BVC 3 (50) -
COVR - 3 (10)
DASA 3 (50) -
HCR-20 - 13 (45)
PCL-R - 4 (14)
PCL:SV - 1 (3)
START - 3 (10)
VRAG - 4 (14)
VRS - 1 (3)
Sample characteristics
Male participants (n (%)) 1115 (80) 1549 (81)
Age (years; mean (SD)) 37.0 (2.5) 36.4 (3.8)
Psychotic disorder (n (%)) 508 (37) 931 (81)
Personality disorder (n (%)) 122 (9) 449 (36)
Violent index offence (n (%)) 715 (51) 1089 (73)
Study design
Sample size (mean (SD)) 232 (233) 66 (54)
Temporal design
Retrospective 0 (0) 12 (41)
Prospective 6 (100) 15 (52)
Pseudo-prospective 0 (0) 2 (7)
Length of follow-up (days; mean (SD)) 1.0 (0.0)) 692.2 (978.6)
Outcome
Violent outcome measured
Only interpersonal physical violence 2 (33) 19 (66)
Including verbal aggression 4 (67) 10 (34)
Rate of violence during study (mean (SD)) 23.8 (15.3) 32.6 (16.2)

Note: Data are number (%) of samples, unless stated otherwise. Percentages are reported in relation to only those samples where information
was available for the variable in question. SD = standard deviation.

period (mean = 31% of the sample being violent; SD = 16.1). Each 0.83), while for longer-term instruments, it was 0.75 (95% CI: 0.65–
risk assessment instrument had between one and four studies 0.83). The summary specificity for imminent tools was 0.99 (95%
assessing predictive validity, with the exception of the HCR-20, CI: 0.80–1.00) and for longer-term tools was 0.56 (95% CI: 0.46–
which was investigated in 13 studies. Studies were conducted in 12 0.66). A summary DOR for imminent tools could not be accurately
different countries: Australia, Belgium, Canada, Denmark, Hong calculated due to the number of zero-value categories (2 of the 6
Kong, Ireland, Japan, the Netherlands, Norway, Spain, the UK and samples included had one or more cells with zero values). The
the USA. summary diagnostic odds ratio (DOR) for longer-term tools was 4.0
(95% CI: 3.0-6.0). The median PPV for imminent instruments was
3.1.1. Comparison between groups 0.36 (Interquartile range [IQR]: 0.10–0.93) and the median NPV
In the meta-analysis of all performance measures, there were was 0.99 (IQR: 0.85-1.00). The median PPV for longer-term
1,394 patients in the 6 imminent tool samples (reported in 4 instruments was 0.55 (IQR: 0.30-0.75) and the median NPV was
publications), compared to 1,912 patients in the 29 longer-term 0.75 (IQR: 0.58-0.95).
tool samples (15 publications). Both sample groups had approxi- Two different summary estimates of AUC values are reported
mately 80% male patients (Table 2) and there was little difference based on different sample sizes. The first were calculated as
in mean age (37.0 and 36.4 years, respectively). Sample sizes for median AUCs from all eligible studies that reported AUC values;
imminent tool studies ranged between 38 and 530 patients, while this amounted to 78 samples and a total of 6,840 patients from 43
for longer-term tool studies, they spanned from 29 to 185. Follow- publications, based on 10 imminent tool samples (1,666 patients)
up length for all imminent tool samples had a 24-hr follow-up, and 68 longer-term tool samples (5,174 patients). The median AUC
while for longer-term tool samples, it was a mean of 692 days for imminent instruments was 0.83 (IQR: 0.71-0.85), while for
(SD = 979). The mean rate of violence over the defined follow-up longer-term instruments it was 0.68 (IQR: 0.62-0.75) (Table 3).
period was 23.8% in the imminent tool sample compared with The second summary AUC value reported is that from the
32.6% for longer-term tools. samples included in the meta-analysis (k = 35), as for the other
reported performance measures. The summary AUC value for
3.2. Predictive accuracy imminent tools in the meta-analysis sample was 0.90 (95% CI:
0.87-0.92) and for longer-term tools it was 0.71 (95% CI: 0.67-0.75).
3.2.1. Summary statistics
The studies included for the production of these summary 3.2.2. HSROC curves
statistics were those for which information on true and false Figs. 1 and 2 show the hierarchical summary receiver operating
positives and negatives was available (k = 35). characteristic (HSROC) curve formed from the meta-analysis of
Predictive accuracy was different for the two groups of imminent and longer-term instruments, respectively. On both
instruments (Table 3). In studies of imminent instruments, curves, the summary sensitivity, specificity point is plotted, along
sensitivity was 0.59 (95% confidence interval [95% CI]: 0.29– with a 95% confidence contour and a 95% prediction contour. The
 T. Ramesh et al. / European Psychiatry 52 (2018) 47–53 51

Table 3
Summary accuracy estimates produced by two categories of violence risk
assessment instruments.

Imminent Instruments (k = 6) Longer-Term Instruments (k = 29)

Summary estimates (95% confidence interval)
Sensitivity 0.59 (0.29 – 0.83) 0.75 (0.65 0.83)
Specificity 0.99 (0.80 1.00) 0.56 (0.46 0.66)
a
PPV 0.36 (0.10 0.93) 0.55 (0.30 0.75)
a
NPV 0.99 (0.85 1.00) 0.75 (0.58 0.95)
DOR - 4.00 (3.00 6.00)
AUCa 0.83 (0.71 0.85) 0.68 (0.62 0.75)

Note: Median AUC values calculated from wider samples (k = 78): 10 samples for
imminent tools and 68 samples for longer-term tools.
a
Median (interquartile range).

HSROC curve for imminent tools is approaching the top left-hand

corner of the graph, indicating high accuracy, but the prediction
contour is large, indicating high levels of between-study hetero-
geneity (Fig. 1). For longer-term tools, the HSROC curve is closer to
the y = x diagonal that would indicate an uninformative test than it
is to the top left-hand corner of space (Fig. 2). The prediction
contour is also large, again indicating high levels of between-study
heterogeneity. Fig. 2. Summary receiver operating characteristics (SROC) curve from bivariate
analysis of longer-term violence risk assessment instruments for forensic inpatient
violence.
3.2.3. Individual tool performance
Note: Summary operating point = best fit for sensitivity and specificity. 95%
Within the wider group of 78 samples, the majority of samples confidence contour represents within-study heterogeneity. 95% prediction contour
assessed the performance of the HCR-20 (k = 27) and the PCL-R represents between-study heterogeneity.
(k = 10). These tools performed moderately for the prediction of
inpatient violence with median AUCs of 0.70 (IQR: 0.62-0.80) and
0.64 (IQR: 0.61-0.69), respectively. Imminent instruments had associated with between-study difference in AUCs
higher AUC values; the BVC (k = 5) had a median AUC of 0.83 (IQR: (Appendix Table 3 in Supplementary material). When we used
0.75–0.87) and the DASA (k = 5) also had a median AUC of 0.83 an alternative binning strategy (low/medium vs. high), the
(IQR: 0.65-0.90). See Appendix Table 2 in Supplementary material performance of the longer-term tools was marginally improved
for all accuracy measures for each instrument. with regards to PPV and AUC (Appendix Table 4 in Supplementary
material).
3.3. Investigation of heterogeneity and subgroup analyses
4. DISCUSSION
Meta-regression analyses were only performed for longer-term
instrument samples, as there were too few imminent instrument This systematic review and meta-analysis examined the
samples (k = 6). No study- or sample-related variables were predictive accuracy of 9 violence risk assessment instruments
for inpatient violence in forensic psychiatric hospitals from 78
samples involving 7,705 patients from 14 different countries. The
main finding was that instruments designed for the prediction of
imminent violence performed better at predicting inpatient
violence than instruments designed for longer-term follow-up
periods, based on a range of performance measures. As a measure
of overall accuracy, the median AUC for imminent tool studies was
0.83, compared to a median AUC of 0.68 for longer-term tools.
Generally, AUC values greater than 0.8 indicate a highly accurate
test and those below 0.7 indicate poor to moderate accuracy [42].
Imminent instruments performed particularly well for screening
out low risk individuals: 99% of those who went on to not be
violent were correctly predicted to be low risk (specificity) and 99%
of those who were predicted to be low risk went on to not be
violent (NPV).

4.1. Individual tool performance

The HCR-20 is the most widely-used violence risk assessment

instrument internationally, yet our findings from this review show
that it has at best moderate accuracy across a range of performance
measures, with regard to the prediction of inpatient violence.
Fig. 1. Summary receiver operating characteristics (SROC) curve from bivariate These lower levels of accuracy are likely a consequence of how the
analysis of imminent violence risk assessment instruments for forensic inpatient HCR-20 has been developed, as it is a general violence risk
violence.
assessment instrument with applications and recommendations
Note: Summary operating point = best fit for sensitivity and specificity. 95%
confidence contour represents within-study heterogeneity. 95% prediction contour for use in a broad range of contexts, populations and follow-up
represents between-study heterogeneity. periods. Similarly, the PCL-R and VRAG performed poorly for the
52 T. Ramesh et al. / European Psychiatry 52 (2018) 47–53

prediction of inpatient violence. Although their performance may calibration of the instrument’s predictions with actual future
be acceptable for some populations in the community, the current violence [48]. To address this, we investigated a range of accuracy
evidence does not support their use for the prediction of inpatient measures although none of the included studies reported
violence in forensic psychiatry. calibration measures.
The two instruments designed specifically for imminent One limitation is that only studies reporting true and false
inpatient violence prediction (the BVC and the DASA) performed positives and negatives could be included in the full meta-
with higher accuracy for a number of measures. However, there analysis. However, median AUCs were reported for the wider
were few studies (k = 10) despite being recommended by NICE. sample of eligible studies. Further, we corresponded with authors
There were more studies focused on the poorer performing tools, requesting unpublished data and increased the number of
such as the HCR-20, suggesting a need to move towards research possible samples from 11 to 35 samples that report a range of
examining short-term tools, and possibly optimizing them by performance measures. Another limitation is the large amount of
considering novel risk factors [43]. between-study heterogeneity, perhaps due to variations in cut-off
scores used for risk classifications. A number of other possible
4.2. Clinical implications explanations were investigated in meta-regression and no
associations were found to explain the variation between tools.
Our findings indicate that the use of instruments designed for This heterogeneity is expected, especially in prognostic (as
the imminent prediction of violence over the 24-hour period post- opposed to diagnostic) studies, and the use of a random-effects
assessment yielded higher accuracy for multiple measures of model accounted for this variation. Further, where possible, the
performance. In clinical practice, consideration should be given to same cut-off scores were applied for each sample of the same
the use of the BVC and the DASA, both of which are recommended instrument.
tools in one clinical guideline for short-term management of There were differences between the imminent and longer-term
violence and aggression in inpatient mental health settings [3]. groups of studies with regard to the type of primary outcome used
Furthermore, the narrow 24-hour window within which violence (interpersonal violence only vs. interpersonal violence and verbal
is predicted allows for prevention and management strategies to aggression), which could explain their relative performance.
be implemented when they may be most needed. Both the BVC and Although this was investigated in meta-regression analyses and
DASA are brief checklists (6 and 7 items, respectively), have the found to have no effect on the AUC accuracy estimate for longer-
advantage of scalability and can easily be integrated into routine term tools, this analysis could not be performed for imminent
practice. instruments due to lack of available data. It is possible, therefore,
However, other clinical contexts will exist where longer-term that the better performance of the imminent tools (based on AUCs)
instruments may be more relevant or appropriate; the high is based on higher rates of softer outcomes (i.e. aggression), which
sensitivity (0.75) and moderate PPV (0.55) suggest these will inflate base rates.
instruments may have a role for some patients. Considering We also found marginally improved performance in some
the brevity of the BVC and DASA, they could act as a screen performance measures when we used a different binning strategy
before a longer term tool is used considering the expense (low/medium vs high). Whether this merits a change in how these
involved in administering time-consuming and resource-inten- tools are used in practice and for which inpatient settings requires
sive instruments [44]. further work.
However, for both imminent and longer-term tools, it is
important for there to be a link with clinical interventions and 4.4. Future directions
outcomes to link the risk prediction element with subsequent
management of risk. One randomised controlled trial (RCT) has Future research on violence risk assessment in forensic
been conducted finding a positive effect (reduction in inpatient inpatient settings should focus more on imminent instruments
violent incidents) when the BVC was used in a forensic psychiatric as this meta-analysis found a smaller proportion of the research
sample combined with implementation of a violence management literature based on these instruments. Another useful direction
strategy and training [47]. for research would be further exploration of whether there
should be a screen before longer-term instruments are used
4.3. Strengths and limitations [44]. As the two imminent tools in this study rely predominantly
on dynamic variables, research could investigate the role of
To our knowledge, this is the first comprehensive review and novel dynamic variables to improve risk prediction, and
meta-analysis of violence risk assessment instruments in the whether adding static variables can add incremental perfor-
context of their predictive accuracy for inpatient violence in mance. Further to this, new technologies that have been
forensic psychiatric populations. There has been one previous developed for the use of risk prediction and monitoring should
review of risk assessment for inpatient violence in forensic be examined [49]. From a methodological perspective, future
psychiatric patients [45]. However, it used mean correlation work in this area should report multiple estimates of predictive
coefficients between violence risk assessment scores and inpatient accuracy in order to provide a more complete picture of an
violence, which is limited to examine predictive accuracy. Further, instrument’s performance, including measures of calibration.
only three violence risk assessment instruments (the HCR-20, PCL- Overall, this meta-analysis supports previous recommendations
R and PCL:SV) were included in that review. that future work in violence risk assessment requires the
Recent criticism of risk assessment literature has stated that development and validation of tools designed for specific
there is an insufficient focus on subpopulations in a specific populations [46,50,51].
context [46]. Unlike previous reviews of risk assessment tools, the
current one investigates a particular patient group in one setting. Acknowledgements
In addition, the literature on predictive accuracy of violence risk
assessment has been limited by relying on one or two measures of We thank the following study authors for providing tabular data
accuracy [46]. The AUC value, for example, is often reported in for the analyses: Dr. Kaoru Arai, Dr. Oliver Chan, Professor Geoff
isolation; however, it does not indicate whether this discrimina- Dickens, Dr. Óscar Herrero, Dr. Helen Miles, Professor Robert
tion is clinically useful, nor does it provide any information on the Snowden, Professor Lindsay Thomson and Dr. Vivienne de Vogel.
 T. Ramesh et al. / European Psychiatry 52 (2018) 47–53 53

SF is a Wellcome Trust Senior Research Fellow in Clinical Violent offenders: Appraising and Managing Risk. Washington, DC, US:
Science (202836/Z/16/Z). We would like to disclose no conflicts of American Psychological Association; 2006. p. 155–96.
[24] Quinsey V, Harris G, Rice M, et al. Violent offenders: appraising and managing
interest of funding sources for this review. risk. Washington, DC: American Psychological Association; 2006.
[25] Wong S, Gordon A. Violence Risk Scale (VRS). Saskatoon, Saskatchewan. 2000.
Appendix A. Supplementary data [26] Douglas K, Hart S, Webster C, et al. HCR-20 version 3: assessing risk for
violence. Burnaby, BC, Canada: Mental Health, Law and Policy Institute, Simon
Fraser University; 2013.
Supplementary data associated with this article can be found, in [27] Webster C, Douglas K, Eaves D, et al. HCR-20: Assessing Risk for Violence
the online version, at https://doi.org/10.1016/j.eurpsy.2018.02.007. (Version 2). Burnaby, British Columbia, Canada: Simon Fraser University.
Mental Health, Law, and Policy Institute; 1997.
[28] Webster CD, Martin M, Brink J, et al. START: The Short-term assessment of risk
References and treatability. Hamilton: St Joseph’s Healthcare; 2004.
[29] Webster C, Martin M, Brink J, et al. Manual for the Short-Term Assessment of
[1] Wildgoose J., Briscoe M, Lloyd K. Psychological and emotional problems in staff Risk and Treatability (START) (Version 1.1). Coquitlam, Canada: British
following assaults by patients. The Psychiatrist 2003;27(8):295–7. Columbia Mental Health & Addiction Services; 2009.
[2] Bowers L, Stewart D, Papadopoulos C, et al. Inpatient violence and aggression: [30] Bjorkly S, Hartvig P, Heggen FA, et al. Development of a brief screen for violence
A literature review. Report from the Conflict and Containment Reduction risk (V-RISK-10) in acute and general psychiatry: An introduction with
Research Programme. Kings College London: Institute of Psychiatry; 2011. emphasis on findings from a naturalistic test of interrater reliability. European
[3] National Institute for Health and Care Excellence, Violence and aggression: Psychiatry 2009;24(6):388–94.
short-term management in mental health, health and community settings. [31] Hartvig P, Østberg B, Alfarnes S, et al. Violence Risk Screening-10 (V-RISK-10).
NICE Guideline (NG10) ed, In: 2015. Oslo, Norway: Centre for Research and Education in Forensic Psychiatry; 2007.
[4] Almvik R, Woods P, Rasmussen K. The Brøset Violence Checklist sensitivity, [32] Macaskill P, Gatsonis C, Deeks J, et al. Cochrane handbook for systematic
specificity, and interrater reliability. Journal of interpersonal violence 2000;15 reviews of diagnostic test accuracy. Version 09 0. London: The Cochrane
(12):1284–96. Collaboration; 2010.
[5] Linaker OM, Busch-Iversen H. Predictors of imminent violence in psychiatric [33] Reitsma JB, Glas AS, Rutjes AW, et al. Bivariate analysis of sensitivity and
inpatients. Acta Psychiatrica Scandinavica 1995;92(4):250–4. specificity produces informative summary measures in diagnostic reviews.
[6] Ogloff J, Daffern M. Dynamic appraisal of situational aggression: Inpatient Journal of clinical epidemiology 2005;58(10):982–90.
version. Melbourne, Victoria, Australia: Monash University and Forensicare; [34] Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of
2002. diagnostic test accuracy evaluations. Statistics in medicine 2001;20(19):2865–
[7] Galletly C, Castle D, Dark F, et al. Royal Australian and New Zealand College of 84.
Psychiatrists clinical practice guidelines for the treatment of schizophrenia [35] Jackson D, White IR, Thompson SG. Extending DerSimonian and Laird's
and related disorders. Australian & New Zealand Journal of Psychiatry 2016;50 methodology to perform multivariate random effects meta-analyses. Statistics
(5):410–72. in medicine 2010;29(12):1282–97.
[8] Lehman AF, Lieberman JA, Dixon LB, et al. Practice guideline for the treatment [36] Jackson D, White IR, Riley RD. Quantifying the impact of between-study
of partients with schizophrenia. American Journal of Psychiatry 2004161(2 heterogeneity in multivariate meta-analyses. Statistics in medicine 2012;31
Suppl). (29):3805–20.
[9] Campbell MA, French S, Gendreau P. The prediction of violence in adult [37] Verde PE. Meta-analysis of diagnostic test data: modern statistical approaches.
offenders a meta-analytic comparison of instruments and methods of Deutsche Nationalbibliothek; 2008.
assessment. Criminal Justice and Behavior 2009;36(6):567–90. [38] White IR. Multivariate random-effects meta-regression: updates to mvmeta.
[10] Fazel S, Singh JP, Doll H, et al. Use of risk assessment instruments to predict Stata Journal 2011;11(2):255.
violence and antisocial behaviour in 73 samples involving 24 827 people: [39] Zhou Y, Dendukuri N. Statistics for quantifying heterogeneity in univariate and
systematic review and meta-analysis. BMJ 2012;345:e4692. bivariate meta-analyses of binary data: The case of meta-analyses of
[11] Singh JP, Grann M, Fazel S. A comparative study of violence risk assessment diagnostic accuracy. Statistics in medicine 2014;33(16):2701–17.
tools: A systematic review and metaregression analysis of 68 studies involving [40] Naaktgeboren CA, Ochodo EA, Van Enst WA, et al. Assessing variability in
25,980 participants. Clinical psychology review 2011;31(3):499–513. results in systematic reviews of diagnostic studies. BMC Medical Research
[12] Whittington R, Hockenhull J, McGuire J, et al. A systematic review of risk Methodology 2016;16:6.
assessment strategies for populations at high risk of engaging in violent [41] StataCorp. Stata Statistical Software: Release 14. College Station, TX: StataCorp
behaviour: update 2002–8. Health Technology Assessment 2013;17(50):1– LP.; 2015.
128. [42] Tape TG., The area under an ROC curve. Interpreting diagnostic tests, 2006.
[13] Singh JP, Fazel S. Forensic risk assessment: A metareview. Criminal Justice and Available from: http://gim. unmc. edu/dxtests/roc3.htm.
Behavior 2010;37(9):965–88. [43] Eriksen BMS, Bjørkly S, Lockertsen Ø, et al. Low cholesterol level as a risk
[14] Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic marker of inpatient and post-discharge violence in acute psychiatry—A
reviews and meta-analyses: the PRISMA statement. PLoS med 2009;6(7): prospective study with a focus on gender differences. Psychiatry research
e1000097. 2017;255:1–7.
[15] Hurducas CC, Singh JP, de Ruiter C, et al. Violence risk assessment tools: A [44] Rosenfeld B, Foellmi M, Khadivi A, et al. Determining when to conduct a
systematic review of surveys. International Journal of Forensic Mental Health violence risk assessment: Development and initial validation of the Fordham
2014;13(3):181–92. Risk Screening Tool (FRST). Law and human behavior 2017;41(4):325.
[16] Singh JP, Desmarais SL, Hurducas C, et al. International perspectives on the [45] Hogan N, Ennis L, Assessment F. Assessing risk for forensic psychiatric
practical application of violence risk assessment: A global survey of 44 inpatient violence: A meta-analysis. Open Access Journal of Forensic
countries. International Journal of Forensic Mental Health 2014;13(3):193– Psychology 2010;2:137–47.
206. [46] Douglas T, Pugh J, Singh I, et al. Risk assessment tools in criminal justice and
[17] Singh JP, Desmarais SL, Otto RK, et al. The International Risk Survey: Use and forensic psychiatry: the need for better data. European Psychiatry
perceived utility of structured violence risk assessment tools in 44 countries. 2017;42:134–7.
In: International Perspectives on Violence Risk Assessment. Oxford University [47] Abderhalden C, Needham I, Dassen T, et al. Structured risk assessment and
Press; 2016. violence in acute psychiatric wards: randomised controlled trial. The British
[18] Monahan J, Steadman HJ, Silver E, et al. Rethinking risk assessment: The Journal of Psychiatry 2008;193(1):44–50.
MacArthur study of mental disorder and violence. Oxford University Press; [48] Singh JP. Predictive validity performance indicators in violence risk
2001. assessment: A methodological primer. Behavioral Sciences & the Law
[19] Monahan J, Steadman HJ, Robbins PC, et al. An actuarial model of violence risk 2013;31(1):8–22.
assessment for persons with mental disorders. Psychiatric services 2005;56 [49] Gulati G, Cornish R, Al-Taiar H, et al. Web-based violence risk monitoring tool
(7):810–5. in psychoses: pilot study in community forensic patients. Journal of forensic
[20] Andrews D, Bonta J. LSI-R: The Level of Service Inventory-Revised. Toronto, psychology practice 2016;16(1):49–59.
Ontario, Canada: Multi-Health Systems: Inc; 1995. [50] Fazel S, Wolf A, Larsson H, et al. Identification of low risk of violent crime in
[21] Hare RD. The Hare psychopathy checklist-revised: Manual. Multi-Health severe mental illness with a clinical prediction tool (Oxford Mental Illness and
Systems, Incorporated; 1991. Violence tool [OxMIV]): a derivation and validation study. Lancet Psychiatry
[22] Hart SD, Cox DN, Hare RD. The Hare psychopathy checklist: Screening version 2017;4(6):461–8.
(PCL: SV). MHS-Multi-Health Systems, Incorporated; 1995. [51] Wolf A, Fanshawe T, Sariaslan A, et al. Prediction of violent crime on discharge
[23] Quinsey VL, Harris GT, Rice ME, et al. Actuarial prediction of violence. In: from secure psychiatric hospitals: A clinical prediction rule (FoVOx). European
Quinsey V, Harris G, Rice M, Cormier C, editors. The Law and Public Policy. Psychiatry 2017;47:88–93.