Cameron 2007
Cameron 2007
ORIGINAL ARTICLE
Abstract
Objective. Outcome measurement in mental health services is an area of considerable clinical interest and policy priority.
This study sought to assess the Behaviour and Symptom Identification Scale-24 (BASIS-24#), a brief, patient self-reported
measure of psychopathology and functioning, in a UK sample, including establishing population norms for comparative
purposes. Methods. Participants were 588 adults recruited from psychiatric inpatient, outpatient and primary care settings;
and 630 adults randomly sampled from primary care lists who completed the BASIS-24#, and the Brief Symptom
Inventory (BSI) at two time points. Results. BASIS-24# demonstrated adequate reliability (coefficient a values for
combined clinical sample across subscales ranged from 0.75 to 0.91), validity and responsiveness to change (effect size for
change of the BASIS-24# was 0.56 compared with 0.48 for BSI Global Severity Index). Population norms were established
for the general population and adult in-patients (at in-take). The scale proved straightforward to complete across clinical
settings. Variable rates of questionnaire distribution across clinical settings highlighted the ongoing challenge of
incorporating outcome measures in clinical settings. Conclusion. BASIS-24# is a brief, easily administered, self-complete
measure of mental well-being and functioning that adequately meets the requirements of reliability, validity and
responsiveness to change required of an outcome measure.
Correspondence: Isobel Cameron, Research Fellow, Department of Mental Health, Institute of Medical Sciences, Foresterhill, Aberdeen AB25 2ZD, UK.
Tel: /44 1224 557254. Fax: /44 1224 559320. E-mail: [email protected]
three health care settings: psychiatric in-patient, population sample was analysed for responsiveness
Community Mental Health Teams (CMHTs) and to change, though no change was expected in this
primary care. group.
The internal consistency of the scale was exam-
ined using Cronbach’s a in order to gauge the extent
Measures
to which responses were consistent to items in the
The BASIS-24# is a self-report questionnaire de- total scale, and items purporting to measure a
signed to measure outcome of mental health treat- subscale. Values of a falling between 0.7 and 0.9
ment from the service user’s viewpoint. Responses were considered indicators of adequate internal
fall within six symptom and functioning domains: consistency [8].
Depression/functioning, Interpersonal relationships, How well the scores of the BASIS-24# can
Psychotic symptoms, Alcohol/drug use, Emotional predict external criteria was assessed by considering
lability and Self-harm. Its predecessor, the BASIS- health care setting as an indicator of severity of
32, whilst exhibiting face validity, ease of use and symptoms reported. Data were analysed to assess
sensitivity to change following treatment, also had whether responders in the in-patient setting reported
some limitations in terms of the reliability of some greater problems and symptom distress than respon-
of its subscales, its between group differentiation ders in the CMHT setting, and whether responders
and its accessibility to responders with limited in the CMHT setting reported greater distress than
literacy skills. For these reasons the present, revised responders in the primary care setting. The validity
version was developed. It can be viewed at www. of the scale was further assessed by analysing
basissurvey.org. The Brief Symptom Inventory (BSI) whether the subscales differentiated patients in
is a 53-tem measure of psychometric symptomatol- associated diagnostic groups.
ogy [7]. By asking respondents to complete both Responsiveness to clinical change over the two
measures, the responsiveness to change of the time points was measured by running paired t -tests
BASIS-24# could be considered in the context of on the BASIS-24# total score and the BSI Global
the established BSI. Severity Index (GSI); the effect size of both mea-
sures was then calculated [9].
Data collection Where data were normally distributed t -tests were
used to test for differences in group means and h2
All participants were asked to complete the BASIS- used as a measure of effect size. Where scores were
24# and the BSI on two occasions: at the outset of not normally distributed, Mann Whitney or Wil-
the intervention and 3 months following (clinical coxon signed rank tests were carried out on these
sample), and at two time points 3 months apart data. Analyses were carried out using SPSS.
(general population sample). In-patients met with a
study researcher to receive the information before
completing the questionnaires independently. Power calculation
CMHT participants received the study information Samples of 500 in both patient and general popula-
from their treating clinician following which they tion groups at time point one are sufficient to yield
completed the questionnaire at home and returned it reliability estimates with confidence intervals of
by post. The primary care clinical sample received 0.1 at the appropriate level of significance [10] and
the study information by post from their general to generate stable percentiles for normative purposes
practitioner (primary care physician) shortly after [8]. The sample size also yields 90% power at the 5%
attending the practice. The general population significance level to assess the scale’s responsiveness
sample received their questionnaire by post. Samples to change (minimum detectable effect size of 0.12),
recruited by post were provided with contact details and a mean difference of 0.18 between the groups.
of the research team, should they wish to discuss the
study before consenting. Questionnaires for the
Ethical considerations
second time point were distributed by post to all
samples. For the clinical sample, diagnoses at time of Informed written consent was obtained before pa-
recruitment were obtained from medical notes (or tients participated. This research was conducted
the Continuous Morbidity Recording Register in the with the approval of the Grampian Research Ethics
case of primary care). Demographic information was Committee.
collected at the outset. For the 3-month follow up,
up to two reminders were sent.
Results
Analyses Recruitment
All analyses were carried out on the clinical samples A sample of 588 patients was recruited from the three
(in-patient, CMHT and primary care). The general health care settings: psychiatric in-patient (n /331;
38 I.M. Cameron et al.
63% of patients approached), CMHTs (n /165; which had more than six item responses missing
37% of patients approached) and primary care (n / were excluded. As the scores of the subscales were
92; 31% of patients approached). Figure 1 details not normally distributed, medians are represented in
how the in-patient sample was derived. In the Table II. Observational comparisons of scores can be
CMHT sample 156 (26% of newly referred patients) made between samples, as well as between time
did not receive packs where clinicians did not, or points, both of which are subjected to statistical
found it inappropriate to distribute the study infor- analyses further on in the results (see sections
mation. In the primary care sample 41% of eligible Concurrent criterion validity and Responsiveness to
patients did not receive study information where change).
doctors felt it inappropriate (37%) or for other
unspecified reasons (4%). A general population (i.e.
Reliability internal consistency
non-clinical) sample of 630 (42% of 1513 adults
(1865 years) randomly selected from three general Time point one a coefficients for the six subscales
practice lists) participated. and total score from the clinical samples are shown
The follow up questionnaire was completed in Table III. Coefficient a values for the total scale
by 418 (71%) of participants in the clinical arm are robust and comparable across the clinical set-
(in-patient /219 (66%), CMHT /124 (75%) and tings and time points. Values of a are also adequate
primary care /75 (82%)) and 506 (80%) of parti- for subscales across clinical groups with the excep-
cipants in the general population sample. tion of the psychosis and interpersonal relationships
subscales in the primary care setting. Overall,
reliability is higher in the in-patient and CMHT
Sample characteristics
samples.
Sample characteristics of all participants are shown
in Table I. As might be expected, differences were
Concurrent criterion validity
observed between the groups in terms of demo-
graphic characteristics (educational qualifications, Patients in the in-patient sample scored higher than
living arrangements and employment status) and patients in the CMHT sample; mean /2.04, sd /
primary diagnosis. Participants with a schizophrenic 0.69 versus 1.49, sd /0.62, p B/0.001, df /488,
illness or bipolar affective disorder are almost h2 /0.132. Patients in the CMHT sample scored
entirely represented in the in-patient setting and higher than patients in the primary care sample;
predictably, almost the entire primary care sample mean /1.49, sd /0.62 versus 1.18, sd /0.52, p B/
had been consulting their General Practitioner about 0.001, df /254, h2 /0.061. The depression/func-
a depressive or anxiety disorder. tioning subscale did not distinguish the sample with
The BASIS-24# raw item scores range from 0 (no a diagnosis of anxiety/depression from other patient
difficulty/symptoms never present) to 4 (extreme samples; median /2.6 (IQR /1.5, 3.3) versus 2.83
difficulty/symptoms always present). Six items re- (IQR /2, 3.5), p /0.09. The substance misuse scale
quire reverse scoring. Following this the scale can be successfully distinguished the substance misuse
scored by calculating a mean score for each subscale sample; median /2.5 (IQR /1.75, 3.25) versus
and for the total scale. Mean scores have been found 0.25 (IQR /0, 1), p B/0.001. The psychosis subscale
to be highly correlated with weighted mean scores successfully distinguished the psychotic sample;
developed more recently, and currently recom- median /1.5 (IQR /0.75, 2.5) versus 0.75 (IQR /
mended for the BASIS-24# instrument [11]. Cases 0, 1.5), p B/0.001.
psychiatric
admissions
n=966
Ineligible
Eligible n=158
n=808 (146 readmitted previous
recruits; 12 not in age range)
* Interquartile range.
Responsiveness to change was 0.54 (0.29, 0.87) and at time point two 0.46
(0.25, 0.75), p /0.29. Similarly there was no differ-
Paired t -tests on the combined clinical samples
ence between the BSI GSI scores at the two time
indicated a significant change from time point one
to time point two in both the BASIS-24# and BSI, points. At time point one the median score was 0.21
reflecting a reduction in psychopathology. The total (0.08, 0.49) and at time point two 0.17 (0.06, 0.43),
score of the BASIS-24 was 1.68 (sd /0.71) at time p/0.34. In a non-clinical, non-intervention sample,
point one, and 1.28 (sd /0.74) at time point two no significant difference was to be expected.
(p B/0.001), df /405. The BSI Global Severity
Index (GSI) was 1.6 (sd /0.85) at time point one, Percentile tables
and 1.19 (0.9) at time point two (p B/0.001), df /
397. The effect size for change of the BASIS-24# To provide normative data for the BASIS-24#, a
was 0.56 compared with 0.48 for the BSI GSI, percentile table was constructed based on overall
indicating that the BASIS-24# is slightly more mean scores of the general population sample and
responsive to change than the 53-item BSI. the in-patient sample (Table IV). To illustrate the
Total scores in the general population sample were use of the percentile tables, suppose an individual’s
positively skewed, therefore BASIS-24# total scores score on the BASIS-24# is 2.00 on admission. By
at the two time points were subjected to Wilcoxon referring to Table IV, the percentiles derived from
signed rank test. At time point one the median score the general population sample show that the indivi-
40 I.M. Cameron et al.
Follow-up (3 months)
General Population sample Median (IQR)
dual is at the 97th percentile; that is, their score is
1..00)
1..20)
0.25)
0.25)
1.67)
0.00)
0.75)
Time/95 days
worse than 97% of the general population. Compar-
(92, 107)
N/505
ing this same raw score to the in-patient sample, it
(0.16,
(0.00,
(0.20,
(0.33,
(0.00,
(0.00,
(0.25,
can be seen that this score is at the 62nd percentile;
i.e. such a score is not unusual for inpatient
0.50
0.60
0.00
0.00
1.00
0.00
0.46
admissions but is towards the more severe end.
Suppose also that the individual’s BASIS-24# score
at follow-up was 0.67. It can be seen that, in the
1.20)
0.50)
0.25)
2.00)
0.00)
0.87)
1.17)
Time point 1
(0.20,
(0.17,
(0.00,
(0.00,
(0.67,
(0.00,
(0.29,
become much less marked; they are now broadly
scoring in the normal range although still above the
0.60
0.67
0.00
0.00
1.33
0.00
0.54
average of the general population (the score is at the
61st percentile).
Follow-up (3 months)
1.80)
1.92)
0.50)
0.50)
2.33)
0.00)
1.25)
Primary Care sample Median (IQR)
Time/97 days
Discussion
(92, 106)
N/74
(0.40,
(0.58,
(0.00,
(0.00,
(1.00,
(0.00,
(0.46,
1.67
0.00
0.88
2.00)
0.50)
1.49)
(0.80,
(1.33,
(0.00,
(0.00,
(1.00,
(0.00,
(0.76,
1.33
0.00
1.19
2.67)
1.00)
1.82)
Time/99 days
(92, 106)
N/122
(0.95,
(1.00,
(1.33,
(0.00,
(0.77,
2.00
0.00
1.21
2.00)
1.00)
1.96)
Time point 1
Representativeness of sample
N/164
(1.50,
(1.20,
(0.00,
(0.00,
(1.00,
(0.00,
(1.04,
1.80
0.25
0.50
1.33
0.00
1.45
3.00)
2.58)
1.25)
1.25)
2.33)
1.50)
1.94)
(1.00,
(1.00,
(0.00,
(0.00,
(1.00,
(0.00,
(0.79,
1.78
0.25
0.50
2.00
0.50
1.34
BASIS-24 Scale In-patient (min n /258) CMHT (min n /146) Primary care (min n/80) All Clinical (min n /484)
Coefficient a (95% CI*) Coefficient a (95% CI) Coefficient a (95% CI) Coefficient a (95% CI)
Depression/functioning 0.88 (0.85, 0.90) 0.90 (0.88, 0.93) 0.87 (0.83, 0.91) 0.90 (0.88. 0.91)
Substance misuse 0.85 (0.82, 0.88) 0.80 (0.75, 0.85) 0.68 (0.55, 0.78) 0.85 (0.82, 0.87)
Psychosis 0.76 (0.72, 0.80) 0.70 (0.62, 0.77) 0.57 (0.41, 0.70) 0.79 (0.76, 0.81)
Interpersonal relationships 0.78 (0.74, 0.82) 0.77 (0.71, 0.82) 0.52 (0.33, 0.66) 0.75 (0.72, 0.79)
Emotional lability 0.70 (0.64, 0.75) 0.73 (0.65, 0.79) 0.71 (0.59, 0.80) 0.71 (0.67, 0.75)
Self-harm 0.91 (0.89, 0.93) 0.83 (0.76, 0.87) 0.85 (0.77, 0.90) 0.91 (0.90, 0.93)
Total scale 0.88 (0.86, 0.90) 0.90 (0.88, 0.92) 0.84 (0.79, 0.89) 0.91 (0.89, 0.92)
Percentile In-patient sample General population sample Percentile In-patient sample General population sample
Note: where a raw score corresponds to more than one percentile take the higher percentile.
The third concern, that of resources, is an completing the questionnaire. Additionally, current
important consideration. The challenges inherent ethics policy dictates that individuals have up to 24 h
in introducing routine outcome measurement into to consider whether to participate or not. In the case
clinical practice were particularly highlighted in the of routine audit, written consent is not required and
data collection for the primary care sample. It may perhaps therefore participation appears less daunting
be that where outcome measures are introduced for than in the research context. Conversely, in some
routine audit purposes there will be less obstacles settings the research process may have enhanced
present than are necessary in the conduct of research participation. In the in-patient setting, a researcher
as patients will have less to complete in routine was present daily to facilitate involvement. In
practice where concurrent measures are not used. Grampian, BASIS-24# has recently been intro-
Also, in research, potential participants have to duced as part of the clinical audit of a new urgent
consider giving informed written consent before referral service. The practical application of the
Psychometric properties of the BASIS-24# 43
measure will be observed with keen interest. If [2] Secretary of State for Health. National Service Framework
mental health services are serious about the imple- Mental Health. London: HSMO; 1999.
[3] Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns
mentation of routine outcome measures, protected A. Health of the Nation Outcome Scales (HoNOS). Br J
time to assess and use data is also required. Psychiatry 1998;172:11 8.
/ /
complete measure is not feasible [7] Derogatis LR. The Brief Symptom Inventory: Administra-
. It provides valuable information from the pa- tion, scoring and procedures. Minneapolis, MN: National
tient perspective Computer Systems; 1993.
[8] Nunally JC, Bernstein IH. Psychometric theory, 3rd ed. New
. Clinician perspectives should also be consid- York: McGraw-Hill; 1994.
ered to form a more complete picture [9] Deyo RA, Diehr P, Patrick DL. Reproducibility and respon-
siveness of health status measures: statistics and strategies for
evaluation. Control Clin Trials 1991;12:142S 58S.
Acknowledgements
/ /
tices in Grampian are gratefully acknowledged for [15] Holloway F. Outcome measurement in mental health
facilitating with recruitment. welcome to the revolution. Br J Psychiatry 2002;181:1 2. / /
Susan V Eisen, receives a percentage of licensing [18] Marshall M, Lockwood A, Green G, Zajac-Roles G, Roberts
C, Harrison G. Systematic assessments of need and care
fees received by McLean Hospital for licensing of the
planning in severe mental illness; Cluster randomised trial.
instrument. Br J Psychiatry 2004;185:163 8. / /
39:649 51.
/
[1] Scottish Office. A framework for mental health services in [20] http://basissurvey.org/webscore24/
Scotland. Edinburgh: Scottish Office; 1997.