Psychosis Risk Screening With The Prodromal Questionnaire
Psychosis Risk Screening With The Prodromal Questionnaire
Author Manuscript
Schizophr Res. Author manuscript; available in PMC 2012 June 1.
Published in final edited form as:
NIH-PA Author Manuscript
Abstract
NIH-PA Author Manuscript
In this study, we examined the preliminary concurrent validity of a brief version of the Prodromal
Questionnaire (PQ-B), a self-report screening measure for psychosis risk syndromes. Adolescents
and young adults (N=141) who presented consecutively for clinical assessment to one of two early
psychosis research clinics at the University of California, San Francisco and UC Los Angeles
completed the PQ-B and the Structured Interview for Prodromal Syndromes (SIPS) at intake.
Endorsement of three or more positive symptoms on the PQ-B differentiated between those with
prodromal syndrome and psychotic syndrome diagnoses on the SIPS versus those with no SIPS
diagnoses with 89% sensitivity, 58% specificity, and a positive Likelihood Ratio of 2.12. A
Distress Score measuring the distress or impairment associated with endorsed positive symptoms
increased the specificity to 68%, while retaining similar sensitivity of 88%. Agreement was very
similar when participants with psychotic syndromes were excluded from the analyses. These
results suggest that the PQ-B may be used as an effective, efficient self-report screen for
prodromal psychosis syndromes when followed by diagnostic interview, in a two-stage evaluation
process in help-seeking populations.
1. Introduction
NIH-PA Author Manuscript
individuals were distressed by the psychotic experience compared to those who were not.
Undergraduate students endorsed PQ items at very high rates in our own study, but fewer
endorsed items as distressing or impairing (Loewy, et al, 2007).
Although the ultimate target group for the PQ-B is the general help-seeking population, the
first step of measure development is to assess preliminary validity of the PQ-B in a selected
help-seeking group that is highly “enriched” for the target diagnoses (McGorry, et al, 2003).
In the current study, we administered the PQ-B along with the SIPS to all adolescent and
young adult patients consecutively presenting to two prodromal psychosis research clinics in
California. We hypothesized that: 1) The PQ-B would show good concurrent validity with
symptomatic syndromes on the SIPS, similar to the original PQ and 2) Assessing frequency
of experiences and related distress/impairment would improve specificity of the PQ-B
related to these SIPS diagnoses.
2. Methods
2.1 Participants
Study participants were 141 individuals age 12-35 who presented consecutively for
NIH-PA Author Manuscript
evaluation at one of two prodromal psychosis research clinics: the Prodrome Assessment,
Research and Treatment program at the University of California, San Francisco (UCSF)
(N=47) and the Staglin Music Festival Center for Assessment and Prevention of Prodromal
States at the University of California, Los Angeles (UCLA) (N=94). Subjects were referred
from community clinicians, schools, family members, and self-referred from seeing
information about the programs on the internet. Participants at the two sites did not
significantly differ from each other on any demographic, psychosocial functioning or
diagnostic grouping variables (see Table 1.)
A sample of age-matched healthy control participants (HCs) at both sites were recruited for
comparison to the patient group through advertisements placed on websites and at local
schools. HCs (N=46) were not significantly different from the patient group on age,
ethnicity or socioeconomic status, as measured by years of parental education, but had a
higher proportion of females than the patient group (p=.045). The control subjects at UCLA
were assigned GAF scores, which were significantly higher than those of the patient group,
as expected (p<.0001). Details of demographic characteristics are presented in Table 1.
NIH-PA Author Manuscript
2.2. Measures
2.2a. SIPS—The SIPS is semi-structured interview designed to be administered by trained
clinicians (Miller, et al, 2003). The interview includes a biopsychosocial history and ratings
along four major symptom dimensions on the Scale of Prodromal Symptoms (SOPS):
positive, negative, disorganized and general/affective symptoms. The SIPS/SOPS diagnoses
three types of prodromal syndromes, listed in order of typical sample prevalence: 1)
Attenuated Positive Symptom Prodromal Syndrome (APS): Attenuated positive psychotic
symptoms present at least once per week, started or worsened in that past year (unusual
thought content/delusional ideas, suspiciousness/persecutory ideas, grandiosity, perceptual
abnormalities/distortions, and conceptual disorganization; 2) Brief Intermittent Psychosis
Prodromal Syndrome (BIPS): Brief and intermittent fully psychotic symptoms that have
started recently; 3) Genetic Risk and Deterioration Prodromal Syndrome (GRDS): Either a
family history of a psychotic disorder in any first-degree relative and decline of at least 30%
in the past 12 months on the GAF scale, or, meets criteria for schizotypal personality
disorder and has had a decline of 30% on the GAF in the past year.
After SIPS assessment, 44% of subjects were diagnosed with a UHR syndrome, 42% were
diagnosed as being fully psychotic, and 13% received no psychotic-spectrum diagnosis.
NIH-PA Author Manuscript
Among UHR subjects, 39 (95%) met APS criteria and two (5%) met BIPS criteria. One
GRDS subject was excluded from analyses, as the PQ-B is intended to capture symptomatic
at-risk syndromes.
about frequency and related distress or impairment. See Appendix A for a copy of the PQ-B
and Appendix B for details on scoring.
2.3. Procedures
Participants or their parents (for subjects age 12-17) completed a brief phone screen prior to
being scheduled for a clinic intake in order to exclude cases of well-established psychosis,
mental retardation, substance dependence, and neurological disorders such as temporal lobe
epilepsy. Upon arrival at the clinic, participants provided informed consent or assent with
parental consent for the study, then completed the PQ-B, followed by the SIPS. Whenever
possible, collateral information was obtained by interviewing parents, significant others and
relevant clinicians.
A sample of age-matched healthy control participants (HCs) at both sites completed the PQ-
B and the SCID to rule out the presence of current Axis I diagnoses. Healthy control
subjects at UCLA (N=26) also completed the SIPS.
NIH-PA Author Manuscript
Clinical interviewers at both sites were MA, PhD or MD- level clinicians who underwent a
standard training procedure. Inter-rater reliability was excellent at both sites; ICCs for UCSF
staff were 0.94 for SIPS diagnoses and 0.70 to 0.97 for SOPS ratings. All ICCs were above
0.80 at UCLA. Participant diagnoses were discussed in regular reliability rounds to limit
rater drift. All study procedures were approved by the human subjects review committees at
UCSF and UCLA.
Agreement between PQ-B scores and SIPS diagnoses was used to assess concurrent validity
by generating receiver operating characteristic (ROC) curves and calculating areas under the
curve (AUCs). Values for sensitivity, specificity, positive predictive value, negative
predictive value and likelihood ratios were computed. Correlation analyses were performed
between PQ-B scores and SOPS positive symptom scores using Spearman’s correlation
coefficient. Cronbach’s coefficient alpha was used to examine internal consistency of the
PQ-B. The Kruskal-Wallis test is a non-parametric rank test that was used to compare PQ-B
scores across the three SIPS diagnostic groups (no SIPS diagnosis, prodromal syndrome,
psychotic syndrome). When significant differences were detected across groups, post-hoc
Mann-Whitney U tests then examined paired group contrasts.
3. Results
3.a. Concurrent validity of Prodromal/psychotic versus no SIPS diagnosis
All PQ-B scores predicted SIPS diagnoses of prodromal/psychotic syndromes versus no
SIPS diagnosis with statistically significantly AUC values. The two functioning items did
not improve prediction above and beyond the 23 positive symptom items. Furthermore, only
3 of the 5 positive symptom items that were added to the original 18 items (which emerged
NIH-PA Author Manuscript
from the analyses described in section 2.2b above) provided additional predictive power
above and beyond the 18 items. Therefore, the Total, Distress and Frequency scores were
calculated using only the 21 positive symptom items. Content of the items on the measure
that were excluded from scoring are included in Appendix B, along with scoring details.
A Total Score of 3 or more endorsed items balanced the greatest sensitivity (89%) with the
greatest specificity (58%) and had a positive Likelihood Ratio of 2.12. Comparatively, the
Frequency score lost substantial sensitivity, while the Distress score heightened specificity.
Table 2 presents the cutoff values and accuracy of the PQ-B scores that showed the most
agreement with SIPS symptomatic syndromes. Compared to the Total and Frequency scores,
the Distress score with a cutoff of 6 or more showed the greatest specificity (68%) while
retaining high sensitivity (88%); its performance was very similar when fully psychotic
patients were excluded from the analysis. The Distress Score cutoff of 6 or more showed
similar sensitivity across sites (87-90%) with some variation in specificity (63-73%),
although all scores overlapped within their 95% confidence intervals. Figure 1 compares the
ROC curves for the best-performing scores.
NIH-PA Author Manuscript
4. Discussion
Overall, the PQ-B showed good preliminary concurrent validity with interview-based SIPS
diagnoses in our help-seeking sample of adolescents and young adults. It effectively
differentiated between participants with SIPS diagnoses of prodromal and psychotic
syndromes versus non-psychotic spectrum patients. The brief version of the PQ maintained
the sensitivity of the original, while adding questions about related distress and impairment
that improved specificity. However, assessing frequency of experiences or asking about
NIH-PA Author Manuscript
functioning was not additionally helpful. The false-positive rate of 11% when using the
Distress Score in the healthy control group suggests that performance of the PQ-B is similar
across samples; however, this rate would result in a great number of interviews if screening
large samples. Taking these results altogether, we recommend only using the PQ-B in help-
seeking samples, especially with those individuals who are already suspected to have
attenuated psychotic symptoms.
As discussed previously (Loewy, et al, 2005), this instrument is designed to function as the
first step in a two-stage screening process that relies on clinician interview to obtain a
diagnosis. Therefore, PQ-B users should be careful not to equate a high score with
prodromal psychosis or unavoidable development of schizophrenia. In order to minimize
unnecessary stigma and distress that may be associated with the diagnosis of a an attenuated
psychosis syndrome (Yang, et al., 2010), we recommend discussing results in light of the
need for a more thorough clinical interview and the importance of early detection and
intervention in improving long term outcomes (Corcoran, et al., 2010).
NIH-PA Author Manuscript
In order for the PQ-B to be used in a general help-seeking population, future studies should
assess concurrent validity of the two-stage screening process and in a general mental health
sample. The results of the current study suggest that pursuing such a study is warranted. We
have an ongoing study tracking 2-year outcomes of the SIPS-positive participants, to assess
predictive validity of the PQ-B. Pending the outcome of future studies in unselected
samples, we hope that the PQ-B can be used to increase access to care for help-seeking
youth whose attenuated psychotic symptoms might otherwise go unnoticed or misdiagnosed.
Supplementary Material
Refer to Web version on PubMed Central for supplementary material.
Acknowledgments
Role of the Funding Source This work was supported by a Young Investigator award from the National Alliance
for Research on Schizophrenia and Depression (NARSAD) and NIH grant K23 MH086618 to Dr. Loewy; a gift
from the Lazslo N. Tauber Family Foundation to Dr. Vinogradov; and NIH Grants MH65079 and P50 MH066286
to Dr. Cannon. Additional support was provided by gifts to the UCLA Foundation by Garen and Shari Staglin and
the International Mental Health Research Organization. The funding sources had no further role in study design; in
NIH-PA Author Manuscript
the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the
paper for publication.
We would like to thank Ashley Lee, Tara Niendam and Danielle Schlosser for their assistance in coordinating this
project, and the reviewers for their thoughtful comments on the manuscript.
References
Cannon TD, Cadenhead K, Cornblatt B, Woods SW, Addington J, Walker E, Seidman LJ, Perkins D,
Tsuang M, McGlashan T. Prediction of psychosis in youth at high clinical risk: a multisite
longitudinal study in North America. Arch. Gen. Psychiatry. 2008; 65(1):28–37. [PubMed:
18180426]
Corcoran CM, First MB, Cornblatt B. The psychosis risk syndrome and its proposed inclusion in the
DSM-V: a risk-benefit analysis. Schizophr. Res. 2010; 120(1):16–22. [PubMed: 20381319]
Hanssen MS, Bijl RV, Vollebergh W, van Os J. Self-reported psychotic experiences in the general
population: a valid screening tool for DSM-III-R psychotic disorders? Acta Psychiatr. Scand. 2003;
107(5):369–77. [PubMed: 12752033]
Loewy RL, Bearden CE, Johnson JK, Raine A, Cannon TD. The Prodromal Questionnaire (PQ):
Preliminary validation of a self-report screening measure for prodromal and psychotic syndromes.
Schizophr. Res. 2005; 79(1):117–125. [PubMed: 16276559]
NIH-PA Author Manuscript
Loewy RL, Johnson JK, Cannon TD. Self-report of attenuated psychotic experiences in a college
population. Schizophr. Res. 2007; 93(1-3):144–151. [PubMed: 17459662]
McGorry PD, Yung AR, Phillips LJ. The “close-in” or ultra high-risk model: a safe and effective
strategy for research and clinical intervention in prepsychotic mental disorder. Schizophr. Bull.
2003; 29(4):771–790. [PubMed: 14989414]
Miller TJ, McGlashan TH, Rosen JL, Cadenhead K, Ventura J, McFarlane W, Perkins DO, Pearlson
GD, Woods SW. Prodromal assessment with the Structured Interview for Prodromal Syndromes
and the Scale of Prodromal Symptoms: predictive validity, interrater reliability, and training to
reliability. Schizophr. Bull. 2003; 29(4):703–715. [PubMed: 14989408]
Yang LH, Wonpat-Borja AJ, Opler MG, Corcoran CM. Potential stigma associated with inclusion of
the psychosis risk syndrome in the DSM-V: an empirical question. Schizophr. Res. 2010; 120(1):
42–48. [PubMed: 20399610]
Yung AR, McGorry PD, Francey SM, Nelson B, Baker K, Phillips LJ, Berger G, Amminger GP.
PACE: a specialised service for young people at risk of psychotic disorders. Med. J. Aust. 2007;
187(7 Suppl):S43–6. [PubMed: 17908025]
NIH-PA Author Manuscript
Yung AR, Nelson B, Stanford C, Simmons MB, Cosgrave EM, Killackey E, Phillips LJ, Bechdolf A,
Buckby J, McGorry PD. Validation of “prodromal” criteria to detect individuals at ultra high risk
of psychosis: 2 year follow-up. Schizophr. Res. 2008; 105(1):10–17. [PubMed: 18765167]
Yung AR, Yuen HP, McGorry PD, Phillips LJ, Kelly D, Dell’Olio M, Francey SM, Cosgrave EM,
Killackey E, Stanford C, et al. Mapping the onset of psychosis: the comprehensive assessment of
at-risk mental states. Aust. N. Z. J. Psychiatry. 2005; 39(11-12):964–971. [PubMed: 16343296]
NIH-PA Author Manuscript
NIH-PA Author Manuscript
Figure 1.
Receiver Operating Characteristic (ROC) curve of PQ-B scores predicting SIPS diagnosis of
prodromal/psychotic syndrome versus no SIPS diagnosis.
NIH-PA Author Manuscript
NIH-PA Author Manuscript
Figure 2.
PQ-B scores of Total symptoms endorsed by SIPS diagnostic group.
NIH-PA Author Manuscript
NIH-PA Author Manuscript
Figure 3.
PQ-B scores of weighted Distress by SIPS diagnostic group.
NIH-PA Author Manuscript
NIH-PA Author Manuscript
Table 1
Demographic and clinical characteristics of the samples.
Age (years, +/−SD) 18.7 (4.8) 19.2 (5.3) 19.1 (3.5) 0.14 0.87
Highest Parental
5.2 (1.9)1 5.5 (1.8)1 5.5 (2.0)1 0.59 0.56
Education
1
Hollingshead Index, means here are equivalent to Bachelor’s degree;
2
UCLA had a greater proportion of Hispanic/Latino participants than UCSF;
3
GAF scores available for UCLA control subjects only (N=27);
4
Post-hoc contracts showed that controls had significantly fewer males than the UCSF patient group;
*
Statistically significant at p<.05.
Table 2
Classification accuracy of PQ-B scores versus SIPS diagnosis of prodromal/psychotic syndrome versus no SIPS diagnosis.
PQ-B Cutoff Sample Sensitivity Specificity PPVa NPVa LR+a AUCa 95% CI p
Loewy et al.
Total Sample
Total Score ≥ 1 96% 16% 88% 38% 1.14 0.78 0.70 – 0.84 0.0001
(N=141)
Total Sample
Total Score ≥ 3 89% 58% 93% 46% 2.12 0.78 0.70 – 0.84 0.0001
(N=141)
Total Sample
Total Score ≥ 6 31% 100% 100% 18% --- 0.78 0.70 – 0.84 0.0001
(N=141)
Total Sample
Distress Score ≥ 4 93% 58% 93% 55% 2.20 0.78 0.70 – 0.84 0.0001
(N=141)
Total Sample
Distress Score ≥ 6 88% 68% 95% 50% 2.83 0.78 0.70 – 0.84 0.0001
(N=141)
Total Sample
Distress Score ≥ 32 32% 95% 98% 18% 6.07 0.78 0.70 – 0.84 0.0001
(N=141)
No Psychotic
85% 68% 90% 59% 2.71 0.76 0.66 – 0.85 0.0001
Subjects (N=82)
UCSF
Distress Score ≥ 6 90% 63% 92% 56% 2.39 0.80 0.66 – 0.90 0.0001
(N=47)
UCLA
87% 73% 96% 42% 3.18 0.77 0.68 - 0.85 0.0001
(N=94)
a
PPV = Positive predictive value, NPV = Negative predictive value, LR+= positive likelihood ratio, AUC= Area Under the Curve