Chapter 20

THE CURRENT STATUS

OF “PROJECTIVE” “TESTS”
Robert E. McGrath, Alec Twibell, and Elizabeth J. Carroll

The term projective tests is often used to encom- for other purposes came to be used as indicators
pass a variety of procedures that allow the target of projection, particularly the Bender Visual
individual to provide free-form responses to Motor Gestalt Test (Hutt, 1985). A 1959 survey
ambiguous stimuli. The participant’s responses found the three most commonly used psycho-
Copyright American Psychological Association. Not for further distribution.

are thought to be sensitive to implicit processes logical tests in clinical practice were projectives
and, consequently, may be somewhat resistant (Sundberg, 1961).
to efforts at misrepresentation. By the 1960s, though, the allure was fading
This class of instruments has had a particularly for two reasons. One was the general critique of
checkered past. Due to concerns about honesty traditional personality assessment that emerged
in responding to self-report measures, and the out of behaviorism. Mischel (1968) questioned
psychoanalytic belief that much of mental activity whether the criterion-related validity of person-
is resistant to self-observation, psychologists ality measures was sufficient to justify their use,
became enamored with the potential of projective while Goldfried and Kent (1972) criticized the
instruments to circumvent misrepresentation by practice of using latent constructs to account
requiring responses to ambiguous stimuli, so that for associations between test behavior and
those responses were thought to be representative behavioral outcomes.
of the individual’s style of engaging with the The second factor was a psychometric critique
external world. The development of the Rorschach of projective methods (e.g., Cronbach, 1949;
Inkblot Test (Rorschach, 1921/1942) preceded Entwisle, 1972; Swensen, 1968). This critique has
formal discussions of projective psychological engendered an enduring negative perception of
tests, but its popularity in the United States was projective instruments in the scientific community.
largely attributable to its presumed projective While surveys in the 1980s and 1990s found more
qualities. The Rorschach was soon joined by other than 75% of clinical doctoral programs required
instruments of similar ambiguity, including the training in projective testing (Piotrowski &
Thematic Apperception Test (TAT; Morgan & Keller, 1984; Piotrowski & Zalewski, 1993),
Murray, 1935; Murray, 1943), the Rosenzweig a more recent summary indicates training in
(1978) Picture Frustration Study, and the Szondi projective instruments has declined in both
Test (Deri, 1949). Even instruments developed clinical doctoral and internship programs

We are grateful to Gregory Meyer for his comments on an earlier version of this chapter.
https://doi.org/10.1037/0000318-020
APA Handbook of Research Methods in Psychology, Second Edition: Vol. 1. Foundations, Planning, Measures, and Psychometrics, H. Cooper
(Editor-in-Chief)
Copyright © 2023 by the American Psychological Association. All rights reserved.

433
APA Handbook of Research Methods in Psychology: Foundations, Planning, Measures, and
Psychometrics, edited by H. Cooper, M. N. Coutanche, L. M. McMullen, A. T. Panter, D. Rindskopf,
and K. J. Sher
Copyright © 2023 American Psychological Association. All rights reserved.
 McGrath, Twibell, and Carroll

(Piotrowski, 2015a), though this rate may have The prototypical projective instrument
stabilized in recent years (Mihura et al., 2017). demonstrates the following features:
A recent attempt to generate a list of discredited
1. Test stimuli are ambiguous in some important
psychological tests was largely dominated by
way. For example, the Rorschach Inkblot
projective instruments (Koocher et al., 2015).
method presents the respondent with a fixed
In contrast to what the publication record
series of inkblots and the question “What
would suggest, training and use of the main might this be?” The TAT requires the respondent
projective instruments seems to have remained to create a story based on a picture in which
relatively stable outside the United States people are engaged in unclear behavior.
(Piotrowski, 2015b). 2. Though some responses are incompatible
The remainder of this chapter summarizes with the instructions, for example, refusing
the current status of projective instruments as to respond to a Rorschach card (in some
scientific instruments. The first section offers a instructional sets) or saying it is an inkblot,
conceptual analysis of the nature of projective the number of acceptable responses to the
assessment. Drawing on discussions of projective stimuli is infinite. Traditional Rorschach
assessment and comparisons with other psycho- practice even allows the individual to decide
logical measurement methods, we suggest that how many responses to make to each inkblot.
Copyright American Psychological Association. Not for further distribution.

the continuing use of the terms projective and The more recent Rorschach Performance
test to refer to these instruments (e.g., Stedman Assessment System (R-PAS; Meyer et al., 2011),
et al., 2018) is problematic and probably should described below, limits the number of responses
be discontinued. per card to four.
Current evidence on each of three projective 3. The use of ambiguous stimuli is intended to
instruments—the Rorschach, TAT, and figure elicit idiosyncratic patterns of responding such
drawings—is also reviewed. Though other as unusual percepts or justification for those
projective instruments are used in clinical percepts on the Rorschach or unusual story
assessment, particularly various forms of content or story structure on the TAT.
incomplete sentences blank (Rotter et al., 4. Because of their free response format, projec-
1992), these three techniques are the most tive instruments often require individualized
extensively researched and effectively reflect administration and specialized training in
the current status of projective instruments administration, scoring, and interpretation.
in general.
The Problem With Projection
The use of the term projective carries with it certain
WHAT WE TALK ABOUT WHEN WE implications about the cognitive process that
TALK ABOUT “PROJECTIVE” “TESTS” determines important test behavior, implications
Murray (1938) and L. K. Frank (1939) that have been questioned in recent years by
provided the seminal works on what is called individuals closely associated with the study of
the projective hypothesis. They proposed that the Rorschach (e.g., Bornstein, 2007; Meyer &
free-format responding to ambiguous, or Kurtz, 2006). To understand why this shift has
“culture-free” (Frank, p. 389), stimuli would occurred, one must recognize there are at least
three problems with calling these instruments
encourage emergence of personal meanings and
projective.
feelings. The labeling of certain instruments
as projective also provided a clever phonetic The ambiguity of the term projective.  The term
contrast to objective measures such as rating projective has multiple connotations in the
scales that restrict the set of acceptable response psychoanalytic literature from which it emerged.
alternatives. Freud (1896/1962) used the term first to refer to

434
 The Current Status of “Projective” “Tests”

a specific defense mechanism characterized by TABLE 20.1

the unconscious attribution of one’s unacceptable
feelings and wishes to some external object or Information Sources Available Through
individual. Later, though, he used the term in a “Projective ” “Tests”
more general sense to encompass any idiosyncratic
construction of environmental stimuli (Freud, Source Examples
1913/1990). It was this latter use of the term Thematic material Morbid themes (R and T); stories that focus
Murray (1938) referenced when he drew the    on achievement (T)
Perceptual Poor form quality (R); preoccupation
connection between responding to psychological   idiosyncracies    with small details (R and T); omission
tests and psychoanalytic theory.    of critical stimulus elements (R and T)
There is nothing uniquely Freudian about the Extratest behavior Card rotation (R); attempts to reject stimuli
general proposition that different people construe    (R and T)
Self-descriptive Indications of task-related discomfort or
stimuli differently, that ambiguity in the stimulus   statements    enjoyment (R and T)
field can contribute to individual differences in Quality of thought Illogical justification of percepts (R);
stimulus responding, and that those differences can    tangentiality (R and T)
Quality of speech Vocabulary, rhyming, or use of clang
reveal something important about the individual.
   associations (R and T)
Once that proposition is couched in terms of
Copyright American Psychological Association. Not for further distribution.

projection, however, it takes on an ambiguous Note. R = Rorschach inkblot method variable;

psychoanalytic connotation. T = Thematic Apperception Test variable.

Characterization of respondent behavior.  

If the concept of projection is not necessary for an important and distinctive element of Rorschach
understanding instruments such as the Rorschach, administration in which the respondent is asked
it is also clearly not sufficient. Based on prior to explain how each response was formulated.
attempts to define the scope of potentially Though perceptual idiosyncrasies play a particu-
interesting respondent behaviors to projective larly central role in the Rorschach, they can be
instruments (e.g., Basu, 2014; Exner, 1989; important for any instrument where the respondent
McGrath, 2008), as well as personal experience is expected to respond to ambiguous stimulus
with these instruments, we suggest there are at materials. For example, clearly ignoring or
least six sources of information that can be distorting a central element of a TAT picture is
observed using a projective instrument (Table 20.1), sometimes taken as a reflection of a disordered
though they tend to be of varying importance perceptual style, perhaps suggestive of thought
across instruments and respondents. disorder, or of issues that are discomforting to
Thematic material refers to the degree to the respondent, though Eron (1950) raised
which responses contain language or phrasing concerns about these interpretations.
that reflects certain attitudes or emotional states. Extratest behavior encompasses anything
This is the information source that comes closest distinct from responses based on the instructional
to the concept of projection, in that respondents set, including the manner in which the person
may respond to the stimuli in a manner reflecting handles the physical stimuli or behaviors such
issues of particular concern for them, but these as odd mannerisms and expressions of resistance.
concerns need not be unavailable to consciousness. Of these, expressly self-descriptive statements
Exner (1989, 2003) argued that Rorschach are significant enough to mention as a distinct
was particularly interested in his instrument as source of information. Because of its free-response
a method for detecting perceptual idiosyncracies. format, the Rorschach or TAT gives the respondent
This emphasis is evident in Rorschach’s original license to make statements providing clues
instructional set, “What might this be?” It is about cardinal traits or distinctive ways in which
also evident in his creation of the inquiry phase, they understand themselves. This source of data

435
 McGrath, Twibell, and Carroll

provides a complementary perspective to the better psycho­metric qualities than the Rorschach.
standardized approach to trait description offered One may hypothesize that a moderate level of
by objective instruments. In practice, though, ambiguity is optimal, in that it challenges the
self-descriptive statements during administration respondent without overtaxing in a manner
of projective instruments are often restricted to likely to result in disengagement from the task.
the respondent’s reactions to the instrument.
Quality of thought refers to the logic or reason- Implications
ableness of the thought processes evidenced Taking these three arguments together, one can
during the administration. Finally, quality of speech conclude that projection is not a necessary con-
encompasses various factors associated with tributor to the clinical value of such instruments.
effectiveness of communication, including the These measures seem to be interesting not
length of responses, complexity and precision of necessarily because they are projective but because
the language used, and so forth. Considering the they are provocative, and so elicit interesting
variety of types of information that can emerge behaviors from the respondent. This conclusion
during the administration of a so-called projective would imply value in retiring the term projective
instrument, emphasizing the first has the potential tests (Meyer & Kurtz, 2006), but then the
to result in underestimation of the value of question arises of the best alternative. Bornstein
Copyright American Psychological Association. Not for further distribution.

the others. (2007) suggested calling them stimulus-attribution

tests, suggesting the common thread is that the
The role of ambiguity.   Finally, it is worth respondent is required to attribute meaning to the
speculating whether the emphasis on projection stimuli. McGrath and Carroll (2012) suggested
has led to misleading conclusions about the broadband implicit techniques. This term was based
relationship between ambiguity and clinical in part on Fowler and Groat’s (2008) proposal
usefulness, though the empirical basis for this that projective instruments are consistent with
point is thin. One of the corollaries of Frank’s other instruments that attempt to draw infer-
(1939) projective hypothesis was that greater ences about personal character without direct
ambiguity was associated with greater potential questioning such as the Implicit Association Test
for projection. Perhaps the most extreme example (Greenwald et al., 1998). However, where that
of this proposition is Card 16 of the TAT, which instrument is limited in focus, providing no other
is simply a white card for which the respondent information than how the individual associates
is instructed to both imagine a picture and tell two constructs, Table 20.1 demonstrates the
a story about that picture. popular projective instruments are broadband
In fact, while there is some evidence that methods that potentially provide information
responding to Card 16 is related to creativity across multiple channels. Because of its broadband
(Wakefield, 1986), clinicians find the stories nature, Weiner (1994) criticized the common
are often less interesting than those provided in practice of referring to the Rorschach as a test.
response to other cards (Groth-Marnat, 2009). In itself, the Rorschach is a method of data
Similarly, it turns out to be the case that Rorschach, collection rather than the standard by which
who was something of an artist, touched up his those data are judged, as is implied by the word
original inkblots in a manner that made them test. For this reason, Weiner recommended
more evocative of certain percepts (Exner, 2003); referring to the Rorschach Inkblot method.
that is, he made them less ambiguous and more His point is relevant to all the projective instru-
culture-bound than they were originally. They are ments, suggesting the term test should be
less amorphous than, for example, the Holtzman generally avoided.
Inkblot Technique (Holtzman et al., 1961). This Despite these concerns, references to projective
difference may help explain why the latter has tests still appear widely in the literature. Stedman
never been popular in clinical use despite much et al. (2018) suggested simply retaining the term

436
 The Current Status of “Projective” “Tests”

projective for historical reasons. If precedent wins length were more complex but allowed for
out over precision, however, it should be with protocols of 50 responses or more. Either of
the understanding that optimal applied use of these procedures could result in cumbersome
projective measures requires using the totality of administrations, even though Exner (2003)
an individual’s behavior during the administration. concluded longer protocols were no more useful
In this regard, the most important recent innova- than shorter ones.
tion in Rorschach methodology is the previously To address the issue of variation in responding,
mentioned R-PAS (Meyer et al., 2011; see also administration instructions were revised as
Meyer & Mihura, 2020). part of the development of the R-PAS. The new
The emphasis on individual variation in R-Optimized instructions limit the protocol to
responding to these instruments does create 40 responses. As a result, some authors raised
a challenge for projectives as psychometric concerns that evidence gathered using the CS may
instruments. The reliance on relatively ambiguous not generalize to the R-PAS (e.g., Khadivi & Evans,
stimuli and/or instructions that allow for infinite 2012; Kivisto et al., 2013), especially as other
diversity in responding creates a challenge in terms differences exist in the two systems. More recently,
of meeting traditional standards for reliability and summaries across multiple samples suggest the
validity. This challenge will be addressed in the two forms of administration have little effect on
Copyright American Psychological Association. Not for further distribution.

context of a general psychometric evaluation of Rorschach variable scores except for variables
the three most popular projectives. directly reflecting the number of responses,
while R-Optimized tends to result in more efficient
administration and less variability in the number
THE PSYCHOMETRIC STATUS
of responses (Hosseininasab et al., 2019; Pianowski
OF THE RORSCHACH
et al., 2021). That said, in the following sections
Of all projective instruments, the Rorschach we focus on results using the more defensible
inkblots remain the most popular in the R-PAS but indicate when results came from the
assessment research literature. The continuing CS instead.
popularity of the Rorschach was founded in the
success of Exner’s (2003) Comprehensive System Reliability
(CS), which brought uniformity in administration Interrater reliability.   Scoring of Rorschach
and scoring, normative data, and interpretation variables involves two steps. The first has to do
to the Rorschach. The R-PAS was subsequently with the assignment of a series of codes to indi-
developed to address limitations of the CS. This vidual responses, the second with the aggregation
process included revising the CS’s instructions of code results into variables. The first phase
for administration, choosing Rorschach vari- can be referred to as response coding, the second
ables for inclusion based on a history of good as protocol scoring. The latter is increasingly
empirical support and gathering an international accomplished by entering response codes into
normative sample. software, reducing the importance of reliability
Before reviewing the evidence, which spans in protocol scoring. On the other hand, inter­
work gathered using both Exner’s CS and R-PAS, pretation is based largely on the R-PAS scores,
the issue of generalizing findings from the former so these represent the more important consid­eration
to the latter should be addressed. Administration in settings where scoring is still done by hand.
instructions for the CS placed few limits on Response coding involves binary decisions
the number of responses per card. As a result, about the presence or absence of a code, so the
it was possible for the individual to generate too appropriate interrater reliability statistic is the
few responses for the results to be considered kappa coefficient. Protocol scores are dimensional
reliable (< 14), in which case the entire instru- and so call for computation of the intraclass
ment was readministered. Rules for limiting correlations (single measure). The R-PAS generates

437
 McGrath, Twibell, and Carroll

raw scores as well as scores adjusted for the remain consistent. While most R-PAS scores
overall complexity of the protocol, thought reflect constructs thought to be relatively stable,
to be an indicator of engagement in the task others focus on states of distress. As a result,
(Ales et al., 2020). For the present purposes, studies examining the stability of an array of
we adopted the recommendation that reliability Rorschach variables are flawed. With this caveat
values of .60 or greater are acceptable for the in mind, a number of studies over the years have
reliability statistics commonly used in this looked at temporal stability in various Rorschach
literature, that is, coefficient alpha and the kappa variables, summarized in a meta-analysis conducted
coefficient (e.g., Cicchetti, 1994; Shrout, 1998). by Grønnerød (2003, 2006). He estimated that
We note, however, that it can be difficult to the mean 6-month correlation for Rorschach
achieve adequate values for kappa when the base variables was > .70, with only three non-CS
rate for one outcome or the other is low. For variables and one CS variable associated with
example, Kivisalu et al. (2017) found accept- correlations < .60. Sultan and Meyer (2009)
able kappa values for 40 of 62 R-PAS variables provided evidence that temporal stability can
from 50 nonpatient protocols with a total of be compromised if the number of responses is
1,168 responses. Using their results, we were high or variable across records, providing further
able to compute the absolute differences between evidence that the R-Optimized administration
Copyright American Psychological Association. Not for further distribution.

the base rate of each variable and .50, and cor- should be preferred to the administration
relate those differences with the corresponding developed for the CS.
kappa value. This correlation proved to be –.42;
as expected, judgments of variables with more Validity
extreme base rates were less reliable. In contrast, One approach to evaluating the validity of
Schneider et al. (2022) meta-analyzed intra­ the Rorschach will be dispensed with quickly.
class correlation coefficients between raters Studies examining convergence between the
across four samples. Out of 60 R-PAS variables, Rorschach and a self-report indicator such as the
only one variable was associated with what Minnesota Multiphasic Personality Inventory
should be considered a questionable reliability (MMPI) have been disappointing, with little
value (< .60). evidence of correlation between variables
that seem to be measuring similar constructs
Internal reliability.   It would technically be
(Meyer et al., 2017; Mihura et al., 2013). Several
possible to compute coefficient alpha or other
hypotheses have been suggested to explain this
indices of internal reliability for Rorschach codes
failure to converge (e.g., Meyer et al., 2000),
if each card is treated as an observation and the
the most compelling of which suggests that
presence-absence or frequency of each code on
measures of different behavioral channels
each card is treated as the outcome. An example
(McGrath, 2005) or methods of measurement
of this approach is provided by Bornstein and
(Bornstein, 2009) generally correlate more
Masling (2005), who found reliabilities of
poorly than anticipated. For example, implicit
.61 to .62 for a measure of dependency. However,
measures correlate poorly with other psycho­
to our knowledge not a single study has evaluated
logical measures (Gawronski et al., 2007) and
the internal reliability of Rorschach variables in
even with other types of implicit measures
either the CS or R-PAS.
(e.g., Nosek et al., 2007; Ziegler et al., 2010).
Test–retest reliability.   The evaluation of A meta-analysis conducted by Mihura and
test–retest reliability is particularly problematic colleagues (2013) examined the validity of
for broadband measures such as the Rorschach. each CS score, including diagnoses, intelligence
It requires setting the interval between admin- test results, and observed behavior as criteria in
istrations so there is a reasonable likelihood addition to self-report as criteria. The authors
that the latent constructs underlying the scores concluded that the mean validity correlation for

438
 The Current Status of “Projective” “Tests”

30 variables was ≥ .21, while 35 had weak or evidence base to date. This is perhaps the most
limited supporting evidence. Variables with the important next step in the future of projective
strongest evidence reflected cognitive processes assessment.
and perceptual styles, while rare variables such Several studies have examined the incremental
as food responses tended to demonstrate weaker validity of the Rorschach. Dao et al. (2008)
validity. Czopp and Zeligman (2016) challenged provided evidence that the Rorschach is a better
some of the conclusions drawn in this meta- predictor of psychosis than the MMPI, and the
analysis, suggesting some of the research reviewed Rorschach has also shown superiority to the MMPI
did not operationalize the variables in the same as a predictor of therapy outcome (Meyer, 2000).
way as the CS, and as a result these variables may Though encouraging, this small literature is an
have been dismissed or restructured prematurely. insufficient basis for concluding the relatively
Despite this concern that some potentially useful demanding Rorschach provides sufficient incre-
variables were poorly represented in the meta- mental validity over other methods to justify
analysis, this study played an important role in its cost. Clearly, more research is needed on this
the subsequent development of R-PAS, influencing topic before there is sufficient evidence of the
decisions about which CS variables to eliminate, Rorschach’s clinical utility.
what non-CS variables to add, and which variables
Copyright American Psychological Association. Not for further distribution.

merited modification (Meyer & Eblin, 2012). General Conclusions

Table 20.2 lists variables with demonstrated validity Research has shown interrater reliability for
according to the meta-analysis and changes made Rorschach scoring is adequate for the majority
to variables in the R-PAS. of variables in their present form, though it is
The R-PAS also introduced two innovations in less than ideal for some variables that are rarely
Rorschach scoring. First, new normative samples scored as positive. While the validity of the
were gathered for both adult and youth (Meyer Rorschach in assessing multiple domains of
et al., 2020; Viglione & Giromini, 2016) popula- functioning has long been accepted, the changes
tions to address various problems with the CS implemented with the introduction of the R-PAS
norms. For example, the adult normative sample creates the potential for a more psychometrically
was collected across 17 countries to reflect defensible instrument. That said, further research
the international popularity of the instrument. is needed to confirm this proposition. Further-
Scores are now standardized based on normative more, a great deal more research is needed to
data to simplify the process of identifying identify situations in which the Rorschach is
extreme scores. In addition, as noted previously, worth using instead of or in addition to other
a measure of complexity of responses is available more cost-effective techniques such as self-report.
as an indicator of engagement in the task, and Certain situations intuitively appear to warrant
standardized scores are generated both with and its use. In particular, forensic evaluations often
without adjustment for complexity. occur in contexts where self-misrepresentation
The innovations described here make the R-PAS is likely, and there is particular interest in the
the most sophisticated instrument development cognitive and perceptual constructs for which
program in the history of projective assessment. the Rorschach is a particularly useful gauge.
The focus on variables with existing empirical
support, more representative normative samples,
PSYCHOMETRIC STATUS OF THE TAT
and consideration of engagement as a moderator all
potentially contribute to the validity of the R-PAS. The consistency in administration and scoring
Unfortunately, research evaluating the success brought to the Rorschach by the CS and R-PAS is
of these efforts at enhancing the validity of the important if reliability results are to be assumed to
Rorschach is limited, with Mihura et al. (2013) be generalizable. Though Murray (1943) initially
having provided the most comprehensive offered a prescribed administration procedure

439
 McGrath, Twibell, and Carroll

TABLE 20.2

Strength of Validity Evidence for Rorschach Comprehensive System Variables

Domain Supported Little or no support No evidence

Controls and Human movement Animal movementb Nonhuman
  situational Experience actuala Adjusted difference Movement
  distress Sum of shading Scoreb Coping style: pervasiveb
Inanimate movements Coping style (extratensive Coping style: ambitentb
Number of responses   vs. introversive)a
Lambdaa
Weighted sum of colora
Diffuse shading
Difference score
Experienced stimulationa
Affective Achromatic color White spacea Color projectionb
  features Form–color ratio Pure color Constriction ratio
Affective ratio
Complexity ratio
Copyright American Psychological Association. Not for further distribution.

Interpersonal Cooperative movement Aggressive movement Interpersonal interest

  perception Whole, realistic Foodb
Humans Isolation indexb
Texture Active to passive ratioa
Good human
Representations
Personal
Poor human
Representations
Self-perception Anatomy and X-ray Form dimension
Morbid Egocentricity indexb
Reflections
Vista
Information Synthesized responsea Processing efficiencyb Aspiration ratiob
  processing Organizational frequencyb Economy index
Vague response
Perseverationb
Cognitive Conventional form White space distortion
  mediation Distorted form
Appropriate form
Popular
Unusual form
Human movement, distorted form
Ideation Critical special scores Human movement, formlessb
Critical special scores, severe
Indices Perceptual-thinking index Hypervigilance indexa Intellectualization index
Suicide constellationa Obsessive style indexb
Depression indexa
Coping deficit index

Note. Supported = mean r ≥ .15 and fail-safe N (number of unpublished studies finding r = .00 needed to render
the mean nonsignificant at .05) ≥ 10, and p < .05. Little support = mean r < .15 or fail-safe N < 10, and p < .05.
No support = p > .05. No evidence = insufficient evidence to draw a conclusion. Adapted from “The Validity
of Individual Rorschach Variables: Systematic Reviews and Meta-Analyses of the Comprehensive System,”
by J. L. Mihura, G. J. Meyer, N. Dumitrascu, and G. Bombel, 2013, Psychological Bulletin, 139(3), p. 570
(https://doi.org/10.1037/a0029406). Copyright 2013 by the American Psychological Association.
aAdjusted in R-PAS. bOmitted from R-PAS.

440
 The Current Status of “Projective” “Tests”

for the TAT, it involved administering 20 cards, for each TAT response and then averaged to
which proved unwieldy in practice. Unfortunately, generate overall scores.
no commonly accepted alternative has emerged. The second is the Defense Mechanism Manual
Researchers continue to use different subsets of (DMM; Cramer, 1991), which was developed
cards (e.g., Bram, 2014; Porcerelli et al., 2010), specifically for use with the TAT and the
in most cases varying between four to six cards. Children’s Apperception Test. It is intended to
Doctoral programs seem to offer little training in detect use of three defense mechanisms: denial,
formal scoring of the TAT (Mihura et al., 2017), projection, and identification. Each of the defense
so that qualitative interpretations of uncertain mechanisms is evaluated on seven dimensions,
validity likely remain the norm in applied settings. with scoring criteria provided for each dimension
In addition, there are alternative apperceptive on certain cards. Research with the DMM has
pictures developed for special populations, such focused on the TAT rather than the Children’s
as the Children’s Apperception Test (Bellak & Apperception Test.
Bellak, 1949), as well as more narrowband picture
sets intended to detect specific motivations Reliability
(e.g., McClelland et al., 1953). There are even Reliability analyses for the two scoring systems
differences in whether the respondent delivers have primarily focused on interrater reliability
Copyright American Psychological Association. Not for further distribution.

the story verbally or in writing, which should and, to a lesser extent, on internal reliability of
have significant effects on productivity. Any scores across cards. Evidence for the interrater
efforts to discuss expected reliability or validity reliability of the SCORS-G is consistently
of scores on the apperception technique in supportive (e.g., Huprich & Greenberg, 2003;
general, or the TAT in particular, are, therefore, Pad et al., 2020). The same is true for the DMM,
problematic. with one exception. Though some studies (e.g.,
Only two scoring systems applicable to the TAT Cramer, 2009; Porcerelli et al., 1998) have reported
have received much research attention in recent adequate interrater reliability for the Denial scale,
years and so provide the most defensible basis for others (Hibbard et al., 1994; Porcerelli et al., 2010)
evaluating the TAT as a research tool. The more have reported interrater reliability estimates for
extensively studied is the Social Cognition and the Denial scale below .60.
Object Relations Scale–Global Rating Method Evidence for the internal reliability of these
(SCORS-G; Stein et al., 2011; Westen, 1995). scoring systems has been weaker. Hibbard et al.
The original version of the instrument consisted (1994) found the internal reliability of DMM
of four scales, with multiple items for each scores across six cards for Denial and Identifi­
dimension, but the SCORS-G involves giving a cation were both < .60. Inslegers et al. (2012)
global 1–7 rating of level of maturity on eight similarly found consistently poor internal
dimensions of interpersonal functioning drawn reliability across cards for the four original
from object relations theory: complexity of SCORS-G scales. Their study also involved
representation of people, affective quality of administering six cards. Two responses have
representations, emotional investment in relation- been offered to these concerns about internal
ships, emotional investment in values and moral reliability. Hibbard et al. (2001) examined internal
standards, understanding of social causality, reliability for the four original SCORS scales in
experience and management of aggressive impulses, samples where four, five, or 10 TAT cards were
self-esteem, and identity and coherence of self. administered. They found reliability values were
Anchors are available for some response options. only consistently acceptable in the samples
Though the SCORS-G is considered applicable to administered 10 cards, suggesting that the four
any self-revelatory narrative material, for example, to six cards typically used in TAT research may
reports of early memories, it is most commonly be insufficient to achieve reliable outcomes. An
applied to TAT stories. A separate score is provided alternative perspective was offered by Schultheiss

441
 McGrath, Twibell, and Carroll

et al. (2008), who proposed that internal reliability (1992) concluded the TAT was a better predictor
analysis is irrelevant to the TAT because it is of achievement motivation than self-report
predicated on the invalid assumption that the measures. Entwisle (1972), however, raised the
cards are intended to represent interchangeable possibility that intelligence could account for the
stimulus objects. Exner (1996) made a similar relationship. Research into the TAT as a predictor
point about the Rorschach, suggesting that each of the motivation for power, affiliation, and
card has unique properties that pull for its own intimacy (McAdams, 1982; McClelland, 1965,
distinctive types of responses. Of course, traditional 1975) has produced similar evidence of validity,
psychometric considerations would suggest though studies of incremental validity are almost
this variability in stimulus materials creates an nonexistent (More & Winch, 1956). It has also
obligation to administer more than four to five been used as a measure of problem-solving using
cards if the goal is to maximize the potential for the Personal Problem-Solving System–Revised
generating test scores that demonstrate desirable (Ronan et al., 2008). This system has been
levels of predictive validity. validated in several studies, some of which have
controlled for intelligence as a possible confound,
Validity but research on this topic has been absent in
Literature from multiple laboratories supports recent years.
Copyright American Psychological Association. Not for further distribution.

the validity of the SCORS-G scales as indicators

of personality pathology and interpersonal General Conclusion
functioning (e.g., Pad et al., 2020; Stein & The evidence base supporting the TAT as a research
Slavin-Mulford, 2018; Stein et al., 2018). However, tool is unfortunately sorely lacking. There is
given that the SCORS-G has particular relevance reasonable evidence that the most extensively
to personality pathology, Pad et al. (2020) found researched scoring systems can generally achieve
the SCORS-G had little incremental validity over acceptable levels of interrater reliability. The
self-report as an indicator of borderline and evidence for internal reliability is weak, and
depressive personality disorder pathology. it remains unclear whether better results are
The best evidence for the validity of the DMM achievable through administration of more
comes from studies by Hibbard et al. (1994) and cards (though this will tend to create practical
Porcerelli et al. (2010). The former found that obstacles to use of the instrument) or whether
undergraduates generated higher scores on internal reliability is an important consider-
identification (considered the most mature of ation in apperceptive techniques. Though some
the defenses evaluated) than psychiatric patients, intriguing evidence exists to support the validity
and lower scores across the three defenses of at least some TAT scales, a good deal more
indicating primitive functioning. The latter work would be needed, particularly on the
similarly found that greater evidence of denial topic of standardization of administration and
and projection in the stories was associated with incremental validity, before the TAT could be
lower ratings by clinician of defensive functioning. recommended as a strategy for collecting data
However, higher scores for identification were rather than as a focus for research on its potential
unrelated to clinician ratings. This represents a as a clinical instrument.
fairly limited evidence base for recommending
these scores as sufficient measures of defensive
PSYCHOMETRIC STATUS
functioning.
OF FIGURE DRAWINGS
Murray (1943) originally intended the TAT
as an indicator of the various motivations he Figure drawings differ from most other projective
referred to as needs, so it is worth noting that techniques in that they depend on physical activity
some evidence exists to support its use for that rather than verbal response. The information
purpose. A meta-analytic review by Spangler sources listed in Table 20.1 are still relevant,

442
 The Current Status of “Projective” “Tests”

though some modifications are in order. Consistent however, have not been encouraging (e.g., Pihl &
elements of drawing style, such as the use of Nimrod, 1976). In reviewing literature on the use
heavily elaborated lines, can be thought of as of figure drawings, Swensen (1968) concluded
themes, in this case a theme often thought to be holistic evaluations were more indicators of
indicative of emotional issues. Unusual details, problems than scoring systems. For example,
such as omitting windows from a house, are Tharinger and Stark (1990) found a holistic
similar to the perceptual idiosyncracies found in evaluation of four characteristics in DAP drawings
TAT and Rorschach responding. The observation (inhumanness, lack of agency, lack of well-being,
of extratest behavior and self-descriptive statements and a hollow or stilted portrayal) was a better
remains potentially useful, and the quality of detector of emotional distress in children than
thought and speech can also be evaluated if the the 30-item Koppitz system.
administrator tests limits by asking questions Naglieri et al.’s (1991) Draw-A-Person:
about the drawings. Screening Procedure for Emotional Disturbance
Several figure drawing techniques have drawn (DAP:SPED) has superseded the Koppitz scores
particular attention. The House-Tree-Person as the most commonly studied scoring procedure.
(H-T-P; Buck, 1948) calls for drawings of the It consists of 55 criteria drawn from the DAP
three objects listed, each on a separate piece of research literature that were expected to identify
Copyright American Psychological Association. Not for further distribution.

paper. The Draw-A-Person test (DAP; Machover, children with emotional difficulties. Though
1949) requires drawing a person, then a person the authors reported strong interrater, internal
of the opposite sex. A more recent alternative reliability, and test–retest reliability, Wrightson
is the Kinetic Family Drawing (KFD; Burns & and Saklofske (2000) questioned the temporal
Kaufman, 1972), which involves drawing a picture stability of scores over 23 to 27 weeks. Studies
of one’s family doing something. suggest correlations between DAP:SPED scores
Figure drawings remain popular clinical and various indicators of emotional difficulties are
instruments. They are easily administered to at best small, resulting in inadequate diagnostic
almost any individual. They also involve a accuracy (e.g., Crusco, 2013; Matto et al., 2005;
familiar task that helps reduce anxiety about the McNeish & Naglieri, 1993; Naglieri & Pfeiffer,
testing, particularly in children. At the same time, 1992; Wrightson & Saklofske, 2000). Given
they have suffered the most radical decline in that the DAP:SPED requires even more judg-
respectability of the three instruments discussed ments than the Koppitz criteria, an investigation
in this chapter. Early work relied heavily on a of the incremental validity of the DAP:SPED over
sign approach, where unusual drawing details Tharinger and Stark’s (1990) holistic evaluation
were individually taken as evidence of a latent would seem to be a useful topic for future research.
construct in a manner that relied heavily on Various scoring systems have also been
psychoanalytic assumptions about the projection developed for using the DAP to estimate
of unconscious conflicts onto ambiguous intelligence. As in the case of emotional distress,
stimuli. This approach has been largely rejected these methods tend to demonstrate good
( Joiner et al., 1996; Swensen, 1968), even by interrater reliability. However, evidence again
proponents of figure drawings as a clinical tool tends to suggest at best small correlations
(e.g., Riethmiller & Handler, 1997). between scores based on drawings and results
Several systems have emerged for scoring from intelligence tests (e.g., Rehrig & Stromswold,
elements of drawings. These consistently 2018; Troncone et al., 2021).
demonstrate excellent interrater reliability, but At this point, no defensible standardized
evidence for validity and incremental validity is scoring system has emerged for the H-T-P. Though
questionable. Koppitz (1968) developed perhaps Burns and Kaufman (1972) offered guidelines for
the most popular early scoring system for detecting scoring the KFD, these are unsupported by any
emotional distress in the DAP. Research results, research. Various scoring systems were suggested

443
 McGrath, Twibell, and Carroll

in subsequent years. Though these systems Though we have accepted continued use of
demonstrate adequate interrater reliability, the term projectives to refer to this class of
evidence that they can identify children with instruments, our first important conclusion is
emotional difficulties is weak (Cummings, 1986; that this must be understood as an anachronistic
Knoff & Prout, 1985). term. We hope that over time these will be
In contrast to the Rorschach, where research increasingly accepted as a set of performance-
suggests an extensive scoring system is capable based measures, founded on the assumption that
of generating valid information about a variety provocative stimuli will encourage responding
of psychological variables, the evidence is that in a manner reflecting important features of
simpler is generally better in the case of figure the respondent. This perspective places these
drawings. Evidence consistently suggests holistic instruments in a category with commonly used
evaluation of the drawing is a more useful research tools such as the Implicit Association
indicator of emotional difficulties in children Test. Conceptualizing the projective measures
than complex scoring systems based on discrete in this way allows one to draw several valuable
drawing elements. This level of effort is also conclusions about how they are best understood.
more justifiable given the limited validity of First, there is room yet for building a better
figure drawings as predictors of distress. The use mousetrap. The optimal projective technique
Copyright American Psychological Association. Not for further distribution.

of drawings to estimate intelligence is even more would consistently use stimuli that are obscure
questionable and probably should be avoided enough to encourage individualized responding
even in clinical settings. There is no current basis but evocative enough to engage the respondent.
for recommending the use of figure drawings Second, the most successful performance-based
as a research tool, and no evidence suggests the measures of personality, those that are thought to
need for further development beyond the simple demonstrate adequate fidelity, tend to demonstrate
procedure suggested by Tharinger and Stark a clear intuitive connection to the implicit
(1990). Future research might also look into process they attempt to gauge. For example,
whether increasing the number of pictures offers bizarre responses to a Rorschach inkblot, TAT
any increment in validity, but doing so undermines picture, or even a figure drawing task are likely
the simplicity that is one of the most attractive to be accurate indicators either of significant
features of figure drawings as an icebreaker in pathology or active misrepresentation. The
clinical evaluation. purely actuarial approach has largely failed as
a means of identifying reliable relationships
between behavior on projective instruments
SUMMARY AND CONCLUSION
and psychological variables (McGrath, 2008).
It is likely that projective techniques will Even when intuitively reasonable relationships
continue to play an important role in clinical are demonstrated, however, cross-validation
assessment. As Table 20.1 indicates, the potential is essential.
array of sources of information resulting from Third, standardization in administration is
administration of projective instruments can important for any future program of research on
contribute to clinical understanding of the these instruments. Thanks to the CS and the R-PAS,
individual (e.g., Nakamura, 2012). As research this goal has been achieved for the Rorschach.
tools, though, only the Rorschach in its most TAT administration on the other hand remains
recent iteration in the R-PAS meets criteria for idiosyncratic. A review of research on the TAT
potential use as a research tool. One may expect suggests there are four cards that are used in
continuing research to appear on broadband almost every study on the instrument: cards 1,
implicit techniques, and the Rorschach in 3BM, 4, and 13MF. However, existing evidence
particular, as clinical tools. suggests that adequate levels of internal reliability

444
 The Current Status of “Projective” “Tests”

have only been achieved with administration of of Personality Assessment, 102(4), 538–550.
10 cards. To our knowledge there is no research https://doi.org/10.1080/00223891.2019.1575227
extant that evaluates this issue in administrations Basu, J. (2014). Psychologists’ ambivalence toward
of seven to nine cards, so it is possible that ambiguity: Relocating the projective test debate
for multiple interpretative hypotheses. SIS Journal
adequate levels of reliability can be achieved with of Projective Psychology & Mental Health,
fewer than 10. The identification of a standard 21(1), 25–36.
set of 10 cards to be used in any future work on Bellak, L., & Bellak, S. (1949). The Children’s
the TAT (unless future research suggests seven Apperception Test. C.P.S.
to nine cards is sufficient) would be a valuable Bornstein, R. F. (2007). Toward a process-based
step forward. framework for classifying personality tests:
Fourth, the association of projective instruments Comment on Meyer and Kurtz (2006). Journal
with other performance-based instruments has of Personality Assessment, 89(2), 202–207.
https://doi.org/10.1080/00223890701518776
important implications for what has been a
common strategy in the validation of projectives. Bornstein, R. F. (2009). Heisenberg, Kandinsky,
and the heteromethod convergence problem:
Many studies have examined correlations between Lessons from within and beyond psychology.
self-report measures and projective instruments Journal of Personality Assessment, 91(1), 1–8.
over the years. We have limited our focus on https://doi.org/10.1080/00223890802483235
Copyright American Psychological Association. Not for further distribution.

such studies in this review. Research with implicit Bornstein, R. F., & Masling, J. M. (2005). The Rorschach
measures provides little evidence of convergence Oral Dependency Scale. In R. F. Bornstein &
with self-report measures, and, in most cases, J. M. Masling (Eds.), Scoring the Rorschach:
Seven validated systems (pp. 135–157). Erlbaum.
the latter are better predictors of important https://doi.org/10.4324/9781410612526
criteria than the former. However, there is also
Bram, A. D. (2014). Object relations, interpersonal
reasonable evidence that performance-based functioning, and health in a nonclinical sample:
measures offer incremental validity over self-report Construct validation and norms for the TAT
in the prediction of such outcomes (Greenwald SCORS-G. Psychoanalytic Psychology, 31(3),
et al., 2009). Comparative studies that focus on 314–342. https://doi.org/10.1037/a0036286
correlations between projective methods and Buck, J. N. (1948). The H-T-P technique, a qualitative
and quantitative scoring manual. Journal of
self-report measures rather than incremental
Clinical Psychology, 4(4), 317–396. https://doi.org/
validity of each over the other potentially result 10.1002/1097-4679(194810)4:4<317::AID-
in underestimating the potential value of JCLP2270040402>3.0.CO;2–6
projectives. Burns, R. C., & Kaufman, S. H. (1972). Actions, styles
With the possible exception of the Rorschach, and symbols in Kinetic Family Drawings (K-F-D):
projective instruments are likely to remain An interpretive manual. Brunner-Routledge.
primarily instruments of clinical interest rather Cicchetti, D. V. (1994). Guidelines, criteria, and rules
than research tools used for their measurement of thumb for evaluating normed and standardized
assessment instruments in psychology. Psychological
accuracy. We consider the R-PAS an important
Assessment, 6(4), 284–290. https://doi.org/10.1037/
model for using research to steer the future of 1040-3590.6.4.284
their applied use, and we hope those individuals
Cramer, P. (1991). The development of defense
who value their clinical contributions will continue mechanisms: Theory, research and assessment.
to work towards a more empirically founded Springer-Verlag. https://doi.org/10.1007/978-1-
standard of practice. 4613-9025-1
Cramer, P. (2009). The development of defense
mechanisms from pre-adolescence to early
References adulthood: Do IQ and social class matter?
Ales, F., Giromini, L., & Zennaro, A. (2020). A longitudinal study. Journal of Research in
Complexity and cognitive engagement in the Personality, 43(3), 464–471. https://doi.org/
Rorschach task: An eye-tracking study. Journal 10.1016/j.jrp.2009.01.021

445
 McGrath, Twibell, and Carroll

Cronbach, L. J. (1949). Statistical methods applied to Freud, S. (1962). Further remarks on the neuro-
Rorschach scores; a review. Psychological Bulletin, psychoses of defence. In J. Strachey (Ed.),
46(5), 393–429. https://doi.org/10.1037/h0059467 The standard edition of the complete psychological
Crusco, M. (2013). Draw-A-Person: Screening procedure works of Sigmund Freud (Vol. 3, pp. 159–188).
for emotional disturbance: An investigation of Hogarth. (Original work published 1896)
the sensitivity of this method to internalising and Freud, S. (1990). Totem and taboo: The standard edition.
externalising behavioural problems identified by the W. W. Norton. (Original work published 1913)
Rutter Parent Questionnaire at age 7 in the 1958
Gawronski, B., LeBel, E. P., & Peters, K. R. (2007).
National Child Development Study. University
What do implicit measures tell us?: Scrutinizing
of London.
the validity of three commonplace assumptions.
Cummings, J. A. (1986). Projective drawings. In H. M. Perspectives on Psychological Science, 2(2), 181–193.
Knoff (Ed.), The assessment of child and adolescent https://doi.org/10.1111/j.1745-6916.2007.00036.x
personality (pp. 199–244). Guilford Press.
Goldfried, M. R., & Kent, R. N. (1972). Traditional
Czopp, S. T., & Zeligman, R. (2016). The Rorschach versus behavioral personality assessment:
Comprehensive System (CS) psychometric validity A comparison of methodological and theoretical
of individual variables. Journal of Personality assumptions. Psychological Bulletin, 77(6),
Assessment, 98(4), 335–342. https://doi.org/ 409–420. https://doi.org/10.1037/h0032714
10.1080/00223891.2015.1131162
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L.
Dao, T. K., Prevatt, F., & Horne, H. L. (2008). (1998). Measuring individual differences in
Differentiating psychotic patients from non­ implicit cognition: The implicit association test.
Copyright American Psychological Association. Not for further distribution.

psychotic patients with the MMPI-2 and Journal of Personality and Social Psychology,
Rorschach. Journal of Personality Assessment, 74(6), 1464–1480. https://doi.org/10.1037/
90(1), 93–101. https://doi.org/10.1080/ 0022-3514.74.6.1464
00223890701693819
Greenwald, A. G., Poehlman, T. A., Uhlmann, E. L., &
Deri, S. (1949). Introduction to the Szondi test. Grune Banaji, M. R. (2009). Understanding and using
& Stratton. the Implicit Association Test: III. Meta-analysis
Entwisle, D. R. (1972). To dispel fantasies about of predictive validity. Journal of Personality and
fantasy-based measures of achievement motivation. Social Psychology, 97(1), 17–41. https://doi.org/
Psychological Bulletin, 77(6), 377–391. https:// 10.1037/a0015575
doi.org/10.1037/h0020021 Grønnerød, C. (2003). Temporal stability in the
Eron, L. D. (1950). A normative study of the Thematic Rorschach method: A meta-analytic review.
Apperception Test. Psychological Monographs: Journal of Personality Assessment, 80(3), 272–293.
General and Applied, 64(9), i–48. https://doi.org/ https://doi.org/10.1207/S15327752JPA8003_06
10.1037/h0093627 Grønnerød, C. (2006). Reanalysis of the Grønnerød
Exner, J. E., Jr. (1989). Searching for projection in (2003) Rorschach temporal stability meta-analysis
the Rorschach. Journal of Personality Assessment, data set. Journal of Personality Assessment,
53(3), 520–536. https://doi.org/10.1207/ 86(2), 222–225. https://doi.org/10.1207/
s15327752jpa5303_9 s15327752jpa8602_12
Exner, J. E., Jr. (1996). Critical bits and the Rorschach Groth-Marnat, G. (2009). Handbook of psychological
response process. Journal of Personality Assessment, assessment (5th ed.). Wiley.
67(3), 464–477. https://doi.org/10.1207/ Hibbard, S., Farmer, L., Wells, C., Difillipo, E., Barry, W.,
s15327752jpa6703_3 Korman, R., & Sloan, P. (1994). Validation of
Exner, J. E., Jr. (2003). The Rorschach: A comprehensive Cramer’s Defense Mechanism Manual for the TAT.
system: Vol. I. Basic foundations and principles of Journal of Personality Assessment, 63(2), 197–210.
interpretation (4th ed.). Wiley. https://doi.org/10.1207/s15327752jpa6302_1
Fowler, J. C., & Groat, M. (2008). Personality Hibbard, S., Mitchell, D., & Porcerelli, J. (2001).
assessment using implicit (projective) methods. Internal consistency of the object relations
In M. Hersen & A. M. Gross (Eds.), Handbook and social cognition scales for the Thematic
of clinical psychology: Vol. 1. Adults (pp. 475–494). Apperception Test. Journal of Personality
Wiley. Assessment, 77(3), 408–419. https://doi.org/
10.1207/S15327752JPA7703_03
Frank, L. K. (1939). Projective methods for the
study of personality. The Journal of Psychology, Holtzman, W. H., Thorpe, J. S., Swartz, J. D., &
8(2), 389–413. https://doi.org/10.1080/ Herron, E. W. (1961). Inkblot perception and
00223980.1939.9917671 personality. University of Texas Press.

446
 The Current Status of “Projective” “Tests”

Hosseininasab, A., Meyer, G. J., Viglione, D. J., Koppitz, E. M. (1968). Psychological evaluation of
Mihura, J. L., Berant, E., Resende, A. C., Reese, J., children’s human figure drawings. Grune & Stratton.
& Mohammadi, M. R. (2019). The effect of Machover, K. (1949). Personality projection in the
CS administration or an R-optimized alternative drawing of the human figure. Thomas.
on R–PAS variables: A meta-analysis of findings
from six studies. Journal of Personality Assessment, Matto, H. C., Naglieri, J. A., & Claussen, C. (2005).
101(2), 199–212. https://doi.org/10.1080/ Validity of the Draw-A-Person: Screening
00223891.2017.1393430 Procedure for Emotional Disturbance (DAP:SPED)
in strength-based assessment. Research on Social
Huprich, S. K., & Greenberg, R. P. (2003). Advances Work Practice, 15(1), 41–46. https://doi.org/
in the assessment of object relations in the 1990s. 10.1177/1049731504269553
Clinical Psychology Review, 23(5), 665–698.
https://doi.org/10.1016/S0272-7358(03)00072-2 McAdams, D. P. (1982). Experiences of intimacy and
power: Relationships between social motives and
Hutt, M. L. (1985). The Hutt adaptation of the autobiographical memory. Journal of Personality
Bender-Gestalt Test: Rapid screening and intensive and Social Psychology, 42(2), 292–302. https://
diagnosis (4th ed.). Grune & Stratton. doi.org/10.1037/0022-3514.42.2.292
Inslegers, R., Vanheule, S., Meganck, R., Debaere, V., McClelland, D. C. (1965). N achievement and
Trenson, E., Desmet, M., & Roelstraete, B. (2012). entrepreneurship: A longitudinal study. Journal
The assessment of the social cognition and of Personality and Social Psychology, 1(4),
object relations scale on TAT and interview data. 389–392. https://doi.org/10.1037/h0021956
Journal of Personality Assessment, 94(4), 372–379. McClelland, D. C. (1975). Power: The inner experience.
Copyright American Psychological Association. Not for further distribution.

https://doi.org/10.1080/00223891.2012.662187 Irvington.
Joiner, T. E., Jr., Schmidt, K. L., & Barnett, J. (1996). McClelland, D. C., Atkinson, J. W., Clark, R. A., &
Size, detail, and line heaviness in children’s Lowell, E. L. (1953). The achievement motive.
drawings as correlates of emotional distress: Appleton-Century-Crofts. https://doi.org/10.1037/
(More) negative evidence. Journal of Personality 11144-000
Assessment, 67(1), 127–141. https://doi.org/
10.1207/s15327752jpa6701_10 McGrath, R. E. (2005). Conceptual complexity
and construct validity. Journal of Personality
Khadivi, A., & Evans, F. B. (2012). The brave new world Assessment, 85(2), 112–124. https://doi.org/
of forensic Rorschach assessment: Comments on 10.1207/s15327752jpa8502_02
the Rorschach special section. Psychological Injury
McGrath, R. E. (2008). The Rorschach in the context
and Law, 5(2), 145–149. https://doi.org/10.1007/ of performance-based personality assessment.
s12207-012-9134-7 Journal of Personality Assessment, 90(5), 465–475.
Kivisalu, T. M., Lewey, J. H., Shaffer, T. W., & Canfield, https://doi.org/10.1080/00223890802248760
M. L. (2017). Correction to: An investigation of McGrath, R. E., & Carroll, E. J. (2012). The current
interrater reliability for the Rorschach Performance status of “projective” “tests.” In H. Cooper, P. M.
Assessment System (R-PAS) in a nonpatient U.S. Camic, D. L. Long, A. T. Panter, D. Rindskopf, &
sample. Journal of Personality Assessment, 99(5), K. J. Sher (Eds.), APA handbook of research methods
558–560. https://doi.org/10.1080/00223891. in psychology: Vol. 1. Foundations, planning, measures,
2017.1325244 and psychometrics (pp. 329–348). American
Kivisto, A. J., Gacono, C., & Medoff, D. (2013). Psychological Association. https://doi.org/10.1037/
Does the R-PAS meet standards for forensic use? 13619-018
Considerations with introducing a new Rorschach McNeish, T. J., & Naglieri, J. A. (1993). Identification
coding system. Journal of Forensic Psychology of individuals with serious emotional disturbance
Practice, 13(5), 389–410. https://doi.org/10.1080/ using the Draw-A Person: Screening Procedure
15228932.2013.838106 for Emotional Disturbance. The Journal of Special
Knoff, H. M., & Prout, H. T. (1985). The kinetic Education, 27(1), 115–121. https://doi.org/
drawing system: Family and school. Western 10.1177/002246699302700108
Psychological Services. Meyer, G. J. (2000). Incremental validity of the
Koocher, G. P., McMann, M. R., Stout, A. O., & Rorschach Prognostic Rating Scale over the MMPI
Norcross, J. C. (2015). Discredited assessment Ego Strength Scale and IQ. Journal of Personality
and treatment methods used with children and Assessment, 74(3), 356–370. https://doi.org/
10.1207/S15327752JPA7403_2
adolescents: A Delphi poll. Journal of Clinical
Child and Adolescent Psychology, 44(5), 722–729. Meyer, G. J., & Eblin, J. J. (2012). An overview of the
https://doi.org/10.1080/15374416.2014.895941 Rorschach Performance Assessment System (R-PAS).

447
 McGrath, Twibell, and Carroll

Psychological Injury and Law, 5(2), 107–121. Murray, H. A. (1938). Explorations in personality.
https://doi.org/10.1007/s12207-012-9130-y Oxford University Press.
Meyer, G. J., & Kurtz, J. E. (2006). Advancing Murray, H. A. (1943). Manual for the Thematic
personality assessment terminology: Time to Apperception Test. Harvard University Press.
retire “objective” and “projective” as personality
test descriptors. Journal of Personality Assessment, Naglieri, J. A., McNeish, T. J., & Bardos, A. N. (1991).
87(3), 223–225. https://doi.org/10.1207/ Draw A Person: Screening Procedure for Emotional
s15327752jpa8703_01 Disturbance: Examiner’s manual. Pro-Ed.
Meyer, G. J., & Mihura, J. L. (2020). Performance- Naglieri, J. A., & Pfeiffer, S. I. (1992). Performance
based techniques. In M. Sellbom & J. A. Suhr of disruptive behavior disordered and normal
(Eds.), The Cambridge handbook of clinical assess- samples on the Draw-A-Person: Screening Pro-
ment and diagnosis (pp. 278–290). Cambridge cedure for Emotional Disturbance. Psychological
University Press. Assessment, 4(2), 156–159. https://doi.org/10.1037/
1040-3590.4.2.156
Meyer, G. J., Riethmiller, R. J., Brooks, R. D., Benoit,
W. A., & Handler, L. (2000). A replication of Nakamura, N. (2012). Rorschach-based psychotherapy:
Rorschach and MMPI-2 convergent validity. Collaboration with a suicidal young woman.
Journal of Personality Assessment, 74(2), 175–215. In S. E. Finn, C. T. Fischer, & L. Handler (Eds.),
https://doi.org/10.1207/S15327752JPA7402_3 Collaborative/therapeutic assessment: A casebook
Meyer, G. J., Viglione, D. J., & Mihura, J. L. (2017). and guide (pp. 269–290). Wiley.
Psychometric foundations of the Rorschach Nosek, B. A., Greenwald, A. G., & Banaji, M. R.
Performance Assessment System (R-PAS).
Copyright American Psychological Association. Not for further distribution.

(2007). The Implicit Association Test at age 7:

In R. Erard & B. Evans (Eds.), The Rorschach A methodological and conceptual review. In J. A.
in multimethod forensic practice (pp. 23–91). Bargh (Ed.), Automatic processes in social thinking
Routledge. and behavior (pp. 265–292). Psychology Press.
Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., Pad, R. A., Huprich, S. K., & Porcerelli, J. (2020).
& Erdberg, P. (2011). Rorschach Performance Convergent and discriminant validity of self-report
Assessment System: Administration, coding, inter­ and performance-based assessment of object
pretation, and technical manual. Rorschach relations. Journal of Personality Assessment,
Performance Assessment System. 102(6), 858–865. https://doi.org/10.1080/
Meyer, G. J., Viglione, D. J., Mihura, J. L., & 00223891.2019.1625909
Giromini, L. (2020, October 8). 2018 update to
Pianowski, G., Meyer, G. J., de Villemor-Amaral, A. E.,
R-PAS transitional child and adolescent norms.
Zuanazzi, A. C., & do Nascimento, R. S. G. F.
R-PAS. https://r-pas.org/UpdateYNorms.aspx
(2021). Does the Rorschach Performance
Mihura, J. L., Meyer, G. J., Dumitrascu, N., & Assessment System (R-PAS) differ from the
Bombel, G. (2013). The validity of individual Comprehensive System (CS) on variables
Rorschach variables: Systematic reviews and meta- relevant to interpretation? Journal of Personality
analyses of the comprehensive system. Psychological Assessment, 103(1), 132–147. https://doi.org/
Bulletin, 139(3), 548–605. https://doi.org/10.1037/ 10.1080/00223891.2019.1677678
a0029406
Pihl, R., & Nimrod, G. (1976). The reliability and
Mihura, J. L., Roy, M., & Graceffo, R. A. (2017). validity of the Draw-A-Person Test in IQ and
Psychological assessment training in clinical personality assessment. Journal of Clinical
psychology doctoral programs. Journal of Personality Psychology, 32(2), 470–472. https://doi.org/
Assessment, 99(2), 153–164. https://doi.org/ 10.1002/1097-4679(197604)32:2<470::AID-
10.1080/00223891.2016.1201978
JCLP2270320257>3.0.CO;2-I
Mischel, W. (1968). Personality and assessment. Wiley.
Piotrowski, C. (2015a). Clinical instruction on
More, D. M., & Winch, R. F. (1956). Does TAT add projective techniques in the USA: A review of
information to interviews? Statistical analysis academic training settings 1995–2014. SIS Journal
of the increment. Journal of Clinical Psychology, of Projective Psychology & Mental Health, 22(2),
12(4), 316–321. https://doi.org/10.1002/1097- 83–92.
4679(195610)12:4<316::AID-JCLP2270120403>
3.0.CO;2-P Piotrowski, C. (2015b). Projective techniques usage
worldwide: A review of applied settings 1995–2015.
Morgan, C., & Murray, H. A. (1935). A method for Journal of the Indian Academy of Applied Psychology,
investigating fantasies: The Thematic Apperception 41(3), 9–19.
Test. Archives of Neurology and Psychiatry, 34(2),
289–306. https://doi.org/10.1001/archneurpsyc. Piotrowski, C., & Keller, J. W. (1984). Psychodiagnostic
1935.02250200049005 testing in APA-approved clinical psychology

448
 The Current Status of “Projective” “Tests”

programs. Professional Psychology, Research and retest reliability, and ipsative stability. Journal
Practice, 15(3), 450–456. https://doi.org/10.1037/ of Research in Personality, 42(6), 1560–1571.
0735-7028.15.3.450 https://doi.org/10.1016/j.jrp.2008.07.008
Piotrowski, C., & Zalewski, C. (1993). Training in Shrout, P. E. (1998). Measurement reliability and
psychodiagnostic testing in APA-approved PsyD agreement in psychiatry. Statistical Methods in
and PhD clinical psychology programs. Journal Medical Research, 7(3), 301–317. https://doi.org/
of Personality Assessment, 61(2), 394–405. 10.1177/096228029800700306
https://doi.org/10.1207/s15327752jpa6102_17 Spangler, W. (1992). Validity of questionnaire and
Porcerelli, J. H., Cogan, R., Kamoo, R., & Miller, K. TAT measures of need for achievement:
(2010). Convergent validity of the Defense Two meta-analyses. Psychological Bulletin,
Mechanisms Manual and the Defensive Functioning 112(1), 140–154. https://doi.org/10.1037/
Scale. Journal of Personality Assessment, 92(5), 0033-2909.112.1.140
432–438. https://doi.org/10.1080/00223891. Stedman, J. M., McGeary, C. A., & Essery, J. (2018).
2010.497421 Current patterns of training in personality
Porcerelli, J. H., Thomas, S., Hibbard, S., & Cogan, R. assessment during internship. Journal of Clinical
(1998). Defense mechanisms development Psychology, 74(3), 398–406. https://doi.org/
in children, adolescents, and late adolescents. 10.1002/jclp.22496
Journal of Personality Assessment, 71(3), 411–420. Stein, M., Hilsenroth, M., Slavin-Mulford, J., &
https://doi.org/10.1207/s15327752jpa7103_9 Pinsker, J. (2011). Social Cognition and Object
Rehrig, G., & Stromswold, K. (2018). What does Relations Scale: Global Rating Method (SCORS-G)
Copyright American Psychological Association. Not for further distribution.

the DAP:IQ measure?: Drawing comparisons (4th ed.) [Unpublished manuscript]. Massachusetts
between drawing performance and developmental General Hospital and Harvard Medical School.
assessments. The Journal of Genetic Psychology, Stein, M. B., & Slavin-Mulford, J. (2018). Social
179(1), 9–18. https://doi.org/10.1080/00221325. Cognition and Object Relations Scale: A comprehensive
2017.1392281 guide for clinicians and researchers. Routledge.
Riethmiller, R. J., & Handler, L. (1997). Problematic Stein, M. B., Slavin-Mulford, J., Sinclair, S. J., Chung,
methods and unwarranted conclusions in DAP W.-J., Roche, M., Denckla, C., & Blais, M. A.
research: Suggestions for improved research (2018). Extending the use of the SCORS–G
procedures. Journal of Personality Assessment, composite ratings in assessing level of personality
69(3), 459–475. https://doi.org/10.1207/ organization. Journal of Personality Assessment,
s15327752jpa6903_1 100(2), 166–175. https://doi.org/10.1080/
00223891.2016.1195394
Ronan, G. F., Gibbs, M. S., Dreer, L. E., & Lombardo,
J. A. (2008). Personal Problem-Solving System- Sultan, S., & Meyer, G. J. (2009). Does productivity
Revised. In S. R. Jenkins (Ed.), A handbook of impact the stability of rorschach scores? Journal
clinical scoring systems for thematic apperceptive of Personality Assessment, 91(5), 480–493.
techniques (pp. 181–207). Erlbaum. https://doi.org/10.1080/00223890903088693
Rorschach, H. (1942). Psychodiagnostics: A diagnostic Sundberg, N. (1961). The practice of psychological
test based on perception. Hans Huber. (Original work testing in clinical services in the United States.
published 1921) American Psychologist, 16(2), 79–83. https://doi.org/
10.1037/h0040647
Rosenzweig, S. (1978). Rosenzweig Picture-Frustration
Study (P-F) (rev. ed.). Psychological Assessment Swensen, C. H. (1968). Empirical evaluations of human
Resources. figure drawings: 1957–1966. Psychological Bulletin,
70(1), 20–44. https://doi.org/10.1037/h0026011
Rotter, J. B., Lah, M. I., & Rafferty, J. E. (1992).
Manual: The Rotter Incomplete Sentences Blank: Tharinger, D. J., & Stark, K. (1990). A qualitative
College form. Psychological Corporation. versus quantitative approach to evaluating the
Draw-A-Person and Kinetic Family Drawings:
Schneider, A. M. de A., Bandeira, D. R., & Meyer, A study of mood- and anxiety-disordered
G. J. (2022). Rorschach Performance Assessment children. Psychological Assessment, 2(4), 365–375.
System (R-PAS) interrater reliability in a Brazilian https://doi.org/10.1037/1040-3590.2.4.365
adolescent sample and comparisons with three
Troncone, A., Chianese, A., Di Leva, A., Grasso, M.,
other studies. Assessment, 29(5), 859–871.
& Cascella, C. (2021). Validity of the Draw A
https://doi.org/10.1177/1073191120973075
Person: A quantitative scoring system (DAP:QSS)
Schultheiss, O., Liening, S., & Schad, D. (2008). The for clinically evaluating intelligence. Child Psy-
reliability of a Picture Story Exercise measure of chiatry and Human Development, 52(4), 728–738.
implicit motives: Estimates of internal consistency, https://doi.org/10.1007/s10578-020-01058-6

449
 McGrath, Twibell, and Carroll

Viglione, D. J., & Giromini, L. (2016). The effects of Westen, D. (1995). Social cognition and object relations
using the International versus Comprehensive scale: Q-sort for projective stories (SCORS Q)
System Rorschach norms for children, adolescents, [Unpublished manuscript]. Department of
and adults. Journal of Personality Assessment, 98(4), Psychiatry, The Cambridge Hospital and Harvard
391–397. https://doi.org/10.1080/00223891. Medical School.
2015.1136313 Wrightson, L., & Saklofske, D. (2000). Validity and
Wakefield, J. (1986). Creativity and the TAT blank reliability of the Draw A Person: Screening
card. The Journal of Creative Behavior, 20(2), Procedure for Emotional Disturbance with
127–133. https://doi.org/10.1002/j.2162- adolescent students. Canadian Journal of School
6057.1986.tb00427.x Psychology, 16(1), 95–102. https://doi.org/
10.1177/082957350001600107
Weiner, I. B. (1994). The Rorschach Inkblot Method
(RIM) is not a test: Implications for theory Ziegler, M., Schmukle, S., Egloff, B., & Bühner, M.
and practice. Journal of Personality Assessment, (2010). Investigating measures of achievement
62(3), 498–504. https://doi.org/10.1207/ motivation(s). Journal of Individual Differences, 31(1),
s15327752jpa6203_9 15–21. https://doi.org/10.1027/1614-0001/a000002
Copyright American Psychological Association. Not for further distribution.

450