Journal of Research in Personality 45 (2011) 631–641

Contents lists available at SciVerse ScienceDirect

Journal of Research in Personality

journal homepage: www.elsevier.com/locate/jrp

Assessing interpersonal functioning: Views from within and without

Daniel Leising a,⇑, Sabrina Krause a, Doreen Köhler a, Kai Hinsen a, Allan Clifton b
a
Department of Psychology, University of Halle-Wittenberg, Germany
b
Department of Psychology, Vassar College, NY, USA

a r t i c l e i n f o a b s t r a c t

Article history: In order to better integrate research on personality pathology, interpersonal problems, and social skills,
Available online 7 September 2011 we applied the traditional methods of these three research strands (questionnaires, interviews, and inter-
personal role-plays) to the same sample. Participants who attributed higher levels of interpersonal prob-
Keywords: lems to themselves in general were also more critical of their own role-play performances, but these
Social skills impressions were not mirrored by observer-ratings. Self-observer agreement in judging overall role-play
Observation performance was essentially zero. Interviewer-ratings of personality pathology had incremental validity
Assessment
over self-ratings in predicting observer-rated role-play performance. Self-reports of interpersonal func-
Interpersonal problems
Personality disorders
tioning leave relevant behavioral variance untapped and thus should be complemented by other sources
of information.
Ó 2011 Elsevier Inc. All rights reserved.

1. Introduction sonal functioning. Second, there are researchers who focus on

‘‘personality disorders’’ (cf. APA, 2000). These researchers rarely
One of the most important ways in which people differ from use terms like ‘‘social skills’’, although interpersonal impairment
each other is their ability ‘‘to act wisely in human relations’’ (that is, a lack of social skills) is certainly a core element of most
(Thorndike, 1920, p. 228). Although several research communities personality disorders (Widiger & Frances, 1985). A related research
investigate this domain of psychological functioning, the connec- tradition focuses on ‘‘interpersonal problems’’ (e.g., Conroy, Elliot,
tions between them have remained loose. In the present paper & Pincus, 2009; Horowitz, 1979).
we attempt to integrate some of these strands of research with Research on personality disorders and interpersonal problems
each other, by applying their preferred assessment methods to only rarely utilizes behavior observation techniques, instead rely-
the same sample of research participants. By comparing the pre- ing primarily on retrospective self-report measures (Bornstein,
ferred assessment methods of the different research traditions, 2003; Oltmanns & Turkheimer, 2006). The present study is in-
we also hope to shed some light on their respective advantages tended to demonstrate that all of the above-named research tradi-
and limitations. tions investigate similar phenomena: They all address people’s
Two major strands of research have addressed individual differ- dispositions to behave in certain ways when interacting with oth-
ences in the domain of interpersonal functioning in particular. ers, including evaluations of those behaviors as being more or less
First, there are researchers who focus on ‘‘social skills’’ and ‘‘social competent.
competence.’’ These researchers primarily study children and ado- The predominant use of self-report measures in research on
lescents (e.g., Rantanen et al., 2009; Rhoades, Greenberg, & Domi- personality disorders and interpersonal problems presupposes that
trovich, 2009; Stacks & Oshio, 2009), or adults with severe individuals should be accurate reporters of their own interpersonal
psychiatric disorders such as schizophrenia (e.g., Sayers & Bellack, functioning. There are, however, a number of problems associated
1995). Interestingly, this research tends to utilize social compe- with this assumption. First, most self-assessments are retrospec-
tence ratings by people other than the target persons (e.g., the chil- tive, and therefore depend on the accuracy of people’s memories.
dren’s parents or teachers, or the patients’ therapists or nurses), as Second, self-assessments necessarily rely on a single source of
well as observer-ratings of people’s behavior in interpersonal role- information (Klonsky, Oltmanns, & Turkheimer, 2002), which usu-
plays. The use of such techniques seems to be based on the (tacit) ally leads to rather modest reliability. Third, self-assessments may
assumption that members of these specific populations may be un- be subject to response biases, both formal (e.g., a tendency to en-
able to provide accurate self-assessments of their own interper- dorse extreme statements) and content-related (e.g., socially desir-
able responding). These potential biases are particularly
problematic because (unlike peer-ratings) self-ratings may not be
⇑ Corresponding author. Address: Department of Psychology, University of Halle-
Wittenberg, 06099 Halle (Saale), Germany.
averaged across sources of information in order to cancel out idio-
E-mail address: [email protected] (D. Leising). syncratic response styles (cf. Hofstee, 1994). In addition to these

0092-6566/$ - see front matter Ó 2011 Elsevier Inc. All rights reserved.
doi:10.1016/j.jrp.2011.08.011
632 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641

general problems, which are associated with all kinds of self-report (2006) found that avoidant and dependent participants (as as-
measures, there is reason to believe that self-reports of interper- sessed by clinical interviews) had difficulties being assertive in
sonal functioning may have some more specific problems associ- brief dyadic role-plays. Second, Leising, Rehbein, and Sporberg
ated with them: For example, judgments of interpersonal (2007) found that the predictive validity of self-assessed submis-
competence have to be evaluative, because judging a person as siveness (using the Inventory of Interpersonal Problems; Horowitz,
‘‘competent’’ requires comparing a person’s actual behavior with Strauß, & Kordy, 2000) with regard to behavior in these role-plays
some imagined behavior that would be ideal, or at least sufficient. was rather limited (r < .30). The present study builds directly on
Because of this evaluativeness, judgments of competence are likely these previous ones. However, in the present study, we broaden
to be affected by people’s (overly) positive or negative views of the scope considerably, by assessing more of the DSM-IV personal-
themselves, or by people’s attempts to present themselves in a po- ity disorders, and using a much broader range of interpersonal
sitive or negative light (John & Robins, 1993; Leising, Erbs, & Fritz, interaction situations.
2010; Paulhus, 1984; Vazire, 2010). Moreover, self-reports of inter- Assessing different perspectives on the interpersonal function-
personal functioning may have reduced validity due to the phe- ing of the same persons also enables a test of incremental validity.
nomenon of ‘‘ego-syntonicity’’ (i.e., people may interpret their Studies in basic personality psychology have shown that other-rat-
own interpersonal problems as adaptive), and some interpersonal ings of personality may make independent contributions in pre-
problems may even be defined in terms of a lack of awareness dicting various outcome variables, beyond the predictions that
regarding one’s own interpersonal behavior (e.g., a man who keeps are possible based on self-reports alone (Connelly & Ones, 2010).
driving people away by being boastful and condescending, but is We are only aware of one such study that explicitly addressed per-
not aware of this, because nobody cares enough to honestly tell sonality pathology: Oltmanns and Turkheimer (2006) showed that
him how he comes across). For these reasons, comparing the typi- peer-ratings of (antisocial) personality pathology had incremental
cal retrospective self-reports of interpersonal functioning to other validity over self-ratings in predicting early discharge from the
sources of information seems desirable (cf. Clifton, Turkheimer, & military. In the present study, we investigated whether inter-
Oltmanns, 2004). viewer-ratings of personality pathology have incremental validity
Studies that have actually compared different perspectives have over self-ratings in predicting people’s performances in interper-
yielded mixed results: On the one hand, self-other agreement in sonal role-plays. This question is relevant because interviewers
judging personality pathology and interpersonal functioning is have to rely heavily on the information that interviewees are will-
generally modest to weak (e.g., Clifton, Turkheimer, & Oltmanns, ing to share with them. As a consequence, interviewer-ratings may
2005; Klonsky et al., 2002; Oltmanns & Turkheimer, 2006). Accord- be highly redundant with what the interviewed persons would
ing to Clifton et al. (2004), some of these divergences may be due to also report in self-report questionnaires, and thus ultimately
systematic discrepancies in how the targets and other people de- expendable. To the best of our knowledge, this issue has not been
scribe the same personality traits of the targets. In the present empirically investigated before.
study, we therefore expected to find modest to weak self-other A wealth of observational measures of social skills and social
agreement for judgments of interpersonal functioning. On the competence is already in existence (e.g., Curran, 1982; Donahoe,
other hand, studies have shown that pathological personality fea- Carter, Bloem, & Leff, 1990; Eisler, Hersen, Miller, & Blanchard,
tures may be recognized by strangers who watch the respective 1975; Goldsmith & McFall, 1975; McFall & Marston, 1970; Patter-
persons for very limited amounts of time only. For example, Fried- son, Moscona, McKibbin, Davidson, & Jeste, 2001; Sayers, Bellack,
man, Oltmanns, and Turkheimer (2007) found that various person- Wade, Bennett, & Fong, 1995; Tsang & Pearson, 2000). Without
ality disorder characteristics (which had been assessed by peer- exception, these measures were developed within the first re-
report and self-report) could be accurately assessed by strangers search tradition that we referred to above. When we inspected
who watched the targets describe themselves for 30 s on video. the various measures and compared them with one another, it be-
Likewise, Oltmanns, Friedman, Fiedler, and Turkheimer (2004) came obvious that none of them were suited for our purposes:
found that research participants with elevated levels of various Most measures only comprise a relatively small number (e.g.,
forms of personality pathology were accurately judged (mainly in two, six, eight) of interpersonal situations, which we considered
terms of higher or lower extraversion) by strangers who watched insufficient for covering the domain of interpersonal (dys-)function
brief (30 s) excerpts from diagnostic interviews. In a study by Fow- comprehensively (e.g., the various interpersonal deficits described
ler, Lilienfeld, and Patrick (2009), elevated psychopathy levels in in the DSM-IV personality disorder criteria may not be assessed by
prison inmates could be judged with some accuracy based on even means of such a small range of situations). On the other hand, mea-
shorter excerpts from diagnostic interviews. Finally, Leising and sures that comprise a larger number of situations tend to focus on
Müller-Plath (2009) found that participants with avoidant person- only one kind of interpersonally competent behavior. For example,
ality features (as assessed by clinical interviews) who gave brief the Behavioral Role Playing Test (McFall & Marston, 1970) com-
self-presentations in front of a camera were judged as being more prises sixteen, and the Behavioral Assertiveness Test (Eisler et al.,
insecure by unacquainted judges. 1975) comprises 32 different interpersonal situations, but all of
A shortcoming of all of these latter studies is that the assess- them are supposed to assess a person’s level of assertiveness. The
ment of the targets’ self-images and the assessment of their actual one exception in which a large number (25) of relatively diverse
interpersonal behavior were confounded. This is because in all of interpersonal situations is employed, the Interpersonal Behavior
these studies the video material for the ‘‘thin slice’’ ratings de- Role-Playing Test (Goldsmith & McFall, 1975) presents the situa-
picted targets who were talking about themselves. The strangers’ tions on audio-tape and only a single response from each partici-
ratings were therefore based on a combination of what the targets pant is recorded, with no real interaction between people taking
said about themselves and how they said it. In the present study, place. As we intended to observe participants’ behavior in actual
we tried to disentangle these two kinds of information by having interactions, with several behavioral interchanges taking place,
unacquainted raters judge the actual behavior of research partici- while at the same time standardizing the responses of the partici-
pants in interpersonal role-plays that did not require them to re- pants’ interaction partners as much as possible, we decided to de-
port their views of their own personalities. We only know of two sign our own set of interpersonal role-plays. This set of role-plays
previous studies that investigated the predictive validity of person- should enable direct observations of those interpersonal compe-
ality pathology assessments with regard to such ratings of overt tencies that feature prominently in the social skills and personality
interpersonal behavior: First, Leising, Sporberg, and Rehbein disorders literature.
 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641 633

2. Method to the individual scale scores, we also computed the IIP-64 total
score (a = .91), by averaging the scale scores.
2.1. Sample
2.2.3. SCID-II
The sample comprised 100 white, German research volunteers The Structured Clinical Interview for DSM IV Personality Disorders
(67 female) who were recruited by public advertisement on campus (SCID-II; First, Spitzer, Gibbon, & Williams, 1996; German version:
and on the internet. Approximately half of them were university Fydrich, Renneberg, Schmitz, & Wittchen, 1997) is a widely used
students. The mean age of the sample was 23.0 years (SD = 3.8, measure of personality pathology based on DSM-IV (APA, 2000) cri-
Range = 18–36). Participants were paid 15 Euro for their participa- teria. The procedure comprises a screening questionnaire in which
tion. The average level of education in the sample was high: participants are to report on various problematic personality fea-
Ninety-five participants had ‘‘Abitur’’ (i.e., the highest secondary tures, and an interview in which they are asked to talk in more depth
education degree in Germany. Abitur is only attained by about 40% about those questionnaire items that they endorsed. Based on the
of a birth cohort and is a prerequisite for attending university). We interview, the interviewer assigns scores between 1 and 3 for each
denied participation to persons who were currently in psychiatric PD criterion, where 1 = ‘not present’, 2 = ‘partly present’ and 3 =
treatment or using psychotropic medication, in order to keep vulner- ‘definitely present’. We computed dimensional scores for each per-
able persons from being exposed to the potentially stressful inter- sonality disorder by summing up the individual criteria. The internal
personal role-plays at this exploratory stage of the research process. consistencies of the SCID-II questionnaire and interviewer-rating
scales were: avoidant: a = .44 (questionnaire), .62 (interviewer);
dependent: a = .45, .60; obsessive–compulsive: a = .36, .51; para-
2.2. Measures
noid: a = .48, .48; schizoid: a = .55, .74; histrionic: a = .63, .44; narcis-
sistic: a = .57, .43; borderline: a = .71, .68; childhood conduct
2.2.1. Standard interaction tasks
disorder: a = .74, .77. These relatively low alphas probably reflect
Our first challenge in the present study was to create a rela-
the often criticized heterogeneity of the DSM-IV PD concepts (cf.
tively large set of role-plays for assessing interpersonal competen-
Bornstein, 1998). Antisocial PD features were assessed only by inter-
cies. We will call these role-plays ‘‘standard interaction tasks’’
view (a = .77), and schizotypal PD was not assessed. Twenty-six
(SITs) in the following. The SITs we developed are very similar to
interviews were recorded on video and judged independently by a
the role-plays that are routinely used in social skills training. They
second rater in order to enable a test of inter-rater reliability. The
always begin with a simple instruction of a few sentences, describ-
agreement (ICC(1, 1)) between the interviewer and the second rater
ing a hypothetical interpersonal situation and a behavior that the
was .93 for avoidant, .80 for dependent, .81 for obsessive–compul-
participant is expected to show. For example, one task (Task 2,
sive, .77 for paranoid, .99 for schizoid, .76 for histrionic, .60 for narcis-
see Appendix A) requires participants to role-play apologizing to
sistic, .85 for borderline, and .92 for antisocial personality features.
someone for embarrassing him or her in front of others. Complet-
Categorical diagnoses were also derived, by adding up the num-
ing an SIT usually takes no longer than 90 s. Standard interaction
bers of ‘definitely present’ criteria for each PD, and then comparing
tasks are tests of maximum performance, rather than typical per-
them to the respective critical thresholds (e.g., five out of nine). Gi-
formance. That is, they assess whether a person is capable of dis-
ven the widespread criticism of the categorical approach to diag-
playing a particular interpersonal behavior when necessary, not
nosing PDs (e.g., Widiger & Samuel, 2005) these diagnoses will
whether the person displays that behavior on a regular basis.
only briefly be reported here. Six persons in the sample received
Most of the sets of interpersonal role-plays that are used in so-
categorical personality disorder diagnoses. Three of these were
cial skills research were created ad hoc, with no particular refer-
obsessive–compulsive, two were schizoid, and one was borderline.
ence to any specific theory (McFall, 1982). In creating our own set
The number of PD cases in our sample was significantly lower
of interaction tasks, we drew from several different sources: For
(p = .015, Binomial Test) than the number one would expect based
example, we studied the DSM-IV personality disorder criteria and
on Torgersen, Kringlen, and Cramer’s (2001) estimate of PD preva-
the social skills literature (e.g., Buhrmester, Furman, Wittenberg,
lence (13.4%) in a community sample. Overall, the numbers speak
& Reis, 1988; Riggio, 1986), as well as German manuals for Social
to a relatively low level of personality pathology in the sample,
Skills Training (e.g., Hinsch & Pfingsten, 2002; Ullrich & DeMuynck,
which was probably an effect of the exclusion criteria that we ap-
2006) to incorporate the most common conceptions of which inter-
plied in recruiting participants (see above).
personal competencies are important. After pilot-testing many dif-
ferent tasks, we ultimately arrived at the final set of 17 tasks that
2.3. Procedures
were used in the present study (see Appendix A). For each task,
we also developed a set of four standard answers, which the partic-
All participants completed three kinds of assessment in the fol-
ipants’ interaction partners were supposed to use in order to stan-
lowing order: First, they filled out the Inventory of Interpersonal
dardize the situational circumstances as much as possible across
Problems and the SCID-II questionnaire. Second, they took part in
participants (see Appendix A). Some of these standard answers
individual SCID-II interviews, based on their responses to the
were also supposed to make the tasks more challenging.
SCID-II questionnaire. Third, they took part in the standard interac-
tion tasks. A team of three research assistants (two female, one
2.2.2. IIP-64 male) helped with the data collection for the study. For each partic-
The Inventory of Interpersonal Problems (IIP-64; Horowitz, ipant, one research assistant took on the role of clinical inter-
Alden, Wiggins, & Pincus, 2000; German version: Horowitz, Strauß viewer, and another took on the role of interaction partner in the
et al., 2000) is one of the most widely used self-report measures of role-plays. The research assistants were advanced psychology stu-
difficulties in dealing with other people. Its eight scales assess differ- dents who had been enrolled in a full-time study of the subject for
ent blends of dominance and affiliation, which are the two basic at least 3.5 years. They received intensive interviewer training by
dimensions of the Interpersonal Circumplex Model (Leary, 1957; the first author. The research assistants were not previously ac-
Wiggins, 1979). In the present study, the scales had the following quainted with the participants, and did not communicate with
internal consistencies: domineering: a = .73, vindictive: a = .77, one another about their impressions of the participants.
cold: a = .76, socially avoidant: a = .79, submissive: a = .79, exploit- In the role-plays, a given participant always interacted with the
able: a = .73, overly nurturant: a = .68, intrusive: a = .65. In addition same interaction partner. Forty-nine participants interacted with a
634 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641

male interaction partner, and 51 participants interacted with a fe- bottom of the table reflect two different kinds of averages. For com-
male interaction partner. Participant sex and partner sex were coun- puting the first (Avg1), the judgments by the individual raters (self,
terbalanced to ensure that they were uncorrelated, r(98) = .01, observers) were first averaged across tasks (for each target) and then
p = .943. The 17 interaction tasks were presented to each participant the respective index was derived (across targets). For computing the
in random order. At the beginning of each task, the research assistant second (Avg2), the respective index was first computed separately
handed the participant a sheet of paper containing the written for each task (across targets), and then averaged across tasks. Most
instruction for the task and an item for rating his or her own perfor- participants performed quite successfully in the tasks, as indicated
mance afterwards (see below). Then the research assistant started by average observer-ratings of 5.8 and higher on the eight-point
the video camera, and the interaction began. If a participant declared scale. The participants’ self-rated performances were lower than
that he or she would rather skip or postpone one of the tasks, this the averaged observer-ratings for 15 tasks, and equal to the
was allowed and noted. During the interaction, the interaction part- averaged observer-ratings for the remaining two tasks, indicating
ners only used the standard replies (see Appendix A), to the extent that the participants were generally more skeptical of their own
possible. After the interaction, the camera was stopped, and the performances.
research assistant asked the participant to judge his or her own
performance immediately. The participants rated their own perfor- 3.2. Inter-rater agreement
mance in each task by means of an item asking ‘‘How well do you
think you performed in this task?’’ They were to check one of eight The four data columns on the right of Table 1 display different
circles, the outer two of which were marked ‘‘Bad’’ (on the left) kinds of inter-rater agreement. Several things are noteworthy here:
and ‘‘Good’’ (on the right). First, the average agreement between the three observers in judg-
The participants’ videotaped performances were judged inde- ing the participants’ performances in a single task (Avg2) was
pendently by three female observers, who also were advanced psy- acceptable (.75), and comparable to levels of inter-observer agree-
chology students. Like the participants, they used an eight-point ment that have been reported in similar studies (e.g., Robins &
scale and had the instructions for the interaction tasks in front of John, 1997). The agreement between the observers in judging the
them when making their ratings. If the observers declared that they participants’ overall performances (that is, across tasks), was even
had seen a target before, the respective ratings were excluded from better (Avg1 = .84). Second, the average self-observer agreement in
analyses. By using the observers we aimed to obtain more objective judging the participants’ performances in individual tasks
ratings of the targets’ actual performances in the interaction tasks, (Avg2 = .17) was much lower than the respective agreement be-
because (a) unacquainted observers will have no motive to judge tween individual observers (Avg2 = .51). Third, the average level
particular targets too leniently or critically, and (b) by averaging of agreement between the participants’ self-ratings and the aver-
across observers, idiosyncratic judgment styles may be expected aged observer-ratings for individual tasks (Avg2 = .20) was larger
to average out to some extent. than zero, but this agreement vanished almost completely when
Research has shown that, in studying interpersonal functioning, we aggregated across tasks before correlating the self- with the ob-
it is useful to distinguish between two different kinds of analysis server-ratings (Avg1 = .02). Possible explanations for these findings
(cf. Gresham, Elliott, Cook, Vance, & Kettler, 2010; McFall, 1982; are presented in the discussion section.
Rose-Krasnor, 1997). First, one may assess whether a person is
‘‘competent’’ in dealing with certain interpersonal tasks. ‘‘Compe- 3.3. Associations of task performance ratings with personality
tence’’ means that a person is able to perform properly, to meet measures
the challenges and solve the problems that are posed by the situa-
tion. Second, the specific behaviors underlying the judgment of We will now report associations between the participants’ per-
competence or incompetence may be analyzed (e.g., how much formances in the interaction tasks and the other measures that
the person smiled or talked). In the present study, we only focused were used in the present study (i.e., questionnaires and inter-
on judgments of competence, and did not investigate the concrete viewer-ratings). In doing so, we will not discuss each individual
behaviors leading to those judgments. coefficient, but rather interpret the overall pattern of findings,
and only discuss a few examples where appropriate. Observer-rat-
3. Results ings were averaged across the three observers.

In the interest of space, the interaction tasks are indexed by key- 3.3.1. Associations with the IIP-64
words in the tables. We do not report associations between IIP-64 When the self-rated performances of the participants in the 17
scores and personality pathology, because this issue has already interaction tasks were predicted from the eight scales of the IIP-
been investigated in depth by others: Clifton et al. (2005) reported 64 (136 correlations), a relatively large number (N = 45) of signifi-
considerable overlap between these measurement domains, as long cant (p < .05) negative correlations (range: r = .20 to r = .35)
as judgments of both domains came from the same perceivers. emerged. The correlation matrix is displayed in Table 2. All eight
IIP scales predicted the average self-ratings of performance across
3.1. Mean performance ratings tasks (bottom row), and self-rated performance in the majority
(N = 9) of tasks was predicted by the IIP total scores (right column).
The five data columns on the left of Table 1 display the average The correlation between the IIP-64 total scores and the average
performance judgments by the participants and the observers, for self-ratings of task performance was r(98) = .50 (p < .001), indi-
the individual tasks and across tasks1. The two data rows at the cating that participants who attributed high levels of interpersonal
problems to themselves in general were also more skeptical of
1
There were a few systematic effects of participant sex on the performance ratings: their own performances across the interaction tasks.
The observers judged female participants (M = 6.07, SD = 1.17) as being more However, when the averaged observer-ratings of task perfor-
successful than male participants (M = 5.35, SD = 1.33) at showing empathy, mance were predicted from the IIP-64 scales, the number of signif-
t(96) = 2.74, p = .007. They also judged female participants (M = 6.53, SD = 1.03) as icant correlations dropped dramatically, to 8 (range: r = .36 to
being more successful than male participants (M = 5.91, SD = 1.40) at expressing
affection, t(97) = 2.48, p = .015. Both of these effects, with women appearing to be
r = .33). The correlation matrix is displayed in Table 3. The correla-
more emotionally responsive and expressive than men, are consistent with previous tion between the IIP-64 total scores and the average observer-rat-
research findings (e.g., Leising, Müller, & Hahn, 2007; Riggio, 1986). ings of performance across tasks was r(97) = .04 (p = .664),
 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641 635

Table 1
Means, standard deviations and inter-rater agreement for judgments of task performance.

Task Keyword Mean (SD) Inter-Rater Agreement

Self Rater A Rater B Rater C Avg. rater S–O S–Ō O–O Ō Rel
1 Initiate 5.7 (1.8) 6.2 (1.5) 5.6 (1.7) 6.5 (1.6) 6.1 (1.4) .39 .42 .60 .83
2 Apologize 5.8 (1.5) 6.7 (1.5) 5.7 (1.6) 6.7 (1.3) 6.4 (1.3) .13 .14 .59 .81
3 Thank 6.5 (1.1) 6.6 (1.2) 6.0 (1.4) 7.0 (1.0) 6.5 (1.0) .15 .21 .47 .73
4 Assert 5.8 (1.4) 6.7 (1.2) 6.3 (1.2) 6.8 (1.1) 6.6 (0.8) .17 .19 .35 .65
5 Confront 6.1 (1.3) 6.7 (1.2) 6.2 (1.4) 6.9 (1.1) 6.6 (1.0) .06 .06 .40 .68
6 Positively 3.7 (1.8) 5.3 (1.7) 5.8 (1.9) 6.3 (1.6) 5.8 (1.5) .44 .49 .62 .83
7 Self-critical 5.2 (1.6) 5.8 (1.7) 5.9 (1.8) 6.8 (1.4) 6.1 (1.5) .08 .10 .67 .86
8 Empathy 5.5 (1.6) 6.0 (1.5) 5.3 (1.7) 6.2 (1.4) 5.8 (1.3) .17 .19 .50 .75
9 Reprimand 5.7 (1.6) 6.5 (1.4) 6.1 (1.5) 6.2 (1.4) 6.3 (1.3) .10 .09 .62 .84
10 Convince 6.4 (1.2) 6.6 (1.1) 6.3 (1.3) 7.0 (0.9) 6.6 (1.0) .15 .16 .48 .73
11 Instructions 6.1 (1.3) 6.0 (1.4) 6.0 (1.5) 6.3 (1.5) 6.1 (1.2) .23 .26 .47 .74
12 Compliment 6.4 (1.1) 6.6 (1.2) 6.0 (1.5) 7.2 (1.0) 6.6 (1.0) .10 .12 .41 .70
13 Emo. support 5.7 (1.6) 6.5 (1.4) 5.9 (1.7) 6.6 (1.5) 6.3 (1.4) .32 .35 .69 .87
14 Encourage 6.0 (1.3) 6.0 (1.3) 5.7 (1.6) 6.8 (1.2) 6.1 (1.1) .18 .26 .45 .70
15 Bargain 5.8 (1.5) 6.4 (1.3) 6.2 (1.2) 6.8 (1.1) 6.4 (0.9) .08 .10 .29 .55
16 Affection 6.1 (1.4) 6.2 (1.6) 5.9 (1.5) 6.9 (1.1) 6.3 (1.2) .16 .20 .55 .79
17 Instr. support 6.5 (1.1) 6.6 (1.2) 6.4 (1.3) 6.8 (1.2) 6.6 (1.0) .07 .09 .44 .71
Avg1 5.8 (0.6) 6.3 (0.6) 6.0 (0.8) 6.7 (0.6) 6.3 (0.6) .03 .02 .62 .84
Avg2 5.8 (0.7) 6.3 (0.4) 6.0 (0.3) 6.7 (0.3) 6.3 (0.3) .17 .20 .51 .75

Note. Avg1 = Measures were first averaged across tasks (for each target), and then the respective index was computed (across targets). Avg2 = Indices were first computed
separately for each task (across targets), and then averaged across tasks. S–O = Average agreement between self and a single observer (ICC(2, 1)). S–Ō = Agreement between
self and the average observer (ICC(2, 1)). O–O = Average agreement between two individual observers (ICC(2, 1)). Ō Rel = Reliability of averaged observer judgment (ICC(2, 3)).

Table 2
Correlations between IIP-64 scales and self-rated task performance.

Task Keyword IIP-64 Scale

1 2 3 4 5 6 7 8 Total
1 Initiate .18 .22 .19 .25 .29 .25 .21 .20 .34
2 Apologize .10 .09 .11 .07 .01 .11 .10 .09 .08
3 Thank .01 .02 .10 .02 .04 .21 .18 .22 .15
4 Assert .08 .19 .27 .29 .25 .24 .17 .10 .30
5 Confront .18 .11 .17 .17 .17 .15 .15 .12 .23
6 Positively .22 .13 .21 .33 .23 .34 .27 .18 .36
7 Self-critical .06 .09 .06 .04 .12 .24 .22 .16 .15
8 Empathy .06 .09 .19 .27 .21 .23 .06 .01 .22
9 Reprimand .01 .05 .12 .19 .27 .12 .26 .15 .22
10 Convince .09 .11 .21 .30 .18 .05 .06 .01 .19
11 Instructions .05 .06 .08 .11 .11 .09 .09 .05 .10
12 Compliment .24 .25 .20 .02 .12 .08 .01 .13 .12
13 Emo. support .23 .23 .28 .35 .26 .16 .22 .11 .34
14 Encourage .11 .20 .20 .09 .05 .04 .08 .06 .15
15 Bargain .13 .16 .19 .14 .18 .15 .21 .25 .26
16 Affection .11 .13 .08 .02 .09 .10 .19 .14 .16
17 Instr. support .22 .21 .15 .27 .22 .16 .27 .20 .32
Total .25 .30 .35 .40 .36 .33 .38 .29 .50

Note. IIP-64 scales: 1 = Dominant, 2 = Vindictive, 3 = Cold, 4 = Socially Avoidant, 5 = Submissive, 6 = Exploitable, 7 = Overly nurturant, 8 = Intrusive. Total = Total score. Bold-
print: p < .05.

indicating that the participants’ general images of themselves as ability of interpersonal skills (deficits) from outside, we will only
being socially competent or incompetent were not reflected at all report results that were obtained with the averaged observer-rat-
by how their performances across tasks were evaluated by the ings of task performance as dependent variables. Analyses using
observers. The association between the IIP-64 total scores and self-rated task performances as dependent variables are available
the averaged self-ratings of performance across tasks (r = .50, from the first author upon request. Antisocial PD was excluded
see above) was significantly stronger than the respective associa- in these analyses, because there is no self-rating scale that
tion between the IIP-64 total scores and the averaged observer-rat- corresponds to the respective interviewer-rating scale for this
ings of performance across tasks (r = .04), t(96) = 4.302, p < .001. disorder.
Tables 4 and 5 display the zero-order correlations between self-
3.3.2. Associations with the SCID-II rated (Table 4) and interviewer-rated (Table 5) personality pathol-
Unlike the IIP-64, the SCID-II comprises not only a self-assess- ogy on the one hand, and observer-rated task performances on the
ment by means of a questionnaire, but also an assessment by an other hand. The number of significant associations was rather
interviewer. This makes it possible to study the relative incremen- modest in both tables, but somewhat larger in Table 5 (17) than
tal validity of the two perspectives in predicting the participants’ in Table 4 (12). Some associations emerged with both self-ratings
task performances. As we were primarily interested in the observ- and interviewer-ratings of personality pathology as predictors.
636 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641

Table 3
Correlations between IIP-64 scales and observer-rated task performance.

Task Keyword IIP-64 Scale

1 2 3 4 5 6 7 8 Total
1 Initiate .07 .04 .09 .10 .01 .02 .03 .11 .03
2 Apologize .06 .01 .08 .12 .04 .09 .14 .15 .08
3 Thank .04 .13 .36 .15 .04 .10 .01 .15 .09
4 Assert .06 .02 .10 .19 .08 .06 .07 .10 .09
5 Confront .10 .18 .03 .13 .03 .09 .12 .25 .11
6 Positively .06 .12 .16 .19 .07 .07 .08 .03 .15
7 Self-critical .21 .04 .04 .00 .03 .13 .19 .19 .13
8 Empathy .14 .05 .14 .03 .02 .21 .26 .33 .14
9 Reprimand .06 .17 .07 .05 .12 .05 .03 .06 .03
10 Convince .04 .07 .14 .03 .02 .07 .06 .13 .01
11 Instructions .04 .12 .03 .14 .01 .18 .14 .13 .04
12 Compliment .20 .07 .01 .13 .08 .13 .12 .16 .17
13 Emo. support .10 .14 .17 .16 .11 .00 .05 .08 .15
14 Encourage .06 .02 .09 .04 .09 .12 .06 .16 .07
15 Bargain .15 .12 .05 .04 .08 .03 .03 .09 .05
16 Affection .01 .09 .22 .06 .03 .04 .00 .05 .06
17 Instr. Support .04 .01 .01 .01 .13 .19 .22 .10 .13
Total .08 .02 .17 .11 .04 .15 .13 .22 .04

Note. IIP-64 scales: 1 = Dominant, 2 = Vindictive, 3 = Cold, 4 = Socially Avoidant, 5 = Submissive, 6 = Exploitable, 7 = Overly nurturant, 8 = Intrusive. Total = Total score. Bold-
print: p < .05.

Table 4
Correlations between SCID-II self-ratings and observer-rated task performance.

Task Keyword SCID-II Scale

Av Dep OC Par Sch His Nar Bor Total
1 Initiate .09 .02 .07 .01 .17 .19 .05 .08 .04
2 Apologize .13 .05 .05 .01 .13 .01 .08 .03 .03
3 Thank .05 .04 .03 .21 .17 .10 .09 .05 .06
4 Assert .26 .12 .08 .22 .21 .05 .05 .02 .19
5 Confront .11 .10 .06 .09 .22 .28 .06 .20 .05
6 Positively .13 .14 .07 .14 .04 .05 .13 .05 .14
7 Self-critical .08 .02 .18 .12 .27 .23 .15 .10 .15
8 Empathy .02 .09 .01 .16 .04 .17 .12 .11 .08
9 Reprimand .06 .13 .05 .05 .09 .13 .09 .11 .04
10 Convince .05 .07 .05 .05 .03 .08 .05 .03 .02
11 Instructions .06 .00 .03 .03 .05 .08 .17 .07 .03
12 Compliment .14 .02 .03 .08 .03 .05 .01 .03 .09
13 Emo. support .22 .22 .14 .21 .08 .01 .07 .27 .28
14 Encourage .04 .03 .00 .01 .13 .18 .02 .10 .03
15 Bargain .00 .11 .06 .11 .06 .01 .10 .08 .02
16 Affection .02 .07 .03 .08 .12 .03 .07 .14 .02
17 Instr. support .05 .07 .09 .01 .17 .12 .07 .12 .02
Total .07 .10 .03 .05 .21 .19 .04 .04 .04

Note. SCID-II scales: Av = Avoidant, Dep = Dependent, OC = Obsessive–Compulsive, Par = Paranoid, Sch = Schizoid, His = Histrionic, Nar = Narcissistic, Bor = Borderline. Bold-
print: p < .05.

For example, participants with more avoidant personality features rately predicted observer-rated task performance for each of the
(according to their own and the interviewers’ judgments) showed 17 interaction tasks, and across tasks (=18 regression analyses).
poorer performances in Task 4 (assert a claim). This association di- The predictor variables in each of these analyses were the eight
rectly replicates a previous finding by Leising et al. (2006). Other self-ratings and the eight corresponding interviewer-ratings of
associations were only found with one kind of predictor variables. personality pathology (=16 predictors altogether). The two differ-
For example, interviewer-rated, but not self-rated, paranoid per- ent kinds of predictors (self- and interviewer-ratings) were entered
sonality features predicted poorer performances in Task 15 (bar- block-wise into the regressions.
gain). Regardless of whether personality pathology was judged In a first set of analyses, the block of eight self-ratings of person-
by the interviewers or by the participants themselves, schizoid ality pathology was entered into the regressions first, and the
personality features were the only significant predictor of the par- strongest predictors within this block were selected by means of
ticipants’ average performances across tasks, and poorer perfor- the Stepwise algorithm in PASW. After the self-ratings, the eight
mances in Task 13 (ask for emotional support) were predicted by interviewer-ratings of personality pathology were entered as a sec-
the average of all SCID-II scales. However, only interviewer-ratings ond block, and the strongest predictors within this block were se-
of overall personality pathology (= average of all SCID-II scales) lected accordingly. The goal of using the Stepwise algorithm was to
predicted poorer performances in Task 6 (present yourself determine the most parsimonious set of predictors for each task,
positively). taking into account the considerable overlap between the DSM-
We addressed the issue of incremental validity using multiple IV PD constructs (cf. Bornstein, 1998). As the self-ratings of person-
regressions, as suggested by Vazire and Mehl (2008): We sepa- ality pathology were entered into the regressions first, the
 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641 637

Table 5
Correlations between SCID-II interviewer ratings and observer-rated task performance.

Task Keyword SCID-II Scale

Av Dep OC Par Sch His Nar Bor Total
1 Initiate .07 .07 .15 .02 .24 .13 .20 .13 .14
2 Apologize .08 .03 .00 .01 .26 .04 .02 .07 .03
3 Thank .01 .04 .03 .17 .23 .10 .22 .06 .09
4 Assert .31 .03 .12 .22 .06 .15 .07 .07 .16
5 Confront .15 .14 .10 .09 .17 .25 .02 .21 .02
6 Positively .12 .23 .02 .12 .18 .07 .31 .08 .24
7 Self-critical .11 .02 .18 .08 .28 .08 .03 .13 .06
8 Empathy .02 .12 .01 .15 .06 .14 .13 .09 .02
9 Reprimand .13 .13 .05 .06 .09 .11 .07 .07 .07
10 Convince .10 .07 .05 .05 .05 .06 .05 .06 .06
11 Instructions .09 .12 .06 .09 .08 .02 .28 .06 .13
12 Compliment .02 .01 .00 .05 .07 .02 .07 .10 .00
13 Emo. support .23 .19 .15 .06 .02 .12 .08 .19 .25
14 Encourage .09 .10 .13 .03 .11 .23 .08 .20 .15
15 Bargain .15 .17 .06 .20 .08 .02 .06 .09 .09
16 Affection .00 .06 .08 .07 .12 .03 .06 .13 .02
17 Instr. support .05 .07 .02 .11 .23 .02 .12 .12 .04
Total .09 .13 .04 .08 .27 .10 .15 .08 .13

Note. SCID-II scales: Av = Avoidant, Dep = Dependent, OC = Obsessive–Compulsive, Par = Paranoid, Sch = Schizoid, His = Histrionic, Nar = Narcissistic, Bor = Borderline. Bold-
print: p < .05.

interviewer-ratings could only account for variance that was not ing that a considerable proportion of relevant behavioral
already accounted for by the self-ratings. If, under these circum- information would be lost by using self-ratings of personality
stances, the interviewer-ratings did not add anything to the predic- pathology only.
tion, they could be considered expendable. In a second set of analyses, we reversed the order of entry and en-
The results are displayed in the first five data columns of Table 6. tered the block of interviewer-ratings of personality pathology first,
The table displays the F values for the overall regressions, and the followed by the block of self-ratings. This allowed us to investigate
differences in R2 that resulted from entering each of the two blocks whether there was any relevant information included in the self-
of predictors. Empty cells indicate that the respective block of pre- ratings that was not already included in the interviewer-ratings.
dictors did not contribute significantly to the prediction of perfor- Again, the strongest predictors within each block were selected by
mance in a task. The self-ratings contributed significantly to the the Stepwise algorithm. The results are displayed in the last five
prediction of performance in five tasks. However, the inter- data columns of Table 6. Interviewer-ratings of personality signifi-
viewer-ratings showed incremental validity for eight tasks, imply- cantly predicted performance in 12 of the 17 tasks. The self-ratings

Table 6
Hierarchical multiple regressions of task performance on measures of personality pathology (SCID-II).

Task Keyword SCID-II self-ratings entered first SCID-II interviewer-ratings entered first
F Self Interviewer F Interviewer Self
DR2 Pred. DR2 Pred. DR 2 Pred. DR2 Pred.
1 Initiate 5.72 .06 Sch ( .24) 5.72 .06 Sch ( .24)
2 Apologize 6.78 .07 Sch ( .26) 6.63 .07 Sch ( .35) .06 Av (.26)
3 Thank 4.11 .04 Par ( .18) .04 Nar ( .20) 4.86 .10 Sch ( .21)
Nar ( .20)
4 Assert 6.92 .07 Av ( .26) 10.05 .10 Av ( .31)
5 Confront 8.53 .08 His (.28) 5.73 .20 His (.28)
Dep ( .28)
Bor (.26)
Sch ( .19)
6 Positively 7.01 .13 Nar ( .29) 7.01 .13 Nar ( .29)
Dep ( .20) Dep ( .20)
7 Self-critical 6.41 .12 Sch ( .38) 7.81 .15 Sch ( .34)
Av (.25) OC (.26)
8 Empathy
9 Reprimand
10 Convince
11 Instructions 8.41 .08 Nar ( .28) 8.41 .08 Nar ( .28)
12 Compliment
13 Emotional support 7.70 .08 Bor ( .27) 5.21 .05 Av ( .16) .05 Bor ( .23)
14 Encourage 5.17 .05 His (.23) 5.17 .05 His (.23)
15 Bargain 4.08 .04 Par ( .20) 4.08 .04 Par ( .20)
16 Affection
17 Instrumental support 5.31 .05 Sch ( .23) 5.31 .05 Sch ( .23)
Total 4.40 .04 Sch ( .21) 7.86 .08 Sch ( .27)

Note. DV = observer-rated task performances in the 17 tasks, and across tasks (bottom row). SCID-II scales: Av = Avoidant, Dep = Dependent, OC = Obsessive–Compulsive,
Par = Paranoid, Sch = Schizoid, His = Histrionic, Nar = Narcissistic, Bor = Borderline. The eight self-ratings and the eight interviewer-ratings of personality pathology were
entered block-wise, and the strongest predictor in each block was selected using the stepwise algorithm. DR2 = R2 increase for entering a block of variables. Only coefficients
from significant regressions (p < .05) are displayed.
638 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641

showed incremental validity in only two tasks, implying that almost investigated was relatively large, introducing a certain level of type
all of the relevant behavioral information that was contained in the I error inflation. Therefore, the associations that we found and dis-
self-ratings was also contained in the interviewer-ratings. cussed above would clearly have to be replicated with an indepen-
dent sample.
4. Discussion On the other hand, two factors made it relatively difficult to find
associations between the different assessment methods in the
The present study aimed to establish a closer connection be- present study: First, due to the exclusion criteria that we applied
tween social skills research and research on personality disorders in recruiting our participants, the sample was relatively healthy
and interpersonal problems. So far, these research traditions have (see above). As a consequence, variance in both the predictor and
existed more or less independently of each other. To achieve a bet- criterion variables was probably restricted, lowering the chances
ter integration, we applied their preferred assessment methods to of finding significant associations between them. Second, it has
the same sample. long been acknowledged that the DSM-IV personality disorder con-
Our first challenge was to develop a new set of standard inter- cepts are lacking in construct validity (Bornstein, 1998; Widiger &
action tasks that would enable a relatively comprehensive observa- Trull, 2007), which was also reflected in the relatively low Alphas
tional assessment of interpersonal capacities that feature in both of the SCID-II scales in the present study. Internal heterogeneity
the personality pathology and the social skills literature. In our of predictor variables makes it more difficult to find significant cor-
opinion, the development of such tasks in the present study may relations with other variables, because heterogeneous groups of
be considered quite successful: Participants were able and willing people may receive the same scores on a predictor variable. Taking
to complete the 17 interaction tasks in less than 30 min, and these issues into account, we do think that the observational ap-
observers could judge the participants’ performances in the tasks proach is a promising pathway for future research on personality
with good agreement. Thus, the set of tasks that we created seems pathology and interpersonal problems, particularly when using
to be a reliable tool for investigating interpersonal functioning less selected samples, and more valid predictors (which, hopefully,
from an observer’s point of view. will become available with the publication of DSM-5).
We investigated the clinical relevance of the tasks by correlat- Our analyses pertaining to incremental validity exemplify how
ing the participants’ task performances with a traditional measure the observational approach may yield insights that would not be
of personality pathology, the SCID-II. The average of all task perfor- available otherwise. These analyses revealed that almost all of
mances correlated negatively with schizoid personality features. the relevant behavioral information from the SCID-II questionnaire
Several authors (e.g., Argyle & Lu, 1990; Riggio, 1986; Schneider, was contained in the interviewer-ratings (see right half of Table 6),
Ackerman, & Kanfer, 1996) have argued that ‘‘overall social compe- but that much of the information provided by the interviewers was
tence’’ may be related to extraversion, and our study supports this not contained in the self-ratings (see left half of Table 6). It should
view, because schizoid PD marks the low end of the extraversion be noted that, in computational terms, this was a fair comparison:
continuum (Reynolds & Clark, 2001). In addition, we also found For each target, there was just one self-rating and one interviewer-
several more specific associations between individual PD scales rating. Therefore, the higher validity of the interviewer-ratings
and performances in specific tasks (see Table 6). All of these made may not be attributed to (e.g.) aggregation across several inter-
good sense, from a clinical point of view. For example, avoidant PD viewers. However, it is also true that the interviewers had an
features were the strongest predictor of (poor) performance in Task advantage, because they could not only base their judgments on
4 (assert a Claim), which seems understandable, given that fear of their own observations, but also on the participants’ self-reports
rejection features so prominently in the DSM-IV concept of this PD (via questionnaire responses and utterances during the inter-
(APA, 2000). Likewise, as arrogance is such a central element of views). Obviously, the interviewers did take most of that informa-
DSM-IV’s definition of narcissism, it seems straightforward that tion into account, as indicated by the fact that the participants’
participants with higher (interviewer-rated) narcissism were self-ratings in the SCID-II questionnaire had very little incremental
judged as being less successful at thanking someone (Task 3), pre- validity over the respective interviewer-ratings. The bottom line of
senting themselves positively (Task 6), and giving instructions these analyses is that relying exclusively on self-reports of person-
(Task 11). ality pathology leaves a substantial amount of relevant behavioral
Interestingly, some of the more specific associations that we variance untapped, and that the use of clinical interviews as an
found between interviewer-rated personality pathology and obser- additional source of information beyond self-reports is certainly
ver-rated task performances were positive. For example, partici- justified. To the best of our knowledge, this incremental validity
pants with more histrionic personality features were better at has not been empirically demonstrated before.
confronting someone (Task 5), and at encouraging someone (Task Another striking finding emerged in comparing the predictive
14), whereas more obsessive–compulsive participants were better validity of the participants’ self-ratings in the IIP-64 with regard
at discussing their own shortcomings (Task 7). Again, these associ- to self-rated and observer-rated task performance: There were
ations appear to be in accordance with prominent themes of the many significant associations between the participants’ general
respective DSM-IV PD concepts (e.g., transgressing interpersonal interpersonal self-images (as assessed by the IIP-64), and how they
boundaries in HPD, high personal standards in OCPD). They suggest evaluated their own performances in the interaction tasks. How-
that (moderate degrees of) some personality characteristics that ever, most of these associations vanished when observer-ratings
are commonly regarded as ‘‘pathological’’ may actually be advanta- of task performance were used instead of self-ratings: The observ-
geous under certain circumstances (cf. Apt & Hurlbert, 1994). ers agreed well with each other in judging the participants’
However, these associations must be interpreted with caution, performances in the role-plays, but the participants’ interpersonal
for two reasons: First, they emerged from exploratory analyses. self-images had very little predictive validity regarding these
We did not explicitly articulate a priori hypotheses regarding asso- judgments.
ciations between specific SCID-II scales and performances in cer- In our view, this finding may be attributable to the operation of a
tain tasks. Rather, at this stage of the research process, our general stylistic self-evaluation factor (cf. Oltmanns & Turkheimer,
primary goal was to determine whether there were any associa- 2006): People may differ from each other in how much they tend to
tions between the two measures at all. Once a specific association attribute positive or negative characteristics to themselves,
is found, it is often relatively easy to come up with an a posteriori regardless of specific content. Such differences would inflate
explanation for it. Second, the number of associations that we associations among all kinds of self-reported variables, to the
 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641 639

extent that those variables are evaluative (cf. Leising & Borkenau, with. Second, our tasks involve only dyadic situations, so abilities
2011; Leising et al., 2010). The correlations displayed in Table 2 that are needed when interacting with several people at once
may have been particularly susceptible to such a ‘‘shared evalua- (e.g., settling a conflict between others) are not assessed either. Fu-
tion effect’’ because both the IIP-64 items and the self-ratings of ture research should investigate to what extent observational
task performance required participants to judge whether they are assessments of interpersonal capacities based on dyadic, activity-
‘‘good’’ at something. As long as independent assessments of the oriented tasks like ours are representative for interpersonal com-
targets’ interpersonal functioning are missing (as in most studies), petence as a whole.
one will never know to what extent associations between self- Despite these limitations, we do recommend the use of stan-
reported interpersonal functioning and other self-reported vari- dard interaction tasks in research on social competence, interper-
ables (e.g., self-assessed efficiency at work) reflect reality, or only sonal problems and personality pathology. For the immediate
a common susceptibility of the measures to be affected by the tar- future of this line of research, we would suggest using standard
gets’ general self-evaluations. Therefore it seems important to use interaction tasks in longitudinal studies, and with less selected
other measures of interpersonal functioning to complement self- samples. In our view, the most important research questions at
reports. present are (a) how stable interpersonal competencies and compe-
The agreement between the observers and the participants in tence deficits are over time, and (b) whether, and which, important
judging the overall task performances of the latter (Table 1, life outcomes (e.g., career, income, marriage, divorce, loneliness,
Avg1) was essentially zero (see Riggio, Widman, & Friedman, social support) may be longitudinally predicted from the presence
1985, for a similar finding). It should be noted, however, that the or absence of certain interpersonal competencies at an earlier time.
average self-other agreement in judging the participants’ perfor- In the present study, performance in standard interaction tasks
mances in individual tasks (Avg2) was higher than zero, though still was used as a criterion measure, demonstrating (e.g.) that inter-
very modest (.20). Thus, it may not be concluded that people are viewer-ratings of personality pathology capture important behav-
completely unable to assess their own performances in socially ioral variance that is not captured by self-ratings. However, it
challenging situations. Only when averaging across situations be- would also be possible to use interaction task performance as a
fore computing self-other agreement, self-other agreement predictor, to study its incremental validity over other sources of
dropped to zero. The most likely explanation for this is that all of information. Such studies would be useful to further elucidate
the participants’ self-assessments were similarly affected by their the extent to which the greater effort that is associated with
(overly) positive or negative views of themselves (‘‘style’’), whereas behavior observation is justifiable in terms of additional insights
the actual competencies that were assessed by the individual tasks that may be obtained with this method.
(‘‘substance’’) differed considerably, and may even have averaged
out to some extent. As a consequence, the participants’ average Appendix A
self-assessments of their task performances primarily reflected
their positive or negative self-views, and did not correlate much A.1. Instructions for Standard Interaction Tasks [standard replies in
with the views of external observers. brackets]
The pattern of (non-)associations between the different mea-
sures of interpersonal functioning that were used in the present Task 1: Initiate a conversation. You are at a party, but you do not
study may partly be explained in terms of the shared and unshared know a single person there except for the host. Another person
features of these measures (cf. Kenny, 1994; Vazire, 2010): The who seems to be in the same situation is standing close to you. Ap-
self-ratings of interpersonal problems, personality pathology and proach the person, introduce yourself, and initiate a conversation.
task performance represent ‘‘views from within,’’ whereas the ob- [Standardized replies for roleplay partner: Actually, I am waiting
server-ratings of task performance represent ‘‘views from with- for someone; So you are here all alone?; Seems to be a nice party;
out.’’ The interviewer-ratings of personality pathology are I do not really know the host of this party too well.]
somewhere in between, because the interviewers also saw the tar- Task 2: Apologize to someone. You have embarrassed a friend in
gets from outside (as did the observers), but obviously took the tar- front of others by saying something inconsiderate. Now the person
gets’ self-reports into account when making their ratings. The is angry and disappointed in you. Apologize to her for your behav-
observers had two features in common with one another, but not ior. [I would not have expected that from you; So what do you in-
with the targets: First, they shared exactly the same visual per- tend to do now?; What you said hurt me a lot; Well, this does not
spective, as they only saw the targets on film. Second, they had really make it any better.]
no particular interest in protecting or harming the targets’ sense Task 3: Thank someone. A friend has helped you move. She has
of self-esteem by judging them too leniently or too harshly. For sacrificed an entire day helping you transport your furniture and
these reasons, one would expect the observer-ratings to correlate boxes of stuff into your new apartment. Thank her for that. [Don’t
more strongly with each other than with the targets’ self-ratings, mention it!; If not me, somebody else would have done it; You
which is exactly what we found. The interviewers also had no would have done the same for me; It was fun.]
motivation to protect or derogate the targets, and their visual per- Task 4: Assert a claim. A friend has borrowed money from you
spective was more similar to the observers’ than to the targets’. repeatedly, but never paid you back. Go to him and ask him to give
Thus one would expect closer associations between interviewer- you the money back. [I thought we were friends; I did not know
and observer-ratings than between self- and observer-ratings. you would need the money back so quickly; Well, it makes me feel
Again, this is what we found. uncomfortable too; But I am broke right now.]
Although the number of interaction tasks we used in the pres- Task 5: Confront someone. You have learned that a friend has
ent study was relatively large, and the tasks were quite diverse, spoken badly about you behind your back. Confront the person
two limitations need to be acknowledged: First, all of the tasks as- and demand an explanation from her. Tell her that you find her
sess people’s abilities to actively do something. However, the behavior unacceptable, and that you expect her to never do any-
capacity to not do something (e.g., express one’s anger) may cer- thing like that again. [Don’t be so touchy; I didn’t mean it that
tainly be important under some circumstances. It is considerably way; Was what I said so wrong?; I don’t understand why this both-
more complicated to assess such ‘‘inhibitory’’ skills, because this ers you so much.]
would require validly distinguishing people who successfully inhi- Task 6: Present yourself positively. You have entered a competi-
bit an impulse from people who do not have the impulse to begin tion to become ‘‘person of the year’’. Now you are standing before
640 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641

a member of the jury. Convince the person that YOU deserve to be how much you will miss her. [It’s just for one year; You still got
the winner. Emphasize all your valuable traits and try to make a your other friends; Now don’t dramatize this thing; After all, I will
really good impression. You are allowed to brag, but please stick be back soon.]
with the truth. Do not lie (e.g., claim to have abilities that you do Task 17: Ask for instrumental support. You plan to move to a new
not really have). [Anything more?; Is that all?; So what is so special apartment next weekend, but you still need some more people to
about you?; What else do you think you should tell me?] help you move. You happen to meet a friend you have not seen
Task 7: Be self-critical. You are talking to a counselor. Right now, in a while. Ask her to help you. [How much stuff do you have?;
the focus is on your personal weaknesses. Describe the things that Does nobody else have time?; I am pretty clumsy and I’m sure I
you do not like about yourself. What mistakes have you made? would wreck some of your stuff; Actually, I just wanted to relax
What would you like to change about yourself if you could? [Is that this weekend.]
really so bad?; And this bothers you – why?; Are there any other
difficulties?; That does not sound like a problem to me at all.]
References
Task 8: Show empathy. A person you know has just been dumped
by her boyfriend. Now she is very sad and unhappy. Comfort her American Psychiatric Association (2000). Diagnostic and statistical manual of mental
and be empathetic. [I am not ready to talk about it yet; I am not disorders (4th ed., text revision). Washington, DC: American Psychiatric
sure whether you understand how I feel; We were together for Association.
Apt, C., & Hurlbert, D. F. (1994). The sexual attitudes, behavior, and relationships of
so long; What am I supposed to do now?] women with histrionic personality disorder. Journal of Sex and Marital Therapy,
Task 9: Reprimand someone. You have been waiting in line at 20, 125–133.
an ATM for several minutes. Two more people are standing in Argyle, M., & Lu, L. (1990). The happiness of extraverts. Personality and Individual
Differences, 11, 1011–1017. doi:10.1016/0191-8869(90)90128-E.
front of you. Suddenly another person enters the room and goes Bornstein, R. F. (1998). Reconceptualizing personality disorder diagnosis in the
directly to the ATM, ignoring the line. Reprimand the person and DSM-V: The discriminant validity challenge. Clinical Psychology: Science and
make her wait in line like everybody else. [I am in a hurry; Don’t Practice, 5, 333–343.
Bornstein, R. F. (2003). Behaviorally referenced experimentation and symptom
wet your pants over this; I’ll be finished in a minute; Just a validation: A paradigm for 21st-century personality disorder research. Journal of
moment.] Personality Disorders, 17, 1–18. doi:10.1521/pedi.17.1.1.24056.
Task 10: Convince someone. You would like to drive to a nearby Buhrmester, D., Furman, W., Wittenberg, M. T., & Reis, H. T. (1988). Five domains of
interpersonal competence in peer relationships. Journal of Personality and Social
lake with a friend, to go swimming. The friend is not really in the Psychology, 55, 991–1008. doi:10.1037//0022-3514.55.6.991.
mood, and is undecided. Convince the friend to come to the lake Clifton, A., Turkheimer, E., & Oltmanns, T. F. (2004). Contrasting perspectives on
with you. [Oh, I don’t know. . .; I have a lot of other stuff to do; That personality problems: Descriptions from the self and others. Personality and
Individual Differences, 36, 1499–1514. doi:10.1016/j.paid.2003.06.002.
would cost me a whole day; I am not sure the weather will stay
Clifton, A., Turkheimer, E., & Oltmanns, T. F. (2005). Self- and peer-perspectives on
this nice.] pathological personality traits and interpersonal problems. Psychological
Task 11: Give instructions. You work for a catering company. A Assessment, 17, 123–131. doi:10.1037/1040-3590.17.2.123.
party is scheduled, and you have been assigned an intern who is Connelly, B. S., & Ones, D. S. (2010). An other perspective on personality: Meta-
analytic integration of observers’ accuracy and predictive validity. Psychological
supposed to help you arrange the tables, chairs, and sound system. Bulletin, 136, 1092–1122. doi:10.1037/a0021212.
Tell the person what to do. [I am not sure I understand; And how Conroy, D. E., Elliot, A. J., & Pincus, A. L. (2009). The expression of achievement
exactly. . .?; And why is that?; Hmmm. . . (hesitating).] motives in interpersonal problems. Journal of Personality, 77, 495–526.
doi:10.1111/j.1467-6494.2008.00556.x.
Task 12: Pay a compliment. Yesterday evening you attended a Curran, J. P. (1982). A procedure for the assessment of social skills: The simulated
party that you really enjoyed. Now you happen to meet the host social interaction test. In J. P. Curran & P. M. Monti (Eds.), Social skills training: A
of that party. Compliment him for how well the party was orga- practical handbook for assessment and treatment. New York, NY: The Guilford
Press.
nized and for the wonderful atmosphere. [I have been to better Donahoe, C. P., Carter, M. J., Bloem, W. D., Hirsch, G. L., Laasi, N., & Wallace, C. J.
parties myself; I was not totally satisfied with it; Who could not (1990). Assessment of interpersonal problem-solving skills. Psychiatry, 53,
throw a party like that?; Hey, it was just an ordinary party.] 329–339.
Eisler, R. M., Hersen, M., Miller, P. M., & Blanchard, E. B. (1975). Situational
Task 13: Ask for emotional support. Imagine you went to the doc- determinants of assertive behavior. Journal of Consulting and Clinical Psychology,
tor and received a diagnosis that will affect your life considerably. 43, 330–340. doi:10.1037/h0076717.
You would rather not be alone right now, and think you need First, M.B., Spitzer, R.L., Gibbon, M., & Williams, J.B.W. (1996). User’s guide for the
structured clinical interview for DSM-IV personality disorders (SCID-II).
someone to talk to. Ask a friend to meet with you and be there
Washington, DC: American Psychiatric Press.
for you this evening. [Today is not such a good day; Tomorrow Fowler, K. A., Lilienfeld, S. O., & Patrick, C. J. (2009). Detecting psychopathy from thin
evening would be much better; I have a job interview tomorrow slices of behavior. Psychological Assessment, 21, 68–78. doi:10.1037/a0014938.
morning; It can’t really be that urgent, can it?] Friedman, J. N. W., Oltmanns, T. F., & Turkheimer, E. (2007). Interpersonal
perception and personality disorders: Utilization of a thin slice approach.
Task 14: Encourage someone. A friend of yours will have an Journal of Research in Personality, 41, 667–688. doi:10.1016/j.jrp. 2006.07.004.
important job interview tomorrow morning, but is very pessimistic Fydrich, T., Renneberg, B., Schmitz, B., & Wittchen, H.U. (1997). SKID-P. Strukturiertes
about his chances of getting the job. He even considers canceling Klinisches Interview für DSM-IV Achse II. Persönlichkeitsstörungen (Structured
clinical interview for DSM-IV personality disorders). Göttingen: Hogrefe.
the interview altogether. Encourage him to attend the interview, Goldsmith, J. B., & McFall, R. M. (1975). Development and evaluation of an
and restore his trust in himself. [The other applicants will be much interpersonal skill-training program for psychiatric inpatients. Journal of
better for sure; I have nothing to offer them, really; It won’t work Abnormal Psychology, 84, 51–58. doi:10.1037/h0076264.
Gresham, F. M., Elliott, S. N., Cook, C. R., Vance, M. J., & Kettler, R. (2010). Cross-
out; Why should this interview go better than all the others I’ve informant agreement for ratings of social skill and problem behavior ratings: An
had?] investigation of the social skills improvement system-rating scales.
Task 15: Bargain. You are at a flea market and have just discov- Psychological Assessment, 22, 157–166. doi:10.1037/a0018124.
Hinsch, R., & Pfingsten, U. (2002). Gruppentraining sozialer Kompetenzen (2. Aufl.)
ered an item that you would like to buy (just think of something). (Group training of social skills (2nd ed.)). Weinheim: Beltz.
The price is 25 Euro. Talk to the seller and try to bargain for a lower Hofstee, W. K. B. (1994). Who should own the definition of personality? European
price. [Well, the item is in really good condition; At that price I Journal of Personality, 8, 149–162. doi:10.1002/per.2410080302.
Horowitz, L. M. (1979). On the cognitive structure of interpersonal problems treated
would lose money myself; I’d rather wait for someone who is
in psychotherapy. Journal of Consulting and Clinical Psychology, 47, 5–15.
willing to pay more; How about a little concession from your doi:10.1037//0022-006X.47.1.5.
side?] Horowitz, L. M., Alden, L. E., Wiggins, J. S., & Pincus, A. L. (2000). Inventory of
Task 16: Express affection. A close friend of yours is about spend a interpersonal problems: Manual. San Antonio, TX: The Psychological Corporation.
Horowitz, L.M., Strauß, B., & Kordy, H. (2000). Inventar zur Erfassung Interpersonaler
year in New Zealand. The evening before she leaves, the two of you Probleme – Deutsche Version (IIP-D) (Inventory of Interpersonal Problems –
meet to say goodbye. Tell her how much you care about her and German version). Weinheim: Beltz Test.
 D. Leising et al. / Journal of Research in Personality 45 (2011) 631–641 641

John, O. P., & Robins, R. W. (1993). Determinants of interjudge agreement on Rhoades, B. L., Greenberg, M. T., & Domitrovich, C. E. (2009). The contribution of
personality traits: The Big Five domains, observability, evaluativeness, and the inhibitory control to preschoolers’ social–emotional competence. Journal of
unique perspective of the self. Journal of Personality, 61, 521–551. doi:10.1111/ Applied Developmental Psychology, 30, 310–320. doi:10.1016/j.appdev.2008.
j.1467-6494.1993.tb00781.x. 12.012.
Kenny, D. A. (1994). Interpersonal perception. A social relations analysis. New York: Riggio, R. E. (1986). Assessment of basic social skills. Journal of Personality and Social
The Guilford Press. Psychology, 51, 649–660. doi:10.1037//0022-3514.51.3.649.
Klonsky, E. D., Oltmanns, T. F., & Turkheimer, E. (2002). Informant-reports of Riggio, R. E., Widman, K. F., & Friedman, H. S. (1985). Actual and perceived
personality disorder: Relation to self-reports and future research directions. emotional sending and personality correlates. Journal of Nonverbal Behavior, 9,
Clinical Psychology – Science and Practice, 9, 300–311. doi:10.1093/ 69–83.
clipsy.9.3.300. Robins, R. W., & John, O. P. (1997). Effects of visual perspective and narcissism on
Leary, T. (1957). Interpersonal diagnosis of personality. New York: Ronald Press. self-perception. Psychological Science, 8, 37–42. doi:10.1111/j.1467-
Leising, D., & Borkenau, P. (2011). Person perception, dispositional inferences and 9280.1997.tb00541.x.
social judgment. In L. M. Horowitz & S. Strack (Eds.), Handbook of interpersonal Rose-Krasnor, L. (1997). The nature of social competence. A theoretical review.
psychology: Theory, research, assessment, and therapeutic interventions Social Development, 6, 111–135. doi:10.1111/1467-9507.00029.
(pp. 157–170). Hoboken, NJ: John Wiley & Sons. Sayers, M. D., & Bellack, A. S. (1995). An empirical method for assessing social
Leising, D., Erbs, J., & Fritz, U. (2010). The letter of recommendation effect in problem solving in schizophrenia. Behavior Modification, 19, 267–289.
informant ratings of personality. Journal of Personality and Social Psychology, 98, doi:10.1177/01454455950193001.
668–682. doi:10.1037/a0018771. Sayers, M. D., Bellack, A. S., Wade, J. H., Bennett, M. E., & Fong, P. (1995). An
Leising, D., Müller, J., & Hahn, C. (2007). An adjective list for assessing emotional empirical method for assessing social problem solving in schizophrenia.
expressivity in psychotherapy research. Clinical Psychology and Psychotherapy, Behavior Modification, 19, 267–289. doi:10.1177/01454455950193001.
14, 377–385. doi:10.1002/cpp. 552. Schneider, R. J., Ackerman, P. L., & Kanfer, R. (1996). To ‘‘act wisely in human
Leising, D., & Müller-Plath, G. (2009). Person–situation integration in research on relations’’: Exploring the dimensions of social competence. Personality and
personality problems. Journal of Research in Personality, 43, 218–227. Individual Differences, 15, 469–481. doi:10.1016/0191-8869(96)00084-0.
doi:10.1016/j.jrp. 2009.01.017. Stacks, A. M., & Oshio, T. (2009). Disorganized attachment and social skills as
Leising, D., Rehbein, D., & Sporberg, D. (2007). Validity of the Inventory of indicators of Head Start children’s school readiness skills. Attachment and
Interpersonal Problems (IIP-64) for predicting assertiveness in role-play Human Development, 11, 143–164. doi:10.1080/14616730802625250.
situations. Journal of Personality Assessment, 89, 116–125. doi:10.1080/ Thorndike, E. L. (1920). Intelligence and its use. Harper’s Magazine, 140,
00223890701468428. 227–235.
Leising, D., Sporberg, D., & Rehbein, D. (2006). Characteristic interpersonal behavior Torgersen, S., Kringlen, E., & Cramer, V. (2001). The prevalence of personality
in dependent and avoidant personality disorder can be observed within very disorders in a community sample. Archives of General Psychiatry, 58, 590–596.
short interaction sequences. Journal of Personality Disorders, 20, 319–330. doi:10.1001/archpsyc.58.6.590.
doi:10.1521/pedi.2006.20.4.319. Tsang, H., & Pearson, V. (2000). Reliability and validity of a simple measure for
McFall, R. M. (1982). A review and reformulation of the concept of social skills. assessing the social skills of people with schizophrenia necessary for seeking
Behavioral Assessment, 4, 1–33. and securing a job. Canadian Journal of Occupational Therapy, 67, 250–259.
McFall, R. M., & Marston, A. R. (1970). An experimental investigation of behavior Ullrich, R., & DeMuynck, R. (2006). ATP 2 – Einübung von Selbstvertrauen. Grundkurs
rehearsal in assertive training. Journal of Abnormal Psychology, 76, 295–303. (Self-confidence training. Basic course). Stuttgart: Klett-Cotta.
doi:10.1037/h0030112. Vazire, S. (2010). Who knows what about a person? The self-other knowledge
Oltmanns, T. F., Friedman, J. N. W., Fiedler, E. R., & Turkheimer, E. (2004). asymmetry (SOKA) model. Journal of Personality and Social Psychology, 98,
Perceptions of people with personality disorders based on thin slices of 281–300. doi:10.1037/a0017908.
behavior. Journal of Research in Personality, 38, 216–229. doi:10.1016/S0092- Vazire, S., & Mehl, M. R. (2008). Knowing me, knowing you: The accuracy and
6566(03)00066-7. unique predictive validity of self and other ratings of daily behavior. Journal of
Oltmanns, T. F., & Turkheimer, E. (2006). Perceptions of self and others regarding Personality and Social Psychology, 95, 1202–1216. doi:10.1037/a0013314.
pathological personality traits. In R. F. Krueger & J. L. Tackett (Eds.), Personality Widiger, T. A., & Frances, A. J. (1985). The DSM-III personality disorders:
and psychopathology (pp. 71–111). New York: The Guilford Press. Perspectives from psychology. Archives of General Psychiatry, 42, 615–623.
Patterson, T. L., Moscona, S., McKibbin, C. L., Davidson, K., & Jeste, D. V. (2001). Social Widiger, T. A., & Samuel, D. B. (2005). Diagnostic categories or dimensions? A
skills assessment among older patients with schizophrenia. Schizophrenia question for the diagnostic and statistical manual of mental disorders – Fifth
Research, 48, 351–360. doi:10.1016/S0920-9964(00)00109-2. edition. Journal of Abnormal Psychology, 114, 494–504. doi:10.1037/0021-
Paulhus, D. L. (1984). Two-component models of socially desirable responding. 843X.114.4.494.
Journal of Personality and Social Psychology, 46, 598–609. doi:10.1037//0022- Widiger, T. A., & Trull, T. J. (2007). Plate tectonics in the classification of personality
3514.46.3.598. disorder: Shifting to a dimensional model. American Psychologist, 62, 71–83.
Rantanen, K., Timonen, S., Hagstrom, K., Hamalainen, P., Eriksson, K., & Nieminen, P. doi:10.1037/0003-066X.62.2.71.
(2009). Social competence of preschool children with epilepsy. Epilepsy and Wiggins, J. S. (1979). A psychological taxonomy of trait-descriptive terms: The
Behavior, 14, 338–343. doi:10.1016/j.yebeh.2008.10.022. interpersonal domain. Journal of Personality and Social Psychology, 37, 395–412.
Reynolds, S. K., & Clark, L. A. (2001). Predicting dimensions of personality disorder doi:10.1037//0022-3514.37.3.395.
from domains and facets of the five-factor model. Journal of Personality, 69,
199–222. doi:10.1111/1467-6494.00142.