JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2020, 53, 25–43 NUMBER 1 (WINTER)

Evaluating the boundaries of analytic efficiency and control: A

consecutive controlled case series of 26 functional analyses
JOSHUA JESSEL
QUEENS COLLEGE

RACHEL METRAS
UNIVERSITY OF NORTH TEXAS

GREGORY P. HANLEY
WESTERN NEW ENGLAND UNIVERSITY

CATHERINE JESSEL
UNIVERSITY OF NORTH TEXAS

EINAR T. INGVARSSON
UNIVERSITY OF NORTH TEXAS & CHILD STUDY CENTER

We conducted this study to determine if the efficiency of the functional analysis could be
improved without detrimental effects on control. In Experiment 1, we reanalyzed functional
analyses conducted for the problem behavior of 18 children. We analyzed rates of problem
behavior during the first 5 min and first 3 min of the original 10-min sessions and evaluated if
changes in the level of control over problem behavior by the programmed contingency were evi-
dent from the analyses of shorter session duration. In Experiment 2, we conducted 8 consecutive
functional analyses with 3-min sessions to further evaluate the utility of brief session durations.
We found that control over problem behavior was demonstrated when conducting functional
analyses with sessions as brief as 3 min.
Key words: analytic control, efficiency, functional analysis, problem behavior

Functional analysis was designed to improve reinforcement and eliminated when those same
the treatment of severe problem behavior by reinforcers are provided noncontingently in the
first demonstrating control over problem control condition (Hanley, 2012). Researchers
behavior by the suspected maintaining environ- have evaluated the results of behavior analytic
mental variables (Hanley, Iwata, & McCord, treatments in multiple quantitative reviews and
2003). Control during a functional analysis is have found that an overall greater reduction in
demonstrated when problem behavior is reli- problem behavior was achieved when treatment
ably evoked during a condition of contingent procedures were informed by a functional anal-
ysis (Campbell, 2003; Heyvaert, Saenen,
Rachel Metras is now at Western New England Univer- Campbell, Maes, & Onghena, 2014; Kahng,
sity. Catherine Jessel is now at Long Island ABA. Einar
T. Ingvarsson is now at the Virginia Institute for Autism. Iwata, & Lewin, 2002).
We would like to thank Holly Gover, Laura Hanratty, In spite of this empirical support for a pre-
Robin Landa, Adithyan Rajaraman, Kelsey Ruppel, and treatment functional analysis, as well as the
Jessica Slaton for their expert contributions to the devel-
opment of this manuscript. numerous replications in the literature
Address correspondence to: Joshua Jessel, Queens Col- (Beavers, Iwata, & Lerman, 2013; Hanley
lege, Department of Psychology, 65-30 Kissena Blvd., et al., 2003), professional behavior analysts
Queens, NY 11367. Email: [email protected]
doi: 10.1002/jaba.544 have reported using less effective functional
© 2019 Society for the Experimental Analysis of Behavior
25
26 JOSHUA JESSEL et al.

assessment tools in their practice (Oliver, not negatively influenced by looking at only
Pratt, & Normand, 2015; Roscoe, Phillips, the first 10 minutes of each session. With
Kelley, Farber, & Dube, 2015). Oliver et al. 5-min sessions, lack of correspondence
(2015) retrieved online surveys from 682 prac- occurred in only three of the 46 analyses. Thus,
ticing Board Certified Behavior Analysts the authors suggested that sessions might be as
(BCBAs) and found that 90% of responders short as 5 min without detrimental effects on
indicated that they routinely used various func- the interpretations of functional control.
tional assessments. However, those functional Interpretation of functional control in a
assessments were likely to be indirect and single-subject design requires visual inspection
descriptive, with responders reporting their use of level, variability, and trends with repeated
71% and 83% of the time, respectively. This measures (Bourret & Pietras, 2013). Multiple
and other similar reports (e.g., Love, Carr, models have been developed to aid in the visual
Almason, & Petursdottir, 2009; Roscoe et al., analysis process (e.g., Fisher, Kelley, & Lomas,
2015) suggest that the functional analysis has 2003; Hagopian et al., 1997; Roane, Fisher,
paradoxically become a widely researched tool Kelley, Mevers, & Bouxsein, 2013), often
for identifying the environmental determinants focusing on the interpretation of the control
of problem behaviors (Beavers et al., 2013; within functional analyses. Roane et al. (2013)
Hanley et al., 2003), while being sparsely used modified previously developed structured cri-
by practitioners and clinicians. teria (Hagopian et al., 1997) to improve the
Oliver et al. (2015) attempted to identify the interpretation of functional analyses of varying
reason for this gap between research and prac- lengths. The participants ranged from postbac-
tice by asking responders to identify barriers to calaureate to postdoctoral behavior analysts,
the use of functional analysis. By and large, and their binary (yes or no) interpretations of
responders reported that lack of time for con- functional control were compared to those of
ducting functional analyses served as their big- expert judges. The structured criteria for visual
gest barrier. The time required to conduct a inspection involved calculating two criterion
functional analysis (i.e., analytic efficiency) has lines: The upper criterion line was set at one
been commonly recognized as a limitation of standard deviation above the mean rate of
the functional analysis (e.g., Cooper, Heron, & problem behavior during the control condition
Heward, 2007; Repp, Singh, Olinger, & and the lower criterion line was set at one stan-
Olson, 1990), with some reporting functional dard deviation below the mean rate. The func-
analysis to require a mean of 6.5 hr and a range tional analysis sessions were 10 min and each
of up to 16.5 hr to conduct (Iwata functional analysis was determined to have con-
et al., 1994). trol if more than half of the sessions in the test
Wallace and Iwata (1999) reanalyzed condition fell above the upper criterion line
46 functional analyses conducted with 15-min rather than below the lower criterion line.
sessions. The results of the first 5 and 10 min Roane et al. found that agreement coefficients
of the 15-min sessions were evaluated sepa- between the participants and the experts were
rately to gauge the level of correspondence in only above .9 when the participants used the
identified functions between shorter and longer structured criteria as opposed to subjective
sessions. A group of six doctoral students with judgments. Evaluations of the possible variabil-
experience conducting and interpreting func- ity imposed by shorter session durations may
tional analyses then made judgements of con- be enhanced by incorporating structured cri-
trol without knowing the session duration in teria in addition to judgements made by an
each instance. Identification of function was expert panel.
 BOUNDARIES OF EFFICIENCY & CONTROL 27

Current structured criteria for visual analysis from original 10-min functional analysis ses-
might not be sensitive to fluctuations in control sions to interpretations resulting from the first
introduced by relatively shorter functional anal- 3 and 5 min of the original 10-min sessions.
ysis sessions, especially when evaluating analyses Furthermore, the session duration evaluation
that tend to involve fewer data points (e.g., two was conducted with a recently described type
to three data points per condition as in Hanley, of functional analysis (Hanley et al., 2014),
Jin, Vanselow, & Hanratty, 2014; Jessel, Han- referred to as an interview-informed, synthe-
ley, & Ghaemmaghami, 2016). With shorter sized contingency analysis (IISCA; Jessel et al.,
sessions (e.g., 5 min) in brief analyses, experi- 2016). In addition to the binary judgements of
mental control could degrade due to the pro- trained panelists, we incorporated structured
portionally greater effect of variability, data criteria (Hagopian et al., 1997; Roane et al.,
overlap, and suboptimal trends. Nonparametric 2013) to produce a second set of binary judge-
statistics have been developed for describing the ments. Because improvements in efficiency
difference between baseline and treatment con- may result in degraded control rather than the
ditions (i.e., effect sizes) for single-subject absolute loss of functional control (Jessel et al.,
design research, but have not been used to aid 2016), we also describe and applied a multilevel
in the interpretation of functional analysis out- system involving differing degrees of control
comes. For example, the percentage of nono- (i.e., none, weak, moderate, strong), and
verlapping data (PND; Scruggs, Mastropieri, & applied the PND statistic to the same 18 ana-
Casto, 1987) is a conservative approach (Carr, lyses. In Experiment 2, we conducted eight
2015) that identifies the number of points dur- additional functional analyses with 3-min ses-
ing treatment that do not overlap with the sions to determine if the results of the reanaly-
highest baseline point and divides that sum of sis were consistent with outcomes of functional
nonoverlapping points by the total number of analyses relying on the briefest session
treatment points to provide an effect size duration.
between 0 and 100%. Effect sizes calculated
using simple statistics like PND have been
found to be useful in treatment efficacy reviews EXPERIMENT 1: REANALYSIS OF
(e.g., Campbell, 2003; Carr, Severtson, & Lep- 18 INTERVIEW-INFORMED
per, 2009; Heyvaert et al., 2014); however, it is SYNTHESIZED CONTINGENCY
unknown if PND effect sizes between the test ANALYSES
and control conditions of a functional analysis Because the purpose of this experiment was
would correspond to, or be more or less sensi- to evaluate improvements in efficiency, we con-
tive to, a structured criteria or visual analysis of ducted a functional analysis specifically
functional analysis data. designed to be quick and practical. Hanley
We attempted to extend the literature on et al. (2014) described a relatively brief func-
improving analytic efficiency by reanalyzing the tional analysis of problem behavior format with
data from 18 functional analyses of varying ses- three children diagnosed with autism. The
sion duration. Previous research (Wallace & analysis included a single test condition with
Iwata, 1999) included a minimum session synthesized establishing operations and synthe-
duration of 5 min extracted from traditional sized reinforcement contingencies (e.g., escape
functional analyses (Iwata et al., 1994) that to tangibles) designed to simulate typically
were originally conducted using 15-min ses- occurring antecedents and consequences of
sions. In the current study, we compared the problem behavior as reported by caregivers. A
interpretations of functional control resulting matched control condition, in which the same
28 JOSHUA JESSEL et al.

reinforcers were freely available, alternated with with the mode being nonverbal. Sessions were
the test sessions. In a study describing 30 repli- conducted by trained therapists in 3-m x 6-m
cations of this type of analysis, Jessel et al. and 3-m x 4-m treatment rooms.
(2016) reported that the mean analysis dura- Response measurement. Observers scored mul-
tion was 25 min, with some analyses involving tiple topographies of problem behavior, which
sessions as short as 3 min. In Experiment 1, we varied across participants (see Table 1). Com-
describe the results of 18 IISCAs that initially mon forms of problem behavior displayed
relied on 10-min sessions, and then extracted included aggression (e.g., hitting, kicking,
samples from the beginning of each session so scratching, biting others), self-injurious behav-
that outcomes from relatively shorter sessions ior (e.g., hitting, scratching, biting self), disrup-
could be reanalyzed. tion (e.g., tearing, throwing, hitting items),
tantrums (e.g., dropping and crying or whining
for more than 30 s), and loud vocalizations
Method (e.g., yelling, screaming, swearing). Less com-
Participants and settings. All participants mon forms included inappropriate sexual
received assessment and treatment services for behavior (ISB) and elopement. All problem
their problem behavior in a 2-week outpatient behavior was measured as responses per minute
program supervised by the first and second and calculated by dividing the total frequency
authors of this study. The data from four of by the duration of the session. Duration of
the participants’ (Ari, Dace, Annie, and Joe) access to reinforcers was also measured. An
functional analyses were previously published observer scored the onset (offset) of reinforce-
in Jessel, Ingvarsson, Kirk, Whipple, and ment when (a) the antecedent variables
Metras (2018a). Their data were included in intended to evoke problem behavior were
the current reanalysis because their IISCAs removed (presented), (b) the consequent vari-
were conducted with 10-min sessions. A fifth ables were presented (removed), or (c) both
participant (Aaron) for whom 10-min sessions (a) and (b) in cases in which positive and nega-
were also conducted in Jessel et al. was not tive reinforcement was synthesized.
included in the current experiment because the Interobserver and interrater agreement. Two
delay from the evocative situation observers independently scored live sessions or
(i.e., transitioning to and from different rooms) videotaped sessions for each IISCA. The second
during the test condition of his functional anal- observer scored a mean of 40% of the sessions
ysis precluded the type of reanalysis applied in (range, 20% to 67%). We segmented each ses-
the current experiment. The remaining partici- sion into 10 s intervals and calculated percent-
pants were the next 14 consecutive clients age of agreement by dividing the smaller value
admitted to a clinic for this consecutively con- by the larger value in each interval, adding the
trolled case-series (Rooker, Jessel, Kurtz, & proportions for all intervals together, dividing
Hagopian, 2013). The participant characteris- by the total number of intervals in each session,
tics are presented in Table 1. The median age and multiplying by 100. The mean IOA for
was 7.5 years and the majority was male problem behavior and reinforcer access across
(16 males, 2 females). Sixteen of the partici- the 18 analyses was 99% (range, 95% to
pants had a diagnosis of autism, with many 100%) and 97% (range, 87% to 100%),
having additional diagnoses of intellectual dis- respectively.
ability and/or attention deficit hyperactivity dis- To evaluate reliability of the evaluations of
order (ADHD). The language abilities of the functional control, a second rater used the
participants ranged from nonvocal to fluent, structured criteria developed by Hagopian et al.
 BOUNDARIES OF EFFICIENCY & CONTROL 29

Table 1
Characteristics of Experiment 1 Participants

Participant Characteristics
Participant Age Sex Diagnosis Language Ability* Problem Behavior
Andy 11 M ASD, ID 1 Agg, dis, tantrums
Max 7 M ASD, ADHD 4 Agg, dis, SIB, tantrums
Gene 10 M ASD 1 Agg, dis, SIB, loud voc, tantrums
Rina 2 F DS 2 Agg, SIB, tantrums
Eli 5 M ASD 3 Agg, loud voc, elope, tantrums
Nick 3 M ASD, ID 1 Agg, dis, loud voc, tantrums
Ari 15 M ASD, ID, ADHD 4 ISB
Jim 8 M ASD 1 Agg
Koa 2 M No diagnosis 2 Agg, dis, loud voc
Jiro 9 M ASD, ADHD 4 Agg, dis, loud voc, elope
Jin 8 M ASD, ID, ADHD 1 Agg, dis, SIB, disrobe, elope
Corey 9 M ASD, ID 2 Agg, dis, loud voc
Annie 5 F ASD, ADHD, ID 3 Tantrums
Tiff 6 M ASD 1 Agg
Dace 11 M ASD, ID 2 Agg, dis, SIB, tantrums
Job 5 M ASD, GAD 2 Dis, SIB, tantrums
Smith 15 M ASD 4 Agg, dis, loud voc
Joe 5 M ASD 1 Agg, dis, tantrums

Note. ASD is autism spectrum disorder. ID is intellectual disability. ADHD is attention deficit/hyperactivity disorder.
GAD is generalized anxiety disorder. DS is Down syndrome. SIB is self-injurious behavior. ISB is inappropriate sexual
behavior.
* 1 = non-verbal; 2 = 1-word utterances; 3 = short diffluent sentences; 4 = full fluency.

(1997) and modified by Roane et al. (2013), alternated (i.e., multielement design). The
and independently rated all of the 54 analyses usual number of sessions was five (i.e., control,
(18 analyses at three different durations each). test, control, test, test), but additional sessions
An agreement was scored if the primary and were conducted when differentiation was not
secondary raters both scored an analysis as hav- evident from the five-session analysis.
ing or not having control. Otherwise, a dis- Procedure. The functional assessment proce-
agreement was scored. The agreements were dures were similar to those described by Hanley
summed and divided by the total number of et al. (2014) and involved an interview, brief
analyses evaluated by both raters; IOA informal contingency probe, and functional
was 100%. analysis. The functional assessment process was
The secondary rater also scored levels of con- designed to discover and then evaluate a single,
trol using the multileveled criteria described contextually-relevant reinforcement contin-
below for all 54 analyses. An agreement was gency for each participant (i.e., evaluating sen-
scored if the level of control (i.e., none, weak, sitivity to distinct classes of reinforcement was
moderate, strong) for an analysis recorded by not the aim of the functional assessment
the primary rater matched that of the secondary process).
rater and a disagreement was scored if the First, the therapist conducted an open-ended
recordings did not match. The number of interview with the caregivers to identify prob-
agreements was divided by the total number of lem behaviors that co-occurred with the prob-
scorings, and IOA was 100%. lem behavior of primary concern, and to
Experimental design. The test and control identify the antecedent and consequent events
conditions of the IISCAs were rapidly that seemed relevant to the problem behavior
30 JOSHUA JESSEL et al.

(see appendix in Hanley, 2012, for the inter- to refine the operational definitions of problem
view). The therapist used the questions from behavior and sometimes enhanced understand-
the interview as a guide and asked only ques- ing of the situations that evoked problem
tions in each section until enough categorical behavior. For example, Rina’s caregivers
information was obtained for the therapist to reported that Rina enjoyed playing with others
(a) operationally define problem behavior and with particular leisure activities. However, if
(b) design the test condition of the functional the playmate were to tell Rina that playtime
analysis. No modifications were made to the was finished and to come with them to a differ-
open-ended interview; however, the therapist ent location, Rina would exhibit problem
did ask follow-up questions to have the care- behavior. The experimenter would then have
givers expand on an answer as needed. The the caregivers watch while the therapist allowed
caregivers were asked questions regarding the Rina to choose an activity and played with her,
specific topographies of problem behavior, and periodically told her the “toys were all fin-
which problem behavior they found most con- ished” and prompted her to leave the area. If
cerning, and which problem behaviors were problem behavior were to have occurred, the
likely to co-occur. The therapist used the transition would have been discontinued and
responses to identify the topographies of prob- the interactive play reinstated. When problem
lem behavior that were of utmost concern and behavior did not occur during those situations,
those that appeared to be functionally related, the caregivers were asked for additional infor-
but less severe. mation to help make the situation more similar
The information obtained from caregivers to the situations occurring in the home
during the interview regarding the antecedent (e.g., suggestions on the specific instructions,
and consequent events was used to develop a suggestions on preferred activities). In other
unique contingency reflecting the situations in words, the probe was used to calibrate the eco-
which problem behavior was reportedly likely logical precision of the contingency before the
to occur for each participant. In other words, systematic evaluation during the IISCA. The
general classes of reinforcement were not evalu- contingency probe lasted from 5 to 30 min
ated and only a single test condition of a syn- across participants and continued until the
thesized contingency implicated by the therapists felt they had enough information
caregivers was included in the subsequent anal- from the interview and contingency probe to
ysis. Had the information provided during the conduct the analysis.
interview implicated multiple contexts of dispa- Third, the therapist conducted the IISCA.
rate contingencies (e.g., problem behavior The putative reinforcers were provided for 30-s
maintained by access to interactive play at access following each instance of problem
home and escape from independent work com- behavior in the test condition and continuously
pletion at school), multiple IISCAs would have in the control condition. The specific rein-
been conducted specifically evaluating each forcers included in the test and control condi-
unique behavior–environment relation. How- tions for each participant are presented in
ever, this was not reported for any of the partic- Table 2. Sixteen of the 18 analyses included
ipants. The interview required 15 to 30 min to the synthesis of positive and negative reinforce-
conduct. ment. The other two analyses included a syn-
Second, the therapist conducted a brief con- thesis (Nick) or isolated (Ari) form of only
tingency probe in the reportedly problematic positive reinforcement. In addition, the number
context. Data were not collected during this of functional analysis iterations required before
time. The contingency probe sometimes helped differentiated results were obtained is included
 BOUNDARIES OF EFFICIENCY & CONTROL 31

Table 2
Experiment 1 Functional Analysis Information

Participant Iteration Test Condition Control Condition

Andy First Escape from adult-directed play to free play with Continuous free play with mom and no adult
mom directions
Max First Escape from academic instructions to independent Continuous independent play and no academic
play instructions
Gene First Escape from instructions to independent play Continuous independent play and no instructions
Rina First Escape from transitions to interactive play Continuous interactive play in the preferred location
Eli First Escape from instructions to interactive play Continuous interactive play and no instructions
Nick First Access to interactive play Continuous interactive play
Ari First Access to iPad Continuous access to iPad
Jim Second Escape from diverted attention to interactive play Continuous interactive play with full attention
Koa First Escape from transitions to interactive play with Continuous interactive play with mom in the
mom preferred location
Jiro First Escape from caregiver instructions to independent Continuous independent play and no caregiver
play instructions
Jin First Escape from gross motor instructions to interactive Continuous interactive play and no gross motor
play instructions
Corey First Escape from adult access to independent access of Continuous independent access of iPad
iPad
Annie First Escape from adult-directed to child-directed play Continuous child-directed play and no adult
directions
Tiff First Escape from blocked access to free access to leisure Continuous free access to leisure items
items
Dace Second Escape from instructions to independent play Continuous independent play and no instructions
Job Third Escape from caregiver-directed play to Continuous child-directed play and no caregiver
child-directed play directions
Smith Second Escape from academic instructions to independent Continuous independent play and no academic
play instructions
Joe First Escape from transitions to iPad Continuous access to iPad in the preferred location

in Table 2. If the IISCA failed to implicate a problem behavior from the first 5 min of each
socially mediated function, the therapist 10-min session and dividing that total by five.
returned to asking the caregivers more open- The rate of problem behavior from the 3-min
ended questions about potential discrepancies IISCA reanalysis was calculated by only includ-
between the IISCA and their typical experience. ing the problem behavior from the first 3 min
The information was used to modify the con- of each 10-min session and dividing that total
tingency probe and functional analysis. Four- by three. We used four methods to evaluate
teen out of 18 analyses (78%) did not require functional control across the analyses of varying
additional iterations (i.e., redesigned analyses), session duration. We also calculated the analysis
whereas three participants (17%) required one duration of each application by multiplying the
set of modifications to the analysis, and one session duration (i.e., 3 min, 5 min, 10 min)
required two sets of modifications to the by the number of sessions conducted. We con-
analysis. sidered five sessions of 3-min each, creating an
Data analysis. We reanalyzed the data from analysis duration of 15 min, to be the most
the first 5 and 3 min of each 10-min session to efficient possible application.
evaluate the impact of shorter sessions on the Binary panelist criterion. We developed a cri-
interpretation of behavioral function. We calcu- terion of agreement between multiple panelists
lated the rate of problem behavior during the to determine if functional control was demon-
5-min IISCA reanalysis by only including the strated in a given functional analysis, similar to
32 JOSHUA JESSEL et al.

Wallace and Iwata (1999). The panel consisted number of data points below the lower criterion
of six BCBAs who were doctoral students with line from the number of data points above the
experience conducting research in functional upper criterion line and divided the difference
analysis. In addition, the panelists completed by the total number of test sessions. Any func-
training on visual analysis prior to their review tional analysis with a quotient at or above 50%
of the data. The visual analysis training was not was considered to be differentiated and to have
specific to the IISCA and included the review control. If the mean rate of problem behavior
of multiple strategies on evaluating functional during the control condition was 0, the upper
control in different experimental designs. The and lower criterion lines were set at 0. During
data from each of the 18 IISCAs were depicted these cases, 50% of the test sessions had to be
in three different ways (i.e., data from 3-min, above 0 to be considered differentiated.
5-min, and 10-min sessions). Therefore, the Multilevel structured criteria. The multile-
BCBAs conducted a total of 54 evaluations of veled structured criteria were considered to be
functional control. The BCBAs were sent an an extension of the binary structured criteria
email with a PowerPoint® and were provided developed by Hagopian et al. (1997). We cate-
with the instructions: gorized the IISCAs according to four levels of
control: strong, moderate, weak, and none (see
Each slide will contain the results of an
examples in Figure 1). The IISCAs categorized
individual functional analysis. Visually
as having strong control did not have any over-
inspect each individual data set and indi-
cate in the notes section whether you lap in the data across the test and control con-
believe the results support the identifica- ditions and did not have any occurrences of
tion of a functional relation or if you problem behavior in the control condition
believe the results are ambiguous (yes if (i.e., immediate, sustained differentiation
functional relation, no if ambiguous). between the test and control condition). The
Complete your review independently. IISCAs that were categorized as having moder-
ate control had some overlap in the data across
The 54 IISCAs of varying session durations the test and control conditions or some occur-
were presented in a mixed order, and there was rences of problem behavior in the control con-
no indication of session duration. We set the dition. The IISCAs that were categorized as
criterion for demonstration of functional con- having weak control had some overlap in the
trol as at least five or six of the ratings data across the test and control conditions and
indicating yes. some occurrences of problem behavior in the
Binary structured criteria. The structured cri- control condition. Lastly, the IISCAs with sub-
teria were developed by Hagopian et al. (1997) stantial overlap in data paths or that failed to
and modified by Roane et al. (2013). We replicate effects in an experimental design were
examined each figure individually and used the categorized as having no control.
rate of problem behavior during the control Percentage of non-overlapping points. To cal-
condition to calculate an upper criterion line culate PND, we counted the number of data
(i.e., one standard deviation above the mean) points in the test condition that were above, and
and a lower criterion line (i.e., one standard did not overlap with, the highest datum in the
deviation below the mean). We then counted control condition. This sum was then divided by
the number of test sessions that fell above the the total number of test sessions conducted and
upper criterion line and the number of test ses- multiplied by 100 to get a percentage. A PND
sions that fell below the lower criterion line. To ranging from 0 to 100% was calculated for each
calculate a percentage, we subtracted the IISCA.
 BOUNDARIES OF EFFICIENCY & CONTROL 33

3 condition across participants in the 10-min

Y Y
S 100
(M = 1.2 RPM; SD = 0.4), 5-min (M = 1.0
2 RPM; SD = 0.5), and 3-min analyses (M = 1.0
RPM; SD = 0.5). The mean analysis duration
1 Test of the 10-min, 5-min, and 3-min IISCAs was
Control 56 min, 28 min, and 17 min, respectively.
0 The results of the binary evaluations of func-
3
tional control are presented in the top half of
Y Y
the pie charts of each IISCA in the figures. The
M 80
2
evaluations completed using the panelist crite-
rion (top right quadrant of pie charts) resulted
Problem behavior per min

1 Legend
in an overall high percentage of IISCAs desig-
nated as demonstrating functional control;
SC PC however, there was a slight decrease in demon-
0
Lvl PND stration of control when the analyses included
3
Y Y briefer sessions. All 18 IISCAs with the full
2
W 60 10-min sessions were identified as having con-
trol, whereas 94% and 83% of the 5-min and
1
3-min IISCAs, respectively, met the panelist
criterion.
The evaluations completed using the binary
0
structured criteria are presented in the top left
3
N N quadrant of the pie charts. The binary struc-
NC 20 tured criteria resulted in 100% of the analyses
2
being designated as demonstrating functional
control, regardless of session duration.
1
The results of the multilevel evaluation of
control are presented in the bottom half of the
0
2 4 6 8 10
pie charts in each IISCA. The majority of the
Sessions IISCAs were determined to have strong control,
with the percentage decreasing across the
Figure 1. Hypothetical data of functional analyses 10-min (89%), 5-min (78%), and 3-min ana-
meeting criteria of strong (no overlap and zero occur- lyses (67%). Similar percentage of analyses with
rences in the control condition), moderate (some overlap moderate control were obtained from the
or some occurrences in the control condition), weak (some
10-min (11%) to the 5-min (11%) and 3-min
overlap and some occurrences in the control), and no con-
trol (substantial overlap). SC refers to structured criteria (17%) analyses. There was only one IISCA that
interpretation of control (Y = Yes, N = No). PC refers to had weak control (Tiff, 5-min sessions). The
panelist criterion. Lvl refers to the level of control shorter durations also resulted in one 5-min
(S = strong, M = moderate, W = weak, NC = no con- IISCA (6%) and three 3-min IISCAs (17%)
trol). PND refers to percentage of nonoverlapping data. being designated as having no control.
Similar patterns in PND effect sizes (bottom
Results and Discussion right quadrant of pie charts) were obtained
The results for all 18 participants are pre- with minimal overlap in the test and control
sented in Figures 2–4. Elevated rates of prob- conditions during the 10-min analyses
lem behavior were observed during the test (M = 98%; SD = 6.8) and a decrease in PND
34 JOSHUA JESSEL et al.

10 min 5 min 3 min

3 Test
Control
2
Andy
1
Y Y Y Y Y Y
Legend
S 100 S 100 S 100
0
SC PC
3
Y Y Y Y Y Y Lvl PND
S 100 S 100 S 100
2
Max
1

2.0
Y Y Y Y Y Y
1.5 S 100 S 100 S 100
Problem behavior per min

1.0 Gene
0.5

0.0

2.0

1.5

1.0 Rina
0.5 Y Y Y Y Y Y
S 100 S 100 S 100
0

2.0

1.5

1.0 Eli
0.5 Y Y Y Y Y Y
S 100 S 100 S 100
0

2.0

1.5

1.0
Nick
0.5 Y Y Y Y Y Y
S 100 S 100 S 100
0
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Sessions

Figure 2. The reanalysis of the first 3 min (right) and 5 min (middle) of a 10-min IISCA (left) for six participants
from Experiment 1. SC refers to structured criteria interpretation of control (Y = Yes, N = No). PC refers to panelist
criterion. Lvl refers to the level of control (S = strong, M = moderate, W = weak, NC = no control). PND refers to per-
centage of nonoverlapping data.
 BOUNDARIES OF EFFICIENCY & CONTROL 35

10 min 5 min 3 min

2.0
Test
1.5 Control

1.0 Ari
0.5 Y Y Y Y Y Y Legend
S 100 S 100 S 100
0.0 SC PC
2.0
Lvl PND
1.5

1.0 Jim
0.5 Y Y Y Y Y Y
S 100 S 100 S 100
0.0

2.0

1.5
Problem behavior per min

1.0 Koa
0.5 Y Y Y Y Y Y
S 100 S 100 S 100
0.0
2.0
Y Y Y Y Y Y
1.5 S 100 S 100 S 100

1.0 Jiro
0.5

0.0
2.0
Y Y Y Y Y Y
1.5 S 100 S 100 S 100

1.0 Jin
0.5

0
2.5
Y Y Y Y Y Y
2.0 S 100 S 100 M 75
1.5
1.0 Corey
0.5
0
2 4 6 2 4 6 2 4 6
Sessions

Figure 3. The reanalysis of the first 3 min (right) and 5 min (middle) of a 10-min IISCA (left) for six additional
Experiment 1 participants. SC refers to the structured criteria interpretation of control (Y = Yes, N = No). PC refers to
panelist criterion. Lvl refers to the level of control (S = strong, M = moderate, W = weak, NC = no control). PND
refers to percentage of non-overlapping data.
36 JOSHUA JESSEL et al.

10 min 5 min 3 min

1.0
Test
0.8 Control
0.6
Annie
0.4
0.2 Y Y Y Y Y Y Legend
S 100 M 75 M 75
0
SC PC
1.0
0.8 Lvl PND
0.6
Tiff
0.4
Y Y Y Y Y Y
0.2
M 100 W 67 S 100
0
2.0
Y Y Y Y Y Y
1.5 M 71 M 57 M 57
Pro blem behavior per min

1.0 Dace
0.5

0
2.0

1.5

1.0 Job
0.5 Y Y Y Y Y N
S 100 S 100 NC 67
0
2.0

1.5

1.0 Smith
0.5 Y Y Y Y Y N
S 100 S 100 NC 67
0
1.0
0.8
0.6
0.4 Joe
Y Y Y N Y N
0.2
S 100 NC 75 NC 75
0
2 4 6 8 10 2 4 6 8 10 2 4 6 8 10
Sessions

Figure 4. The reanalysis of the first 3 min (right) and 5 min (middle) of a 10-min IISCA (left) for six additional
Experiment 1 participants. SC refers to the structured criteria interpretation of control (Y = Yes, N = No). PC refers to
panelist criterion. Lvl refers to the level of control (S = strong, M = moderate, W = weak, NC = no control). PND
refers to percentage of nonoverlapping data.
 BOUNDARIES OF EFFICIENCY & CONTROL 37

during the 5-min (M = 93%; SD = 14.0) and time windows (e.g., the last 5 or 7 min in each
3-min analyses (M = 90%; SD = 15.4). session) can be removed from the data analysis,
The reanalysis of 5-min and 3-min IISCAs the participants still experienced the contingen-
indicates that shorter sessions can be conducted cies in effect during these time windows, and
with a high likelihood of obtaining differenti- this history likely influenced responding in sub-
ated results, independent of the manner in sequent sessions. Differentiated outcomes with
which the data were analyzed. Furthermore, the the briefer session durations may thus be
effects of reducing session duration on the level inflated due to extended contact with the
of control were minimal. These findings sug- arranged contingencies. In Experiment 2, we
gest that a functional analysis can be efficient conducted additional IISCAs with 3-min ses-
without sacrificing control, which is important sions, with additional clients, to address this
for clinicians who may have time constraints. limitation.
Based on these results, a practitioner may
obtain differentiated outcomes in as little as Method
15 min and the entire assessment period Participants and settings. The first eight con-
(i.e., interview through IISCA) could be com- secutive clients who were admitted to the out-
pleted within 35 min under optimal patient clinic following the 18 participants
circumstances. from Experiment 1 served as participants in
Experiment 2. Their median age was 6 years,
and seven participants were boys. Additional
EXPERIMENT 2: EIGHT REPLICATIONS participant characteristics can be located in
OF 3-MIN IISCAS Table 3.
In Experiment 1, we reanalyzed data from Response measurement and interobserver agree-
IISCAs conducted with 10-min sessions to ment. We calculated the rate of problem behav-
determine if brief session durations would affect ior per minute across sessions during the
the interpretability of the results. However, the IISCA. We calculated IOA for problem behav-
reanalysis in Experiment 1 shares the same lim- ior and reinforcement during a mean of 40%
itation as the reanalysis of session durations of (range, 20% to 60%) of each IISCA applica-
the standard functional analysis conducted by tion using a partial agreement coefficient (see
Wallace and Iwata (1999). Whereas particular Experiment 1). The mean IOA obtained for

Table 3
Characteristics of Experiment 2 Participants

Participant Characteristics
Participant Age Sex Diagnosis Language Ability* Problem Behavior
Omi 8 M ASD, ID 3 Agg, dis, SIB, tantrums
Hina 4 F ASD, ID 2 Agg, tantrums, SIB, loud voc
Levi 4 M ASD, GAD 4 Tantrums, loud voc
Dan 4 M ASD, ID 3 Agg, dis, tantrums, loud voc
Matt 10 M ASD, ID, FASD, ADHD 4 Agg, dis, loud voc
Lei 8 M ADHD 4 Agg, dis, tantrums, loud voc
Val 3 M ASD, ID 1 Agg, dis, tantrums, SIB, undressing, loud voc
Duke 13 M ASD 4 Agg, dis, tantrums, loud voc

Note. ASD is autism spectrum disorder. ID is intellectual disability. ADHD is attention deficit/hyperactivity disorder. GAD is
generalized anxiety disorder. DS is Down syndrome. SIB is self-injurious behavior. ISB is inappropriate sexual behavior.
* 1 = non-verbal; 2 = 1-word utterances; 3 = short diffluent sentences; 4 = full fluency.
38 JOSHUA JESSEL et al.

Table 4 of each IISCA was conducted in rapid alterna-

Experiment 2 Functional Analysis Information tion with the control condition in a multiele-
ment design. Additional information on the
Control
Participant Iteration Test Condition Condition individual contingencies and the number of
Omi First Escape from Continuous
analysis iterations for each participant can be
adult access to independent found in Table 4. Seven of the eight analyses
independent access of iPad included the synthesis of positive and negative
access of iPad
Hina First Escape from Continuous reinforcement, whereas only a single analysis
adult-directed child-directed evaluated positive reinforcement alone. No ana-
to play and no
child-directed adult lyses required modifications and differentiated
play directions results were obtained on the first attempt
Levi First Escape from Continuous
adult-directed independent for all.
to play and no Data analysis. We used the same two sets of
independent adult
play directions
binary criteria (i.e., panelist and structured)
Dan First Escape from Continuous from Experiment 1 to determine if the IISCA
adult-directed child-directed had control. For the panelist criterion, we gave
to play and no
child-directed adult the six trained panelists new slides including
play directions each individual IISCA. We evaluated the
Matt First Access to water Continuous
play independent degree of control using the same multilevel cri-
water play teria as in Experiment 1 that provided four pos-
Lei First Escape from Continuous
difficult child-directed sible outcomes: strong, moderate, weak, and no
academic play and no control. Lastly, we calculated effect sizes for
instructions to academic
child-directed instructions
each IISCA using PND.
play
Val First Escape from Continuous free
blocked access access to
to free access leisure items Results and Discussion
to leisure and snacks The results of the eight IISCAs are presented
items and
snacks in Figure 5. Elevated rates of problem behavior
Duke First Escape from Continuous were observed during the test condition
adult-directed child-directed
to math (M = 1.62 RPM; SD = 1.04) in comparison to
child-directed completion the control condition (M = 0.02 RPM; SD =
math with no adult
completion directions
0.08). Duke was the only participant to have
any overlapping data points. In addition, prob-
lem behavior was completely eliminated during
problem behavior and reinforcement was 97% the control condition across participants with
(range, 92% to 100%) and 99% (range, 94% the exception of Val. The average total analysis
to 100%), respectively. In addition, we calcu- duration was 16.1 min.
lated interrater agreement from all of the IIS- The results of the binary evaluations of func-
CAs for the evaluation of control and level of tional control (top half of pie charts) were simi-
control. The interrater agreement was 100% lar to that of Experiment 1. All but one IISCA
for both analyses. (88%) were considered to show functional con-
Procedure and design. The IISCA procedures trol using the panelist criterion (top right quad-
were identical to those of Experiment 1 with rant of pie charts), whereas all eight IISCAs
the exception that session durations were were determined to have control based on the
3 min for each participant. The test condition structured criteria (top left quadrant of pie
 BOUNDARIES OF EFFICIENCY & CONTROL 39

4 4 4
Y Y Y Y
Test
3 S 100 3 3 S 100 Control

2 2 Y Y 2
S 100
1 1 1
P roblem behavior per min

0 Omi 0 Hina 0 Levi

2.0 2.0 3
Y Y Y Y Y Y
1.5 S 100 1.5 S 100 S 100
2
1.0 1.0
1
0.5 0.5

0 Dan 0 Matt 0 Lei

1 2 3 4 5
1.5 3
Y Y Y N
M 100 NC 75 Legend
1.0 2
SC PC
0.5 1
Lvl PND
0 Val 0 Duke
2 4 6 2 4 6
Sessions
Figure 5. The results of the 3 min analyses for the eight participants of Experiment 2. SC refers to the structured
criteria interpretation of control (Y = Yes, N = No). PC refers to panelist criterion. Lvl refers to the level of control
(S = strong, M = moderate, W = weak, NC = no control). PND refers to percentage of nonoverlapping data.

charts). The evaluation of levels of control represented in the excised data. Based on these
using the multilevel structured criteria identi- results, we suggest that practitioners might
fied six IISCAs (75%) with strong control, one obtain useful assessment results using the
IISCA with moderate control, and one IISCA IISCA with 3-min sessions in most cases.
with no control. All but one of the PND statis-
tics was 100%. Duke’s data set had minimal
overlap (PND = 75%). GENERAL DISCUSSION
The majority of the 3-min IISCAs resulted Some boundaries of analytic efficiency and
in differentiated analyses. In addition, those control were evaluated in the current investiga-
that were differentiated generally had strong tion. A decrease in functional analysis session
demonstrations of experimental control. These duration to as short as 3 min had limited
results were similar to those of Experiment impact on interpretations of functional control
1, suggesting that the control shown in the (see Figure 6 for a summary of Experiments
3-min analyses of Experiment 1 were not 1 and 2). Overall, there was no detriment to
largely influenced by participant experiences experimental control in 67% of the IISCAs,
40 JOSHUA JESSEL et al.

Andy Experiments 1 and 2 (Figure 7) was generally

Level of Control
Max high. The two binary criteria (top panel) had a
Gene Strong
Rina Moderate high level of correspondence with only 8% dis-
Eli Weak agreement. The disagreements were specific to
Nick the panelist criterion and may be an artifact of
Ari None
Jim how the conditions during the IISCA were
Koa arranged. The panelists reported that control
Jiro
Jin
could not be properly identified because differ-
Corey entiation was only observed after two control
Participants

Annie sessions and the first test session had been con-
Tiff
Dace ducted. Thus, by only looking at the first
Job 3 min in each condition, one cannot rule out
Smith the possibility that an unknown confounding
Joe Study 1
Omi variable influenced problem behavior across ses-
Hina sions, independent of the control and test con-
Levi
Dan
ditions. The structured criteria were insensitive
Matt to this important aspect of experimental design.
Lei Interestingly, both the panelist criterion and
Val
Duke Study 2 multilevel criteria of control agreed with the
10-min 5-min 3-min panelist determinations of lack of control in all
Session Duration five analyses.
High levels of correspondence in the multile-
Figure 6. Summary of outcomes across all participants vel structured criteria and PND (bottom panel)
from Experiments 1 and 2 regarding level of control were also observed, with 96% of the IISCAs
(strong, moderate, weak, none) across session durations. that had no overlapping data being interpreted
as having strong levels of control. Only one
and only minimal detriments in the level of IISCA was interpreted as having weak control
control when reducing the sessions to 3 min and although this corresponded to a smaller
(17%) and 5 min (11%). The findings from PND, smaller PNDs were obtained in two IIS-
Experiment 1 were supported by the eight con- CAs that had moderate control. These minor
secutive IISCAs conducted with 3-min sessions disagreements may be due to the multilevel cri-
in Experiment 2. The majority of the eight IIS- teria being a more stringent determinant of
CAs (75%) were considered to have strong fluctuations in control (e.g., for the two dis-
control. Based on these results, we suggest that agreements, the multilevel criteria underrated
practitioners may be able to conduct functional the obtained PND).
analyses with sessions as brief as 3 min with Both the multilevel criteria and PND pro-
minimal negative impact on their interpreta- vided a more nuanced interpretation of control
tions of functional control. in comparison to the binary evaluations, and
The level of correspondence between each of both may be useful for predicting general thera-
the four criteria used to evaluate functional peutic outcomes of the function-based treat-
control across the 62 distinct analyses1 from ments. This study did not examine treatment
efforts, but treatments developed from and
1
There were 18 IISCAs analyzed at three different ses-
sion durations (i.e., 10-min, 5-min, 3-min) creating analyses with 3-min sessions were conducted in Experi-
54 distinct analyses in Experiment 1. Eight additional ment 2. This produces a total of 62 distinct analyses.
 BOUNDARIES OF EFFICIENCY & CONTROL 41

Yes
Structured
Criteria
No

No
Panelist
Criteria
Yes
Strong
Moderate
Levels
Weak
No Control

PND 50

100

10 20 30 40 50 60
Analyses

Figure 7. Summary of each of the four evaluations of functional control from Experiments 1 and 2. Each pair of
white and black bars represents an individual analysis. There were 54 distinct analyses in Experiment 1 (18 IISCAs each
represented in 10-min, 5-min, and 3-min sessions) and 8 analyses in Experiment 2, producing a total of 62 analyses.

informed by the IISCA have been indepen- The results of either study would indicate if
dently evaluated in previous research strong control during functional analyses is
(e.g., Beaulieu, Van Nostrand, Williams, & likely to correspond with better treatment
Herscovitch, 2018; Ghaemmaghami, Hanley, & outcomes.
Jessel, 2015; Hanley et al., 2014; Herman, The degree of experimental control obtained
Healy, & Lydon, 2018; Jessel, Ingvarsson, Kirk in a functional analysis is important because
et al., 2018; Jessel, Ingvarsson, Metras et al., inadequate control could lead to at least two
2018; Rose & Beaulieu, 2018; Santiago, Han- complications in subsequent treatment. First,
ley, Moore, & Jin, 2016; Slaton, Hanley, & weak control might be predictive of the use of
Raftery, 2017; Strand & Eldevik, 2018). Evalu- punishment or arbitrary reinforcement contin-
ating the treatment relevance of the PND and gencies because problem behavior continues to
multilevel criteria of control could involve two occur despite the elimination of the putative
different courses of investigation. establishing operation (Jessel et al., 2016). Sec-
First, this study could be replicated and ond, moderate control (e.g., functional analyses
extended to the point that each of the four with inconsistent levels of problem behavior)
evaluations of functional control are repre- may also make it challenging to identify reliable
sented by multiple participants. Subsequent evocative situations for differential reinforce-
function-based treatment outcomes could be ment. Therefore, strong demonstrations of con-
compared and the predictions noted above eval- trol (i.e., clear and stable differentiation in the
uated against those outcomes. Second, studies functional analysis) may be better for treatment
of existing function-based treatments can be planning than weaker demonstrations of con-
collated in a literature review with the multile- trol. Differentiation with variability allows for
vel criteria retrospectively applied to the func- the inference that a controlling variable has
tional analyses that informed those treatments. indeed been identified. However, variability
42 JOSHUA JESSEL et al.

also suggests there may exist other important implementation obstacles, and developing new lore.
controlling variables (Sidman, 1960). Behavior Analysis and Practice, 5, 54-72. https://doi.
org/10.1007/BF03391818
Hanley, G. P., Iwata, B. A., & McCord, B. E. (2003).
Functional analysis of problem behavior: A review.
REFERENCES Journal of Applied Behavior Analysis, 36, 147-185.
https://doi.org/10.1901/jaba.2003.36-147
Beaulieu, L., Van Nostrand, M. E., Williams, A. L, &
Herscovitch, B. (2018). Incorporating interview- Hanley, G. P., Jin, C. S., Vanselow, N. R., &
informed functional analyses into practice. Behavior Hanratty, L. A. (2014). Producing meaningful
Analysis in Practice, 11, 385-389. https://doi.org/10. improvements in problem behavior of children with
1007/s40617-018-0247-7 autism via synthesized analyses and treatments. Jour-
nal of Applied Behavior Analysis, 47, 16-36. https://
Beavers, G. A., Iwata, B. A., & Lerman, D. C. (2013).
doi.org/10.1002/jaba.106
Thirty years of research on the functional analysis of
problem behavior. Journal of Applied Behavior Herman, C., Healy, O., & Lydon, S. (2018). An
Analysis, 46, 1-21. https://doi.org/10.1002/jaba.30. interview-informed synthesized contingency analysis
Bourret, J. C., & Pietras, C. (2013). The five pillars of to inform the treatment of challenging behavior in a
the experimental analysis of behavior. In young child with autism. Developmental
G. J. Madden, W. V. Dube, T. D. Hackenberg, Neurorehabilitation, 21, 202-207. https://doi.org/10.
G. P. Hanley, & K. A. Lattal (Eds.), APA handbook 1080/17518423.2018.1437839
of behavior analysis (pp. 33-64). Washington, DC: Heyvaert, M., Saenen, L., Campbell, J. M., Maes, B., &
American Psychological Association. https://doi. Onghena, P. (2014). Efficacy of behavioral interven-
org/10.1037/13937-002 tions for reducing problem behavior in persons with
Campbell, J. M. (2003). Efficacy of behavioral interven- autism: An updated quantitative synthesis of single-
tions for reducing problem behavior in persons with subject design. Research in Developmental Disabilities,
autism: A quantitative synthesis of single-subject 35, 2463-2476. https://doi.org/10.1016/S0891-4222
research. Research in Developmental Disabilities, 24, (03)00014-3
120-138. doi.org/10.1016/S0891-4222(03)00014-3 Iwata, B. A., Pace, G. M., Dorsey, M. F., Zarcone, J. R.,
Carr, M. E. (2015). A sensitivity analysis of three non- Vollmer, T. R., Smith, R. G., . . . Willis, K. D.
parametric treatment effect scores for single-case (1994). The functions of self-injurious behavior: An
research for participants with autism. Review Journal experimental-epidemiological analysis. Journal of
of Autism and Developmental Disorders, 2, 67-78. Applied Behavior Analysis, 27, 215-240. https://doi.
https://doi.org/10.1007/s40489-014-0037-2 org/10.1901/jaba.1994.27-215
Carr, J. E., Severtson, J. M., & Lepper, T. L. (2009). Jessel, J., Hanley, G. P., & Ghaemmaghami, M. (2016).
Noncontingent reinforcement is an empirically sup- Interview-informed synthesized contingency analyses:
ported treatment for problem behavior exhibited by Thirty replications and reanalysis. Journal of Applied
individuals with developmental disabilities. Research Behavior Analysis, 49, 576-595. https://doi.org/10.
in Developmental Disabilities, 30, 44-57. 1002/jaba.316
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Jessel J., Ingvarsson, E. T., Kirk, H., Whipple, R., &
Applied behavior analysis (2nd ed.). Upper Saddle Metras, R. (2018a). Achieving socially significant
River, NJ: Pearson. reductions in problem behavior following the
Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). interview-informed synthesized contingency analysis:
Visual aids and structured criteria for improving A summary of 25 outpatient applications. Journal of
visual inspection and interpretation of single-case Applied Behavior Analysis, 51, 130-157. https://doi.
designs. Journal of Applied Behavior Analysis, 36, 387- org/10.1002/jaba.436
406. https://doi.org/10.1901/jaba.2003.36-387. Jessel, J., Ingvarsson, E. T., Metras, R., Whipple, R.,
Ghaemmaghami, M., Hanley, G. P., & Jessel, J. (2015). Kirk, H., & Solsbery, L. (2018b). Treatment of
Contingencies promote delay tolerance. Journal of elopement following a latency-based interview-
Applied Behavior Analysis, 49, 548-575. https://doi. informed, synthesized contingency analysis. Behav-
org/10.1002/jaba.333 ioral Interventions, 33, 271-283. https://doi.org/10.
Hagopian, L. P., Fisher, W. W., Thompson, R. H., 1002/bin.1525
Owen-DeSchryver, J., Iwata, B. A., & Wacker, D. P. Kahng, S. W., Iwata, B. A., & Lewin, A. B. (2002).
(1997). Toward the development of structured cri- Behavioral treatment of self-injury, 1964-2000.
teria for interpretation of functional analysis data. American Journal of Mental Retardation, 107, 212-
Journal of Applied Behavior Analysis, 30, 313-326. 221. https://doi.org/10.1901/jaba.1997.30-267
https://doi.org/10.1901/jaba.1997.30-313. Love, J. R., Carr, J. E., Almason, S. M., &
Hanley, G. P. (2012). Functional assessment of problem Petursdottir, A. I. (2009). Early and intensive behav-
behavior: Dispelling myths, overcoming ioral interventions for autism: A survey of clinical
 BOUNDARIES OF EFFICIENCY & CONTROL 43

practices. Research in Autism Spectrum Disorders, 3, Santiago, J. L., Hanley, G. P., Moore, K., & Jin, C. S.
421-428. https://doi.org/10.1016/j.rasd.2008.08.008 (2016). The generality of interview-informed
Oliver, A. C., Pratt, L. A., & Normand, M. P. (2015). A functional analyses: Systematic replications in school
survey of functional behavior assessment methods and home. Journal of Autism and Developmental
used by behavior analysts in practice. Journal of Disorders, 46, 797-811. https://doi.org/10.1007/
Applied Behavior Analysis, 48, 817-829. https://doi. s10803-015-2617-0
org/10.1002/jaba.256 Scruggs, T. E., Maestropieri, M. A., & Casto, G. (1987).
Repp, A. C., Singh, N. N., Olinger, E., & Olson, D. R. The quantitative synthesis of single-subject research:
(1990). The use of functional analyses to test causes Methodology and validation. Remedial and Special
of self-injurious behaviour: Rationale, current status Education, 8, 24-33. https://doi.org/10.
and future directions. Journal of Mental Deficiency 1177/074193258700800206
Research, 34, 95-105.
Roane, H. S., Fisher, W. W., Kelley, M. E., Sidman, M. (1960). Tactics of scientific research. US:
Mevers, J. L., & Bouxsein, K. J. (2013). Using modi- Authors Cooperative, Inc.
fied visual-inspection criteria to interpret functional Slaton, J. D., Hanley, G. P., & Raftery, K. J. (2017).
analysis outcomes. Journal of Applied Behavior Analysis, Interview-informed functional analyses: A comparison
46, 130-146. https://doi.org/10.1002/jaba.13 of synthesized and isolated components. Journal of
Rooker, G. W., Jessel, J., Kurtz, P. F., & Hagopian, L. P. Applied Behavior Analysis, 50, 252-277. https://doi.
(2013). Functional communication training with and org/10.1002/jaba.384.
without alternative reinforcement and punishment: Strand, R. C. W., & Eldevik, S. (2017). Improvements in
An analysis of 58 applications. Journal of Applied problem behavior in a child with autism spectrum
Behavior Analysis, 46, 708-722. https://doi.org/10. diagnosis through synthesized analysis and treatment:
1002/jaba.76 A replication in an EIBI home program. Behavioral
Roscoe, E. M., Phillips, K. M., Kelley, M. A., Interventions, 33, 102-111. https://doi.org/10.1002/
Farber, R., & Dube, W. V. (2015). A statewide sur- bin.1505
vey assessing practitioners’ use and perceived utility
Wallace, M. D., & Iwata, B. A. (1999). Effects of session
of functional assessment. Journal of Applied Behavior
duration on functional analysis outcomes. Journal of
Analysis, 48, 830-844. https://doi.org/10.1002/
Applied Behavior Analysis, 32, 175-183. https://doi.
jaba.259
org/10.1901/jaba.1999.32-175
Rose, J. C., & Beaulieu, L. (2018). Assessing the general-
ity and durability of interview-informed functional
analyses and treatment. Journal of Applied Behavior Received June 13, 2018
Analysis. Advance online publication. https://doi. Final acceptance December 11, 2018
org/10.1002/jaba.504 Action Editor, Linda LeBlanc