See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/7146687

Screening for Autistic Spectrum Disorder in

Children Aged 14–15 Months. II: Population
Screening with the Early Screening of Autistic
Traits Questionnaire (ESAT)

Article in Journal of Autism and Developmental Disorders · September 2006

DOI: 10.1007/s10803-006-0114-1 · Source: PubMed

CITATIONS READS

136 93

5 authors, including:

S. H. N. Swinkels Emma van Daalen

Danone Nutricia Research University Medical Center Utrecht
94 PUBLICATIONS 2,269 CITATIONS 70 PUBLICATIONS 1,461 CITATIONS

SEE PROFILE SEE PROFILE

Jan Buitelaar
Radboud University Medical Centre (Radbo…
904 PUBLICATIONS 26,911 CITATIONS

SEE PROFILE

All in-text references underlined in blue are linked to publications on ResearchGate, Available from: Jan Buitelaar
letting you access and read them immediately. Retrieved on: 28 July 2016
J Autism Dev Disord (2006) 36:713–722
DOI 10.1007/s10803-006-0114-1

ORIGINAL PAPER

Screening for Autistic Spectrum Disorder in Children Aged

14–15 Months. II: Population Screening with the Early Screening
of Autistic Traits Questionnaire (ESAT). Design and General
Findings
Claudine Dietz Æ Sophie Swinkels Æ Emma van Daalen Æ
Herman van Engeland Æ Jan K. Buitelaar

Published online: 22 April 2006

Ó Springer Science+Business Media, Inc. 2006

Abstract A two-stage protocol for screening for autistic interests, activities, and behaviors. Autistic disorder is
spectrum disorders (ASD) was evaluated in a random considered to be the core and generally most severe dis-
population of 31,724 children aged 14–15 months. Chil- order of the group of autistic spectrum disorders (ASD).
dren were first pre-screened by physicians at well-baby These disorders are thought to be caused by neurobiolog-
clinics using a 4-item screening instrument. Infants that ical deficits that are present early in life. As defined by the
screened positive were then evaluated during a 1.5-h home Diagnostic and Statistical Manual of Mental Disorders 4th
visit by a trained psychologist using a recently developed edition (DSM IV) criteria for Autistic Disorder (American
screening instrument, the 14-item Early Screening of Psychiatric Association, 1994), delay or abnormal func-
Autistic Traits Questionnaire (ESAT). Children with 3 or tioning must have started before 3 years of age. Despite
more negative scores were considered to be at high-risk of this very early onset, a considerable delay exists between
developing ASD and were invited for further systematic parents’ first concerns and the psychiatric diagnosis.
psychiatric examination. Eighteen children with ASD were Although more expertise and knowledge has led to earlier
identified. The group of children with false positive results diagnosis in recent years, the diagnosis is still rarely made
had related disorders, such as Language Disorder (N = 18) before 3 years of age. For example, in the study by Howlin
and Mental Retardation (N = 13). and Asgharian (1999), children were on average 5.5 years
old at diagnosis whereas parents reported that they started
Keywords Autistic spectrum disorder Æ Screening Æ worrying when the child was 18 months old. Similarly,
Instrument Æ Early identification other reports described first parental concerns within the
first or second year of development: around 19 months (De
Giacomo & Fombonne, 1998), between 12 and 18 months
Introduction old (Smith, Chung, & Vostanis, 1994) or within the first
year of life (Rogers & DiLalla, 1990). The age of first
Autistic disorder, a serious, disabling disorder, is charac- parental concern was found to be lower when the child was
terized by impairments in social interaction and verbal and mentally retarded, when there was an older sibling in the
non-verbal communication, and by a restricted repertoire of family, when there was a delay in developmental mile-
stones or a medical problem co-existed (De Giacomo &
Fombonne, 1998). Studies based on home movies support
the existence of abnormalities within the first year of life
C. Dietz (&) Æ S. Swinkels Æ E. van Daalen Æ H. van Engeland
(Adrien et al., 1993; Baranek, 1999; Osterling & Dawson,
Department of Child and Adolescent Psychiatry, B01.201,
University Medical Center Utrecht, Post Box 8500, Utrecht 3508 1994; Werner, Dawson, Osterling, & Dinno, 2000).
GA, The Netherlands While these findings suggest that it should be possible to
e-mail: [email protected] diagnose ASD earlier, it is important to consider the factors
that seem to be responsible for the relatively late diagnosis.
J. K. Buitelaar Æ S. Swinkels
Department of Psychiatry, University Medical Center Nijmegen, First of all, few instruments exist to screen for ASD in very
Nijmegen, The Netherlands young children (e.g. toddler age). A pioneering attempt to

123
714 J Autism Dev Disord (2006) 36:713–722

screen for autism at 18 months at the level of the general severity (Howlin, 1997; Rogers, 1996). As long as the effect
population was undertaken by Baron-Cohen, Allen and of early intervention is not proven, the aim of early detec-
Gillberg (1992). These authors developed the CHecklist for tion must lie initially in helping these children and their
Autism in Toddlers (CHAT) as a screening instrument in a parents to cope with the handicap.
genetically high-risk group of 41 toddlers and then used this Opinions vary as to the requirements of ASD screening
instrument in a prospective screening study of 16,000 tod- instruments and procedures. A multidisciplinary consensus
dlers aged 18 months (Baron-Cohen et al., 1996). A follow- panel on behalf of nine professional organizations and four
up study of these children at 7 years showed that a high parent organizations reviewed and evaluated research on
positive predictive value could be reached, however with the screening and diagnosis of autism at an early age
the compromise of low sensitivity (Baird et al., 2000). Since (Filipek et al., 1999). The panel concluded that a dual
the CHAT study excluded children with profound handicap approach to the detection of autism is required: (1) routine
or profound retardation, sensitivity may well be improved developmental surveillance to identify children at risk of
by systematically assessing these children as well. any type of atypical development; and (2) diagnosis and
A second factor that might be responsible for the rela- evaluation of autism. This approach would not include a
tively late detection of ASD is the low prevalence of ASD, universal screen for ASD. Instead, only children at risk
implying a low priori diagnostic probability. However, would be screened with an instrument such as the CHAT
recent studies show that ASD is not as rare as previously (Baron-Cohen et al., 1992), the Pervasive Developmental
thought. An extensive review of 32 epidemiological sur- Disorders Screening Test (Siegel, 1996), or the Australian
veys published between 1966 and 2001, conclude that the Scale for Asperger’s syndrome (Garnett & Attwood, 1998).
prevalence of ASD is 27.5 per 10,000, with more recent The preventive health system in the Netherlands offers the
studies suggesting estimates as high as 60 per 10,000 opportunity to screen at a population level as opposed to
(Fombonne, 2003). screening of high-risk children only.
Lastly, the diagnostic criteria and algorithm of AD as Besides the advantage of a large reach when screening
described in DSM-IV and ICD-10 might have limitations on population level, sensitivity will be inevitably lower due
when applied to infants and toddlers, since some of the to a lower base rate within a random population (Clark &
criteria described relate to behaviors that are not present at Harrington, 1999). Validity indices should therefore be
young age (e.g. developing friendships with peers). evaluated carefully for both ethical and scientific reasons.

Screening for ASD Dutch Screening Study

In most countries, the developmental surveillance of very Because of the lack of evidence that ASD and non-ASD
young children focuses on general and sensorimotor aspects types of developmental problems can be distinguished at
of development. For example, the achievement of motor very early age, we adopted a pragmatic approach. We set
milestones, all of which may appear normal in children with out to develop a screening instrument that was maximally
ASD. There is less emphasis on social-interactional aspects sensitive to ASD, even if sensitivity was achieved at the
of development, which may be abnormal in these children. cost of a slightly lower specificity for other non-ASD
A screening instrument for ASD suitable for infants would developmental disorders, and which was based on both
facilitate the identification of children at high risk and speed parent report and expert observation.
up the subsequent referral to specialized centers for further Since we wanted to use this instrument in a population
diagnostic evaluation (Filipek et al., 1999). First of all, screening protocol to detect ASD in very young children,
establishing the diagnosis can help parents understand the we chose to base our study in the existing primary care
handicaps of their child, take away feelings of anguish, and system for surveillance of developmental problems in the
help them to find ways to support their autistic child in its Netherlands, namely, the well-baby clinics. These clinics
development, even though the disorder cannot be cured. monitor the development and growth of infants and toddlers
Secondly, parents might want to know the diagnosis be- and provide vaccinations. They focus primarily on the more
cause of the genetic risks when considering having another general development of infants and pay less attention to the
child. It is of note that the siblings of a child with autism monitoring of social development. We were supported in
have themselves an increased risk of about 3–7% of our attempts to set up an ASD screening protocol by the
developing ASD (Rutter, Bailey, Simonoff, & Pickles, comments of parents visiting our department, on how
1997). Thirdly, some evidence exists for the benefits of stressful life was in the period before their child was first
early intervention in terms of IQ gains, language gains, diagnosed. Next we were encouraged by the enthusiastic
improved social behavior, and reduction in symptom cooperation of physicians and nurses of well-baby clinics.

123
J Autism Dev Disord (2006) 36:713–722 715

In our attempt to lower the diagnostic threshold of ASD these re-examinations, parents of both screen-positive and
as much as possible, we elected to screen infants at 14– screen-negative children, received a parent questionnaire at
15 months. Moreover, a very practical reason was the high 24- and 42-month follow-up. The questionnaire contained
attendance rate at well-baby clinics around this age because items of the 14-item ESAT. The 42-month follow-up
the measles-mumps-rubella vaccination is administered at questionnaire contained the Autism Screening Question-
that time. naire (ASQ; Berument, Rutter, Lord, Pickles, & Bailey,
Unlike the study of Baron-Cohen et al. (1996), we aimed 1999). Screen-negatives with a high score on the ASQ,
to identify children with ASD across the whole range of were invited for psychiatric evaluation at our department.
intellectual functioning, thus without systematically The screening procedure is given in Fig. 1. The study
excluding children with profound retardation. design and screening procedure were approved by the
Here, we describe the screening procedure and our Medical Ethics Review Board of the University Medical
findings. Centre Utrecht.

Pre-screening at Well-Baby Office

Method
The physicians of the well-baby-clinics were instructed in
The overall design was to screen about 30,000 children. the pre-screening procedure before the start of the screen-
This sample size was chosen on the basis of available ing project. During the screening period we organized
prevalence estimates of ASD and the projected sensitivity annual lectures on the early detection of autism.
and specificity of our screening procedure (see paper I by Parents were informed in advance of the pre-screening
Swinkels et al.). Our aim was to study the biological, program by means of a brochure and an attractive poster in
cognitive, and environmental predictors and correlates of the waiting room of the well-baby clinic. The brochure and
development in this cohort of very young children with the pre-screening test were also available in English,
ASD. Arabic, and Turkish. The parent had to give verbal consent
After a pilot study from January 1999 to October 1999, to participate in the pre-screening procedure. The com-
we started screening children from October 1999 up to pleted screening list was sent to our department with
April 2002 in a geographically defined area, the province minimal information on personal identification (first letter
of Utrecht (the Netherlands). This area is situated in the of the surname of the child, the sex as well as the birth date,
middle of the country and mainly has an urban character. and date of screening), in order to safeguard patient pri-
The total population is 1,139,925 people. The province of vacy. Physicians were instructed to obtain answers to all 4
Utrecht has two large cities, Utrecht with a population of items of the pre-screening test by interviewing parents.
260,625 and Amersfoort with 129,720 inhabitants. Other Parents were advised to continue with the screening pro-
villages and small cities vary in population between 4138 cedure if their child had a negative answer on at least one
and 62,140; 7.5% of the total population is of foreign of four items in the pre-screening test. By signing and
extraction. The population is mostly well educated. 13% of returning a response card, the parents of pre-screening
the population has primary education only (Provincie positive children gave informed consent to be contacted for
Utrecht, 2003). a home visit, which was made within 2 weeks of receipt of
Practical and budgetary constraints led us to choose a the response card.
short, 3-min pre-screening test, carried out at the well-baby
clinic. Children were pre-screened here at age 14–15 months. Screening During Home Visit
Although attendance of well-baby clinics is not compulsory,
most children up to 4 years of age are taken to these clinics. A trained child psychologist administered the 14-item
In the first year, attendance is as high as 98%, with an Early Screening of Autistic Traits; (ESAT paper I by
average of 6 visits in the first year (Centraal Bureau voor Swinkels et al.) to the child’s parents during a 1.5-h home
Statistiek, 2003). visit. Parents were asked additional questions, using the
Children that screened positive were evaluated during a items of the CHAT (Baron-Cohen et al., 1996), the Infant/
home visit using the 14-item screening instrument ESAT. Toddler Checklist for Communication and Language
Next, screen-positive children on the 14-item ESAT were Development (Wetherby & Prizant, 1998), and some items
invited for further investigations at the Department of the ADI-R relevant to this young age (Lord, Rutter, &
of Child Psychiatry. Re-examinations of cognitive devel- Le Couteur, 1994). The cognitive level of the child was
opment were made at age 24 and more extensive clinical measured with the Mullen Scales of Early Learning
re-examinations were done at age 42 months. Apart from (MSEL; Mullen, 1995). For 10% of the children (n = 25)

123
716 J Autism Dev Disord (2006) 36:713–722

Fig. 1 Study design

no MSEL scores were obtained; 19 children that started of very young children and their parents. The assessments
during the pilot phase of our project were evaluated with included a standardized parental interview, developmental
the Dutch translation of the Bayley Scales (BOS 2–30; van history, and the Vineland Social-Emotional Early Child-
der Meulen & Smirkosky, 1982). Six children that were too hood Scales (Sparrow, Balla, & Cicchetti, 1997); stan-
difficult to evaluate with the MSEL due to non-coopera- dardized behavior observation using the Autism Diagnostic
tion, were examined with the Psycho-educational Profile Observation Schedule or ADOS-G (DiLavore, Lord, &
Revised (PEP-R; Schopler, Reichler, Bashford, Lansing, & Rutter, 2000), and pediatric examination and medical work-
Marcus, 1994). All children were videotaped and the up. On the basis of all available information, an experienced
behavior was scored using an experimental observation child psychiatrist made a clinical judgement, on whether the
protocol including items of the 14-item ESAT (Dietz et al., child was likely to meet DSM-IV criteria of specific cate-
not published). gories by about the age of 3.5 years. The clinical diagnoses
The parents of screen-positive children at the home visit, were subgrouped as follows: ASD with or without concur-
were advised to allow their child to undergo extensive rent Mental Retardation; Mental Retardation (without
diagnostic investigations at the Department of Child Psy- ASD); Language Disorder; other disorder according to
chiatry. A series of five tests were scheduled within a period DSM-IV (e.g. attention deficit and hyperactivity disorder,
of 5 weeks. At each weekly visit, the social and commu- reactive attachment disorder); classification within the DC
nicative behavior of the child was observed in a small group 0-3 (Diagnostic Classification of Mental Health and

123
J Autism Dev Disord (2006) 36:713–722 717

Developmental Disorders of Infancy and Early Childhood, of the ESAT were considered to be screen-positive and
1994), or no disorder (normally developing child). Exam- thus at high risk of developing ASD.
ples of DC 0-3 classifications are a regulation disorder or
problems in the parent–child relationship. More details on
the psychiatric diagnoses will be reported elsewhere (Van Results
Daalen et al., in preparation).
The inter-rater reliability for the clinical diagnosis Results Pre-screening and Screening
among three child psychiatrists (HE, JB, ED) was calcu-
lated first for two diagnostic categories; ASD or other than A flowchart indicating the number of children at the dif-
ASD. Agreement was reached in 92% of 38 cases. ferent stages of the screening procedure is presented in
Agreement corrected for chance was .74 (Cohen’s Kappa). Fig. 2. In total 31,724 children (mostly aged 14–15 months)
Second, the inter-rater reliability was measured for all were screened. The compliance rate was very high, with a
diagnostic categories. An agreement was reached of 79% negligible number of 52 parents who did not cooperate. We
of 38 cases. Agreement corrected for chance was .67 excluded another 399 children because screening lists were
(Cohen’s Kappa). Diagnostic discrepancies were resolved either incomplete or the physician was unable to screen the
at a consensus meeting. If appropriate, children and their child, usually because of a language barrier. Because the
parents were offered ‘‘care as usual’’. number of incomplete screening lists was low, the screened
Re-evaluations of the clinical diagnosis were made population can still be considered a true reflection of the
around age 42 months on the basis of a second series of population.
clinical assessments. Children were on average 14.91 months old (SD = 1.37)
at the 4-item ESAT, range 13–23 months, 90% of the
The Screening Instrument children was 16 months or younger. 63% of all children
was male. 255 of 370 parents of pre-screen positives (69%)
The items of the 4-item ESAT and of the 14-item ESAT are consented to a home visit. At the time of the home visit,
described in paper I (Table 1). Items of the ESAT could be children were on average 16.31 months old (SD = 1.86),
answered with ‘‘yes’’ for typical and ‘‘no’’ for atypical range 14–30 months, 91% of the children was 18 months
behavior. The pre-screening instrument, which can be or younger. 66% of the children were male. The delay
completed within 3 min, comprises four items that are easy between pre-screening and 14-item ESAT testing was
to administer. The two first items measure play behavior generally no more than a month (66%): 91% of the children
(interest in different toys and varied play), item 3 measures had a home visit within 2 months and 9% had a home visit
the readability of emotions, and item 4 the reaction to between 3 and 11 months.
sensory stimuli. If a child has a negative answer on at least Of the 255 infants screened with the 14-item ESAT, 100
one of the items using parental information, he or she is (39%) were screen-positive. The compliance rate for further
considered to be pre-screen positive. The 14 items of the diagnostic evaluation at the Department of Child and Ado-
ESAT were answered by both the parent and the child lescent Psychiatry was 73%. Diagnoses were made by three
psychologist during the home visit. When either the parent child psychiatrists (ED, JB, HE). Of the 73 infants examined
or the child psychologist found the behavior of the child on at our toddler unit, 18 were diagnosed as having ASD, at the
a specific item atypical, this resulted in a negative answer. first psychiatric evaluation. Other diagnoses included
Children with negative answers on at least 3 of the 14 items non-ASD developmental disorders, such as developmental

Table 1 Description of developmental level

Overall standard score Low < 70 Below average 70–85 Average >85

Screen-negatives ESAT (n = 155) 5 22 128

Screen-positives ESAT (n = 100) 26 30 44
N = 18 ASD 11 4 3
N = 13 Mental Retardation 11 2* –
N = 22 Language Disorder 1 8 13
N = 7 Other DSM-IV 1 2 4
N = 13 DC: 0-3 1 6 6
Note: The MESL (Mullen, 1995) was used to estimate the overall standard score
*These children were diagnosed with Mental Retardation, despite borderline cognitive levels just above 70 at the initial measurement. They did
however end up with low cognitive scores at follow-up evaluation

123
718 J Autism Dev Disord (2006) 36:713–722

Fig. 2 Screening results Total screen stage I

31,724

Positive
n = 370
1.2%
Refusals
n = 115
31%

Total screen stage II

n = 255

Positive
n = 100
39%
Refusals
n = 27
27%

First evaluation
n = 73

ASD MR LD Other DSM IV zero-to-three

n = 18 n = 13 n = 18 n = 11 n = 13
25% 18% 25% 15% 18%

Language Disorder (n = 18), or general Mental Retardation Parents of children diagnosed with ASD most often
(n = 13). Another 11 children had problems that would fit answered pre-screening item 3 ‘‘emotions understandable’’
other diagnostic categories of the DSM-IV. The remaining (50%) and 4 ‘‘Reaction sensory stimuli’’ (50%) negatively.
children had problems that could be labeled best by a None of the 4 pre-screening items could however differ-
Diagnostic Classification in the DC 0-3 system (Diagnostic entiate between the group of children with and without
Classification of Mental Health and Developmental Disor- ASD diagnosis, using chi-square analysis.
ders of Infancy and Early Childhood, 1994). At the time of
the first psychiatric evaluation, children were on average ESAT Screening
23.26 months old (SD = 8.0), range 15–54 months. 74% of
the children was 24 months or younger. While 67% of the Children who tested positive on the 14-item ESAT had far
children were evaluated within 5 months of the home visit, more ‘‘no’’ answers (M = 5.1 and range 3–14) than did the
others waited up to 39 months. At the clinical re-examina- children who tested negative (M = 1.1 and range 1–2). The
tions, children were on average 43.07 months old group of screen-positives differed from the group of
(SD = 5.15), range 34–64 months. screen-negatives on all individual items (See Table 2).
Table 1 shows the cognitive levels of all children Next, the children were divided into groups with and
measured during the home visit. Children that were screen- without a diagnosis of ASD. The group of children not
positive on the 14-item ESAT had on average a lower diagnosed with ASD contained children that were ASD-
cognitive level compared with children that were screen- negative on the 14-item ESAT and children that were
negative on the 14-item ESAT (M = 81, SD = 16 com- diagnosed with a disorder other than ASD. The children
pared to M = 96, SD = 13, F (1, 254), P < .01). with ASD had more ‘‘no’’ answers on all items except for
item 7 ‘‘Attracts attention’’, item 11 ‘‘ Likes cuddling’’,
Pre-screening Test and item 13 ‘‘Enjoys social play’’ using chi-square anal-
ysis. Odds ratios were calculated to assess whether scores
At the pre-screening test, most parents answered item 3 on each item predicted ASD. The items that were most
‘‘Emotions understandable’’ negatively (49%); a minority related to ASD, with the highest odds ratio for ASD, were
answered item 1 ‘‘Interest different toys’’ negatively (8%). item 10 ‘‘Interest people’’, item 12 ‘‘Smiles directly’’, and
The scores of the pre-screen positive children that did not item 14 ‘‘Reacts when spoken to’’. Percentages of negative
undergo 14-item ESAT screening were compared with scores on the 14-item ESAT are shown in Table 2.
those that did undergo 14-item ESAT screening. No dif- Finally, parental information was compared with expert
ferences in scores for the individual items were found observation on the individual items of the 14-item ESAT.
between the two groups, using chi-square analysis. Experts and parents agreed on most items (agreement over

123
J Autism Dev Disord (2006) 36:713–722 719

Table 2 Percentage of negative scores on the ESAT at screen-II

Item Screen- Screen- Odds ratio Non-ASD ASD Odds ratio
negatives positives for ESAT (n = 228) (n = 18) for ASD (95% CI)
ESAT (n = 155) ESAT (n = 100) positive (95% CI)

ESAT 1: 3 24 9.30**(3.3–26.0) 8 33 5.68**(1.9–17.0)

Interest different toys
ESAT 2: 11 56 5.11**(3.2–8.3) 24 61 4.90**(1.8–13.3)
Varied play
ESAT 3: 34 62 1.85**(1.4–2.4) 40 72 3.98**(1.3–11.6)
Emotions understand.
ESAT 4: 29 61 2.10**(1.6–2.8) 37 67 3.46*(1.2–9.6)
React. sensory stimuli
ESAT 5: 3 26 10.08**(3.6–28.0) 7 28 5.39**(1.7–17.3)
Facial expression
ESAT 6: 2 41 21.18**(6.7–66.6) 11 50 7.75**(2.8–21.4)
Eye contact
ESAT 7: 5 19 4.21**(1.8–9.6) 10 17 1.90 n.s.(.5–7.1)
Attracts attention
ESAT 8: 13 51 4.0**(2.5–6.2) 25 56 3.80**(1.4–10.1)
Stereotypical movem.
ESAT 9: 3 35 10.90**(4.4–26.8) 12 33 3.54*(1.2–10.2)
Brings/shows objects
ESAT 10: 1 28 43.40**(6.0–313.9) 8 56 15.16**(5.2–43.8)
Interest people
ESAT 11: 7 41 6.36**(3.4–12.1) 16 28 2.06 n.s.(.7–6.2)
Likes cuddling
ESAT 12: 3 22 8.53**(3.0–24.0) 6 44 13.20**(4.4–39.5)
Smiles directly
ESAT 13: 2 16 8.27**(2.5–27.6) 6 17 3.03 n.s.(.8–11.8)
Enjoys social play
ESAT 14: 1 26 40.30**(5.6–292.3) 7 44 10.40** (3.6–30.3)
Reacts spoken to
Total ESAT mean 1.1 5.1 – 2.2 6.1 –
Total ESAT range 1–2 3–14 – 0–9 3–14 –
*P < .05 with chi-square analysis, **P < .01 with chi-square analysis, n.s.=not significant

75%) except for item 3 ‘‘emotions understandable’’(59.4%), Mental Retardation without ASD (N=13), Language Dis-
4 ‘‘sensory stimuli (61%) and 8 ‘‘stereotypical movements’’ order (N = 18), another childhood psychiatric disorders like
(72.3%). Parents evaluated the behavior of the child on the ADHD (N=11) and diagnosis according to DC 0-3. No
items in general as more negative than did the children with typically development were found within the
experts. Agreement on the items was 82.4% on average group of false positives.
(range 59.4–94.4%). Some changes in psychiatric evaluations at age 42
The picture was however found to be quite different within months were found. Stability in early ASD diagnosis was
the small group of children diagnosed with ASD. Within this found in 14 of 16 children that were re-examined. Two
group, parents and experts disagreed on most items. Experts children could not be traced for re-evaluations. Two chil-
evaluated the behavior of these children as more negative dren who were diagnosed with ASD at first evaluation left
than parents did. Agreement on the items was 65.9% on the autism spectrum at re-evaluation; one demonstrated
average (range 44.4–88.9%). Interesting is that 3 out of 18 evidence of an articulation disorder, one child was diag-
children diagnosed with ASD, would score below threshold nosed with a regulatory disorder according to DC 0-3.
at the 14-item ESAT when using parent ratings only. Another two children were not diagnosed with ASD at first
evaluation, but were diagnosed in the autistic spectrum at
Detected ASD Cases re-evaluation. One of these children was diagnosed with
Mental Retardation without ASD at first evaluation, the
Eightteen young children with ASD were detected with the other was first diagnosed with a receptive and expressive
14-item ESAT, out of a population of 31,724 children, that Language Disorder. A full report on stability of diagno-
is, 5.7 per 10,000. The Positive Predictive Value would be sis will be presented elsewhere (Van Daalen et al., in
25%. The group of false positives contained children with preparation).

123
720 J Autism Dev Disord (2006) 36:713–722

Discussion parents. Both the young age of the children and the effort
of having preferable five but at least two examinations at
Our study is the first prospective population study to screen our department, turned out to be obstacles for cooperation.
for ASD in children aged 14–15 months. Of 31,724 chil- The number of detected ASD cases in our screening
dren screened, 18 were identified with ASD. This indicates study can probably best be compared with the number
that early identification of ASD is possible, at least in a found in the CHAT study. We detected 5.7 ASD cases per
proportion of cases. 10,000. We would expect to find about 11.7 if the same
In an earlier study by Willemsen-Swinkels et al. (paper proportion of ‘‘cases’’ were found in the refusal groups.
I), the 4-item prescreen was found to select 2% of the However, as to date we do not know in which direction the
random population (confidence interval 0.6–4.7%). In the bias of the refusal would operate. To compare, the CHAT
present population study the percentage of children found study detected 10 cases with Autistic Disorder per 16,236
to fail screen-1 was lower e.g. 1.2%, although within the toddlers, that is, 6.2 per 10,000 at 18 months of age. Using
confidence interval. a medium threshold, the CHAT could even detect 19 cases
While the 14-item ESAT test undoubtedly failed to (Baird et al., 2000). While both studies indicate that early
detect at least some ASD cases, it is difficult to estimate identification of ASD is possible, a comparison of the
how many cases were missed among these 14-month-old number of cases detected with recent prevalence figures in
children. An obvious approach is to compare the yield of the literature suggest that the sensitivity of both screening
the 14-item ESAT with current knowledge about the tests is low.
prevalence of ASD. Prevalence estimates have been The positive predictive value of our screening procedure
obtained, in children of school age or even older was lowered by the large number of false-positives.
(Fombonne, 2003). Autistic Disorder by definition, However, it is worth mentioning that none of these children
according to the DSM IV criteria (American Psychiatric were found to have a ‘‘typically normal’’ development. A
Association, 1994), must be associated with delay or large number of the children with false-positive results had
abnormal functioning before 3 years of age. However, it a Language Disorder (N=18). Moreover, the children with
is unknown which percentage of children with Autistic Mental Retardation without ASD also had high ESAT
Disorder or ASD have observable delays or abnormal scores. Although the differentiation of the 14-item ESAT
functioning already at age 14 months. Therefore, it seems items was examined in the study by Willemsen-Swinkels
plausible that our screening method failed to detect et al. (paper I) for groups of children with Attention Deficit
especially the milder variants of ASD and/or the children and Hyperactivity Disorder and ASD, this was not done for
with a high level of development at age 14 months. The children with other disorders, such as Language Disorder
problem of low sensitivity of tests in detecting develop- and Mental Retardation. It is possible that the symptoms of
mental disorders is recognized as a broader phenomenon these disorders are similar to those of ASD, especially at a
affecting even the most thoroughly researched screening young age.
tests such as the Denver Developmental Screening Test An important difference between the studies described
(Meisels, 1989). in paper I and the population screening described in this
At each stage of the screening procedure, we ‘‘lost’’ a paper is the use of the expert in the rating of the screening
large number of children because parents did not (yet) want items as well as parental rating, whereas the studies in
to cooperate. In a Japanese study by Sugiyama and Abe paper I were based on parental ratings only. In general we
(1989) it was found that especially toddlers that failed a found that experts and parents agreed on most items,
developmental check up at age 18 months, and whose implying the added value of expert observation could be
parents refused further investigations, did show more considered minimal. However, within the small group of
developmental problems at 3-year follow-up than the children diagnosed with ASD, parents were more opti-
children who did participate in further investigations. mistic than experts. Three out of 18 children diagnosed
These results would suggest that the children who did not with ASD would even score below threshold when only
participate in the ESAT test might have had more serious parental ratings were used. This preliminary finding might
disorders. be of clinical value and needs further investigation.
The hesitation on the part of parents to cooperate is also It is of interest that we found much agreement on ESAT-
reflected by the delay between the home visit and the scores between parents and experts within the total group
examinations at the Department of Child and Adolescent of children, as opposed to much disagreement within the
Psychiatry. Parents needed on average 7 months to decide group of children diagnosed with ASD. Although the group
to visit our Department. Because no waiting list or other of children with ASD in this study is too small to draw
restrictions existed for the psychiatric examinations during conclusions on, the finding needs further investigation and
the project, this delay was solely due to the hesitation of the might be of clinical value.

123
J Autism Dev Disord (2006) 36:713–722 721

We tried to identify those items of the 4-item ESAT that Clinical Implications
were the most sensitive, defined as the proportion of cases
that would fail the items, and would discriminate optimally We have shown that it is possible to detect ASD in children
between ASD and normality, using odds ratio. Most parents aged about 14 months. However, the yield of this two-step
answered item 3 ‘‘emotions understandable’ of the pre- screening procedure for ASD seems low at this age and
screening negatively, whereas parents of children diagnosed parents appeared relatively reluctant to cooperate. More-
with ASD more often answered pre-screening item 4 over, the pre-screening test generated a large number of
‘‘Reaction sensory stimuli’’ negatively. None of the items false-positive results and should therefore be used with
of the pre-screening could however differentiate between caution.
children that did and did not receive an ASD diagnosis. The While the 14-item ESAT test resulted in a high number
pre-screening instrument appeared to have the function of of false-positive results, none of these children showed
just selecting roughly between suspected and non-suspected typical development. It can be argued that all ESAT
children, and the 14-items ESAT was necessary for a more screen-positive children would benefit from further diag-
accurate discrimination between children that actually re- nostic evaluation. The 14-item ESAT test is still far from a
ceived or did not receive ASD diagnoses. diagnostic instrument, but it can be viewed as a screening
Items of the 14-item ESAT that showed to be most instrument for children at high risk of ASD. Clinicians
sensitive to ASD were item 6 ‘‘Eye contact’’, item 8 could use the instrument when considering referral for
‘‘Stereotypical movements’’ and item 10 ‘‘Interest peo- diagnostic evaluation.
ple’’. Each item was answered negatively for about 50% of
all children with ASD. Of these items, item 8 was least Acknowledgments This study was supported by grants 940-38-045
and 940-38-014 (Chronic Disease Program), by grant 28.3000-2 of
predictive of ASD. ESAT-items that were most predictive the Praeventiefonds-ZONMW, by the Netherlands Organisation for
of ASD referred to the social-communicative domain; item Scientific Research (NWO), by a grant from the Dutch Ministry of
10 ‘‘Interest people’’, item 12 ‘‘Smiles directly’’, and item Health, Welfare and Culture, and by grants from Cure Autism Now,
14 ‘‘Reacts when spoken to’’. and the Korczak Foundation. We are very grateful to the physicians of
the well-baby clinics of Eemland, Thuiszorg Stad Utrecht, Weides-
The 14-item ESAT was found to detect ASD in children ticht and Zorgservice Vitras, who were all willing to cooperate with
with various levels of cognitive functioning. In the litera- our screening procedure. We also thank the parents and children for
ture, the rate of children with Autistic Disorder without their trust in our expertise when the project was still in an experi-
mental delay has been reported as 30% (e.g. Fombonne, mental stage.
2003). In an earlier study, Chakrabarti and Fombonne
(2001), report 50% of children with PDD-NOS to have
mental delay. In our study we found three children (17%) References
having ASD without mental delay with standard score at or
above 85 (see Table 2). 61% of the children with ASD had Adrien, J. L., Lenoir, P., Martineau, J., Perrot, A., Hameury, L.,
a standard score below 70 on the MSEL (Mullen, 1995). Larmande, C., & Sauvage, D. (1993). Blind ratings of early
The numbers are however too small to derive conclusions symptoms of autism based upon family home movies. Journal of
the American Academy of Child and Adolescent Psychiatry, 32,
on the intellectual functioning of the children. Also future 617–626.
cognitive development should be awaited. American Psychiatric Association (1994). Diagnostic and statistical
manual of mental disorders (4th ed., Rev.). Washington DC:
Author.
Limitations Baird, G., Charman, T., Baron-Cohen, S., Cox, A., Swettenham, J.,
Wheelwright, S., & Drew, A. (2000). A screening instrument for
autism at 18 months of age: a 6 year follow-up study. Journal of
The question remains whether we would have detected
the American Academy of Child and Adolescent Psychiatry, 39,
more children with ASD if we had used different items or 694–702.
extended the pre-screening test with several items. The Baranek, G. T. (1999). Autism during infancy: a retrospective video
study of Swinkels et al. (paper I) selected this combination analysis of sensory-motor and social behaviors at 9–12 months
of age. Journal of Autism and Developmental Disorders, 29,
of four items because of their high sensitivity and effi-
213–224.
ciency referring to the low amount of normal developing Baron-Cohen, S., Allen, J., & Gillberg, C. (1992). Can autism be
false positives they would yield. detected at 18 months? The needle, the haystack, and the CHAT.
To be able to judge the validity of our screening British Journal of Psychiatry, 161, 839–843.
Baron-Cohen, S., Cox, A., Baird, G., Swettenham, J., Nightingale, N.,
instrument, we need data on the false-negative cases. In a
Morgan, K., Drew, A., & Charman, T. (1996). Psychological
future study we will investigate all children in this birth markers in the detection of autism in infancy in a large popu-
cohort again at age 6 years. lation. British Journal of Psychiatry, 168, 158–163.

123
722 J Autism Dev Disord (2006) 36:713–722

Berument, S. K., Rutter, M., Lord, C., Pickles, A., & Bailey, A. Meisels, S. J. (1989). Can developmental screening tests identify
(1999). Autism screening questionnaire: Diagnostic validity. children who are developmentally at risk? Pediatrics, 83, 578–
British Journal of Psychiatry, 175, 444–451. 585.
Centraal Bureau voor Statistiek, Voorburg, Heerlen (2003). Gebruik van der Meulen, B. F., & Smirkosky, M. (1982). BOS 2-30 Bayley
medische voorzieningen. Retrieved June 25, 2003, from http:// Ontwikkelingsschalen. Swets en Zeitlinger.
statline.cbs.nl/statWeb. Mullen, E. M. (1995). Mullen scales of early learning. Circle Pines
Chakrabarti, S., & Fombonne, E. (2001). Pervasive developmental MC: American Guidance Service.
disorders in preschool children. JAMA, 285, 3093–3099. Osterling, J., & Dawson, G. (1994). Early recognition of children with
Clark, A., & Harrington, R. (1999). On diagnosing rare disorders autism: A study of first birthday home videotapes. Journal of
rarely: appropriate use of screening instruments. Journal of Autism and Developmental Disorders, 24, 247–257.
Child Psychology and Psychiatry, 2, 287–290. Provincie Utrecht (2003). Feiten en Cijfers Provincie Utrecht.
De Giacomo, A., & Fombonne, E. (1998). Parental recognition of Retrieved June 25, 2003, from http://www.provincie-utrecht.nl/
developmental abnormalities in autism. European Child & prvutr/internet/kern2000.nsf.
Adolescent Psychiatry, 7, 131–136. Rogers, S. J., & DiLalla, D. L. (1990). Age of symptom onset in
Diagnostic Classification of Mental Health and Developmental Dis- young children with pervasive developmental disorders. Journal
orders of Infancy and Early Childhood (1994). ZERO TO of the American Academy of Child and Adolescent Psychiatry,
THREE: National Center for Infants. Washington DC: Toddlers 29, 863–872.
and Families. Rogers, S. J. (1996). Brief report: Early intervention in autism.
Dilavore, P. C., Lord, C., & Rutter, M. (2000). The pre-linguistic Journal of Autism and Developmental Disorders, 26, 243–246.
autism diagnostic observation schedule. Journal of Autism and Rutter, M., Bailey, A., Simonoff, E., & Pickles, A. (1997). Genetic
Developmental Disorders, 25, 355–379. influences and autism. In D. J. Cohen, & F. R. Volkmar (Ed.),
Filipek, P. A., Accardo, P. J., Baranek, G. T., Cook, E. H., Dawson, G., Handbook of autism and pervasive developmental disorders (pp.
Gordon, B., Gravel, J. S., Johnson, C. P., Kallen, R. J., Levy, S. 370–387). New York: John Wiley & Sons.
E., Minshew, N. J., Prizant, B. M., Rapin, I., Rogers, S. J., Stone, Schopler, E., Reichler, R. J., Bashford, A., Lansing, M. D., & Marcus,
W. L., Teplin, S., Tuchman, R. F., & Volkmar, F. R. (1999). The L. M. (1994). Psychoeducational profile revised: Pro-ED.
screening and diagnosis of autistic spectrum disorders. Journal of Siegel, B. (1996). Pervasive developmental disorders screening test.
Autism and Developmental Disorders, 29, 439–484. Unpublished work.
Fombonne, E. (2003). Epidemiological surveys of autism and other Smith, B. Chung, M. C., & Vostanis, P. (1994). The path to care in
pervasive developmental disorders: An update. Journal of Aut- autism: Is it better now? Journal of Autism and Developmental
ism and Developmental Disorders, 33, 365–382. Disorders, 24, 551–563.
Garnett, M. S., & Attwood, A. J. (1998). The Australian Scale for Sparrow, S. S., Balla, D. A., & Cicchetti, D. V. (1997). Vineland
Asperger’s syndrome. In T. Attwood (Ed.), Asperger’s syn- social-emotional early chilhood scales: Manual. Circle Pines
drome. A guide for parents and professionals (pp. 17–19). MC: American Guidance Service.
London: Kingsley. Sugiyama, T. & Abe, T. (1989). The prevalence of autism in Nagoya
Howlin, P. (1997). Prognosis in autism: Do specialist treatments af- Japan: A total population study. Journal of Autism and Devel-
fect long-term outcome? European Child & Adolescent Psychi- opmental Disorders, 19, 87–96.
atry, 6, 55–72. Werner, E., Dawson, G., Osterling, J., & Dinno, N. (2000). Brief
Howlin, P., & Asgharian, A. (1999). The diagnosis of autism and report: recognition of autism spectrum disorder before one year
asperger syndrome: Findings from a survey of 770 families. of age: A retrospective study based on home videotapes. Journal
Developmental Medicine and Child Neurology, 41, 834–839. of Autism and Developmental Disorders, 30, 157–162.
Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic Wetherby, A. M., & Prizant, B. M. (1998). Infant / Toddler Checklist
interview-revised. Journal of Autism and Developmental Dis- for Communication and Language Development. Chicago,
orders, 24, 659–686. Applied Symbolix.

123