Examining the impact of

afterschool STEM programs

A paper commissioned by the Noyce Foundation

July 2014

Anita Krishnamurthi
Afterschool Alliance

Melissa Ballard
Afterschool Alliance

Gil G. Noam
Harvard University
Executive summary

Recent years have seen a growing recogni on strong STEM learning experiences are making an
that social and economic factors play a crucial impact on par cipa ng youth—youth not only
role in influencing academic success and that become excited and engaged in these fields but
young people require supports that extend develop STEM skills and proficiencies, come to
beyond the school walls. The no on of a value these fields and their contribu ons to
“learning ecosystem” is taking root, a concept society, and—significantly—begin to see
that places students at the center, with resources themselves as poten al contributors to the STEM
and supports organized around them. In such a enterprise.
system, schools are extremely important but are
just one of many influences in a young person’s For example:
life. In addi on to schools, the ecosystem
includes the influence of families and peers;  Program a endance and engagement in the
out‐of‐school‐ me offerings such as a erschool 4‐H Tech Wizards program is high—on
programs; and community resources such as average 95 percent of enrolled youth a end
science centers, libraries and media. sessions regularly and 95 percent of
par cipants stayed in the program for three
A erschool programs have long aimed to years to build successively on their tech skills.
posi vely influence students’ personal
 Girlstart A er School par cipants engage
development and support their social and
weekly in rigorous problem‐solving. Using
emo onal growth. Studies of high‐quality
inquiry and itera ve design, they generate
a erschool programs have found that
predic ons and hypotheses, document
par cipa ng children see a significant
observa ons, and prototype solu ons. At the
improvement in their self‐percep on, increased
end of the year, 91 percent of par cipants
posi ve social behavior and a decrease in
demonstrated mastery of scien fic inquiry
problem behaviors—changes that ul mately
and the engineering design process.
extend to school‐related behavior. Hence,
a erschool programs are increasingly seen as a  Every week at Science Club, an a erschool
key social and academic support for youth. program run as a partnership between
Northwestern University and the Boys & Girls
A erschool researchers and providers have Club of Chicago, youth work alongside
recognized that hands‐on, inquiry‐driven science, scien st mentors on fun, challenge‐based
technology, engineering and math (STEM) curricula. These units focus on health and
programs are in line with a erschool’s overall biomedical careers and development of
approach to educa on and have increasingly scien fic skills: experimental design,
begun to include STEM in their a erschool variables, data analysis and evidence‐based
offerings. A erschool programs that provide conclusions. As a result, 100 percent of

Examining the impact of a erschool STEM programs | 1

 elec ng advanced math and/or science
classes. Eighty‐five percent reported they find
engineering more interes ng and 83 percent
said they find science more interes ng. Eighty
‐one percent of par cipants said they can see
themselves working in technology, science or
engineering.
Photo courtesy of Techbridge

While the a erschool field has made great strides

Science Club youth see science as important
toward offering strong a erschool STEM
for their future careers, 30 percentage points
programming, much work remains to be done.
higher than control youth. Two independent,
Support systems are rapidly being assembled to
well‐controlled assessment methods reveal
support program providers, however assessing
increases in par cipants’ scien fic skills
and documen ng the impact of a erschool STEM
ranging from 25‐30 percent over
experiences remains a challenge due to the
non‐par cipa ng students.
diversity of programming. Efforts are being made
 Par cipants in Project GUTS, an a erschool to develop common frameworks for defining
compu ng program, come to understand that outcomes and assessments. Some common
STEM is integral to everyday life and can be assessment tools have been developed to
used to study and poten ally solve local support the evalua on needs of many youth‐
community problems. Project GUTS serving organiza ons and to aggregate data
curriculum covers real‐world topics of across many programs.
societal concern in fields such as ecology,
biology, social sciences, resource As we move forward, it is vital that local, state,
management and public health. When youth and federal educa on policies are inclusive of the
were asked how they would inves gate a various contributors to the learning ecosystem;
community problem, 80 percent suggested distribu ng the educa onal burden and
using computer modeling and simula on as a opportunity across a range of appropriate
technique to inves gate the issue. organiza ons so it does not rest so heavily on
schools. Policies must incen vize partnerships
 Career explora on is a primary focus of the and encourage data sharing between partners.
girl‐focused Techbridge program; all ac vi es Resources must be allocated to reflect an
integrate a career explora on component. understanding that the system as a whole is
Techbridge extensively uses professional accountable for success and individual players
female role models who visit programs and are accountable for their role in contribu ng to
share their experiences with par cipants; this success. Fortunately, good progress has been
students also go on worksite field trips. Eighty made in the past few years toward these goals,
percent of par cipants planned to take on though much remains to be done to pursue this
addi onal STEM learning opportuni es by essen al agenda.

Examining the impact of a erschool STEM programs | 2

Introduction
The last decade has seen growing interest in for children and youth as they grow into adults.
building the capacity of young people in science, The no on of a learning ecosystem, influenced by
technology, engineering and math (STEM) in the Bronfenbrenner’s ecological theory of
United States. The world has changed such that development (Bronfenbrenner, 1979), is not new.
ci zens require a greater level of STEM literacy to However, it is enjoying a renewed interest,
make decisions about complex issues such as par cularly in the context of STEM learning (see
global climate change, renewable energy sources the Research+Prac ce Collaboratory ini a ve2;
and gene cally modified foods. Increasingly, Barron, 2006; Hedegaard, 2009; Traphagen &
more and more jobs require proficiency in STEM Traill, 2014).
and there is great concern that without access to
adequate educa onal experiences, large Hence, as we consider redesigning learning to
segments of the popula on will be unable to become more student‐centered, we have to
par cipate effec vely in the modern workplace. draw on our knowledge of how people learn and
U.S. students are being outperformed in science where people learn in order to design a system
and math by young people in many other that more fully u lizes community assets, cultural
industrialized countries1, which has alerted many ins tu ons, and technological and media
of the need to improve this country’s K‐12 STEM resources. The landmark 2009 study from the
educa on. Na onal Research Council, Learning Science in
Informal Environments, highlighted this issue and
Efforts to address the situa on have primarily examined informal learning se ngs and out‐of‐
focused on educa on reform ini a ves targe ng school‐ me (OST) programs. The report
schools. While schools and teachers are a cri cal described an outcomes framework that included
part of young people’s STEM educa on, several strands of learning, two of which are
increasingly they are seen as one part of a larger par cularly relevant for a erschool programs—
“learning ecosystem,” one in which students are one strand states that learners in informal
at the center of the system rather than a environments will experience excitement,
par cular ins tu on of learning taking center interest, and mo va on to learn about
stage. In an ecosystem, the many components phenomena in the natural and physical world;
each have a differen ated and valuable role. The and the other states that learners will begin to
ins tu ons that represent the components of a think about themselves as science learners and
learning ecosystem work inten onally and develop iden es as young people for whom
collabora vely to create a rich educa onal world science ma ers.
1
29 countries outperformed the U.S. in mathema cs and 22 countries outperformed the U.S. in science, as measured by
the 2012 Program for Interna onal Student Assessment, or PISA.
2
The Research+Prac ce Collaboratory ini a ve seeks to engage researchers and prac oners to address gap between
educa onal research and prac ce in STEM educa on. The ini a ve takes an explicitly ecological perspec ve on learning,
and hence has a primary focus on furthering research and prac ce that address learning across se ngs. For more
informa on see www.researchandprac ce.org

Examining the impact of a erschool STEM programs | 3

Some of the hesita on to change current models Photo courtesy of The A er‐School Corpora on (TASC)
of teaching and learning may arise from a lack of
knowledge, trust, and perhaps a lack of
centralized control of the various spaces that
children and youth inhabit. And yet, spaces such
as a erschool and summer programs, libraries,
science centers, and other such OST se ngs offer
great resources and capabili es as partners in
providing STEM learning experiences for youth.
Many of these partners are sophis cated and
experienced in designing programs that not only
engage young people in STEM fields, but also
help them to develop tangible skills. It is vital to As schools have focused on measureable
acknowledge a erschool and other OST se ngs academic results in recent years, other aspects of
as necessary partners to the formal educa on schooling like music, art, learning healthy
system, and not just a luxury. behaviors and engaging in physical ac vity are
reduced or cut out of the school day due to
budget pressures. These subjects, o en
The role of a erschool in STEM learning considered ‘enrichment’ programming, are not
A erschool programs (used here to broadly refer seen as key aspects in preparing children for
to before‐school, a erschool and summer success. A erschool programs have stepped up
learning programs) are an important and growing their efforts, anchored in a holis c youth
part of the STEM educa on ecosystem in the development approach, to offer arts, sports and
United States. Roughly 8.4 million children other learning programs that offer youth a voice,
par cipate in these programs each year, many choice and control over their own learning.
from popula ons that are under‐represented in
STEM fields and careers. Children from ethnic Youth‐serving organiza ons such as 4‐H, Girls
backgrounds historically underrepresented in Inc., and Girl Scouts have long embraced STEM in
STEM a end in higher numbers—24 percent of their program models. And in the past few years,
African‐American, 21 percent of Hispanic and 16 the larger a erschool field has begun to
percent of Na ve American children a end recognize STEM as an important area of
a erschool programs, compared to the na onal enrichment. Historically, a erschool educators
average of 15 percent (A erschool Alliance, have experimented with a learning approach that
2009). Data from the Department of Educa on’s links school‐day academic science and math with
21st Century Community Learning Centers enriched learning in the a erschool hours.
program suggest that girls a end a erschool
programs in equal numbers to boys. These O en these connec ons were established
programs serve an important role in providing a through homework support, but increasingly
safe space for children while their parents are at STEM ac vi es in the a erschool space are hands
work, but they are increasingly offering ‐on or project‐based, serving to simultaneously
innova ve learning opportuni es. engage young people in STEM topics, teach them

Examining the impact of a erschool STEM programs | 4

scien fic reasoning and apply concepts learned evalua on data from a selec on of strong
during the school day. Today, many a erschool a erschool STEM programs and describes the
and summer programs are including STEM types of substan ve impacts these programs are
educa on as a standard part of their having on par cipa ng youth. The paper
comprehensive programming. Thus, STEM has concludes with recommenda ons for how we
become an important element in a learning can construct an integrated approach to
ecosystem rich in explora on, fun and hands‐on educa on that includes in‐school and OST
ac vi es that can lead to greater knowledge and experiences. Appendices provide details of the
interest in STEM coursework and careers. programs whose outcomes are featured in the
narra ve.
Does a erschool STEM have an impact?
Prac oners closest to students typically answer
this ques on affirma vely—they see youth
engaged in and excited about STEM ac vi es,
asking significant ques ons, and wan ng to learn
more. However, funders and policy makers o en
seek data that substan ates such claims and
demonstrates posi ve changes in a variety of
outcomes—interest and engagement in science,
greater knowledge of STEM careers, elec on of
school science classes, and, some mes, improved
test scores in science and math. While some
programs are able to achieve these outcomes
and document their successes, it is unrealis c to
expect that the a erschool field, in aggregate,
will be able deliver on all of these outcomes
immediately. Investments in a erschool STEM
programs are s ll growing, as is the infrastructure
that supports programs to offer more and
higher‐quality STEM opportuni es. We can get a
glimpse at the poten al for what high‐quality
a erschool and summer STEM programs can
offer by taking a look at what outcomes strong
a erschool STEM programs are currently
delivering.

This paper begins by presen ng some of the

research findings about the importance of
a erschool and other OST experiences for STEM
learning. It then proceeds to summarize

Examining the impact of a erschool STEM programs | 5

Research on afterschool and out-of-school-time learning

A well‐established research base indicates the social behavior and a decrease in problem
importance of cross‐sector learning and behaviors. Vandell, Reisner and Pierce (2007)
underscores the importance of such experiences looked at 35 quality a erschool programs and
and ac vi es outside of the classroom, especially found that students regularly par cipa ng in the
for under‐privileged popula ons (e.g., see Posner programs improved their work habits;
& Vandell, 1999; Bransford et al., 2006; Covay & demonstrated higher levels of persistence; and
Carbonaro, 2011). saw reduc ons in reports of misconduct, such as
skipping school.
The capability of OST learning experiences, such
as a erschool programs, to support the social In addi on to suppor ng a child’s development
and emo onal growth of students, and the and sense of worth, building social skills, and
genesis of these programs as ini a ves to igni ng his or her passion for learning,
posi vely influence students’ personal a erschool programs do have the ability to
development, is o en overlooked in an posi vely impact a child’s academic performance.
educa onal culture that devotes significant Both the CASEL meta‐analysis and the Vandell et
a en on to test scores and academic al. (2007) study found that, on average, students
achievement. There are many studies that par cipa ng in quality a erschool programs
document the myriad ways a erschool programs show gains in their school‐day performance.
impact youth. An o ‐cited meta‐analysis (Durlak Several addi onal evalua on studies (see
& Weissberg, 2007) by the Collabora ve for A erschool Alliance, 2013b) also demonstrate
Academic, Social and Emo onal Learning (CASEL), the ability of a erschool programs to support the
found that children par cipa ng in a erschool learning that takes place during the school day
programs saw a significant improvement in their and help boost students’ academic performance,
percep ons of themselves, improved posi ve and increase the likelihood of gradua ng from
high school—especially for students who have
fallen behind in school and need extra support
and mentoring.

New research by Deborah Vandell and colleagues

(Auger, Pierce, & Vandell, 2013) finds that not
only do students in a erschool programs see
academic gains, but a erschool programs are
also helping to close the achievement gap. The
disparity in academic performance that has been
documented between high‐ and low‐income
students disappears when low‐income
Photo courtesy of Tech Reach
elementary school students have high levels of

Examining the impact of a erschool STEM programs | 6

par cipa on in a erschool programs. The and thus was not something they could imagine
findings3 also show that students par cipa ng in for their own lives. Most of these students and
a erschool programs, especially those who their families also did not understand that
regularly par cipate, see gains in their math science could lead them to a diverse range of
achievement and other academic performance, promising careers. This result is in keeping with
improve their work habits, and have be er the findings of an important paper by Tai et al.
school day a endance. (2006), which showed that for students with
average achievement in math, an early interest
More specifically, the math achievement gap in pursuing physical science or engineering
between low‐ and high‐income students careers was a be er predictor of whether they
narrows when low‐income students a end would follow through on that interest compared
a erschool programs with greater frequency. to their peers who had higher achievement in
Conversely, the less o en low‐income students math but did not express a desire for such
par cipate in a erschool programs, the larger careers.
the math achievement gap between them and
their higher‐income peers. This has significant Genera ng interest and engagement in STEM is
implica ons for STEM educa on, as math a stated goal of many a erschool STEM
proficiency is o en the gatekeeper for many programs. The focus is thus on nurturing
STEM fields. curiosity and engagement with certain topics or
fields (Krapp et al., 1992; Noam & Shah, 2013),
Research on the role of STEM in and not necessarily on increased academic
a erschool achievement. Of course, the theory of change
implicit in much of the work is that increased
Lack of a STEM “iden ty” is o en cited as one of interest and engagement will translate into
the main reasons that young people do not improved achievement. Many a erschool
pursue STEM fields, an issue highlighted in a providers do not see it as their direct goal to
recent study from King’s College London improve test score results, but instead they
(ASPIRES Project, 2014). The ASPIRES study strive to increase involvement and explora on
found that most of the par cipa ng young with STEM, decrease anxiety around STEM, and
people held rela vely high career aspira ons, energize mo va on.
but a very small percentage of the study sample
(15 percent) aspired to be scien sts. They The approach taken by these program providers
reported liking science4, having posi ve views of is supported by theory and research. Hidi &
science and scien sts, and doing well in school Renninger (2006) have described how curiosity
in those subjects—but imagined scien sts to be and enjoyment are not only cri cal first steps,
mostly white, middle‐class, male and “brainy” but integral to increased and con nued

3
The full report is forthcoming, however findings are summarized in “The achievement gap is real: New research shows
a erschool is a real solu on linked to closing the gap” retrieved from www.expandinglearning.org/docs/The%
20Achievement%20Gap%20is%20Real.pdf.
4
Historically, science interest and iden ty have been studied more extensively than the other fields in STEM. Although
research is now catching up in engineering and technology educa on, science con nues to be used as a proxy for STEM.
Examining the impact of a erschool STEM programs | 7
 Photo courtesy of Science Ac on Club

engagement with STEM ac vi es and are part of so that young people can update their ideas
a cycle where interest and proficiency feed and about the types of people who work in these
build on each other. The Na onal Research fields.
Council’s framework (discussed previously; 2009)
begins with interest and engagement and A study by Dabney et al. (2011) supports this
progresses to a science iden ty. Development of hypothesis: they conducted a retrospec ve
a science or STEM iden ty involves mul ple examina on of the associa on between
pieces: ge ng young people interested in STEM university students’ current interest in STEM
topics and professions; developing competence careers and their par cipa on in secondary OST
and a sense of confidence; and ge ng youth to STEM ac vi es. The inves gators found a strong
envision themselves as contributors and posi ve correla on that showed par cipa on in
par cipants in this enterprise. OST STEM ac vi es was associated with STEM
career interests, for both young men and young
Such studies highlight the importance of women, and by a similar amount. The ability of
providing young people with opportuni es to OST programs to engage young girls in STEM has
engage in STEM fields and with professionals on been noted in other studies as well. For example,
their own terms and in ways that may not always a study by Tan et al. (2013), which examined
be possible during the school day: to design middle school girls’ experiences in school‐day
open‐ended projects that are not guaranteed to science classes and OST science clubs, suggests
succeed, to work on hands‐on projects that build that the posi ve science iden ty development
STEM skills and hence confidence in their that takes place within OST environments may
abili es, and to interact with STEM professionals impact girls’ science trajectories and career goals.

Examining the impact of a erschool STEM programs | 8

In the course of leading an a erschool STEM
program for middle school youth, Barton and Tan
(2010) inves gated the rela onship between
science learning, science iden ty, and student
agency (which refers to the ability of individuals
to engage in a field or prac ce as they choose).
Their study underscores the need to provide
opportuni es for young people to exercise STEM
agency in a socially appropriate context so they
Photo courtesy of Science Club
develop a sense of iden ty linked to the STEM
enterprise.
Noam et al. (2014) have taken this one step
The importance of increased STEM “dosage” (or further and studied the impact of higher doses of
dura on of exposure) was studied by Wai et al. STEM opportuni es within a erschool
5
(2010), who found that added exposure to programs . Program dosage (hours per week and
opportuni es for STEM engagement and number of weeks) was consistently found, across
learning, through individualized experiences such mul ple large datasets and analyses, to correlate
as compe ons, led mathema cally talented with science interest scores. On average, children
students to greater STEM achievement than their who spent more me involved in science
counterparts who lacked such opportuni es. The ac vi es reported significantly higher science
results from the 2009 Na onal Assessment of scores than children with less par cipa on.
Educa onal Progress (NAEP) in science also Addi onally, self‐reported interest in science and
showed that 4th graders who par cipated in improved skills, like cri cal thinking, increased in
“hands‐on science ac vi es” and 8th and youth program par cipants.
12th graders who did “science‐related ac vi es
outside of school” showed a significant increase
in test scores compared to those who did not.
Further, it appears that the dosage ma ered.
Students involved almost daily in hands‐on
science, both in‐ and out‐of‐school, scored be er
on the 4th grade assessment than those who only
par cipated once or twice a month. A erschool
programs offer an important opportunity to
increase the dosage of STEM experiences for
young people.

5
For this study, Noam et al. (2014) used the assessment tool, the Common Instrument of Interest in Science, which is a
survey used to assess student interest and engagement in science. For more informa on, see www.pearweb.org/tools/
commoninstrument.html.

Examining the impact of a erschool STEM programs | 9

What impacts do strong afterschool STEM programs
have on youth?

A erschool programs that offer high‐quality However, there is no widely‐adopted measure of

STEM learning experiences have a significant program quality or of related outcomes used in
impact on par cipa ng youth—they cul vate a erschool and summer programs. So in order to
students’ interest in STEM and build STEM skills examine the poten al range of the impacts and
and proficiencies among diverse groups of young outcomes a erschool STEM programs can have,
people. Specifically in rela onship to interest, we gathered outcomes data from a sample of
students in a recent study (Robertson & Noam, programs that have a reputa on for excellence.
2014) reported significantly greater science This is not intended as a representa ve sample of
interest in a erschool and summer programs the a erschool field as a whole, but it does allow
rated high in quality compared to programs rated for an understanding of what strong programs
average or low in quality. Program quality in can achieve. This is especially important because
these two studies was measured by the the OST STEM field is rela vely young, and strong
Dimensions of Success (DoS) tool6, a program outcomes should only be expected from strong
quality assessment rubric. This parallels findings programs (Noam & Shah, in press). Table 1
about the effect of quality programming in presents brief descrip ons of the featured
rela onship to other, non‐STEM outcomes. programs. More extensive descrip ons can be
found in Appendix 2.

Table 1. Brief descrip ons of the programs featured in this paper.

Program Brief descrip on

4‐H Tech Wizards A 4‐H a erschool program that provides targeted student groups with technology training. It is
Mul ple states delivered in partnership with schools, volunteers, libraries, nonprofits and other community
partners.

Build IT An a erschool and summer curriculum for middle school youth offered by Girls Inc. to develop
U.S. and Canada informa on technology (IT) fluency, interest in mathema cs and knowledge of IT careers.

Computer An OST program in which youth from underserved communi es work with adult mentors to
Clubhouse explore their own ideas, develop new skills and build self‐confidence through the use of
Interna onal technology.

FUSE A strategy used in seven ci es to ins tu onalize engaging, inquiry‐based informal STEM
(Fron ers in Urban educa on na onally. FUSE is a project of Every Hour Counts and The A er‐School Corpora on.
Science Explora on)

6
Read more about the Dimensions of Success (DoS) tool here www.pearweb.org/tools/dos.html
Examining the impact of a erschool STEM programs | 10
Table 1, con nued. Brief descrip ons of the programs featured in this paper.

Program Brief descrip on

Girlstart An intensive a erschool and summer program for girls that provides free STEM programming
Aus n, Texas every week throughout the school year at partner schools.

Project GUTS An a erschool program in which middle school students learn cu ng‐edge compu ng
Santa Fe, New Mexico methods to solve modern‐day problems.

Science Ac on Club An a erschool program that offers hands‐on science inves ga ons and par cipa on in ci zen
San Francisco, California science projects all in a fun, club‐like environment.

Science Club An a erschool program run in partnership between Northwestern University and the Boys &
Chicago, Illinois Girls Club of Chicago in which graduate student mentors lead small groups of students
through designing and running hands‐on science experiments.

Science Minors Club An outreach ini a ve of the Museum of Science and Industry aimed at increasing interest in
Chicago, Illinois science in underserved neighborhoods by engaging students in places where they already
spend their me a er school, such as community‐based organiza ons and schools.

Tech Reach An outreach program of the Thinkery, a hybrid science and technology center/children’s
Aus n, Texas museum, focused on crea ve compu ng.

Techbridge An a erschool and summer program focusing on hands‐on projects and career explora on to
Bay Area, California inspire girls in science, technology and engineering.

Each program had its own methodology for 1. Ac vely par cipa ng in STEM learning
collec ng data on outcomes and impacts, ac vi es;
however we have organized the findings of their
evalua on results according to a framework of 2. Demonstra ng curiosity about STEM topics,
youth outcomes for STEM learning in a erschool concepts or prac ces;
that was developed in a recent consensus study
3. Developing abili es to produc vely engage in
(A erschool Alliance, 2013a), represen ng the
STEM processes of inves ga on;
opinions of expert a erschool prac oners and
other stakeholders. 4. Developing abili es to exercise STEM‐
relevant life and career skills;
The consensus study concluded that a erschool
STEM programs can help young people to: (a) 5. Coming to understand the role that STEM
develop an interest in STEM and STEM learning plays in solving societal issues; and
ac vi es; (b) develop a capacity to produc vely
6. Developing an increased awareness of STEM
engage in STEM learning ac vi es; and (c) come
professions.
to value the goals of STEM and STEM learning
ac vi es. The indicators of progress toward such Next, we summarize the main findings with
outcomes include: respect to these six indicators.

Examining the impact of a erschool STEM programs | 11

  Across more than 100 drop‐in based
Summary of impacts and outcomes
Computer Clubhouse sites, 83 percent of
Despite the specific nature of each of the par cipants visited their clubhouse at least
a erschool programs and their evalua ons, there weekly, and 47 percent every day. Ninety‐one
were several themes that emerged that help us percent visited for at least one hour and 37
be er understand the impacts of OST STEM in percent of youth visited their clubhouse for
general. more than three hours at a me.

1. The programs are successful in engaging and  In Project GUTS, middle school students are
retaining large numbers of students from presented with a challenging compu ng
diverse popula ons in STEM. curriculum and 82 percent persist in the
program to complete a working computer
All programs featured in this paper focus on simula on model (22 to 26 weeks).
engaging underserved youth in high‐need
communi es. On average, 75 percent of students 2. Young people in these programs express
served are eligible for free or reduced price curiosity and interest in STEM subjects, in
lunch7. High percentages of minority students are ways that extended that interest in school
enrolled; an average of 82.5 percent of youth are and out of school.
of La no, African‐American, Na ve American or
Asian/Pacific Islander8 ethnic background9. Youth For example:
have many op ons for what they do a er school,
and young people consistently choose to a end  Par cipa on in Girlstart resulted in the
these programs. They par cipate in high numbers extension of interest in science outside the
and return regularly, reflec ng a keen interest in program. Eighty‐six percent of par cipants
the hands‐on STEM learning they’re being agreed with the statement “I want to try
exposed to. more science ac vi es.” Eighty‐four percent
reported interest in taking further STEM
For example: classes in middle or high school. In 2012, 58
percent a ended Girlstart’s Girls in STEM
 Program a endance and engagement in the conference.
4‐H Tech Wizards is high: on average 95
percent of enrolled youth a end sessions  Sixty‐five percent of students in Project GUTS
regularly and 95 percent of par cipants strongly agreed that par cipa ng in the
stayed in the program for three years to program made them more excited to do and
complete all three skill levels. learn science and technology in school.

7
7 of 11 programs reported data on the eligibility status for free or reduced price lunch of their par cipants.
8
While Asian‐Americans overall are not underrepresented in STEM, we chose to include Asian/Pacific Islander in this
calcula on because students in these programs came from such high‐poverty backgrounds. The Na onal Center for
Science and Engineering Sta s cs at the Na onal Science Founda on places Pacific Islanders in the “Other” category,
which is underrepresented.
9
8 of 11 programs reported data on the ethnic background of par cipants.

Examining the impact of a erschool STEM programs | 12

 opportuni es for youth to use tools like a
hand lens, calorimeter and rulers to make
observa ons, take measurements or collect
data.

 Science Club’s curriculum strongly

emphasizes teaching scien fic skills through
the scien fic method and engineering design
process. Youth design their own experiments
each week and collect data to make
evidence‐based conclusions. As a result,
Science Club youth outperform their matched
Photo courtesy of Science Ac on Club
peers in Chicago Public Schools’ annual
science fair. Youth are assigned an ini al
 Seventy‐seven percent of youth in Science ap tude category (low/middle/high) and the
Ac on Club reported that “being in Science effect of Science Club par cipa on is
Ac on Club makes me want to learn more equivalent to raising youth scores by a full
about science outside of school.” ap tude category (e.g. scores for Science
Club youth in the middle ap tude category
are sta s cally indis nguishable from non‐
3. As they par cipate in these programs, young par cipa ng high ap tude youth).
people gain real skills and the ability to
produc vely engage in STEM processes of  Youth in 4‐H Tech Wizards are closely
inves ga on. mentored by professionals specializing in
emerging technologies and spend three years
Par cipants in these programs become familiar progressively developing their technology
with concepts such as the engineering design skills. As a result, 95 percent of 4‐H Tech
process, scien fic enquiry and problem solving. Wizards par cipants demonstrate mastery of
They apply these ideas to hands‐on projects that skills in website development, video and
include computer coding; Web development; podcast produc ons, GIS and GPS
designing hovercra and solar cars; and building technologies, and LEGO robo cs.
robots, geographic informa on systems (GIS) and
other such technologies.  Techbridge girls consistently u lize the
engineering design process. For example,
For example: they brainstorm on how to make a device to
assist a disabled person, design a prototype
 Observa ons of Science Minors Club’s staff and redesign the model based on use.
revealed that 80 percent encouraged youth Ninety‐five percent of par cipants said they
to formulate testable ques ons and 93 understand it can take many tries to solve a
percent fostered the collec on of data and problem; 91 percent said they try harder to
recording of observa ons. At 93 percent of overcome a challenge; and 81 percent said
sites, staff was observed providing they are be er at problem‐solving. More

Examining the impact of a erschool STEM programs | 13

 than 93 percent said they know more about supported youth in making connec ons
how things work, like circuits and simple between their work and their everyday lives.
machines, and 80 percent said they are
be er at using new computer programs.  Tech Reach students made sta s cally
significant improvements in communica on,
collabora on and compu ng skills on an
4. Youth learn essen al STEM‐relevant life and
assessment of 21st century problem solving
career skills, such as working in teams and
skills. Par cipants improved their confidence
collabora ng effec vely, as well as making
with technology—90.6 percent of students
presenta ons to audiences.
reported that “I am good at computers” and
more than 88 percent said “I like to figure
Youth become demonstrably more confident
out how something works.”
about their abili es to work on complex tasks
and projects as well as persistence in the face of  Computer Clubhouse youth who visited
ini al failures. For example, 90 percent of more frequently and stayed longer exhibited
Girlstart par cipants responded posi vely to the higher levels of collabora on. Ninety‐two
statement “I understand that it is okay if my percent of youth plan to use skills acquired
Girlstart ac vity does not work on the first try.” in the clubhouse in their future careers.
Youth in these programs also start to use Eighty‐two percent of alumni say they are
scien fic terminology appropriately, consider currently using the tools and technologies
mul ple perspec ves, and even begin to they learned at the clubhouse professionally
understand the importance of peer‐review in and/or personally.
scien fic research.

5. Equally importantly, par cipants in many of

Other examples:
these programs come to understand the
 All programs observed in FUSE received a top value of STEM in contribu ng to society and
score (5/5) in an observa on‐based solving global and local problems. They begin
evalua on tool for providing opportuni es to to see how STEM in mately connects to
prac ce group process skills, such as ac vely their everyday lives.
listening, contribu ng ideas or ac ons to a
group, doing a task with others, or taking Other examples:
responsibility for a part of the project.
 In observa ons of Science Minors Club sites,
 Observa ons of Science Minors Club staff 86 percent of facilitators supported youth in
indicated that 100 percent u lized making connec ons to their everyday lives
coopera ve groups and individual roles to and 78 percent of par cipa ng youth
promote collabora on between youth indicated that they use science in their
par cipants. Eighty percent of observed sites everyday lives.
provided opportuni es for youth to report
 Project GUTS curriculum covers real‐world
out their findings and communicate their
topics of societal concern in fields such as
ideas to the broader group and 86 percent

Examining the impact of a erschool STEM programs | 14

 ecology, biology, social sciences, resource
management and public health. Par cipants
demonstrate their understanding that STEM
relates to everyday life and can be used to
study and poten ally solve local community
problems through their individual projects
and presenta ons. When asked how they
would inves gate a new community issue, 80 Photo courtesy of Girlstart
percent suggested using computer modeling
and simula on as a technique to inves gate nuanced understanding of the types of careers
the issue. that require knowledge and skills in technology
and math, and—for many girls—a clearer picture
 Science Ac on Club youth are engaged with
of the coursework involved in training for an IT‐
increasing scien fic understanding of their
influenced career. Results from the other
local environments by contribu ng to
featured programs echo these outcomes. Youth
authen c science in ci zen science projects.
not only develop an interest in STEM topics, but
Seventy‐five percent of youth respondents
also gain an understanding of what related
agreed that “in Science Ac on Club, I do
careers might involve.
ac vi es that contribute to real science
research.” Sixty‐two percent agreed that
Other examples:
“Science Ac on Club makes me wonder and
ask ques ons about the natural world.”  Science Club par cipants are exposed to a
wide array of STEM professions through
curricular units covering topics ranging from
6. Students in these programs display an
neuroscience to food science. Graduate
increased awareness of career op ons, as
student mentors, who lead these curricular
well as a nuanced understanding of those
units, similarly come from diverse scien fic
careers.
disciplines. Youth par cipants o en ask their
mentors ques ons about their career path,
The Build IT curriculum, delivered through Girls,
life in the lab, and their lives outside of
Inc., gives a powerful demonstra on of the
school. Par cipants regularly describe their
impact a erschool programs can have on young
favorite school subject, their dream jobs and
people’s choice of careers. Girls in the program
their hobbies in program surveys. One
showed sta s cally significant improvement on
hundred percent of students felt science was
the survey items related to knowing what classes
relevant to their future careers a er
to take in high school for an IT career. Interviews
par cipa on in Science Club, up from 70
with female par cipants as well as a erschool
percent before par cipa on. Students could
staff document that par cipa ng in Build IT has
also more specifically describe STEM careers.
made a no ceable difference in how girls view
technology careers, including having a more  Each Girlstart A er School lesson includes a
posi ve image of technology careers, a more discussion of a STEM career, including

Examining the impact of a erschool STEM programs | 15

 educa onal prerequisites and how to enter A select number of programs have been able to
that career. Ninety‐four percent of track their students longitudinally. An evalua on
par cipants demonstrated awareness that of Project Explora on, a Chicago a erschool and
success in STEM can broaden their career summer STEM program, found that high‐school
op ons. Sixty‐eight percent expressed strong gradua on rates for par cipants are higher
interest and 93 percent indicated at least compared their peers, and their aspira ons to
moderate interest in entering a STEM career. a end college and major in a STEM field are
Almost all par cipants—97 percent— higher (Chi et. al., 2010). An evalua on of
expressed an intent to a end college a er Techbridge focused on long‐term educa onal
high school, which is notable among a outcomes (2006‐2012) found that par cipa on
popula on in which 55 percent are first helped girls achieve greater academic success. In
genera on college aspirants. comparisons made between girls who
par cipated in Techbridge and those who did not
 A key component of Techbridge is its use of (at the same school), findings indicated that:
STEM professionals as role models, which
provides opportuni es for youth par cipants  Techbridge girls have a higher weighted total
to engage in career explora on. As a result, cumula ve GPA at high school gradua on
94 percent knew more about different kinds (3.32) than girls who have not par cipated in
of jobs. Eighty‐one percent of girls said that Techbridge (2.94).
because they engaged with role models and
 Techbridge girls scored an average of 26
par cipated in field trips, they were more
interested working in technology, science or points higher (321 vs. 295) than non‐
engineering. Techbridge girls on the California Standards
Algebra II test, and an average of 43 points
 High numbers of FUSE par cipants believe higher (365 vs. 322) on the California
the program contributed to their science Standards Biology test.
career aspira ons—79.5 percent reported
 Girls who par cipated in Techbridge are 12
that par cipa ng in their a erschool science
program(s) “made the idea of a job in science percent more likely to enroll in Advanced
when I am older seem more possible”; 73.2 Placement (AP) Calculus (AB or BC) than girls
percent reported that it “made me more who have not. Furthermore, Techbridge
interested in a science job when I am older”; par cipants have a higher average grade in
and 69 percent reported that it “made me calculus than non‐par cipants (87 percent vs.
feel more sure that I want a job in science 82 percent).
when I am older.” Eighty‐eight percent of
students reported that par cipa ng in FUSE Recently, Girlstart released an analysis of the
“made me more confident that I could do impact of their programs (Bussiere & Hudgins,
well in science classes in college.” 2014). The primary ques ons for the analysis
were: (1) Does Girlstart help girls perform be er
7. Finally, a erschool STEM programs can have on standardized math and science tests? (2) Do
an impact on academic performance. Girlstart par cipants choose advanced STEM

Examining the impact of a erschool STEM programs | 16

classes and elec ves at higher rates than non‐  Girlstart par cipants are also more likely to
par cipa ng peers? enroll in advanced and pre‐AP math and
science—1.58 advanced courses per girl,
 Girlstart A er School par cipant test scores compared to 1.00 per non‐par cipant girl. In
show that Girlstart girls are doing be er than conjunc on with Techbridge’s findings, this
their non‐par cipant peers in both math and provides strong evidence that par cipa on in
science. OST STEM programs can be a significant
factor in encouraging girls to pursue
◦ In spring 2013, 76 percent of Aus n advanced math and science classes in school.
Independent School District (AISD)
Girlstart A er School par cipants passed
Many other programs observe that student
the 5th grade State of Texas Assessments
impacts extend to academics or the school day.
of Academic Readiness (STAAR) science
However, such measures are difficult to capture
test, while only 41 percent of comparison
due to the lack of resources available for
group members passed (64 percent of all
longitudinal tracking of students and the
students AISD partner schools passed–
difficulty in having school districts share student‐
including both boys and girls).
level data. The most cri cal limita on in
◦ Over a two‐year period, 71 percent of evalua on efforts of a erschool and other OST
STEM programs is the availability of validated
AISD Girlstart par cipants passed the 5th
tools (Noam and Shah, 2013), as well as the
grade STAAR science test, while only 48
amount of me and resources available for
percent of comparison group members
planning and implemen ng evalua on.
(and 62 percent of all students) passed.

As a final note, the featured programs had

◦ On the STAAR math test, 85 percent of
chosen, prior to the wri ng of this paper, which
Girlstart par cipants passed, compared to
outcomes to assess and evaluate based on their
70 percent of comparison group members
defined programma c goals and funder
(and 73 percent of all students at AISD
requirements. Addi onally, they did not form
partner schools). their evalua on plans around the youth
outcomes framework discussed above. The goal
 This impact is not limited to AISD. In 2013, 87
of this sec on was not to describe a
percent of Girlstart A er School par cipants
representa ve sample of programs and students,
at the Georgetown Independent School
but a nominated collec on of strong programs
District (GISD) partner schools passed 5th
and their impacts. Appendix 3 presents addi onal
grade math, compared to 77 percent of
evalua on data from these programs set within
students overall at partner schools. Over a
the youth outcomes framework.
two‐year period, 82 percent of Girlstart
par cipants passed the 5th grade STAAR test
in science compared to 77 percent of
students overall at GISD partner schools.

Examining the impact of a erschool STEM programs | 17

Conclusions

A erschool programs present a great and document successes and challenges more
opportunity to engage a diverse group of young systema cally.
people in the STEM fields. It may be temp ng to
view such programs as compensa ng for needs The program evalua ons and studies that are
not currently met via formal educa on, but it is included in this paper point to important
crucial to remember that these programs serve a contribu ons a erschool STEM programs can
unique role. A erschool programs complement make. However, the sample of programs
the learning that occurs during the school day described here is not representa ve of the
with experiences rooted in youth development a erschool field as a whole, but instead serves as
principles and have many impacts that go beyond a glimpse of the poten al outcomes possible if
STEM‐specific outcomes. As discussed here, in‐ the a erschool field is further engaged and
school and OST programs are one set of strengthened. Leading youth‐serving
components in a learning ecosystem that organiza ons such as 4‐H, Big Brothers Big
includes family, media, and cultural ins tu ons. Sisters, Boys & Girls Clubs of America, Girls Inc.,
and the YMCA of the USA have recently joined
A erschool providers have embraced STEM in forces to make STEM a priority. This ini a ve has
part because the hands‐on learning it affords fits the poten al to bring quality curricula, ac vi es
well with the core tenets of youth development, and mentoring to millions of children and youth.
such as offering youth a voice and affording them
opportuni es to discover and grow their own It is difficult to isolate various environmental
interests. STEM programming in a erschool has factors and a ribute cause and effect specifically
grown at an extremely rapid pace in the past few to a erschool interven ons. Hence, there is a
years. Several support systems are now in place need to conduct further research to be er
to aid both the growth in the number of understand these issues, specifically research
programs offering STEM, as well as the quality of that uses control groups in order to strengthen
such programs. City‐based intermediaries and the evidence that posi ve changes can be
statewide a erschool networks are able to a ributed to the learning experience, and not
coordinate resources and support programs; simply a difference in interest among students
blended professional development models such who choose to par cipate in a erschool STEM.
as Click2Science PD10 are helping to meet the Absent such efforts, it will always be difficult to
professional development needs of a erschool understand the selec on bias of students who
educators; and investments in developing join programs and how it may differ from those
outcomes frameworks and assessment tools are who do not. In addi on, these groups have to be
greatly helping the field to set appropriate goals studied within the same schools and districts in

10
Click2Science is an online STEM professional development tool for out‐of‐school‐ me professionals and is available at
www.click2sciencepd.org

Examining the impact of a erschool STEM programs | 18

order to dis nguish effects that come from system as a whole is accountable for success and
schools and those that are related to a erschool all of the various players are accountable for their
programs and other parts of the learning role in contribu ng to this success. The
ecosystem. educa onal burden and opportunity cannot rest
solely on schools; it must be distributed across a
Nonetheless, the consistent findings across range of appropriate organiza ons. Policies must
increasing numbers of a erschool programs give also incen vize partnerships between these
us confidence that we are seeing a real organiza ons and encourage data to be shared
phenomenon: students par cipa ng in OST among partners. Resources must be allocated for
programs have immediate and long‐term gains professional development to support program
on a number of STEM‐related dimensions. providers. Finally, there is a tremendous need to
develop assessment instruments that can
As the idea of cul va ng learning ecosystems measure the impacts and outcomes of the
gains ground and a erschool programs are system and the individual players appropriately.
increasingly included as a key strategy in STEM Progress has been made toward this goal, though
educa on improvement, it is necessary to remain much remains to be done to pursue this essen al
vigilant about the roles and values assigned to agenda.
the components of an ecosystem. A true
ecosystem assigns different roles to the different
players, only some of which are duplica ve.
Local, state and federal policies and resource
alloca on must reflect an understanding that the

Photo courtesy of Project GUTS

Examining the impact of a erschool STEM programs | 19

References

A erschool Alliance. (2013a). Defining youth outcomes for STEM learning in a erschool. Washington,
D.C. Retrieved from www.a erschoolalliance.org/STEM_Outcomes_2013.pdf

A erschool Alliance. (2013b). Evalua ons backgrounder: A summary of formal evalua ons of
a erschool programs’ impact on academic, behavior, safety and family life. Washington, D.C.
Retrieved from www.a erschoolalliance.org/documents/Evalua ons_Backgrounder_2013.pdf

A erschool Alliance. (2009). America A er 3 PM. Washington, D.C. Retrieved from

www.a erschoolalliance.org/AA3_Full_Report.pdf

ASPIRES Project (2014). ASPIRES: Young people’s science and career aspira ons, age 10‐14. London:
King’s College London. Retrieved from www.kcl.ac.uk/sspp/departments/educa on/research/aspires/
ASPIRES‐final‐report‐December‐2013.pdf

Auger, A., Pierce, K., & Vandell, D. L. (2013). Narrowing the achievement gap: Consistency and
intensity of structured ac vi es during elementary school. Presented at the biennial mee ng of the
Society for Research in Child Development. Sea le, WA.

Barron, B. (2006). Interest and self‐sustained learning as catalysts of development: A learning ecology
perspec ve. Human Development, 49, 193–224. DOI:10.1159/000094368

Bransford, J., Stevens, R., Schwartz, D., Meltzoff, A., Pea, R., Roschelle, J., Vye, N., Kuhl, P., Bell, P.,
Barron, B., Reeves, B., & Sabelli, N. (2006). Learning theories and educa on: Toward a decade of
synergy. In: P. A. Alexander & P. H. Winne (Eds.), Handbook of educa onal psychology. Second edi on
(pp. 209–244). Mahwah, NJ: Lawrence Erlbaum Associates.

Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design.
Cambridge, MA: Harvard University Press.

Bussiere, L., & Hudgins, T. (2014). Girlstart A er School: Analysis of program impact on par cipants’
academic achievement. Retrieved from h p://issuu.com/girlstart/docs/a er_school_impact_analysis

Calabrese Barton, A., & Tan, E. (2010). “We be burnin!": Agency, iden ty, and science learning.
Journal of the Learning Sciences, 19(2), 187–229. DOI: 10.1080/10508400903530044

Chi, B., Snow, J. Z., Goldstein, D., Lee, S., & Chung, J. (2010). Project Explora on 10‐year retrospec ve
program evalua on summa ve report. The Center for Research, Evalua on & Assessment, Lawrence
Hall of Science. Retrieved from www.projectexplora on.org/wp‐content/uploads/2013/04/lhs‐final‐
report‐10‐year‐eval.pdf

Examining the impact of a erschool STEM programs | 20

Covay, E., & Carbonaro, W. (2010). A er the bell: Par cipa on in extracurricular ac vi es, classroom
behavior, and academic achievement. Sociology of Educa on, 83(1), 20–45. DOI:
10.1177/0038040709356565

Dabney, K., Tai, R., Almarode, J., Miller‐Friedmann, J., Sonnert, G., Sadler, P., & Hazari, Z. (2011). Out‐
of‐school me science ac vi es and their associa on with career interest in STEM. Interna onal
Journal of Science Educa on, Part B: Communica on and Public Engagement, 2(1), 63‐79. DOI:
10.1080/21548455.2011.629455

Durlak, J. A., & Weissberg, R. P. (2007). The impact of a er‐school programs that promote personal
and social skills. Chicago, IL: Collabora ve for Academic, Social, and Emo onal Learning. Retrieved
from www.casel.org

Hedegaard, M. (2009). Children's development from a cultural‐historical approach: Children's ac vity

in everyday local se ngs as founda on for their development. Mind, Culture, and Ac vity, 16(1), 64–
82. DOI: 10.1080/10749030802477374

Hidi, S., & Renninger, K. A. (2006). The four phase model of interest development. Educa onal
Psychologists, 41(2), 111–127. DOI: 10.1207/s15326985ep4102_4

Krapp, A., Hidi, S., & Renninger, K. A. (1992). Interest, learning, and development. In K. A. Renninger,
S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 3–25). Hillsdale, NJ:
Lawrence Erlbaum Associates, Inc.

McCreedy, D., & Dierking, L. D. (2013). Cascading influences: Long‐term impacts of informal STEM
experiences for girls. Philadelphia, PA: The Franklin Ins tute. Retrieved from www.fi.edu/sites/
default/files/cascading‐influences.pdf

Na onal Research Council. (2009). Learning science in informal environments: People, places, and
pursuits. Washington, D.C.: The Na onal Academies Press. Retrieved from www.nap.edu/
catalog.php?record_id=12190

Noam, G. G., & Shah, A. M. (2013). Game changers and the assessment predicament in a erschool
science. Belmont, MA: Program in Educa on, A erschool, and Resiliency. Retrieved from
www.a erschoolalliance.org/documents/STEM/Noam_Shah.pdf

Noam, G. G., & Shah, A. M. (In press). Informal science and youth development: Crea ng convergence
in out‐of‐school me. Chapter to appear in the Na onal Society for the Study of Educa on (NSSE)
Yearbook 2013.

Examining the impact of a erschool STEM programs | 21

Noam, G. G., Robertson, D. L., Papazian, A. & Guhn, M. (2014, in prep). The development of a brief
measure for assessing science interest and engagement in children and youth: Structure, reliability
and validity of the Common Instrument. Manuscript in prepara on by Program in Educa on,
A erschool and Resiliency (PEAR), Harvard University.

Organisa on for Economic Co‐opera on and Development. (2014). PISA 2012 results: What students
know and can do–Student performance in mathema cs, reading and science (Volume I, Revised
edi on, February 2014), PISA, OECD Publishing. Retrieved from www.oecd.org/pisa/keyfindings/pisa‐
2012‐results‐volume‐i.htm

Posner, J., & Vandell, D. L., (1999). A er‐school ac vi es and the development of low‐income urban
children: A longitudinal study. Developmental Psychology, 35(3), 868‐879.

Robertson, D. L. & Noam, G. G. (2014). U lizing the PEAR Common Instrument (CI) and the
Dimensions of Success (DoS) program observa on tools: A report to the Private Industry Council (PIC)
and the Boston STEM Network. Belmont, MA: Program in Educa on, A erschool and Resiliency
(PEAR).

Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science, 312
(5777), 1143‐1144.

Tan, E., Calabrese Barton, A., Kang, H., & O’Neill, T. (2013). Desiring a career in STEM‐related fields:
How middle school girls ar culate and nego ate iden es‐in‐prac ce in science. Journal of Research
in Science Teaching, 50(10), 1143–1179. DOI: 10.1002/tea.21123

Traphagen, K., & Traill, S. (2014). How cross‐sector collabora ons are advancing STEM learning. Los
Altos, CA: The Noyce Founda on. Retrieved from www.noycefdn.org/documents/
STEM_ECOSYSTEMS_REPORT_140128.pdf

Vandell, D. L., Reisner, E. R., & Pierce, K. M. (2007). Outcomes linked to high‐quality a erschool
programs: Longitudinal findings from the study of promising a erschool programs. Report to the
Charles Stewart Mo Founda on. Retrieved from www.gse.uci.edu/childcare/pdf/a erschool/PP%
20Longitudinal%20Findings%20Final%20Report.pdf

Wai, J., Lubinski, D., Benbow, C. P., & Steiger, J. H. (2010). Accomplishment in science, technology,
engineering, and mathema cs (STEM) and its rela on to STEM educa onal dose. Journal of
Educa onal Psychology, 102(4), 860‐871.

Examining the impact of a erschool STEM programs | 22

Appendix I: Framework of Youth Outcomes
The youth outcomes framework for a erschool STEM was developed in a consensus study by expert
a erschool prac oners and stakeholders (A erschool Alliance, 2013a).

Outcome Indicators Sub‐Indicators

Through a erschool You know or can see If you had appropriate tools, you could document the following types of evidence:
STEM programs, that children and
children and youth: youth demonstrate:

Ac ve engagement and focus in STEM learning ac vi es

Persis ng in a task or program; sharing knowledge and ideas; expressing enthusiasm, joy, etc.
Ac ve
Pursuit of other out‐of‐school‐ me STEM learning opportuni es
A. Develop an par cipa on in
Enrolling in programs; a ending programs regularly; repor ng performing STEM‐related ac vi es at home
interest in STEM learning
Pursuit of school STEM learning opportuni es
STEM and opportuni es Par cipa ng more ac vely in school STEM ac vi es; enrolling in courses; selec ng special programs or
STEM learning schools; improving academic achievement
ac vi es.
Curiosity about Ac ve inquiries into STEM topics, concepts or prac ces
“I like to do this.” Exploring ideas verbally or physically; ques oning, hypothesizing, tes ng
STEM topics,
Ac ve informa on‐seeking about mechanical or natural phenomena or objects
concepts or Conduc ng internet searches for more informa on; ge ng books/journals about STEM; watching TV
prac ces programs on science, etc.

Demonstra on of STEM knowledge

Demonstra ng increase in knowledge in specific content areas; making connec ons with everyday world;
Ability to
using scien fic terminology
produc vely
Demonstra on of STEM skills
engage in STEM Formula ng ques ons; tes ng, exploring, predic ng, observing, collec ng and analyzing data
B. Develop a processes of
capacity to Demonstra on of an understanding of STEM methods of inves ga on
inves ga on Demonstra ng understanding of the nature of science; using evidence‐based reasoning and argumenta on;
produc vely demonstra ng engineering design prac ces
engage in
STEM learning Demonstra on of mastery of technologies and tools that can assist in STEM inves ga ons
ac vi es. Developing capacity to use measurement and other scien fic instruments; running computer programs for
Ability to data analysis; developing effec ve methods to communicate findings
“I can do this.” exercise STEM‐ Demonstra on of ability to work in teams to conduct STEM inves ga ons
relevant life Communica ng effec vely with team members; collabora ng effec vely with team members;
demonstra ng leadership on the team
and career skills
Demonstra on of applied problem‐solving abili es to conduct STEM inves ga ons
Engaging in cri cal thinking; ques oning, sequencing, reasoning

Demonstra on of an understanding of relevance of STEM to everyday life, including personal life

Referencing examples of STEM in everyday life: everyday problems
Understanding Demonstra on of knowledge of important civic, global and local problems that can be addressed by STEM
of value of Contribu ng to projects that address a community need; developing awareness of how STEM is implicated in
C. Come to value STEM in society larger societal issues
the goals of Demonstra on of awareness of opportuni es to contribute to society through STEM
STEM and Engaging in a service‐learning project
STEM learning
ac vi es. Development of an understanding of the variety of STEM careers related to different fields of study
Gaining knowledge about relevant professions; gaining knowledge of where such jobs and careers exist
“This is important Demonstra on of knowledge of how to pursue STEM careers
Awareness of
to me.” Acquiring knowledge of what courses are needed to prepare for or pursue STEM degrees; declaring STEM
STEM interests or majors
professions
Demonstra on of awareness that STEM is accessible to all
Expressing a desire to meet role models; declaring STEM interests and majors; desiring to become a role
model to pave the way for others

Examining the impact of a erschool STEM programs | 23

Appendix II: Descriptions of selected afterschool
STEM programs with strong youth outcomes
4-H Tech Wizards Build IT Computer Clubhouse

Location Originally in Oregon; now in 26 states U.S. and Canada International (100 locations in 20 countries),
headquartered at Museum of Science, Boston

Description 4-H Tech Wizards is an afterschool, small-group Build IT is an afterschool and summer The Computer Clubhouse provides a creative, safe
mentoring program that trains youth on various curriculum for middle school youth to out-of-school learning environment in which youth
technologies within a bilingual, bicultural develop IT (information technology) from underserved communities work with adult
learning environment. The program is fluency, interest in mathematics, and mentors to explore their own ideas, develop new
implemented through the 4-H National knowledge of IT careers. The program— skills, and build self-confidence through the use of
Mentoring Program, a partnership between co-developed by SRI International and technology.
National 4-H Council and the Office of Juvenile the Girls Inc. of Alameda County—
Justice and Delinquency Prevention. is designed to engage girls and African-
American and Latino/a youth.

Age Level Ages 8-18; older youth involved as teen teachers Middle school Ages 10-18

Students Served 8,623 3,500 25,000

Demographics Open to all, but focus is on Hispanic and other 95% girls; more than 80% are African- Demographics depend on the local community.
under-served, high-need youth. American and Latina and the majority All clubhouses focus on reaching under-served
are from low-income households. communities and youth.

Evaluation Outcome data is collected by a variety of Staff collects ongoing outcomes data As a drop-in program, the clubhouse’s first
Methods methods, including formal skill assessment tools; through performance tasks, concept measure of success is participation. A sign-in
surveys; service logs; observations; and surveys and attitude surveys. Evaluation system collects data about participants and
interviews with youth, mentors, staff and family data is collected during specific research attendance. An annual survey gathers
members. projects and includes participant, demographic data; clubhouse visiting patterns;
facilitator, staff and parent surveys, as and attitudes related to technological competence,
well as interviews and observations with academic engagement, social-emotional well-
youth and facilitators. being, and aspirations for the future.

Examining the impact of afterschool STEM programs | 24

 Project GUTS
FUSE Girlstart
Santa Fe Institute
New York, New York; Providence, Rhode Island; Oakland, Austin, Texas, and other high-need Santa Fe, New Mexico
Location California; Baltimore, Maryland; Boston, Massachusetts; communities
Chicago, Illinois; and Palm Beach County, Florida

FUSE is a strategy to institutionalize engaging, inquiry- Girlstart After School is an intensive Project GUTS (Growing Up Thinking
based, informal STEM education nationally. The goal is to intervention program providing free weekly Scientifically) is an afterschool program
stimulate a culture shift among afterschool leaders and STEM programming throughout the year at in which middle school students learn
staff to increase the demand for and capacity to deliver partner schools. Girlstart programs are cutting-edge computing methods to
Description high-quality informal STEM education. designed to promote girls’ early engagement solve modern-day problems.
and academic success in STEM, encourage Participants design, create and test
their aspirations and persistence in STEM computer models to simulate “what if”
education and careers, and incubate a scenarios for real-world questions of
talented and diverse STEM workforce. community and societal concern.

Age Level Grades K-12 Ages 8-13; Grades 4-8 Ages 11-13; Grades 6-8

Students Served 32,000 1,000 2,000

Varies by city 100% girls; 70% qualify for federal free or 35% qualify for federal free or reduced
reduced price lunch; 33% are Limited English price lunch; 7% are Limited English
Proficient; 5% have special needs or Proficient; 5% have special needs or
Demographics disabilities; 12% African-American, 5% Asian- disabilities; 1% African-American, 2%
American/Pacific Islander, 17% Caucasian, Asian-American, 24% Caucasian, 58%
63% Latina, 4% identify as multiracial; 55% Hispanic/Latino, 2% Native American,
first-generation college aspirants. 13% Other.

The FUSE initiative is evaluated by TASC (The After School Girlstart uses a system of pre- and post- Project GUTS measures student
Corporation), which conducts interviews with stakeholders; surveys to assess girls’ STEM skills and demographics and attendance;
collects surveys from staff, students and intermediaries’ knowledge; self-reported attitudes toward collects pre- and post-surveys of
Evaluation partners; and observes science activities using the STEM Girlstart; and interest and confidence in knowledge, skills and self-efficacy;
Methods Program Quality Assessment (PQA). Staff confidence is undertaking future STEM activities, courses, teacher participation in professional
examined via the Science Teaching Efficacy Belief majors and careers. A framework for development; growth and self-efficacy
Instrument (STEBI), and youth science attitudes via the tracking program alumnae’s academic of teachers as GUTS club leaders; and
Science Attitude Change Tool and Common Instrument. progress assesses long-term impact. partnerships developed.

Examining the impact of afterschool STEM programs | 25

 Science Action Club Science Minors Clubs
Science Club
California Academy of Sciences Museum of Science and Industry

Location San Francisco, California Chicago, Illinois Chicago, Illinois

In Science Action Club (SAC), youth conduct Science Club is a partnership between Science Minors Clubs is an outreach initiative of
dynamic and authentic investigations by Northwestern University and the Boys & the Museum of Science and Industry aimed at
exploring their local environment, making Girls Club of Chicago, utilizing long-term increasing interest in science in underserved
observations about the natural world, and mentoring relationships to engage low- neighborhoods by engaging students in places
Description contributing data to real scientific research. income urban youth in science. Every week, where they already spend their time after school,
SAC also provides resources for other Northwestern graduate students lead small such as community-based organizations and
afterschool partners to implement the groups of students through designing and schools. Participants work together on STEM
program. running hands-on science experiments. projects and activities that build curiosity and
encourage teamwork.

Age Level Ages 11-14; Grades 6-8 Grades 6-8 Ages 8-12

Students Served 430 (expansion planned for fall 2014) 60 6,370

66% qualify for federal free or reduced price 97% qualify for federal free or reduced price 87% qualify for federal free or reduced price lunch;
lunch; 25% are Limited English Proficient; lunch; 35% are Limited English Proficient; 48% African-American, 6% Asian-American/Pacific
18% have special needs or disabilities; 15% 15% have special needs or disabilities; Islander, 8% Caucasian, 36% Hispanic/Latino.
Demographics
African-American, 50% Asian-American/ 40% Asian-American/Pacific Islander,
Pacific Islander, 13% Caucasian, 33% African-American, 5% Caucasian,
13% Hispanic/Latino, 9% Other. 22% Hispanic/Latino.

SAC has partnered with a professional Using a case-control methodology, Science The museum’s in-house research and evaluation
evaluation firm to assess programmatic Club measures changes in youth skills with team leads the design of the evaluation plan.
impact on participating youth and activity a scenario-based skills interview and Data sources include student intake forms, weekly
leaders. Outcome data is collected through science fair scores. Youth attendance, attendance reporting, surveys, facilitator reflections
Evaluation pre- and post-session surveys, site visits, interviews and surveys are also collected. and site observations. Program implementation
Methods interviews and focus groups. In addition to Mentors participate in focus groups and across diverse sites, effectiveness and usability of
developing original assessment strategies, self-report on changes in communication the curriculum and pedagogical approaches, impact
the program employs validated evaluation and teaching skills as well as attitudes of the professional development trainings, and
tools. toward science outreach. community perceptions of the museum have also
been evaluated.

Examining the impact of afterschool STEM programs | 26

 Tech Reach
Techbridge
The Thinkery

Austin, Texas Bay Area, California; expanding to Seattle in fall 2014 and two additional
Location
cities by 2016

Tech Reach is an outreach program of the Thinkery, a hybrid Techbridge offers afterschool and summer programs with hands-on
science and technology center/children’s museum formerly known projects and career exploration to inspire girls in science, technology and
as the Austin Children’s Museum. Tech Reach engages students engineering. Curricula are developed with girls in mind, and designed to
Description
through a concept called “creative computing”—where the bridge spark and sustain an interest in these fields as well as to make a connection
to computing concepts is achieved through projects and challenges with STEM careers. Career exploration is facilitated through STEM role
that tap into young students’ creativity and inventiveness. model visits and field trips to local engineering and technology companies.

Age Level Grades 3-5 Grades 5-12

Students Served 150 400

95% qualify for federal free or reduced price lunch; 10% are 100% girls; 94% of school partners receive Title I funding; 75% of students
Limited English Proficient; 15% have special needs or disabilities; are eligible for free or reduced price lunch; 40% Limited English Proficient;
Demographics
10% African-American, 2% Asian-American/Pacific Islander, 10% African-American, 30% Asian-American/Pacific Islander, 8% Caucasian,
5% Caucasian, 80% Hispanic/Latino. 42% Hispanic/Latino, 2% Native American, 8% Other.

An external evaluator designs pre- and post-program assessments, Techbridge uses quantitative and qualitative evaluation methods that
embedded assessments, and surveys for students to measure include pre- and post-surveys; focus groups with girls, teachers, and
attitudes toward STEM and the program. Focus groups are families; and program observations and coaching. Surveys examine
Evaluation convened for participating teachers to help determine program technical and scientific ability, career awareness, aspirations, teamwork
Methods strengths in design and implementation and identify areas for and adult influence. Comparison groups are utilized to evaluate outcomes
improvement. with groups not participating in Techbridge. In addition, an observation
rubric is used to ensure fidelity of implementation and identify areas for
program improvement.

Examining the impact of afterschool STEM programs | 27

Appendix III: Selected evalua on results of
a erschool STEM programs with
strong youth outcomes
Each a erschool program u lized the youth outcomes framework (A erschool Alliance, 2013a; also
shown in Appendix I) to describe their youth‐level impacts. Programs collected a variety of evalua on
data, however only data that fit the framework is shown here. Addi onally, many programs found
anecdotal evidence suppor ng the outcomes and indicators, but did not collect data on all of them.
This evidence is not reflected in the following tables.

Outcome A: Develop an interest in STEM and STEM learning ac vi es

“I like to do this.”
Indicators: Ac ve par cipa on in STEM learning opportuni es Curiosity about STEM topics, concepts or prac ces

Program a endance is high and consistent—on Youth report that they are a racted to 4‐H Tech
average, 95% of enrolled youth a end 4‐H Tech Wizards because of the opportuni es to work with
4‐H Tech
Wizard sessions and 95% of par cipants have stayed cu ng edge technology and the mentors.
Wizards
in the program for three years to complete all three
skill levels.

Reten on was o en 100% at sites that had girls and

parents commit to girls' ac ve par cipa on in Build
Build IT IT. Girls reported a sta s cally significant increase in
their confidence in math, belief in its usefulness,
and plans to take computer courses.

83% of par cipants visited their clubhouses at least Clubhouse alumni reported that they ini ally
weekly, and 47% every day. 37% of youth visited started a ending to learn about technology (61%)
their clubhouse for more than three hours at a me, and work on interes ng projects (45%). Con nued
Computer
and 91% visited for at least one hour. par cipa on gave them the opportunity to work
Clubhouse
on projects of their own (38%), learn important
skills (34%), and gave them access to valuable
technology (31%).

FUSE students par cipated in addi onal science‐ Student a tudes toward science increased
related opportuni es—43.6% played a math or significantly in terms of agreement with the
science game at home; 42% par cipated in following statements: “I get excited to find out
discussions about science topics with friends; that I will be doing a science ac vity”; “Science is
FUSE
55% watched TV, movies or online videos related something I get excited about”; “I like to work on
to science topics; and 30% read a book about a science ac vi es”; “I like to par cipate in science
science topic. projects”; and “I am curious to learn more about
science, computers, or technology.”

Examining the impact of a erschool STEM programs | 28

 Outcome A: Develop an interest in STEM and STEM learning ac vi es
“I like to do this.”
Indicators: Ac ve par cipa on in STEM learning opportuni es Curiosity about STEM topics, concepts or prac ces

Par cipa on resulted in the extension of interest in 87% report enjoying science, and par cularly the
science outside the program—86% agree with the scien fic ac vi es they did at Girlstart.
statement “I want to try more science ac vi es.”
Girlstart
84% reported interest in taking further STEM
classes in middle or high school. In 2012, 58%
a ended Girlstart’s Girls in STEM Conference.

82% of Project GUTS youth persist in the program to

successfully complete a working computer model.
Project GUTS 65% of students strongly believed that par cipa ng
in Project GUTS made them more excited to do and
learn science and technology in school.

69% of par cipants a end the program regularly 74% agree that “I get excited to find out that I will
and 86% indicated that they would recommend it to be doing a science ac vity in a erschool.” 63%
a friend. More than 60% of youth who enrolled in agree that “science is one of my favorite subjects in
Science
two sessions also par cipated in a third session. a erschool.”
Ac on Club
77% of youth reported that “Being in Science Ac on
Club makes me want to learn more about science
outside of school.”

Youth choose to go to Science Club among an array 94% of youth prefer learning hands‐on science with
of op ons at a Boys & Girls Club—84% of Science Club over school‐based science classes.
Science Club
par cipants a end weekly, par cipa ng for 1.5 82% of youth express a desire to con nue in the
years on average, with a 92% reten on rate. program a er as a high school mentor.

87% of youth surveyed indicated that they enjoy 85% of site facilitators reported an interest in
science and 92% expressed interest in doing more addi onal STEM topics to meet youth par cipant
Science
science ac vi es. Family engagement is strong—on demand. 57% of site facilitators accessed addi onal
Minors Club
average, 1266 par cipant family members a end curriculum kits to expand offerings for their club
Museum of Science and Industry Family Days. par cipants.

A er par cipa ng in Tech Reach, 88% of students

Tech Reach believed that science was fun and 86% said that
math was fun.

A er par cipa on, 80% of girls planned to pursue 85% of girls reported they find engineering more
Techbridge addi onal STEM learning opportuni es by taking interes ng and 83% said they find science more
advanced math and/or science classes. interes ng a er par cipa ng in Techbridge.

Examining the impact of a erschool STEM programs | 29

 Outcome B: Develop a capacity to produc vely engage in STEM learning ac vi es
“I can do this.”
Ability to produc vely engage in STEM
Indicators: Ability to exercise STEM‐relevant life and career skills
processes of inves ga on

95% of 4‐H Tech Wizards par cipants

4‐H Tech demonstrate mastery of skills in website
Wizards development, video and podcast produc ons,
GIS and GPS technologies, and LEGO robo cs.

The computer engineering design process is Build IT mo vates girls to use technology to strengthen
embedded in every curriculum unit as a and build their technology fluency. The project
Build IT method of problem solving. Girls showed achieved sta s cally significant improvements in
sta s cally significant improvements in their frequency of computer use, computer skills, and
understanding and use of the design process. conceptual understanding of compu ng.

Youth who visited more frequently showed Clubhouse youth who visit more frequently and stay
higher levels of problem‐solving competence longer show higher levels of collabora on. 92% of
Computer and technology competence. Of students in youth plan to use skills acquired in the clubhouse in
Clubhouse that group, 61% of girls and 55% of boys their future careers. 82% of alumni say they are
scored above the median on “technology currently using the tools and technologies they learned
competence.” at the clubhouse professionally and/or personally.

Youth saw gains in science knowledge, FUSE received a top score (5/5) on an observa on‐
mo va on and confidence across all FUSE based evalua on for providing opportuni es to prac ce
sites par cipa ng in the evalua on. At least group process skills, which includes ac vely listening,
85% of youth reported that par cipa ng in contribu ng ideas or ac ons to a group, doing a task
their a erschool science program: “Improved with others, or taking responsibility for a part of a
FUSE
my understanding of science”; “Helped me project.
learn things that I need to answer science
ques ons”; and “Gave me experience that
will help me in the future with science
projects and ac vi es.”

100% engage weekly in itera ve design and 100% par cipate in weekly collabora ve problem‐
scien fic inquiry to solve problems, including solving challenges that require cri cal thinking. 90%
genera ng observa ons, predic ons and respond posi vely to the statement “I understand that
Girlstart hypotheses, and tracking experiments and it is okay if my Girlstart ac vity does not work on the
outcomes. As a result, 91% of par cipants first try.” All par cipants present their experiments and
demonstrated mastery of scien fic inquiry results to parents, teachers and others in an end‐of‐
and the engineering design process. semester showcase.

64% of youth agreed they had learned how to

use computer models to conduct scien fic
inves ga ons—a skill that requires following
Project GUTS
an itera ve problem solving process to
design, implement, test and debug computer
models.

Examining the impact of a erschool STEM programs | 30

 Outcome B: Develop a capacity to produc vely engage in STEM learning ac vi es
“I can do this.”
Ability to produc vely engage in STEM
Indicators: Ability to exercise STEM‐relevant life and career skills
processes of inves ga on

100% of par cipants contribute directly to a The SAC environment emphasizes teamwork and
na onal ci zen science project by making leadership, and evalua ons show gains in youth self‐
Science observa ons, collec ng data and considering percep on and leadership. As a result of an emphasis
Ac on Club the implica ons of their findings. 72% of on using technology for data collec on, 89% of
youth report that “The data I collect in par cipants agreed that, “I feel confident using
Science Ac on Club is scien fically relevant.” technology to do science ac vi es in SAC.”

The Science Club curriculum emphasizes the Twice as many Science Club students described
scien fic method and the engineering design experiments as a way to help them learn and find out
process. Science Club youth significantly new things compared to youth not in the program
outperform their ap tude‐matched peers in (64% vs. 33%). 81% of Science Club youth described
two independent, well‐controlled oral using science outside of school compared to the
Science Club assessments of science skills. These control group.
assessments include constructs of
experimental design, use of variables, and
data analysis. Science Club youth are also
more confident conduc ng experiments than
non‐par cipa ng youth (56% vs. 32%).

Site observa ons revealed that 80% of Observa ons indicated that 100% of program sites
facilitators encouraged youth to formulate u lized coopera ve groups and individual roles to
testable ques ons and 93% fostered the promote collabora on between youth par cipants.
collec on of data and recording of 80% of sites observed provided opportuni es for youth
Science
observa ons. 93% of observed sites provided to report out their findings and communicate their
Minors Club
opportuni es for youth to use STEM tools ideas to the broader group.
such as a hand lens, calorimeter and rulers
to make observa ons, take measurements
or collect data.

All students must use cri cal thinking and Students made sta s cally significant improvements
crea ve problem solving needed to design, in communica on, collabora on and compu ng skills
program and build technology in Tech on an assessment of 21st century problem solving
Tech Reach
Reach—77% of students report “I am good at skills. 90.6% of students reported that “I am good at
building things using technology.” computers” and more than 88% percent said “I like to
figure out how something works.

95% of girls say they understand it can take 70% said they are more comfortable speaking in
many tries to solve a problem. 93.5% said front of a group of people. 91% said they try harder
they know more about how things work, like to overcome a challenge. 81% said they are be er at
Techbridge
circuits and simple machines. 80% said they problem‐solving. 92% said they feel more confident
are be er at using new computer programs. using technology. 89% said they feel more confident
in science.

Examining the impact of a erschool STEM programs | 31

 Outcome C: Come to value the goals of STEM and STEM learning ac vi es
“This is important to me.”
Indicators: Understanding the value of STEM in society Awareness of STEM professions

85% of 4‐H Tech Wizards par cipants completed Evaluators report that 70% of gradua ng 4‐H Tech
4‐H Tech 15 hours of community service learning by Wizards par cipants pursued post‐high school
Wizards teaching technology to others. educa on and careers in science, technology,
engineering or math.

Interviews document that par cipa on has made a

no ceable difference in how girls view technology
careers. Many who ini ally reported IT as solitary
and boring later reported that they found IT to be
Build IT
collabora ve, fun, intellectually s mula ng and a
possible career. Girls showed sta s cally significant
improvements in knowing what classes to take in
high school to prepare for an IT career.

74% of alumni give back to their communi es Among alumni, 80% reported that the clubhouse
through con nued interac on with clubhouse had been the most important source of support for
Computer
staff and youth. pursuing a career. 97% of alumni said that it was
Clubhouse
also the most important source of support for
se ng high goals and expecta ons for themselves.

A er par cipa ng in FUSE, student a tudes 79.5% reported that par cipa ng in FUSE “made
increased significantly in terms of agreement with the idea of a job in science when I am older seem
the statement: “I pay a en on when people talk more possible”; 73% reported that it “made me
about recycling to protect our environment.” more interested in a science job when I am older”;
FUSE and 69% reported that it “made me feel more sure
that I want a job in science when I am older.” 88.6%
reported that par cipa ng in FUSE “made me more
confident that I could do well in science classes in
college.”

A er par cipa ng in Girlstart, 70% report that 94% of par cipants demonstrate awareness that
they believe their ideas can solve problems in success in STEM can broaden their career op ons.
their own neighborhoods. 87% realize that they 68% express strong interest and 93% indicate at
Girlstart use science frequently outside of Girlstart. 91% least moderate interest in entering a STEM career.
agree with the statement, “If I do well in STEM in In a popula on in which 55% of par cipants are
college, I am more likely to get a be er job.” first genera on college aspirants, 97% expressed
intent to a end college a er high school.

Examining the impact of a erschool STEM programs | 32

 Outcome C: Come to value the goals of STEM and STEM learning ac vi es
“This is important to me.”
Indicators: Understanding the value of STEM in society Awareness of STEM professions

When asked how they would inves gate a new

community issue, 80% of Project GUTS
Project GUTS
par cipants suggested using computer modeling
and simula on.

In SAC, youth are engaged with increasing SAC is central to the California Academy of
scien fic understanding of their local Science’s pathway of youth programs, which
environments. 75% of youth agreed that fosters an awareness of STEM professions. 40% of
Science “In [SAC], I do ac vi es that contribute to survey respondents agreed that “Being in SAC
Ac on Club real science research.” 62% agreed that “[SAC] makes me think about ge ng a science job when
makes me wonder and ask ques ons about the I am older.”
natural world.” 68% agree that “I care about
the topics we learn about in [SAC].”

Science Club youth are be er able to describe 100% of students felt science was relevant to their
how science is used in everyday life in comparison careers a er par cipa on in Science Club, up from
Science Club
to their non‐par cipa ng peers. 70% at the start of the program. Students could
also more specifically describe science careers.

In site observa ons, 86% of facilitators supported An average of 1,350 youth and family members
youth in making connec ons to their everyday par cipated in Museum of Science and Industry
lives and 78% of par cipa ng youth indicated that STEM career fairs. 100% of youth a ending the
Science
they use science in their everyday lives. Of the fairs completed a STEM career journal, which
Minors Club
sites implemen ng the “Green Energy” guided their interviews of STEM professionals and
curriculum, 100% of facilitators reported encouraged them to describe STEM careers of
par cipa on in a recycling program. personal interest.

Thinkery staff members promote awareness of

STEM careers and futures by rou nely encourag‐
ing students to imagine themselves as engineers
Tech Reach
solving problems. 76% of par cipants reported
“Someday I would like a job where I can use math
and science.”

Techbridge provides extensive opportuni es to

engage in career explora on. As a result, 94% of
Techbridge par cipants knew more about different kinds of
jobs. 81% said they can see themselves working in
technology, science or engineering.

Examining the impact of a erschool STEM programs | 33

About the authors
Anita Krishnamurthi
Dr. Anita Krishnamurthi is the Vice President for STEM Policy at the A erschool Alliance, where she
leads efforts to advance policies, research and partnerships so youth can have rich STEM educa on
experiences in their a erschool programs. An astronomer by training, Dr. Krishnamurthi received her
PhD from The Ohio State University, conduc ng her postdoctoral work at the University of Colorado
at Boulder. For more than a decade, she has been immersed in science educa on, outreach and policy
through a variety of roles at the Na onal Academy of Sciences, NASA and the American Astronomical
Society.

Melissa Ballard
Melissa Ballard is the Research Associate at the A erschool Alliance, where she works to advance the
research efforts of the organiza on focusing on STEM in a erschool. Previously, she worked at a
children's science center developing a variety of STEM programs, training educators and teaching
students. Melissa has a background in industrial and opera ons engineering and liberal studies,
earning a B.S.E. and a B.G.S. from the University of Michigan.

Gil G. Noam
Dr. Gil Noam is the Founder and Director of the Program in Educa on, A erschool & Resiliency (PEAR)
and an Associate Professor at Harvard Medical School and McLean Hospital. Trained as a clinical and
developmental psychologist and psychoanalyst in both Europe and the United States, Dr. Noam has a
strong interest in suppor ng resilience in youth, especially in educa onal se ngs. Dr. Noam and his
team created a number of widely used STEM research and evalua on tools, and established the
Assessment Tools in Informal Science (ATIS) website. He served on the Na onal Research Council's
Commi ee on Learning Science in Informal Environments.

Acknowledgements
We would like to thank the Noyce Founda on for commissioning this paper and helping to bring
together its many collaborators. Immense thanks to the leaders of the programs featured in this
paper for generously sharing their evalua on results with us and using the specified outcomes
framework to describe their impacts. We are very grateful to all of those who reviewed the paper,
including Cary Sneider, Dylan Robertson, Ellen Le vin, Jen Rinehart, Nikki Yamashiro and Ron
O nger. This paper is stronger as a result of their comments and contribu ons. Finally, we wish to
thank Sarah Simpson for lending her considerable edi ng skills to hone and sharpen the paper.

Cover photo courtesy of Woodcraft Rangers

 Photo courtesy of McREL

The Noyce Founda on supports the informal science community to

develop work that addresses the gaps that exist in outcomes
measurement, research and evalua on, program scale up,
leadership development, policy issues, and pathways or pipeline
design. To read more about the Founda on’s projects in informal
science, visit www.noycefdn.org.

The A erschool Alliance is a nonprofit public awareness and

advocacy organiza on working to ensure that all children and youth
have access to quality a erschool programs. More informa on is
available at www.a erschoolalliance.org.

PEAR's mission is to create and foster school and a erschool se ngs

in which all young people can be successful. Dedicated to “the
whole child; the whole day; the whole year,” PEAR con nuously
integrates research, theory, and prac ce for las ng connec ons
between youth development, school reform and mental health.
For more informa on, visit www.pearweb.org.