Comparing Two Low Middle School Social

and Emotional Learning Program Formats:

A Multilevel Effectiveness Study

Vitor Alexandre Coelho & Vanda Sousa

Journal of Youth and Adolescence

A Multidisciplinary Research Publication

ISSN 0047-2891
Volume 46
Number 3

J Youth Adolescence (2017) 46:656-667

DOI 10.1007/s10964-016-0472-8

1 23
Your article is protected by copyright and all
rights are held exclusively by Springer Science
+Business Media New York. This e-offprint is
for personal use only and shall not be self-
archived in electronic repositories. If you wish
to self-archive your article, please use the
accepted manuscript version for posting on
your own website. You may further deposit
the accepted manuscript version in any
repository, provided it is only made publicly
available 12 months after official publication
or later and provided acknowledgement is
given to the original source of publication
and a link is inserted to the published article
on Springer's website. The link must be
accompanied by the following text: "The final
publication is available at link.springer.com”.

1 23
 Author's personal copy
J Youth Adolescence (2017) 46:656–667
DOI 10.1007/s10964-016-0472-8

EMPIRICAL RESEARCH

Comparing Two Low Middle School Social and Emotional

Learning Program Formats: A Multilevel Effectiveness Study
Vitor Alexandre Coelho1 • Vanda Sousa1

Received: 15 December 2015 / Accepted: 18 March 2016 / Published online: 23 March 2016
Ó Springer Science+Business Media New York 2016

Abstract Understanding which factors are relevant for Introduction

the effectiveness of social and emotional learning programs
is of key importance in designing interventions. This study In today’s society, the successful acquisition of social and
investigated the effectiveness of a universal, school-based, emotional competencies during childhood provides an
social and emotional learning program for low middle important foundation for healthy development and future
school students (5th and 6th grades) and compared two successful adult lives (Durlak et al. 2011; Greenberg et al.
program delivery formats for the intervention: curriculum 2003). The enhancement of social and emotional compe-
(a semi-structured format) and pre-package (fixed struc- tencies promotes achievement (Caemmerer and Keith,
ture).The program also analyzed differential gender effects. 2015), substance abuse prevention (Tobler et al. 2000), and
There were 982 participants (Mage = 11.22; SD = 1.02, bullying, disruptive behavior and interpersonal violence
46.8 % girls): 318 in the control condition, 355 students in reduction (Cook et al. 2010; Greenberg et al. 2003; Hahn
the intervention delivered with a curriculum format and et al. 2007). This area garnered additional attention after
309 in the intervention delivered in a pre-package format, studies demonstrated that social and emotional learning
assessed at pretest, posttest and follow-up eight months programs have the best cost/benefit ratio in mental health
later. Multilevel analyses identified positive intervention promotion (Knapp et al. 2011). Given these benefits, there
results in social awareness, self-control, self-esteem, social has been a steady increase in several countries of inter-
isolation and social anxiety. The pre-packaged format led ventions enhancing these competencies (Coelho et al.
to better results in self-esteem and social isolation. Boys 2015; Kimber et al. 2008; Sklad et al. 2012; Wolpert et al.
gained more in social awareness in both program delivery 2015).
formats. These findings support the program’s effective-
ness and the importance of designing the intervention to Social and Emotional Learning (SEL) Framework
match program delivery formats with developmental needs.
The Collaborative for Academic, Social, and Emotional
Keywords Social and emotional learning
Program Learning (CASEL) defines social and emotional learning
delivery format
Gender differences
Program as ‘‘the processes through which children and adults
effectiveness acquire and effectively apply the knowledge, attitudes, and
skills necessary to understand and manage emotions, set
and achieve positive goals, feel and show empathy for
others, establish and maintain positive relationships, and
& Vitor Alexandre Coelho make responsible decisions’’ (Collaborative for Academic,
[email protected]
Social, and Emotional Learning 2012, p. 9). Social and
Vanda Sousa emotional competence supports children’s ability to man-
[email protected]
age their emotions, establish healthy relationships and
1
Académico Torres Vedras, Travessa do Quebra-Costas, 9, develop socially responsible behavior throughout the
2564-910 Torres Vedras, Portugal lifespan (Greenberg et al. 2003), with most programs

123
 Author's personal copy
J Youth Adolescence (2017) 46:656–667 657

focusing on universal prevention—that is, preventing boys to girls ratio. Given all these findings, several authors
behavioral problems by enhancing social and emotional (Hahn et al. 2007; Sklad et al. 2012; Wigelsworth and
competence (Collaborative for Academic, Social, and Lendrum, 2013) concluded that research in this area should
Emotional Learning 2012). focus on what works for whom.
There have been some mixed results concerning the
effectiveness of universal social and emotional learning Gender Differences
programs. Extensive meta-analyses conducted in the USA
(Durlak et al. 2011) and Europe (the Netherlands; Sklad While some studies (e.g. Kimber et al. 2008) have found no
et al. 2012) have supported the conclusion that adequately differential impact by gender from participating in uni-
implemented school-based universal social and emotional versal social and emotional learning programs, most stud-
learning programs positively impact social and emotional ies (Coelho et al. 2015; Conduct Problems Prevention
competencies, mental health, academic achievement, pro- Research Group 2010; Holsen et al. 2008; Shapiro et al.
social behaviors and attitudes about self, while also con- 2002) report different impacts for each gender. Shapiro
tributing to the reduction of antisocial behavior and sub- et al. (2002), in a study with 6th grade students reported
stance abuse. This broad range of results has led to an that, after the application of the Peacemakers program,
increase in the implementation of social and emotional violent behavior had decreased and knowledge of social
learning programs by schools (Wigelsworth and Lendrum skills had increased, particularly among boys, whereas
2013). However, Durlak et al. (2011) reported that 77 % of Holsen et al. (2008), using the Norwegian version of
social and emotional learning program evaluations last less Second Step, indicated that in 6th grade only girls bene-
than 1 year, and not all implementation efforts have fitted in social competence from participation in the pro-
resulted in the expected positive outcomes. In England, a gram, while only boys reported lower levels of
strategy to implement a national curriculum of social and externalizing problem behaviors when compared to control
emotional aspects of learning has been considered a failure students. A broader study by the Conduct Problems
by several authors (Wigelsworth et al. 2013; Wigelsworth Prevention Research Group (2010) reported that, after the
and Lendrum 2013). intervention, peers only identified a decrease in aggres-
The evidence base responsible for the growth of social siveness for boys. In Portugal, Coelho et al. (2015) reported
and emotional learning programs has enabled the identifi- that girls gained more in social awareness, whereas boys
cation of some key elements that increase the probability of showed bigger decreases in social anxiety after participa-
the programs achieving the expected positive results. tion in a high middle school social and emotional learning
Among these key elements are high-quality implementa- program.
tion (Durlak et al. 2011) and developmental adequacy
(Hahn et al. 2007; Sklad et al. 2012). Despite fast advances Program Structure
in the field, there are some important issues that need
addressing. Wolpert et al. (2015) concluded that programs Some school programs are more effective than others,
implemented on a routine, ongoing basis in schools are which led Hahn et al. (2007) to consider that determining
only rarely considered in the evaluation literature, while the program characteristics that make them more or less
none of the more recent meta-analysis (Durlak et al. 2011; effective is an important question for research. In their
Sklad et al. 2012) has addressed the issue of how program meta-analysis, Durlak et al. (2011) concluded that social
delivery format may impact program efficacy or effec- and emotional learning programs which used a set of
tiveness. This is a relevant issue given that Wigelsworth procedures (identified by the acronym SAFE) resulted in
et al. (2013) considered that the failure of the social and better student outcomes. These procedures included a
emotional aspects of learning (SEAL) curriculum to impact sequenced step-by-step training approach (Sequenced), an
on students’ mental health outcomes could, to a certain emphasis on active forms of learning that require students
extent, be attributed to its flexibility. This intervention was to practice new skills (Active), focusing specific time and
conceived as a loose framework for school improvement attention on skill development (Focused), and an explicit
rather than a prescriptive, manualized intervention, more definition of the social and emotional skills they are
typical of universal social and emotional learning pro- attempting to promote (Explicit). The authors, furthermore,
grams. Another issue that requires attention, according to suggested that it would be preferable to evaluate these
several authors (Coelho et al. 2015; Diekstra and Gra- procedures as continuous rather than dichotomous vari-
vesteijn 2008; Hahn et al. 2007), is accounting for gender ables. Additionally, there is a consensus regarding program
differences when designing social and emotional learning implementation. Programs should be carefully monitored
programs. Diekstra and Gravesteijn (2008) note that sev- to ensure that they are delivered as intended by their
eral meta-analyses do not even mention the participants’ developers (Collaborative for Academic, Social, and

123
 Author's personal copy
658 J Youth Adolescence (2017) 46:656–667

Emotional Learning 2012; Durlak et al. 2011; Domitrovich than in high middle school (9 vs. 12) and there are dif-
et al. 2010). Program implementation is typically assessed ferences in teacher training (low middle school teacher
in terms of fidelity and dosage; however Wigelsworth et al. training is more focused on pedagogy, while high middle
(2013) concluded that there was no single core model of school teacher training is more focused on the specific
implementation against which to assess schools’ fidelity or subject that they will be teaching).
dosage. Given the similarities between low and high middle
There are, however, other questions that have not been school, the Positive Attitude low middle school social and
properly addressed in the literature, when determining on emotional learning program was originally delivered
what works for whom. For instance, there is a lack of (during the first 4 years) in a curriculum format, similar to
studies analyzing which program delivery format is more the high middle school program. As such, it was composed
appropriate for elementary or middle school. Durlak et al. by 13 1-h weekly sessions, tailored according to each class
(2011) concluded that younger students may need more profile. The first two sessions were dedicated to the initial
time to acquire complex skills and the Collaborative for assessment, and ice-breaking activities allowing for the
Academic, Social, and Emotional Learning (2015) guide introduction of the psychologist, the program and the stu-
for effective social and emotional learning recognized that dents. In the ten subsequent sessions, two out of four
there are diverse developmental needs for middle school possible modules were implemented based on each class’s
students and recommended that programs should be initial assessment results and a meeting with the Class
adapted to meet those needs. Director. The four possible themes (five sessions each)
were: self-awareness and self-management, social aware-
Positive Attitude Low Middle School Social ness, relationship skills, and responsible decision-making.
and Emotional Learning Program The last session (13th) was dedicated to program evalua-
tion. In total, 23 sessions were created (and described in the
Positive Attitude is a project developed in the municipality manual), of which 13 were implemented in each class, with
of Torres Vedras, district of Lisbon, Portugal and imple- each module focused more explicitly in one or two social
mented since 2004 (Coelho and Figueira 2011). It includes and emotional competencies, despite the fact that all of
three developmentally appropriate universal social and them were developed within that unit (as recommended by
emotional learning programs: 4th grade, low middle school the Collaborative for Academic, Social, and Emotional
(5th and 6th grade) and high middle school (7th–9th grade). Learning 2012).
The programs are designed to improve children’s social After analyzing several yearly reports that were issued
and emotional competencies by helping them develop to the Municipality, a decision was made to change the
social awareness, self-awareness, self-control, relationship program delivery format, mostly due to the better results
skills and responsible decision making. All the programs obtained with the elementary school social and emotional
are classroom-based (including all students in each class), learning Positive Attitude Program (Coelho et al., manu-
delivered weekly by an educational psychologist (in the script submitted for publication) and to better align the
presence of the class director), and infused into the school program recommendations by the Collaborative for Aca-
curriculum as part of a school subject named Civic Edu- demic, Social, and Emotional Learning (2005) regarding
cation. Their contents and activities are based on the Col- developmental adequacy. Therefore, for the next 4 years,
laborative for Academic, Social, and Emotional Learning the low middle school program was applied as a pre-
(2005) theoretical framework and described in the respec- packaged format (the same as the elementary school ver-
tive training manual (Coelho and Figueira 2011). Quality sion). This program delivery format has a fixed structure
of implementation was monitored through weekly meet- for all classes, composed by 15 1-h weekly sessions. The
ings with all the program’s psychologists who presented first, second and last sessions were similar to the corre-
class reports; a progress report was also sent monthly to the sponding sessions in the previous format, while the other
municipality and a more extensive report, including an 12 focused on self-awareness and self-management (four
analysis of the results, was sent yearly to the municipality. sessions), self-awareness and self-esteem enhancement
In Portugal, schools are organized into school groupings (two sessions), relationship skills (three sessions), and
composed by a middle school, several elementary schools social awareness (three sessions). This format was less
and kindergartens, and administrated by a School Board. sensible to class profile differences in arising from the
Low middle school (5th–6th grade) and high middle school individual characteristics of its elements but guaranteed
(7th–9th grade) are situated in the same school building, that a larger number of social and emotional competencies
have similar teacher distribution (one teacher per subject) were directly targeted, albeit with less time dispensed
and class sizes. However, in low middle school the number overtly on each one. This format also introduced a unit
of subjects (e.g. Portuguese, History, English) is smaller focused explicitly on self-esteem.

123
 Author's personal copy
J Youth Adolescence (2017) 46:656–667 659

In short, both program delivery formats employed the not accepting to randomly assign classes to the intervention
set of practices recommended by Durlak et al. (2011) and, or control conditions. Therefore, school groupings nomi-
therefore, were sequenced, active, focused and explicit. nated the intervention classes in which the program would
However, programs differed in how these two last dimen- be implemented, and conceded to assign matching classes
sions were operationalized: the curriculum format had a with similar characteristics (achievement levels, class size
narrower focus, with an explicit focus on two competen- and socio-economic status). In the intervention condition,
cies, while the pre-packaged had a broader focus, devoting two program delivery formats were implemented: cur-
less time to a wider range of competencies. riculum (during the first four waves) and pre-packaged
(during the last four waves), control groups were estab-
lished in each wave of implementation. All groups were
Present Study natural groups (i.e., full classes). Students in the control
groups received the usual Civic Training contents officially
The present study follows suggestions by Carroll et al. issued by the Portuguese Ministry of Education to all
(2009) that practitioners should contribute to the schools in the country, taught by their regular teacher.
advancement of school psychology, by conducting empir- These contents do not include social and emotional
ical research in their applied settings and through the training.
application of effective and sustainable evidence-based Teacher reports were not included since for many
practices in education. It also follows Hahn et al. (2007), classes they would only be available for pre and posttest,
who considered that it is important to understand what and not follow-up. From 6th to 7th grade, teacher turnover
maximizes the effectiveness and sustainability of ongoing is 100 percent (even though students remain in the same
programs since many programs assessed in meta-analysis school building); so the teachers filling the reports at fol-
are not ongoing, standing programs, but instead conducted low-up would not be the same.
for research purposes. Additionally, the present study
emerges from the original lower results of the low middle Participants
school Positive Attitude social and emotional learning
program when compared with the high middle school The sample was a convenience sample, composed by 982
program (Coelho et al. 2014, 2015) and the elementary low middle school students (Mage = 11.22; SD = 1.02),
school program (Coelho et al., manuscript submitted for from five public school groupings (in the district of Lis-
publication), which raised the question of whether the bon). Participant characteristics are shown in Table 1.
program format was the most adequate. Intervention and control groups did not differ in terms of
Therefore, this research analyses the impact of a uni- gender, ethnicity and school location (as seen in Table 1).
versal social-emotional learning program applied to low Attrition from pre to posttest was low, with only 19
middle school Portuguese students and, more specifically, it students (15 from the intervention, four from the control
compares the effectiveness of two program delivery for- group) dropping out of the program because they changed
mats: curriculum and pre-packaged. It was hypothesized schools; dropout was homogenous during the 8 years of the
that students who participated in the Positive Attitude pro- program implementation.
gram (intervention groups) would show improvements in
social and emotional competencies and internalizing prob- Instruments
lems when compared with students in the control groups
(Hypothesis one), and also that groups who received the Social and Emotional Competencies
intervention in a pre-packaged program delivery format
would yield more positive results (Hypothesis two). Based The evaluation of social and emotional competencies was
on the literature review, it would also be expected that boys conducted with the Portuguese version of the Bateria de
present greater gains in self-control, while girls will present Socialização 3 (BAS-3; Portuguese adaptation by Ferreira
added gains in social awareness (Hypothesis three). and Rocha 2004). This self-report instrument for adoles-
cents (aged 11–18 years old) is composed by 75 items
rated in a two point scale (yes or no). The questionnaire
Method consists of 5 dimensions: Social Awareness (14 items;
Cronbach’s a = .82 in the Portuguese adaptation; assesses
Research Design the degree of sensitivity or concern for others), Self-control
(14 items; a = .73; evaluates the acceptance of rules and
This research is a quasi-experimental design as sampling social norms and the manifestations of aggressive or
was not totally random, due to the school grouping boards imposing behaviors), Social Isolation (14 items; a = .72;

123
 Author's personal copy
660 J Youth Adolescence (2017) 46:656–667

Table 1 Student characteristics across groups and waves of implementation

Characteristic Total (%) Control group Curriculum intervention group Pre-packaged intervention group
N = 318 (32.3 %) N = 355 (36.2 %) N = 309 (31.5 %)

Gender v2 (1) = 0.55; p [ .05

Male 522 (53.2 %) 180 (56.6 %) 342 (51.5 %)
Female 460 (46.8 %) 138 (43.4 %) 322 (48.5 %)
Middle school location v2 (1) = 0.98; p [ .05
Rural 319 (32.5 %) 93 (29.2 %) 226 (34.0 %)
Urban 663 (67.5 %) 225 (70.8 %) 438 (66.0 %)
Cohort
Wave 2 101 (10.3 %) 24 (7.5 %) 77 (21.7 %)
Wave 3 149 (15.2 %) 43 (13.5 %) 106 (29.8 %)
Wave 4 120 (12.2 %) 37 (11.6 %) 83 (23.4 %)
Wave 5 147 (15.0 %) 58 (18.2 %) 89 (25.1 %)
Wave 6 113 (11.5 %) 60 (18.9 %) 53 (17.2 %)
Wave 7 135 (13.7 %) 12 (3.8 %) 123 (39.8 %)
Wave 8 120 (12.2 %) 59 (18.6 %) 61 (19.7 %)
Wave 9 97 (9.9 %) 25 (7.9 %) 72 (23.3 %)
N = 982

allows detecting passive or active isolation), Social Anxiety intervention group, questionnaires were administered in the
(12 items; a = .73; assesses expressions of anxiety and first and last sessions of the program, control groups were
shyness in social situations) and Leadership (12 items, assessed in the same period, but did not received any social
a = .69; assesses the degree of popularity, initiative, self- and emotional learning training. In each format, four edu-
confidence and availability towards others). The instrument cational psychologists implemented the program, with
also included a 10-item sincerity scale. three of them delivering both the curriculum and pre-
packaged formats. Test applications for both groups were
Self-Esteem carried out under the same conditions, with the psycholo-
gist responsible for each class reading questionnaire
Assessed through the Global Self scale of the Self-De- instructions out loud to the students. If a student was not
scription Questionnaire I (SDQ I, Marsh 1989; Portuguese present during evaluation the questionnaires were admin-
version; Faria and Fontaine 1990). This scale evaluates istered in another class within 2 weeks (n = 27).
global self-worth, and is composed by 8 items (e.g., ‘‘In Program implementation was monitored in weekly team
general I have a lot to be proud of’’) rated in a five-point meetings in which each of the psychologists submitted
scale (1—false, 2—mostly false, 3—nor true or false, 4— class reports, and a progress report was sent every month to
mostly true and 5—true). The scale’s internal consistency the municipality. For the pre-packaged format program, the
is adequate, with Cronbach’s a = .70 (for the Portuguese psychologists reported that, in four of the 15 classes
adaptation). (26.7 %) where the program was delivered, an extra ses-
sion was needed to deal with all the material in the lesson
Procedure plans for every session. There were no sessions added in
the 18 classes where the program was delivered in a cur-
The educational psychologists who implemented the pro- riculum format. Thus, the psychologists reported a fairly
gram were present in the meetings in back-to-school high degree of fidelity in implementing the program as
meetings (mandatory for parents), to explain the program conceived.
and to answer questions. All schools used passive informed
consent, because the program could be considered part of Data Analysis
the school curriculum, following national legislation.
Self-reports were filled at baseline, posttest and follow- Multilevel linear modeling (MLM) with a repeated mea-
up (eight months after the conclusion of the intervention), sures design was used (SPSS, mixed models) to evaluate
while demographic data was recorded at pretest. In the the effectiveness of the program. There are advantages to

123
 Author's personal copy
J Youth Adolescence (2017) 46:656–667 661

the MLM approach to repeated measures when compared differences between control and intervention groups in
to other analyses, the main one being that it controls for self-esteem, t(980) = 2.08, p \ .05, with control groups
non-independence among the repeated observations for presenting higher initial levels. No significant differences
each individual (Heck et al. 2013). In this repeated mea- were found in social awareness, t(729) = 1.29, p [ .05;
sures study design, individual scores are nested within self-control, t(980) = 1.96, p [ .05; social isolation,
individuals, and nested data is more likely to correlate t(695) = -1.78, p [ .05; social anxiety, t(980) = -1.61,
within the group (class). Originally, a three-level model p [ .05; and leadership, t(980) = 0.99, p [ .05.
was run to account for the three measurements nested Regarding gender the descriptive data for the control
within 982 students, and that these students were nested in and intervention groups is shown in Tables 4 and 5. Girls
49 classes. However, for social anxiety, the Intraclass presented higher initial levels of social awareness,
Correlations (reported in Table 2) show that there is no t(916.19) = -7.32, p \ .001; self-control, t(980) =
need to include a 3rd level (class) in the models, since that -5.57, p \ .001; and social anxiety, t(980) = -3.44,
there is not sufficient variance explained at that level p \ .001, whereas boys presented initial higher levels of
(\.05), following the suggestion by Heck et al. (2013). social isolation, t(968.76) = 5,19, p \ .001. There were no
No time-varying covariates were included in the model. significant differences observed in self-esteem,
After several analyses, the best fit was achieved with a t(980) = -1.08, p [ .05; and leadership, t(980) = 0.56,
linear measure of time and auto-regressive as the covari- p [ .05.
ance structure for level 1. The intercept was used as a
random effect in the models, and the best model fit was
Program Effects on Social and Emotional
achieved with scaled identity as covariance structure for
Competencies
levels 2 and 3. The next model included an interaction
between time and condition (dummy-coded as a level 3
The effects of condition and the interaction between con-
variable). In model 3, condition was removed and replaced
dition and time were tested in the condition model (dis-
by group (modeled by adding dummy-coded level 3 vari-
played in Table 6), with significant interaction effects
ables). The final model included all significant interactions
between condition and time found for social awareness
between condition, time, and gender.
(b = -0.36, SE = 0.08; t = -4.58, p \ .001); self-con-
The alpha level was set to \ .05 and effect sizes (Co-
trol (b = -0.37, SE = 0.10; t = -3.73, p \ .001); self-
hen’s) were calculated following the procedure established
esteem (b = -0.69, SE = 0.18; t = 3.86, p \ .001),
by Tomic and Klauer (1996), that includes an adjustment
social isolation (b = 0.31, SE = 0.08; t = 3.81, p \ .001);
for any previous intervention differences between groups on
and social anxiety (b = 0.74, SE = 0.10; t = 7.58,
each outcome measure. Cohen (1988) suggests that d values
p \ .001). The fit of the model improved significantly
of 0.2, 0.5, and 0.8 should be interpreted as small, medium
when condition was added as a factor: social awareness,
and large effects, respectively. IBM SPSS Statistics for
Dv2 (2) = 11.15, p \ .01; self-control, Dv2 (2) = 14.34,
Windows, Version 20 (IBM Corp, Armonk, NY) was used.
p \ .01; self-esteem, Dv2 (2) = 34.23, p \ .01; social iso-
lation, Dv2 (2) = 12.83, p \ .01; social anxiety, Dv2
Results (2) = 94.32, p \ .001. At class level, there is some
reduction in variance by adding the interaction of time and
Preliminary Analysis condition, this reduction is different by competence: social
awareness (-1.4 %); self-control (-2.7 %); self-esteem
Descriptive data for the control and intervention groups are (-1.3 %); social isolation (-1.1 %); social anxiety
displayed in Table 3. There were only significant (-36.7 %).

Table 2 Intraclass correlations

Self-reports
(ICC) for social and emotional
competencies Repeated measures Students Class School grouping

Social awareness .367 .564 .026 .043

Self-control .368 .568 .021 .042
Social isolation .375 .572 .018 .034
Social anxiety .364 .626 .007 .002
Leadership .362 .608 .030 \.001
Self-esteem .474 .464 .059 .003

123
 Author's personal copy
662 J Youth Adolescence (2017) 46:656–667

Table 3 Social and Emotional competencies—pre-test, post-test and follow-up means and standard deviations for control and intervention
group, effect sizes for the interaction group 9 time
Control group Intervention group d pre-post d pre-
test follow up
Pre-test (N = 318) Post-test Follow-up Pre-test Post-test Follow-up
M (SD) (N = 314) (N = 290) (N = 664) (N = 649) (N = 603)

Social 12.42 (1.94) 12.37 (2.11) 12.25 (1.86) 12.24 (2.30) 12.53 (2.05) 12.58 (1.59) 0.16 0.28
awareness
Self-control 10.76 (2.67) 10.67 (2.75) 10.40 (2.40) 10.48 (2.73) 10.64 (2.69) 10.77 (1.92) 0.09 0.28
Social 2.10 (2.12) 2.16 (2.16) 2.25 (1.98) 2.35 (2.35) 2.19 (2.26) 1.90 (1.78) -0.10 0.30
isolation
Social 4.23 (2.88) 4.31 (2.68) 4.65 (2.44) 4.54 (2.83) 3.94 (2.84) 3.46 (2.29) -0.24 -0.62
anxiety
Leadership 7.18 (2.34) 7.14 (2.32) 7.41 (1.75) 7.02 (2.40) 7.34 (2.46) 7.43 (1.96) 0.15 0.08
Self-esteem 29.13 (4.70) 28.79 (4.92) 28.49 (4.31) 28.43 (5.04) 29.01 (4.95) 29.38 (4.20) 0.19 0.35

Table 4 Social and emotional competencies—descriptive statistics for control and intervention group, pre, post-test and follow-up, girls
Control group (N = 138) Intervention group (N = 322)
n = 138 n = 136 n = 130 n = 322 n = 316 n = 304
Pre-test M (SD) Post-test M (SD) Follow-up M (SD) Pre-test M (SD) Post-test M (SD) Follow-up M (SD)

Social awareness 12.84 (1.53) 13.15 (1.27) 12.85 (1.35) 12.80 (1.71) 13.15 (1.38) 12.98 (1.19)
Self-control 11.22 (2.46) 11.21 (2.54) 10.84 (2.24) 11.01 (2.49) 11.28 (2.34) 11.12 (1.82)
Social isolation 1.78 (1.86) 1.75 (1.74) 1.98 (1.78) 1.92 (2.00) 1.80 (2.02) 1.69 (1.72)
Social anxiety 4.57 (2.80) 4.59 (2.72) 4.85 (2.48) 4.86 (2.85) 4.14 (2.84) 3.74 (2.36)
Leadership 7.02 (2.28) 7.02 (2.35) 7.22 (1.72) 7.03 (2.36) 7.29 (2.37) 7.37 (1.91)
Self-esteem 29.41 (4.87) 29.14 (5.40) 28.28 (4.55) 28.60 (5.03) 29.30 (5.01) 29.56 (4.28)

Table 5 Social and emotional competencies—descriptive statistics for control and intervention group, pre, post-test and follow-up, boys
Control group Intervention group
n = 180 n = 178 n = 160 n = 342 n = 333 n = 299
Pre-test M (SD) Post-test M (SD) Follow-up M (SD) Pre-test M (SD) Post-test M (SD) Follow-upM (SD)

Social awareness 12.09 (2.16) 11.77 (2.42) 11.76 (2.06) 11.70 (2.64) 11.95 (2.38) 12.18 (1.83)
Self-control 10.41 (2.77) 10.26 (2.83) 10.05 (2.48) 9.98 (2.84) 10.04 (2.85) 10.41 (1.95)
Social isolation 2.34 (2.27) 2.47 (2.40) 2.46 (2.12) 2.75 (2.58) 2.56 (2.42) 2.12 (1.79)
Social anxiety 3.97 (2.91) 4.10 (2.65) 4.48 (2.40) 4.25 (2.78) 3.76 (2.83) 3.18 (2.19)
Leadership 7.31 (2.38) 7.23 (2.29) 7.56 (1.77) 7.01 (2.44) 7.38 (2.55) 7.50 (2.01)
Self-esteem 28.92 (4.56) 28.52 (4.52) 28.53 (4.12) 28.28 (5.05) 28.73 (4.88) 29.21 (4.12)

As can be seen in Table 3 the effect sizes of partici- Program Effects, by Program Format
pating in the program upon the various social and emo-
tional competencies were different. While for social The analysis by group showed differences in the impact
awareness, self-control, self-esteem and social isolation during analyzed time between program delivery formats in
small effect sizes were found at follow-up (according to self-esteem (b = -0.70, SE = 0.15; t = -4.55, p \ .001)
criteria established by Cohen 1988), in social anxiety the and social isolation (b = 0.23, SE = 0.07; t = 3.27,
effect size found was medium. p = .001), with the pre-packaged program presenting a

123
 Author's personal copy
J Youth Adolescence (2017) 46:656–667 663

Table 6 Self-reports: mixed linear model estimates for condition, group and moderating effects for gender
Variable Social awareness Self-control
B SE T 95 % CI p b SE t 95 % CI p

Intercept 11.73 0.17 67.68 11.39, 12.08 \.001 10.11 0.21 48.00 9.69, 10.53 \.001
Time 0.24 0.07 3.56 0.11, 0.37 \.001 0.23 0.08 2.74 0.07, 0.40 .006
Control group 0.21 0.23 0.93 -0.25, 0.67 .355 0.22 0.28 0.78 -0.34, 0.77 .438
Curriculum int. group -0.01 0.22 -0.02 -0.45, 0.44 .984 -0.29 0.27 -1.09 -0.83, 0.25
Gender 1.07 0.12 9.00 0.84, 1.31 \.001 1.04 0.15 6.80 0.74, 1.34 \.001
Control group 9 time -0.41 0.09 -4.47 -0.59, -0.23 \.001 -0.43 0.12 -3.76 -0.66, -0.21 \.001
Curriculum group 9 time -0.10 0.09 -1.26 -0.28, 0.08 .286 -0.13 0.12 -1.11 -0.35, 0.10 .268
Variable Social isolation Social anxiety
B SE T 95 % CI p b SE t 95 % CI p

Intercept 2.51 0.17 14.78 2.49, 3.02 \.001 4.12 0.17 24.11 3.78, 4.46 \.001
Time -0.40 0.07 -5.76 -0.54, -0.27 \.001 -0.61 0.08 -7.30 -0.78, -0.45 \.001
Control Group -0.08 0.22 -0.36 -0.52, 0.36 .718 -0.18 0.21 -0.83 -0.60, 0.25 .411
Curriculum int. group 0.42 0.22 1.89 -0.01, 0.83 .071 0.23 0.21 1.13 -0.18, 0.65 .262
Gender -0.78 0.13 -5.93 -1.03, -0.52 \.001 0.57 0.17 3.39 0.24, 0.90 .001
Control group 9 time 0.47 0.10 4.95 0.29, 0.66 \.001 0.85 0.11 7.40 0.62, 1.04 \.001
Curriculum group 9 time 0.31 0.10 3.24 0.12, 0.50 .001 0.20 0.11 1.78 -0.02, 0.43 .076

Variable Leadership Self-esteem

B SE T 95 % CI p b SE t 95 % CI p

Intercept 6.99 0.17 40.14 6.64, 7.33 \.001 28.17 0.36 78.14 27.45, 28.89 \.001
Time 0.24 0.08 3.14 0.09, 0.38 .002 0.82 0.15 5.39 0.52, 1.11 \.001
Control group -0.13 0.10 -1.26 -0.33, 0.07 .208 0.79 0.47 1.71 -0.14, 1.73 .093
Curriculum int. group -0.02 0.10 -0.22 -0.23, 0.18 .824 0.12 0.45 0.16 -0.79, 1.02 .798
Gender -0.08 0.14 -0.55 -0.35, -0.20 .576 0.43 0.30 1.44 -0.15, 1.01 .150
Control group 9 time 0.08 0.14 0.57 -0.20, 0.36 .591 -1.10 0.21 -5.30 -1.50, -0.69 \.001
curriculum group 9 time 0.10 0.14 0.73 -0.17, 0.37 .466 -0.79 0.21 -3.80 -1.19, -0.38 \.001

(I) intervention group

more positive impact, as seen in Figs. 1 and 2. The dif-

ferent program formats did not present significant differ-
ences in: social awareness (b = -0.08, SE = 0.07;
t = -1.11, p [ .05); self-control (b = -0.08, SE = 0.08;
t = -0.95, p [ .05); leadership (b = 0.03, SE = 0.08;
t = 0.35, p [ .05); and social anxiety (b = 0.08,
SE = 0.08; t = 0.09, p [ .05). Accordingly, the fit of the
model improved significantly when condition was replaced
by group as a factor in self-esteem, Dv2 (2) = 22.22,
p \ .001, and social isolation, Dv2 (2) = 20.52, p \ .001.
Additionally, substituting condition (control vs. interven-
tion) for group (control, curriculum and pre-packaged)
further reduced Level 3 variances, for self-esteem
(-9.2 %), social isolation (-51.3 %) and social anxiety
(-4.2 %).
The curriculum format yielded small effect sizes in self-
esteem and social isolation at follow-up (see Tables 7, 8), Fig. 1 Means for self-esteem by group (control, curriculum, pre-
whereas the pre-packaged format led to medium effect packaged) along time

123
 Author's personal copy
664 J Youth Adolescence (2017) 46:656–667

gender and the gender-time interaction reduced level 2

variance for social awareness (-10.8 %), self-control
(-5.9 %), and social isolation (-4.5 %).

Discussion

Several recent studies and meta-analyses (Conduct Prob-

lems Prevention Research Group 2010; Durlak et al. 2011;
Sklad et al. 2012) have established that universal school-
based social and emotional learning programs positively
impact a range of behavioral, social and emotional out-
comes. During the last decade, the research focus in this
area has shifted to the factors that promote effectiveness
(Durlak et al. 2011; Hahn et al. 2007; Kimber et al. 2008;
Sklad et al. 2012), i.e., the attempt to answer the question
Fig. 2 Means for social isolation by group (control, curriculum, pre- of ‘‘what works for whom’’. Therefore, a universal school-
packaged) along time based social and emotional learning program should now
focus not only on establishing its effectiveness, but also on
sizes. For social anxiety the reported effect size for the pre- identifying the factors that influence it.
packaged format was large, while for the curriculum format In the present study, results showed that the universal
it was medium. social and emotional learning program Positive Attitude
Low Middle School led to gains in several social and
Program Effects, by Gender emotional competencies, namely social awareness, self-
control and self-esteem, while also contributing to the
Further analyses were conducted in order to understand if reduction of social isolation and social anxiety. Effect sizes
the program (and its different delivery program formats) are in line with those reported by Sklad et al. (2012).
had similar impact on both genders (Table 3). Only one However, the results contrasted with Sklad et al. (2012)
competence yielded a different impact by gender: social who reported that positive immediate effects on social and
awareness (b = 0.12, SE = 0.06; t = 2.04, p \ .05), with emotional competencies decreased substantially at follow-
boys benefitting more from the participation in the program up. Also, the higher effect size found in social anxiety
during the total time interval analyzed. There were no contrasts with conclusions by Holsen et al. (2008) that
significant differences in slopes for both genders across universal interventions have more modest effects on
program delivery formats for any dimension. Adding internalizing problems.

Table 7 Social and emotional competencies—pre-test, post-test and follow-up means and standard deviations for control and intervention
groups curriculum format, effect sizes for the interaction group 9 time
Control group Intervention group d pre- d pre-
post follow
Pre-test Post-test Follow-up Pre-test Post-test Follow-up test up
(N = 149) M (N = 146) M (N = 131) M (N = 322) M (N = 314) M (N = 288) M
(SD) (SD) (SD) (SD) (SD) (SD)

Social 12.34 (1.94) 12.30 (2.07) 12.17 (1.81) 12.38 (2.25) 12.55 (2.12) 12.56 (1.69) 0.13 0.25
awareness
Self-control 10.62 (2.81) 10.70 (2.88) 10.32 (2.58) 10.41 (2.70) 10.58 (2.77) 10.56 (2.00) 0.09 0.17
Social 2.33 (2.34) 2.37 (2.35) 2.47 (2.12) 2.46 (2.09) 2.41 (2.30) 2.33 (1.90) -0.04 -0.15
isolation
Social 4.14 (2.70) 4.21 (2.75) 4.37 (2.22) 4.67 (2.86) 4.04 (2.97) 3.64 (2.34) -0.21 -0.48
anxiety
Leadership 7.23 (2.49) 7.11 (2.42) 7.39 (1.75) 7.06 (2.39) 7.55 (2.42) 7.49 (1.92) 0.24 0.11
Self-esteem 29.07 (4.72) 28.72 (4.74) 28.52 (4.06) 28.58 (4.87) 29.79 (4.86) 29.78 (4.21) 0.13 0.18

123
 Author's personal copy
J Youth Adolescence (2017) 46:656–667 665

Table 8 Social and emotional competencies—pre-test, post-test and follow-up means and standard deviations for control and intervention pre-
packaged format, effect sizes for the interaction group 9 time
Control group Intervention group d pre- d pre-
post follow
Pre-test Post-test Follow-up Pre-test Post-test Follow-up test up
(N = 169) M (N = 168) M (N = 159) M (N = 342) M (N = 335) M (N = 315) M
(SD) (SD) (SD) (SD) (SD) (SD)

Social 12.57 (1.82) 12.55 (1.97) 12.33 (1.90) 12.22 (2.21) 12.65 (1.80) 12.61 (1.48) 0.21 0.33
awareness
Self-control 11.07 (2.33) 10.87 (2.53) 10.50 (2.20) 10.67 (2.80) 10.85 (2.53) 11.01 (1.79) 0.16 0.41
Social 1.83 (1.91) 1.81 (1.78) 2.01 (1.81) 2.03 (2.43) 1.69 (1.94) 1.42 (1.46) -0.16 -0.50
isolation
Social 4.30 (3.00) 4.49 (2.69) 4.94 (2.63) 4.36 (2.75) 3.86 (2.70) 3.25 (2.23) -0.29 -0.76
anxiety
Leadership 7.18 (2.25) 7.18 (2.14) 7.43 (1.77) 6.99 (2.38) 7.07 (2.43) 7.36 (2.00) 0.16 0.16
Self-esteem 29.34 (4.84) 29.11 (5.16) 28.30 (4.57) 28.55 (5.32) 29.49 (5.18) 30.08 (4.09) 0.25 0.59

The results revealed some differences between program programs. It was also carried out under regular daily con-
delivery formats, with the pre-packaged delivery format ditions addressing the concerns raised by Carroll et al.
having a more positive impact on self-esteem and social (2009) about the lack of practitionerś work being reported
isolation during the entire time period analyzed, as well as in school psychology.
higher effect sizes at follow-up in social anxiety, thus In general, the study allowed for the identification of the
mostly confirming the second hypothesis. The better results more adequate program delivery format for this population
in self-esteem and social isolation can be partially attrib- and showed that differences between program delivery
uted to the change in program delivery format, since the formats may lead to different effectiveness results. For low
pre-packaged delivery format had a more explicit focus on middle school students, the results supported the advan-
self-esteem and relationship skills. tages of using a pre-packaged program format, with a
Results did not support the third hypothesis given that it broader focus (addressing more competencies, in an
was boys who presented bigger gains in social awareness. explicit way), even with less time dispensed on each one
This result is contrary to most literature findings (Coelho instead of a narrower focused curriculum format (ad-
et al. 2015; Holsen et al. 2008), although this difference dressing fewer competencies explicitly) with more time
can be partially attributed to the fact that boys presented spent on each competency. This pattern of results supports
lower initial levels of social awareness. The other the conclusions of Durlak et al. (2011), who recommended
assumption made in the third hypothesis, regarding boys the use of a set of procedures (sequenced, active, focused
presenting more gains in self-control, was also not sup- and explicit) in designing social and emotional learning
ported even though boys reported lower initial levels of programs, while also supporting their suggestion that the
self-control. Although there were differential results by use of those practices should be evaluated as continuous
gender, the results from the present study contrast with rather than dichotomous. The results are also consistent
previous studies (Conduct Problems Prevention Research with earlier research that showed that these programs have
Group 2010; Shapiro et al. 2001) concerning variables that stronger effects on direct rather than indirect outcomes
rely on self-control (e.g., aggressiveness). Additionally, no (Sklad et al. 2012).
program delivery format was especially effective for girls
or boys. As a whole, the pattern of results from the low
middle school program is distinct from the high middle Limitations of the Study
school program, which can also be partially attributed to
the different program formats and developmental profiles. Some limitations of the study are a result of it being rooted
This study encompassed eight years of program deliv- in practice. One limitation that arose was the impossibility
ery, following the suggestion by Hahn et al. (2007) that of randomly assigning classes to the control and interven-
ongoing social and emotional learning programs should be tion conditions as school grouping boards felt that doing so
analyzed for a longer period than what is usually reported would not serve their needs. Future studies about the pro-
in research-based programs, in order to understand what gram’s effectiveness should attempt to employ random-
maximizes the effectiveness and sustainability of ongoing ization in order to strengthen the validity of the results.

123
 Author's personal copy
666 J Youth Adolescence (2017) 46:656–667

Additionally, it would be desirable to compare both with the conclusion by the Conduct Problems Prevention
program delivery formats with a control group in the same Research Group (2010) that contemplating gender differ-
wave of implementation. Therefore, another limitation of ences is also a crucial point when developing school-based
the study is that the curriculum format was implemented universal social and emotional learning programs, as these
with the first four cohorts (waves of implementation), while differences directly affect how much a child profits from
the pre-packaged format was implemented with the last these programs.
four cohorts, which is not ideal since it raises the possi-
bility of a cohort effect, thus curtailing the generalization Acknowledgments The authors would like to thank Bárbara Soares
for her assistance in the English editing.
of the results.
It would also have been useful to use teacher reports, Authors’ Contributions VAC conceived of the study, participated
which would have allowed the triangulation of the in its design and coordination, performed the statistical analysis and
assessments, as suggested by Wigelsworth et al. (2010). drafted the manuscript; VS conceived of the study, participated in the
design, performed the measurement, the statistical analysis and
However, this was not possible because in the Portuguese interpretation of the data. All authors read and approved the final
school system there is a complete turnover of teachers from manuscript.
6th to 7th grade, which would have resulted in a reduced
percentage of teacher reports available for the follow-up, Conflict of interest The authors report no conflict of interests.
compromising the validity of the findings.
Funding Project Positive Attitude is funded by the Municipality of
Torres Vedras.
Future Directions
Ethical Approval The present study was conducted following the
In future studies it would be particularly important to national professional code of ethics for psychologists, following
national legislation.
analyze the cumulative results of social and emotional
learning programs (i.e., elementary school, low middle Informed Consent All schools used passive informed consent,
school, middle school) over several grades applied to the following national legislation when a program is integrated into the
same students, in accordance with suggestions by the school curriculum.
Collaborative for Academic, Social, and Emotional
Learning (2012) that programs should be pluriannual. It
would also be interesting to look into the role of the References
experience of the psychologists applying the program, as
experience and mastery in program delivery could have a Caemmerer, J. M., & Keith, T. Z. (2015). Longitudinal, reciprocal
effects of social skills and achievement from kindergarten to
prominent role in the results achieved (Castillo et al. 2013). eighth grade. Journal of School Psychology, 53, 265–281.
doi:10.1016/j.jsp.2015.05.001.
Carroll, E., Skinner, C., McCleary, D., vonMizener, B., & Bliss, S.
Conclusion (2009). Analysis of author affiliation across four school
psychology journals from 2000 to 2008: Where is the practi-
tioner research? Psychology in the Schools, 46, 627–635. doi:10.
The results reinforce the need, stressed by several authors 1002/pits.20403.
(Coelho et al. 2015; Durlak et al. 2011; Greenberg et al. Castillo, R., Fernández-Berrocal, P., & Brackett, M. A. (2013).
2003; Sklad et al. 2012), for social and emotional learning Enhancing teacher effectiveness in Spain: A pilot study of The
RULER approach to social and emotional learning. Journal of
programs to be developmentally adequate. Program Education and Training Studies, 1, 263–272. doi:10.11114/jets.
developers should take into account that the program v1i2.203.
delivery format may influence the program’s effectiveness. Coelho, V., & Figueira, A. (2011). Project ‘‘Positive Attitude’’:
Hence, when designing programs, it is important to take promoting school success through social and emotional abilities
development. Design for elementary and middle school students,
developmental profiles into account, not only during the in Portugal. Interamerican Journal of Psychology, 45(2),
content development stage, but also when choosing pro- 185–192.
gram delivery format. The present findings also support Coelho, V., Sousa, V., & Figueira, A. P. (2014). The impact of a
Durlak et al. (2011) in that younger students may need school-based social and emotional learning program on the self-
concept of middle school students. Revista de Psicodidactica,
more time to acquire complex skills. 19(2), 347–365. doi:10.1387/RevPsicodidact.10714.
The narrower gender effects found, especially compared Coelho, V., Marchante, M., & Sousa, V. (2015). ‘‘Positive Attitude’’:
with the wider array of results observed in the high middle A multilevel model analysis of the effectiveness of a social and
school program (Coelho et al. 2015), suggest that gender- emotional learning program for Portuguese middle school
students. Journal of Adolescence, 43, 29–38. doi:10.1016/j.
specific differences in the development of social and adolescence.2015.05.009.
emotional competencies may arise in time, following Cohen, J. (1988). Statistical power analysis for the behavioral
gender role differentiation. The results are in agreement sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.

123
 Author's personal copy
J Youth Adolescence (2017) 46:656–667 667

Collaborative for Academic, Social, and Emotional Learning. (2005). An effectiveness study of 5 years of intervention. Health
Safe and sound: An educational leader’s guide to evidence- Education Research, 23, 931–940. doi:10.1093/her/cyn040.
based social and emotional learning programs—Illinois edition. Knapp, M., McDaid, D., & Parsonage, M. (2011). Mental health
Chicago: Author. promotion and mental illness prevention: The economic case.
Collaborative for Academic, Social, and Emotional Learning. (2012). London: Department of Health.
2013 CASEL guide: Effective social and emotional learning Marsh, H. W. (1989). Age and sex effects in multiple dimensions of
programs (Preschool and elementary school edition). Chicago, self-concept: Preadolescence to early childhood. Journal of
IL: Authors. Educational Psychology, 81, 417–430. doi:10.1037/0022-0663.
Collaborative for Academic, Social, and Emotional Learning. (2015). 81.3.417.
2015 CASEL guide: Effective social and emotional learning Shapiro, J., Burgoon, J., Welker, C., & Clough, J. (2002). Evaluation
programs—Middle and high school edition. Chicago, IL: of the Peacemakers program: School-based violence prevention
Authors. for students in grade through eight. Psychology in the Schools,
Conduct Problems Prevention Research Group. (2010). The effects of 39, 87–100. doi:10.1002/pits.10040.
a multiyear universal social–emotional learning program: The Sklad, M., Diekstra, R., DeRitter, M., Ben, J., & Gravesteijn, C.
role of student and school characteristics. Journal of Consulting (2012). Effectiveness of school-based universal social, emo-
and Clinical Psychology, 78(2), 156–168. doi:10.1037/ tional, and behavioral programs: Do they enhance students’
a0018607. development in the area of skill, behavior and adjustment?
Cook, C. R., Williams, K. R., Guerra, N. G., Kim, T. E., & Sadek, S. Psychology in the Schools, 49, 892–909. doi:10.1002/pits.21641.
(2010). Predictors of bullying and victimization in childhood and Tobler, N., Roona, M., Ochshorn, P., Marshall, D., Streke, A., &
adolescence: A meta-analytic investigation. School Psychology Stackpole, K. (2000). School-based adolescent drug prevention
Quarterly, 25(2), 65–83. doi:10.1037/a0020149. programs: 1998 meta-analysis. Journal of Primary Prevention,
Diekstra, R., & Gravesteijn, C. (2008). A review of meta-analytic 30, 275–336. doi:10.1023/A:1021314704811.
literature reviews in social and emotional education. In C. Tomic, W., & Klauer, K. J. (1996). On the effects of an inductive
Clouder & B. Heys (Eds.), Social and emotional education: An reasoning training: How far does it transfer and how long do the
international analysis (pp. 258–284). Santander, Spain: Fun- effects persist? European Journal of Psychology of Education,
dación Marcelino Botı́n. 11, 283–299. doi:10.1007/BF03172941.
Domitrovich, C., Gest, S. D., Jones, D., Gill, S., & Sanford-DeRousie, Wigelsworth, M., & Lendrum, A. (2013). The evaluation of school-
R. M. (2010). Implementation quality: Lessons learned in the based social and emotional learning interventions: Current issues
context of the Head Start REDI trial. Early Childhood Research and future directions. The Psychology of Education Review,
Quarterly, 25, 284–298. doi:10.1016/j.ecresq.2010.04.001. 37(2), 70–76.
Durlak, J., Weissberg, R., Dymnicki, A., Taylor, R., & Schellinger, K. Wigelsworth, M., Humphrey, N., Lendrum, A., & Kalambouka, A.
(2011). The impact of enhancing students’ social and emotional (2010). A review of key issues in the measurement of children’s
learning: A meta-analysis of school-based universal interven- social and emotional skills. Educational Psychology in Practice,
tions. Child Development, 82, 405–432. doi:10.1111/j.1467- 26, 173–186. doi:10.1080/02667361003768526.
8624.2010.01564.x. Wigelsworth, M., Humphrey, N., & Lendrum, A. (2013). Evaluation
Faria, L., & Fontaine, A. M. (1990). Avaliação do conceito de si of a school-wide preventive intervention for adolescents: The
próprio de adolescentes: adaptação do SDQ I de Marsh à secondary social and emotional aspects of learning (SEAL)
população portuguesa. Cadernos de Consulta Psicológica, 6, programme. School Mental Health, 5, 96–109. doi:10.1007/
97–106. s12310-012-9085-x.
Ferreira, C., & Rocha, A. (2004). BAS-3: Bateria de Socialização. Wolpert, M., Humphrey, N., Deighton, J., Patalay, P., Fugard, A.
Lisboa: CEGOC. J. B., Fonagy, P., et al. (2015). An evaluation of the implemen-
Greenberg, M., Weissberg, R., O’Brien, M., Zins, J., Fredericks, L., tation and impact of England’s mandated school-based Mental
Resnik, H., et al. (2003). Enhancing school-based prevention and Health Initiative in elementary schools. School Psychology
youth development through coordinated social, emotional, and Review, 44, 117–138.
academic learning. American Psychologist, 58, 466–474. doi:10.
1037/0003-066X.58.6-7.466.
Hahn, R., Fuqua-Whitley, D., Wethington, H., Lowy, J., Crosby, A., Vitor Alexandre Coelho has been the coordinator of the Positive
Fullilove, M., et al. (2007). Effectiveness of universal school- Attitude Project since 2004. He received his doctorate in Educational
based programs to prevent violent and aggressive behavior: A Psychology from the University of Coimbra in 2015. His research
systematic review. American Journal of Preventive Medicine, interests include social and emotional learning, self-concept and self-
33(2, Suppl. 1), S114–S129. doi:10.1016/j.amepre.2007.04.012. esteem, bullying and cyberbullying prevention, and professional
Heck, R. H., Thomas, S. L., & Tabata, L. N. (2013). Multilevel and issues.
longitudinal modeling with IBM SPSS (2nd ed.). London:
Routledge. Vanda Sousa has been a researcher in the Positive Attitude Project
Holsen, I., Smith, B. H., & Frey, K. S. (2008). Outcomes of the social for the Académico de Torres Vedras since 2005. She received her
competence program Second Step in Norwegian elementary master’s in Stress and Well-Being from the University of Lisbon in
schools. School Psychology International, 29, 71–88. doi:10. 2007. Her main research interests include social and emotional
1177/0143034307088504. learning, gender equality, and school adjustment.
Kimber, B., Sandell, S., & Bremberg, S. (2008). Social and emotional
training in Swedish schools for the promotion of mental health:

123