ABSTRACT
The purpose of this study was to determine the impact of Itinerario+ program, an intervention aimed at improving the different areas of competence of the social and emotional learning model (i.e., self-awareness, social awareness, self-management, relationship skills and responsible decision-making) among students of Basic Vocational Education and Training from disadvantaged contexts. The sample was composed by 140 first year students (70 experimental group y 70 control group) from four different Basic Vocational Education and Training programs taught in five educational centers in the southern district of the city of Madrid (Spain). Social and emotional learning was assessed by the Social and Emotional Learning Scale. After assigning students to either the experimental or control group according to quasi-experimental design with a non-equivalent control group, it was confirmed that both groups were equivalent around the control variables or covariates and ltinerario+ program was implemented. This intervention was integrated with school instruction, so the curriculum was developed in six transversal projects, including other activities (i.e., peer mentoring, individualized tutoring, vocational and professional guidance, internships in a professional environment), which were carried out by a team of previously trained teachers and educators. The results after comparing the experimental and control groups yield statistically significant differences in favor of the experimental group in social and emotional learning. These results confirm the effectiveness of the program to improve the social and emotional learning of students at risk of social exclusion, so it can be stated that ltinerario+ is an example of evidence-based practice.
Keywords: social and emotional learning, social development, emotional development, emotional intelligence, program evaluation, evidence-based practice
RESUMEN
El proposito de este trabajo de investigacion fue determinar el impacto del programa ltinerario+, una intervencion dirigida a mejorar las diferentes areas de competencia del modelo de aprendizaje social y emocional (i.e., autoconciencia, conciencia social, autocontrol, habilidades para relacionarse y toma de decisiones responsable) en alumnado de Formacion Profesional Basica procedente de contextos desfavorecidos. La muestra estuvo compuesta por 140 estudiantes de primer curso (70 grupo experimental y 70 grupo control) de cuatro titulaciones de Formacion Profesional Basica impartidas en cinco centros educativos del distrito sur de la ciudad de Madrid (Espana). El aprendizaje social y emocional se evaluo mediante la Escala de Aprendizaje Social y Emocional. Despues de asignar al alumnado a la condicion experimental o control de un diseno cuasiexperimental con grupo control no equivalente, se confirmo que ambos grupos eran equivalentes en torno a las variables control o covariantes y se implemento el programa ltinerario+. Esta intervencion se integro con la instruccion escolar, por Io que el curriculo se desarrollo en seis proyectos transversales, incluyendo otras actividades (i.e., mentoria entre alumnado, tutoria individualizada, orientacion vocacional y profesional, practicas en un entorno profesional), que fueron llevados a cabo por un equipo de profesores y educadores previamente formados. Los resultados derivados de las comparaciones entre los grupos experimental y control arrojan diferencias estadisticamente significativas a favor del grupo experimental en aprendizaje social y emocional. Estos resultados confirman la eficacia del programa para mejorar el aprendizaje social y emocional del alumnado en riesgo de exclusion social, por Io que se puede manifestar que ltinerario+ es un ejemplo de practica basada en evidencias.
Palabras clave: aprendizaje social y emocional, desarrollo social, desarrollo emocional, inteligencia emocional, evaluacion de programas, practicas basadas en evidencias
INTRODUCTION
The need to tackle the high incidence rates of social, emotional and behavioural problems among students in compulsory education has led to a significant growth in educational, social and political interest in the last few years in certain protective factors that, according to the available scientific evidence, enhance student performance and well-being (Oberle et al., 2016; Trujillo et al., 2021). These protective factors include social and emotional learning, which is conceived as the process of learners acquiring and effectively using the knowledge, skills and attitudes needed to develop healthy identity, manage their emotions, set and achieve positive personal and collective goals, empathise with and feel for others, establish and maintain positive and supportive interpersonal relationships, manage interpersonal situations in a constructive manner, and make responsible and affective decisions (Collaborative for Academic, Social, and Emotional Learning, 2021; Durlak et al, 2011; Jagers et al., 2019; Mahoney et al., 2020; Payton et al., 2008; Taylor et al., 2017; Weissberg et al., 2015).
Students' social and emotional learning involves the ability to combine behaviour, cognition and affect (Mahoney et al., 2020), providing them with the precise skills to successfully deal with any situation in their daily lives, which is essential to improve their learning, performance and personal satisfaction (Oberle et al., 2016; Organisation for Economic Co-operation and Development, 2021; Weissberg et al., 2015). Indeed, the evidence from research on this essential students' elements about personal and social-emotional development has strongly encouraged the implementation and evaluation of a variety of programmes and intervention strategies that aim to create safe and supportive learning environments in which to promote the following social-emotional competencies (Collaborative for Academic, Social, and Emotional Learning, 2021; Jagers et al., 2019; Mahoney et al., 2020; National Commission on Social, Emotional, and Academic Development, 2019): (a) self-awareness, able to identify own emotions, thoughts, values and how they affect behaviour, including identifying strengths and limitations with confidence, enthusiasm and a growth mindset; (b) social awareness, needed to empathise and understand the perspectives for others, from diverse backgrounds and cultures, including standing up for one's own ideas without putting others aside; (c) self-management or successfully regulating own thoughts, emotions and behaviour in different situations, including setting school goals and working for their achievement, with self-discipline and self-motivation, using strategies for planning and organisation; (d) skills in relationships, establishing and maintaining appropriate relationships with others, including effective communication, active listening and cooperation with others; and (e) responsible and constructive decision-making about personal behaviour and relationships with others based on ethical standards, safety and social norms, including assessment of the consequences of actions and the wellbeing of self and others, as well as identifying problems, proposing solutions and carrying out actions that contribute to improving the most immediate environment.
Many studies have established causal relationships between intervention measures based on social and emotional learning and some improvements at the socioemotional, behavioural and school level of the participating students, regardless of their socio-demographic and educational profile (i.e. racial, ethnic and socio-economic background, from different educational levels and environments, with and without emotional and behavioural problems), the firsts systematic reviews that have been developed to determine the impact of these programmes have confirmed this, racial, ethnic and socio-economic background, from different educational levels and settings, with and without emotional and behavioural problems), as confirmed by the first systematic reviews that have been developed to determine the impact of these programmes and practices (i.e., Diekstra, 2008; Durlak et al., 2010, 2011; Payton et al., 2008; Sklad et al., 2012). In fact, the results of the meta-analytic reviews that have subsequently been conducted for the same purpose are along the same line(i.e., Corcoran etal., 2018; Jagers etal., 2015; Murano et al., 2020; Sabey et al., 2017; Taylor et al., 2017; Wigelsworth et al., 2016; Yang et al, 2019), which have repeatedly demonstrated their ability to improve students' social-emotional skills, self-perceptions, attitudes towards others, commitment and connection to the school, pro-social behaviour and school performance, leading to a decrease in their emotional, behavioural and substance abuse difficulties, and even effects on other members of the educational community (e.g., higher rates of teacher effectiveness and planning achievement).
Those results have contributed to the fact that interventions based on social and emotional learning are amongst the most successful development programmes, which has led to their dizzying and extensive diversification and incorporation into educational institutions and classrooms all over the world (Wigelsworth et al., 2016). More precisely, Spain has been one of the countries that has made a strong commitment over the last decade to these programmesand practices in compulsory education, although it is true that during their implementation the key indicators that guarantee their success have not always been considered (Durlak et al., 2010, 2011; Mahoney et al., 2020), meanwhile their evaluation has been characterised by incorporating qualitative or pre-experimental designs, which limits the power of the evidence available on their efficacy (Fernandez et al., 2021). Therefore, it seems unquestionable that there is a need to increase the quality of scientific production on the design, implementation and evaluation of programmes and practices based on social and emotional learning, which allows for the establishment of a national agenda to promote its adoption throughout the education system, integrating it into existing educational priorities (Fernandez et al., 2021).
In this sense, the ltinerario+ programme which is presented in this paper incorporates in its design, implementation and evaluation a series of elements and characteristics that aim to redress the weaknesses and shortcomings noted above, such as (Durlak et al., 2010, 2011; Fernandez et al., 2021; Mahoney et al., 2020): (a) the explicit instruction of social-emotional competences, based on a sequenced, step-by-step training approach that emphasises active forms of learning, concentrating specific time and attention on skills training and in which goals are clearly defined, i.e., training that is sequenced, active, focused and explicit; (a) the explicit instruction of social-emotional competencies, based on a sequenced, stepby-step training approach that emphasises active forms of learning, concentrating specific time and attention on skills training and in which goals are clearly defined, that is, training that is sequenced, active, focused and explicit; (b) the integration of the programme with school instruction; (c) the active role of the participants; (d) the training of the teachers responsible for implementation; (e) the collaboration and synergies between classrooms, families and communities; (f) a quasi-experimental evaluation methodological design with a non-equivalent control group enhanced with statistical control techniques or conventional pairwise matching; and (g) a system of monitoring and continuous improvement. Itinerario+ is a transformative educational intervention, an holistic model (i.e., it mixes formal and non-formal, social and community, as well as personal and professional elements) and innovative action plan, which aims to change the life trajectory of young people in Basic Vocational Education Training from disadvantaged backgrounds, directing its efforts to improve the competency profile of these young people from a socio-emotional and employment perspective, which will allow them to progress in school, have more opportunities for socio-occupational integration and have a life plan in which they can make informed decisions about their future (Fundacion Tornillo, 2022).
Therefore, the purpose of this research was to test the effectiveness of the Itinerario + programme in improving the different competence areas of the social and emotional learning model (i.e., self-awareness, social awareness, selfmanagement, relationship skills and responsible decision-making) among Basic Vocational Education Training students from disadvantaged backgrounds. The hypotheses that were established were: (1) in the experimental group of students, as a result of their participation in ltinerario+, statistically significant differences will be observed in the different socioemotional competencies in the post-test phase with respect to the pre-test phase, whereas in the case of the control group no such differences will be observed; and (2) there will be statistically significant differences in the different socioemotional competencies in the post-test phase in favour of the experimental group with regard to the control group as a result of their participation in the programme
METHODOLOGY
Participants
This research involved 140 first-year students from four Basic Vocational Education Training qualifications taught in five schools in the southern district of the city of Madrid (Spain). This sample was divided into two equivalent groups, in which the experimental group was made up of 70 students from one of the schools, 14 females and 56 males, with a mean age of 15.89 years (SD = 0.81, range between 15 and 17 years), and a distribution by qualifications including 23 students of Computer Science and Communications, 15 of Electricity and Electronics, 10 of Administrative Services, and 22 of Cooking and Catering. The control group consisted of 70 students from four other schools, with mean and range age the same as the experimental group, as well as the same distribution by gender and qualifications.
Instrument
The Social and Emotional Learning Scale (Fernandez et al., 2022) is a Likerttype scale made of 30 one- to four-point items (i.e., 1 = Never or hardly ever; 2 = Occasionally; 3 = Often; and 4 = Almost always or always), grouped into five areas of social-emotional competence, such as self-awareness, social awareness, self-management, relationship skills, and responsible decision-making. This scale was used as it is one of the few instruments available to evaluate the different competences areas of the social and emotional learning model (Collaborative for Academic, Social, and Emotional Learning, 2021; Jagers et al., 2019; Mahoney et al, 2020; National Commission on Social, Emotional, and Academic Development, 2019) intheSpanish teenager population, as well as for having an adequate reliability (Cronbach's alpha [a] between .70 and .84, and McDonald's omega [ui] between .71 and .84, with composite reliability and average variance indices of higher than .77 and .67, respectively, in the different areas of socioemotional competence) and validity (tests with excellent goodness-of-fit indices that confirm their internal structure and predictive validity on school performance and life satisfaction) (Fernandez et al., 2022) in the different areas of socioemotional competence) and validity (tests with excellent goodness-of-fit indices that confirm their internal structure and predictive validity on school performance and life satisfaction) (Fernandez et al., 2022) (Fernandez et al., 2022). For their part, in that study they yielded an a of .90 and a u) of .90, with scores ranging between .71 and .83 in the different areas of socioemotional competence, as well as compound reliability and average variance extracted indices above .70 and .60, respectively, in these areas of competence. Confirmatory factor analysis showed adequate goodness-of-fit indices and statistics: chi-square (x2) (395) = 402.94; p < .38; x2/degree of freedom = 1.02; comparative fit index = .99; goodness-of-fit index = .95; standardised root mean square residual = .08; root mean square error of approximation = .02 (90% confidence interval = .00 - .04).
The Participation Questionnaire is an ad hoc self-report made up of 28 items with different response alternatives, aimed at collecting socio-demographic information (i.e., age, sex, nationality, immigration background, employment status, family structure, educational level of mothers/parents/guardians, employment status of mothers/parents/guardians, perceived economic sufficiency, age of access to early childhood education and cultural capital) and school information (i.e., educational centre, qualification, year, subjects, subjects or modules enrolled, year of access to the qualification, presence of special education needs (SEN), previous course repetition, previous change of studies and previous drop-out) relevant to the control or covariate variables of the participating students.
Design and process
The methodological design used in this research, which was approved by the Ethics Committee of the University of Granada (1736/CEIH/2020), was quasiexperimental with a non-equivalent control group enhanced with statistical control techniques or conventional matching (Ato et al., 2013; Gertler et al., 2017). The sample selection procedure was based on a non-probabilistic type of sampling, called convenience sampling (Gertler et al., 2017; Kalton, 2020). Consequently, the entity responsible for the management of the programme established that ltinerario+ had to be implemented compulsorily in the first year of the Basic Vocational Education Training qualifications taught in an educational centre in the southern district of the city of Madrid, so its students (four groups, n = 94) were assigned to the experimental condition. In view of this requirement, in order to form the comparison or control group, we contacted the institutional heads of several schools with very similar characteristics to the experimental school (i.e., geographical location, type, educational offer, school and socio-demographic profile of the students) and provided them with the relevant information about the programme and the requirements for participation, finally formalising a collaboration agreement with four of these schools (10 groups, n = 216).
Then, once the appropriate institutional permissions were granted, as well as the students' family consent and authorisation, the instruments were administered, followed by the pairing, i.e., constructing an artificial comparison or control group as similar as possible to the experimental group on the basis of those observable characteristics (i.e., control or covariate variables) that may influence the results and/or be affected by the intervention (Ato et al., 2013; Gertler et al., 2017). From this approach, 89 associated pairs (n = 178) were created based on the following control or covariate variables (Choi & Calero, 2013; Fernandez et al., 2010; Gonzalez et al., 2019; Rivkin et al., 2005): (a) sociodemographic: age, gender, nationality, immigration background, employment status, family structure, educational level of mothers/parents/guardians, employment status of mothers/parents/guardians, perceived economic sufficiency, age of access to early childhood education and cultural capital; (b) school: degree, year, subjects or enrolled modules, year of access to the qualification, presence of SEN, previous courses repeats, previous change of studies and previous dropout; and (c) socio-emotional competences: self-awareness, social awareness, self-control, interpersonal skills and responsible decision making; (d) social-emotional competences: self-awareness, social awareness, self-management, interpersonal skills and responsible decision making; and (e) social-emotional competences: self-awareness, social awareness, selfmanagement, interpersonal skills and responsible decision making.
In this regard, the remaining 132 students (five from the experimental condition and 127 from the control group) were excluded because they did not provide family consent and authorisation and/or did not have a suitable partner. Furthermore, 19 students from the experimental group, for different reasons (e.g., dropping out of the qualification) did not attend at least 80% of the classes during the implementation of the ltinerario+ programme, which meant that the final sample consisted of 70 associated pairs (n = 140), well above the minimum established according to the result obtained after calculating the minimum size required to carry out the evaluation of the ltinerario+, both in total (n = 102) and per group (n = 51).
It was confirmed that both groups, experimental and control, were equivalent in terms of the control or covariate variables, since: (a) some of them presented a single or the same value in both groups, as was the case of employment status (course of studies in Basic Vocational Education Training, not work), course (first year), subjects, subjects or modules enrolled and year of access to the degree (2021); (b) others presented identical proportions in both conditions, such as age (15 years = 38. 57%, 16 years = 34.29%, 17 years = 27.14%), gender (male = 80%, female = 20%), nationality (Spanish = 82.86%, Dominican = 7.14%, Chinese = 2.86%, Moroccan = 5.71%, Nicaraguan = 1.43%), immigration background (Yes = 60%, No = 40%), qualification (Computer and Communications = 32.86%, Electricity and Electronics = 21.43%, Administrative Services = 14.28%, Cooking and Catering = 31. 43%), presence of Specific Educational Support Needs (Yes = 15.71%, No = 84.29%), previous course repetition (Yes = 85.71%, No = 14.29%), previous change of studies (Yes = 70%, No = 30%) and previous dropout (Yes = 14. 29%, No = 85.71%); and (c) non-parametric tests did not show statistically significant differences between the two groups in family structure (Pearson's Chi-square [x2] = 0.86; p >. 05), educational level of mothers/guardians (x2 = 0.43 p > .05), educational level of fathers/guardians (x2 = 0.31; p > .05), employment status of mothers/guardians (x2 = 0.44; p > .05), employment status of fathers/guardians (x2 = 0.14; p > . 05), age of access to early childhood education (x2 = 0.72; p > .05), perceived economic sufficiency (MannWhitney's U test [U] = 2324.50; p > .00), cultural capital (U = 2142.00; p > .00) and socioemotional competencies in the pretest phase (see Table 2).
The ltinerario+ programme was implemented during the 2021/2022 school year, although the training of the teachers responsible for its implementation and the piloting of the programme was carried out during the previous school year. This intervention, in the terms that have been established in the specialised literature (Durlak et al., 2010, 2011; Fernandez et al., 2021; Mahoney et al., 2020), opted for the explicit instruction of socioemotional competences based on a sequenced, active, focused and explicit training and, effectively, it was integrated with school instruction. To this end, the Basic Vocational Education Training curriculum was developed in six transversal projects, the core activity of the programme's logic model, in which the contents of the different modules were addressed through active methodologies (i.e., project-based learning, with a service-learning approach, and cooperative learning), with their corresponding evaluation processes, using tools aimed at favouring reflection, metacognition and conscious learning of the students (i.e., learning portfolio, learning diary, rubrics and self- and co-assessment questionnaires). These projects were carried out by a core team of teachers and educators in the Aula+, a technical, open and flexible space for group and individual work for students, but also involved the development of different significant experiences in which students had the opportunity to actively consolidate learning from a different scenario, such as: (a) the disruptive learning pills, which involved 50 group work sessions of two hours each in a variety of artistic-musical, environmental, sports and technological scenarios; (b) 20 group school outings to promote experiential and experiential learning (e.g.,15 visits to professional work centres and five learning visits to other cities); (c) 10 two-hour group technical sessions given by professionals from companies (e.g., workshops and training sessions given by professionals from their professional profile with the aim of broadening their exposure to the professional world and getting to know different realities of companies such as Iberdrola, Telefonica, Grupo VIPS, etc.); and (d) 20 group field exploration and research sessions (e.g., 100 hours of exploration of the environment and experiential evidence collection and analysis).
Itinerario+ also incorporated into its logic model the implementation of other activities in addition to the development of the transversal projects, such as: (a) a mentoring programme between students and young people from the Tornillo Foundation, to facilitate access and school and personal adjustment of students, in which they carried out together 30 mentoring sessions, with a weekly frequency; (b) actions of individualised tutorial attention aimed at the early detection of potential situations of school failure and socio-educational support to students and families (e.g., nine interviews with students, three per semester, at the beginning, middle and end of the semester; three interviews with families, one at the beginning of each semester), as well as support for personal development and personalisation of the students' learning itineraries (e.g., 10 two-hour group sessions and 10 onehour individual sessions, in which we developed welcoming dynamics to overcome emotional states that do not encourage learning and to determine itineraries according to the students' learning itineraries), 10 two-hour group sessions and 10 one-hour individual sessions, in which welcome dynamics were developed to leave behind emotional states that do not favour learning and to determine itineraries according to vocation and interests); (e.g., nine interviews with students, three per semester, at the beginning, middle and end of the semester; three interviews with families, one at the beginning of each semester), as well as support for personal development and personalisation of the students' learning itineraries (e.g., 10 twohour group sessions and 10 one-hour individual sessions, in which we developed welcoming dynamics to overcome emotional states that do not encourage learning and to determine itineraries according to the students' learning itineraries), 10 twohour group sessions and 10 one-hour individual sessions, in which welcome dynamics were developed to leave behind emotional states that do not favour learning and to determine itineraries according to vocation and interests); (c) 25 one-hour individual sessions of vocational orientation to help students to know their strengths and their passions, in order to make informed decisions about their future; (d) 240 hours of internships in a professional environment, designed in a personalised way for the students, identifying the opportunities for the development of technical skills and socio-emotional competences that each company could offer, in order to make the student-company assignment based on the needs of the company and the students' competence and technical development; and (e) 25 one-hour individual sessions of professional orientation to jointly assess the access to the labour market, job offers, up-skilling or re-skilling programmes, invitation and participation in events, etc.
On the other hand, a monitoring plan was adopted, with several actions aimed at identifying possible deviations of ltinerario+ from its initial approach (Fernandez et al., 2019). To be precise, 3 group follow-up sessions were held (i.e., at the end of the first, second and third trimester of the school year) between those responsible for the evaluation of the programme and the core team of teachers and educators, while 2 group follow-up sessions were held with the participating students (i.e., at the end of the first and second trimester of the school year). These sessions were mainly devoted to an overall assessment of participation in the programme, with emphasis on difficulties in the development of the ltinerario+ and possible solutions. Additionally, with the outcome evaluation plan, measures of the dependent variables were collected in the pre-test and post-test phases, in order to subsequently examine the presence of statistically and educationally significant effects (Fernandez et al., 2019).
Data Analysis
First, a priori calculation of the minimum sample size was performed considering the expected effect size (0.50), the associated probability (.05) and the desired levels of statistical power (.80) (Soper, 2021) (Soper, 2021). Subsequently, following the recommendations of Tabachnick and Fidell (2019), we confirmed the absence of missing, outliers and influential values (Mahalanobis distance) and performed the descriptive analysis of the data collected, and then confirmed the absence of univariate normality (Kolmogorov-Smirnov test) in the distribution of scores for both the control variables and the dependent variables: perceived financial sufficiency (z = 0. 12, p < .00) cultural capital (z = 0.20, p < .00), self-awareness (z = 0.09, p < .00), social awareness (z = 0.17, p < .00), self-managemenet (z = 0.10, p < .00), interpersonal skills (z = 0.09, p < .00) and responsible decision-making (z = 0.12, p < .00). Moreover, the absence of multivariate normality in the distribution of the dependent variable scores was confirmed by Mardia's skewness (60.91, x2 = 1457.44, p < .00) and kurtosis (135.89, x2 = 36.81, p < .00) coefficients.
In this regard, the data were analysed using the Mann-Whitney U test for two independent samples and Pearson's x2 to test the equivalence of the experimental and control groups on the control variables.
Fourth, in order to examine the psychometric properties of the Social and Emotional Learning Scale (Fernandez et al., 2022) in this study, a confirmatory factor analysis of five first-order correlated factors was conducted and estimated by the weighted least squares method, using the indices that are normally used to assess goodness-of-fit (i.e., x2, x2 ratio/degrees of freedom, comparative goodness-of-fit index, standardised mean square residual, and standardised mean square error of approximation) (Kline, Kline, et al., 2022), x2, x2 ratio/degrees of freedom, comparative fit index, goodness-of-fit index, standardised root mean square residual and root mean square error of approximation) (Kline, 2015) (Kline, 2015). In addition, internal consistency (i.e., a and oi) and composite reliability (i.e., composite reliability index and average variance extracted) were calculated (Hair et al., 2014).
Finally, to determine the effects of the ltinerario+ programme on the dependent variables, the data were analysed using Wilcoxon's z-tests (hypothesis 1) and MannWhitney U-tests for two independent samples (hypothesis 2). In additional, Cohen's d value was calculated, while the error rate per family, resulting from the problem of multiple comparisons, given the impossibility of performing multivariate contrasts, was controlled with the Bonferroni correction.
Statistical analyses were performed using Statistical Package for Social Sciences (SPSS) v26 (IBM Corp., Armonk, NY, USA) and STATA vl7 (StataCorp., College Station, TX, USA).
RESULTS
The Bonferroni correction was used to adjust the significance level for each of the multiple comparisons tests, yielding a result of .01 (.05/5).
The results ofthepretest-posttest comparisons on socioemotional competencies, hypothesis 1, reveal no statistically significant differences in the control group, while in the case of the experimental group a statistically significant improvement is observed in the posttest phase with respect to the pretest phase in each of the areas of socioemotional competence (Table 1).
As for hypothesis 2, the results of the intergroup comparisons in the pretest phase do not show statistically significant differences in any of the social-emotional competences established, in contrast to the post-test phase, in which statistically significant differences are observed in favour of the experimental group in each of the social-emotional competences (Table 2).
DISCUSSION AND CONCLUSION
The purpose of the study was to test the kind, direction and magnitude of changes produced by the ltinerario+ programme on the socioemotional development of a sample of Basic Vocational Education Training students at risk of social exclusion, specifically the impact on each of the areas of competence of the social and emotional learning model (i.e., self-awareness, social awareness, self-control, relationship skills and responsible decision-making). Considering the results derived from participation in the programme, the following conclusions can therefore be drawn: (a) since in the intragroup comparisons statistically significant differences are observed in the experimental group in favour of the posttest phase in all areas of social-emotional competence, quite the contrary to the control group, hypothesis 1 is accepted; and (b) since statistically significant differences are observed in the intergroup comparisons in the posttest phase in favour of the experimental group in each of the areas of competence of the social and emotional learning model, not so in the control group, hypothesis 2 is also accepted.
Effectively, participation in the ltinerario+ programme has generated a positive and statistically significant impact for the pupils on the different indicators of socioemotional development, despite the high restrictive nature of the Bonferroni correction. Moreover, if the hypotheses of this study are tested using tests that provide answers on their practical significance, as established in the specialised literature (Hattie, 2009; Kraft, 2020; Ledesma et al, 2008), considering the most demanding guidelines for interpreting the results of analyses complementary to the contrast of means (Cohen, 1988), the magnitude of the effect size that has been achieved in most of the areas of socioemotional competences has been large, that is, the intergroup differences that have been generated in these areas of competences can be detected by simple observation (Coe, 2002), which clearly indicates that they have an important practical relevance (Hattie, 2009; Kraft, 2020). Indeed, effect size values reveal that a hypothetical member of the experimental group, compared to any member of the control group, can achieve scores in these competence areas in excess of 69% (e.g., interpersonal skills), with percentages as high as 88% (e.g., self-awareness) (Coe, 2002; Kraft, 2020).
To sum up, the results obtained confirm the effectiveness of the ltinerario+ programme in increasing the level of socioemotional skills of compulsory education students at risk of social exclusion. This will possibly help them to progress at school, have more opportunities for socio-occupational integration and have a life plan in which they can make more informed decisions about their future. Furthermore, these results are in line with those of other studies that have been developed with the aim of establishing causal links between programmes and practices based on the social and emotional learning model and certain improvements in the socioemotional level of their participants. It can be seen in the meta-analytical reviews that have been carried out in the last decade (Corcoran et al, 2018; Durlak et al., 2010, 2011; Jagers et al., 2015; Murano et al., 2020; Sabey et al., 2017; Taylor et al., 2017; Wigelsworth et al., 2016; Yang et al., 2019), and equally favours increasing the quantity and quality of the limited scientific production on the systematic evaluation of this type of interventions at the national level (Fernandez et al., 2021).
The effectiveness of the ltinerario+ programme seems to be mainly determined by the inclusion in its logic model of the main elements and characteristics that, according to the specialised literature, guarantee to a greater extent the success of these intervention measures, such as, explicit instruction of social-emotional competences (i.e, using a step-by-step sequenced training approach to socialemotional skills, emphasising active forms of learning for students to practice the new skills, focusing specific time and attention on skills training, and clearly defining goals) and their integration with school instruction (Durlak et al., 2010, 2011; Fernandez et al., 2021; Mahoney et al., 2020). However, the active involvement of the educational community in the development of the programme, which was strongly supported by the implementation of other successful actions (e.g. mentoring, tutorial action, vocational and career guidance), seems to have been another key component in explaining the results obtained, although it is difficult to isolate the effects of the different elements of which the programme was composed.
Nevertheless, when interpreting the results obtained in this study, it is necessary to consider certain limitations, mainly linked to the sampling, methodological design, data analysis model and self-reporting measures adopted. In that line, a completely randomised sampling and design for the different activities of the programme's logic model would have allowed for greater control of the possible sources of bias. Despite this, the requirements and resources of the entity responsible for managing the programme made this impossible, in terms similar to the analysis model initially planned (i.e., analysis of covariance), as the confirmation of non-compliance with the basic assumptions for its estimation made it necessary to opt for a non-parametric bivariate model, with the limitations that this entails for the power of the evidence obtained in the study. On the other hand, the equivalence of the groups with respect to a series of control variables was confirmed, which maximises their comparability, and the error rate per family derived from the problem of multiple comparisons was controlled in the analysis model, thus raising the level of statistical requirement.
Despite all this, in future replications of the programme it would be necessary to adopt a methodological design with a greater degree of experimentality, as well as to include measures (e.g., several experimental groups with different levels of exposure to the intervention programme) that allow us to determine the impact of each of the programme's activities or the contribution of each of them to the results obtained. It would also be essential to increase the sample size, even if it was adjusted to standard power and alpha error conditions, but above all to increase the diversity of participating qualifications and courses. At the same time, the number of measures on the competency areas of the social and emotional learning model (e.g., other indicators and instruments) or informants (e.g., teachers, families) should be increased, in addition to using some other analysis model (e.g., generalised linear models) or assessing the possibility of transforming variables (e.g., differences in differences) to estimate other multivariate models (e.g., analysis of covariance). Finally, it is certainly always advisable to change those aspects of the programme that, as a result of the process evaluation, could be improved (e.g. coordination between teachers and educators, mentoring programme).
Furthermore, taking into account the results of the research, it would be appropriate to carry out new studies to verify that the improvements that occur in each and every one of the areas of competence of the social and emotional learning model of the students as a result of participation in the programme promote changes in their immediate environment and surroundings, generating a substantial increase in their performance and well-being, as established in the specialised literature (Oberle et al., 2016). It would also be interesting to determine the school and socio-demographic profile of the students who benefit most from the programme, as well as to confirm that the results derived from participation in the programme are maintained overtime and to establish the impact of this type of programme when it is implemented in non-formal or informal settings.
To sum up, this research, at a theoretical level, contributes to expanding the empirical evidence available on the power and validation of the causal and logical model of a programme based on the social and emotional learning model to generate improvements in the socioemotional competences of compulsory education students at risk of social exclusion, while at a practical level, it provides educational institutions with an efficient educational tool or project that can contribute to changing and improving the life trajectory of their students. The integration of this type of programme into school instruction is usually associated with an increase in the level of demand for the professionals involved, especially in terms of training and support for its implementation and evaluation, but what seems certain is that the adoption of social and emotional learning programmes brings multiple benefits to any educational institution and its community (Corcoran et al., 2018; Jagers et al., 2015; Murano et al., 2020; Sabey et al., 2017; Taylor et al., 2017; Wigelsworth et al., 2016; Yang etal., 2019).
To conclude, taking international standards of quality in educational interventions as a reference (e.g., evidence for ESSA), we can state that ltinerario+, both in terms of its methodological rigour and its results, is a clear example of an evidence-based programme (Slavin, 2017). However, considering the limited number of studies with these characteristics and methodological rigour at the national level (Fernandez et al., 2021), it is advisable to continue with the systematic evaluation of this type of intervention, mainly to accumulate more evidence and improve the impact of these programmes on the socioemotional development of compulsory education students at risk of social exclusion.
ACKNOWLEDGEMENTS
This study has been funded by Porticus and the Tornillo Foundation, through the R&D&I contract called "Evaluation of the ltinerario+ Educational Model - Phase I", established between the Office for the Transfer of Research Results (OTRI) of the University of Granada and the Tornillo Foundation (Reference: CNT4547).
How to reference this article:
Fernandez-Martin, F. D., Hinojo-Lucena, F. J., Aznar-Diaz, I., & Caceres-Reche, M. P. (2024). An intervention to improve social and emotional learning among students at risk of social exclusion. Educacion XX1, 27(1), 303-322. https:// doi.org/1o.5944/educxx1.34579
Date of received: 18/09/2022
Date of acceptance: 15/02/2023
Published online: 02/01/2024
REFERENCES
Ato, M., Lopez, J. J., & Benavente, A. (2013). Un sistema de clasificacion de los disenos de investigacion en Psicologia. Anales de Psicologia, 29(3), 1038-1059. https://doi.org/10.6018/analesps.293.178511
Choi, A., & Calero, J. (2013). Determinantes del riesgo de fracaso escolar en Espana en PISA-2009 y propuestas de reforma. Revista de Education, 362, 562-593. http://doi.org/10.4438/1988-592X-RE-2013-362-242
Coe, R. (12-14 de septiembre de 2002). It's the effect size, stupid. What effect size is and why it is important [Ponencia de Congreso]. Annual Conference of the British Educational Research Association, University of Exeter, England.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2$ ed.). Lawrence Erlbaum Associates.
Collaborative for Academic, Social, and Emotional Learning. (2021). What is the CASEL framework? Collaborative for Academic, Social, and Emotional Learning. https://casel.org/sel-framework/
Corcoran, R. P., Cheung, A. C. K., Kim, E., & Xie, C. (2018). Effective universal school-based social and emotional learning programs for improving academic achievement: a systematic review and meta-analysis of 50 years of research. Educational Research Review, 25, 56-72. https://doi.Org/10.1016/j. edurev.2017.12.001
Diekstra, R. F. W. (2008). Effectiveness of school-based social and emotional education programmes worldwide. In C. Clouder (Ed..), Social and emotional education: an international analysis (pp. 255-312). Fundacion Marcelino Botin.
Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 45, 294-309. https:// doi.org/10.1007/sl0464-010-9300-6
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: a meta-analysis of school-based universal interventions. Child Development, 82(1), 405-432. https://doi.Org/10.llll/j.1467-8624.2010.01564.x
Fernandez, F. D., Arco, J. L., & Hervas, M. (2019). La evaluacion ex-ante de un programa, proyecto o accion de aprendizaje-servicio. In M. Ruiz, & J. Garcia (Eds.), Aprendizaje-servicio. Los retos de la evaluation (pp. 57-70). Narcea.
Fernandez, M., Mena, L., & Riviere, J. (2010). Fracaso y abandono escolaren Espana. Fundacion la Caixa.
Fernandez, F. D., Moreno, A. J., Marin, J. A., & Romero, J. M. (2022). Adolescents' emotions in Spanish education: development and validation of the social and emotional learning scale. Sustainability, 14(7), Articulo 3755. https://doi. org/10.3390/sul4073755
Fernandez, F. D., Romero, J. M., Marin, J. A., & Gomez, G. (2021). Social and emotional learning in the Ibero-American context: a systematic review. Frontiers in Psychology, 12, Articulo 738501. https://doi.org/10.3389/fpsyg.2021.738501
Fundacion Tornillo. (2022). Itinerario +. Fundacion Tornillo, https://tomillo.org/ proyectos/itinerarioplus/
Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2017). La evaluation de impacto en la prdctica (2$ ed.). Banco Internacional para la Reconstruccion y el Desarrollo/Banco Mundial, https://doi.org/10.1596/978-l4648-0888-3
Gonzalez, D., Vieira, M. J., & Vidal, J. (2019). Variables que influyen en la transicion de la educacion primaria a la educacion secundaria obligatoria. Un modelo comprensivo. Bardon. Revista de pedagogia, 71(2), 85-108. https://doi. org/10.13042/Bordon.2019.68957
Hair, J., Black, W., Babin, B. J., Anderson, R., & Tatham, R. (2014). Multivariate data analysis (6g ed.). Pearson International Edition.
Hattie, J. (2009). Visible learning: A synthesis of meta-analyses in education. Routledge.
Jagers, R. J., Harris, A., & Skoog, A. (2015). A review of classroom-based SEL programs at the middle school level. In J. A. Durlak, C. E. Domotrovich, R. P. Weisberg, & T. P. Gullotta (Eds.), Handbook of social and emotional learning: research and practice (pp. 167-180). Guilford Press.
Jagers, R. J., Rivas-Drake, D., & Williams, B. (2019). Transformative Social and Emotional Learning (SEL): Toward SEL in service of educational equity and excellence. Educational Psychologist, 54(3), 162-184. https://doi.org/10.1080 /00461520.2019.1623032
Kalton, G. (2020). Introduction to survey sampling. Sage.
Kline, R. B. (2015). Principles and practice of structural equation modeling (4g ed.). Guilford Publications.
Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241-253. http://doi.org/10.3102/0013189X20912798
Ledesma, R., Macbeth, G., & Cortada, N. (2008). Tamano del efecto: revision tedrica y aplicaciones con el sistema estadistico ViSta. Revista Latinoamericana de Psicologia, 40(3), 425-439.
Mahoney, J. L., Weissberg, R. P., Greenberg, M. T., Dusenbury, L., Jagers, R. J., Niemi, K., Schlinger, M., Schlund, J., Shriver, T. P., VanAusdal, K., & Yoder, N. (2020). Systemic social and emotional learning: promoting educational success for all preschool to high school students. American Psychologist, 76(7), 1128-1142. http://doi.org/10.1037/amp0000701
Murano, D., Sawyer, J. E., & Lipnevich, A. A. (2020). A meta-analytic review of preschool social and emotional learning interventions. Review of Educational Research, 90(2), 227-263. https://doi.org/10.3102/0034654320914743
National Commission on Social, Emotional, and Academic Development. (2019). From a nation at risk to a nation at hope. The Aspen Institute.
Oberle, E., Domitrovich, C. E., Meyers, D. C., & Weissberg, R. P. (2016). Establishing systemic social and emotional learning approaches in schools: a framework for schoolwide implementation. Cambridge Journal of Education, 46, 277-297. https://doi.org/10.1080/0305764X.2015.1125450
Organisation for Economic Co-operation and Development. (2021). Beyond academic learning: First results from the survey of social and emotional skills. OECD Publishing, https://doi.org/10.1787/92all084-en
Payton, J., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Schellinger, K. B., & Pachan, M. (2008). The positive impact of social and emotional learning for kindergarten to eighth-grade students: findings from three scientific reviews. Collaborative for Academic, Social, and Emotional Learning.
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417-458. https://doi.0rg/lO.llll/j. 14680262.2005.00584.x
Sabey, C. V., Charlton, C. T, Pyle, D., Lignugaris-Kraft, B., & Ross, S. W. (2017). A review of classwide or universal social, emotional, behavioral programs for students in kindergarten. Review of Educational Research, 87(3), 512-543. https://doi.org/10.3102/0034654316689307
Sklad, M., Diekstra, R., De Ritter, M., Ben, J., & Gravesteijn, C. (2012). Effectiveness of school-based universal social, emotional, and behavioral programs: do they enhance students' development in the area of skill, behavior, and adjustment? Psychology in the Schools, 49(9), 892-909. https://doi.org/10.1002/pits.21641
Soper, D. S. (2021). A-priori sample size calculator for students t-test [Software]. https://www.danielsoper.com/statcalc/calculator.aspx?id=47
Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7- ed.). Pearson.
Taylor, R. D., Oberle, E., Durlak, J. A., & Weissberg, R. P. (2017). Promoting positive youth development through school-based social and emotional learning interventions: a meta-analysis of follow-up effects. Child Development, 88(4), 1156-1171. https://doi.org/10.llll/cdev.12864
Trujillo, J. M., Caceres, M5. P., Alonso, S., & Gomez, G. (2021). Diseno y pilotaje de la escala de aprendizaje social y emocional. In I. Aznar, C. Rodriguez, M. Ramos, & G. Gomez. Desafios de la investigation y la innovation educativa ante la sociedad inclusiva (pp. 68-77). Dykinson.
Weissberg, R. P., Durlak, J. A., Domitrovich, C. E., & Gullotta, T. P. (2015). Social and emotional learning: past, present and future. In J. A. Durlak, C. E. Domitrovich, R. P. Weissberg, & T. P. Gullotta (Eds.), Handbook of social and emotional learning: research and practice (pp. 3-19). Guilford Press.
Wigelsworth, M., Lendrum, A., Oldfield, J., Scott, A., ten Bokkel, I., Tate, K., & Emery, C. (2016). The impact of trial stage, developer involvement and international transferability on universal social and emotional learning programme outcomes: a meta-analysis. Cambridge Journal of Education, 46, 347-376. https://psycnet. apa.org/doi/10.1080/0305764X.2016.1195791
Yang, W. P., Datu, J. A. D., Lin, X. Y., Lau, M. M., & Li, H. (2019). Can early childhood curriculum enhance social-emotional competence in low-income children? A meta-analysis of the educational effects. Early Education and Development, 30, 36-59. http://doi.org/10.1080/10409289.2018.1539557
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2024. This work is published under https://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
El proposito de este trabajo de investigacion fue determinar el impacto del programa ltinerario+, una intervencion dirigida a mejorar las diferentes areas de competencia del modelo de aprendizaje social y emocional (i.e., autoconciencia, conciencia social, autocontrol, habilidades para relacionarse y toma de decisiones responsable) en alumnado de Formacion Profesional Basica procedente de contextos desfavorecidos. La muestra estuvo compuesta por 140 estudiantes de primer curso (70 grupo experimental y 70 grupo control) de cuatro titulaciones de Formacion Profesional Basica impartidas en cinco centros educativos del distrito sur de la ciudad de Madrid (Espana). El aprendizaje social y emocional se evaluo mediante la Escala de Aprendizaje Social y Emocional. Despues de asignar al alumnado a la condicion experimental o control de un diseno cuasiexperimental con grupo control no equivalente, se confirmo que ambos grupos eran equivalentes en torno a las variables control o covariantes y se implemento el programa ltinerario+. Esta intervencion se integro con la instruccion escolar, por Io que el curriculo se desarrollo en seis proyectos transversales, incluyendo otras actividades (i.e., mentoria entre alumnado, tutoria individualizada, orientacion vocacional y profesional, practicas en un entorno profesional), que fueron llevados a cabo por un equipo de profesores y educadores previamente formados. Los resultados derivados de las comparaciones entre los grupos experimental y control arrojan diferencias estadisticamente significativas a favor del grupo experimental en aprendizaje social y emocional. Estos resultados confirman la eficacia del programa para mejorar el aprendizaje social y emocional del alumnado en riesgo de exclusion social, por Io que se puede manifestar que ltinerario+ es un ejemplo de practica basada en evidencias.