Twitch as a Techno-Pedagogical Resource to Complement the Flipped Learning Methodology in a Time of Academic Uncertainty

Abstract

Flipped learning methodology has the potential to be combined with a multitude of didactic actions and techno-pedagogical resources that are adapted to the needs of the student. Access to resources and training outside the classroom can be carried out from various platforms. Following this line, Twitch is an interactive video transmission platform that is presented as an interesting digital resource to enhance the motivation of students in turning over learning moments. The objective of the study is to analyze the motivation of students in two different methodological approaches, one traditional and the other innovative through the Twitch application. A quasi-experimental pre-post design was carried out. A total of 52 Spanish students from the fourth level of secondary education participated in the research. Data collection was carried out by adapting the Motivated Strategies for Learning Questionnaire (MSLQ) instrument to the Spanish context. The results reflect higher levels of motivation in the implementation of Twitch with respect to its materialization without using said techno-pedagogical complement. This difference is especially significant in intrinsic motivation, since in the case of extrinsic motivation, the difference between both pedagogical proposals is smaller. In short, the implementation of Twitch in the flipped learning supposes an optimization of this pedagogical methodology, increasing the extrinsic and intrinsic motivation of the students and modifying the centers of interest of students from quantitative aspects to other, qualitative ones.

1. Introduction

The change generated in the educational field in recent years has had an indisputable protagonist: the inclusion of information and communication technologies (ICT) in the development of the didactic act. Society does not conceive of its existence without the use of technological devices [1]. This has led to the inclusion of these devices in teaching, with essential didactic tools for teachers and students [2,3].

The use of ICT in education has enabled current students to approach the official educational curriculum more effectively [4]. Consequently, current education is built on the effective use of technologies [5] and the daily use of digital tools in the educational context by students [6].

On the other hand, pedagogical innovation has appeared in the educational context in parallel with the inclusion of ICT in pedagogy. In order to achieve adequate and individualized teaching for the student, innovation and the good use of technology are the protagonists in today’s education [7].

The inclusion of these technologies has favored educational innovation, generating new ways of teaching [8] and new ways of learning for students [9].

Innovation linked to ICTs causes an increase in teaching–learning options since they can be used in any space and time, and its innovative nature is patent [10]. This innovation requires knowledge of these tools to use them [11]. Furthermore, this requires innovations aimed at destroying the “digital divide” [12] and ensuring that all students have access to training [13].

All innovation, together with the application of ICT, causes the emergence of emerging methodologies. They arise since most of the educational actions are not adapted to the particularities of the students [14], being far from the current interests of the students [15,16]. A series of methodologies appear that arise from the need to implement new methodologies, together with innovation and current pedagogical trends, compared to traditional systems and spaces [17,18].

One of these previously mentioned methodologies is the flipped classroom or flipped learning. With this methodology, ICTs become fundamental tools in the educational process [19], giving all the protagonism of the action to the learner [20]. Flipped learning was born as a response to the needs of students with problems of regular attendance at classes. Thanks to the work of Aaron Sams and Jonathan Bergmann, students would receive audiovisual content through telematic channels and through technological tools [21]. Since it was devised in 2012, flipped learning has evolved to become one of the emerging methodologies with the most followers in the educational world [22,23].

Flipped learning takes a much more generic perspective as a teaching and learning methodology. This methodology is specified in learning spaces, such as a physical classroom with the so-called flipped classroom approach [7]. With this approach, the formative moments that have traditionally marked the instructional processes are turned around. Thanks to this training methodology, students break the academic routine that they have followed previously [15]. This allows them to connect with other attitudinal contents such as autonomy, collaboration, teamwork, and inquiry, among other outstanding aspects, to work on the contents of each subject [19]. All this creates an enriching learning environment that encourages student motivation, as found in recent studies [14,16,24].

With flipped learning, the teacher dedicates the time of the face-to-face class to clarifying doubts and developing practical activities with the students [25]. Therefore, the typical magisterial explanations, by teachers, are transferred outside the school environment and into digital media [26]. However, investing in formative moments is not simply the creation and visualization of a video or digital content. Investing in learning moments means reformulating the traditional way of learning, combining new constructivist and active didactic techniques [27].

On the other hand, the student dedicates the extracurricular period in the study and work of conceptual and procedural contents using digital pedagogical tools [28]. Thus, the student becomes the absolute protagonist of this learning process [27,29] and can adapt the teaching rhythm and access to content independently [30,31,32].

The studies carried out and the investigations derived from the implementation of flipped learning in the classrooms have shown a very significant improvement, which translates into high degrees of motivation and participation on the part of the students [33,34,35]. In addition, a high climate of cooperation has also been noted among students who participate, now in a leading way, in the educational act [36]. This undoubtedly causes an increase in the levels of socialization among students [37] and a broad capacity to solve problems in the classroom, both individually and collectively [38]. For this reason, flipped learning gives possibilities of a better acquisition of contents [39], reaching objectives with greater ease [40], which leads to better qualifications of the evaluation processes [41,42].

Despite all the benefits that research attributes to flipped learning, this emerging methodology has some limitations. This can be problematic in students with low abstraction capacity [43] or poor problem solving [38,44]. Furthermore, flipped learning can generate problems with students who suffer from the digital divide and therefore have difficulties in accessing technologies [45,46].

The methodological innovation of flipped learning is complemented by other multimedia and audiovisual innovations, based on the video format [47,48]. Linked to these types of tools, the Amazon-owned Twitch platform was launched in 2014, consolidating itself as one of the main binders of multimedia content in the world [49]. It is a prepaid tool established as a new way of watching television and multimedia content [50]. Furthermore, this platform is used by users to express themselves, socialize, and even as a means for guided learning [51]. Twitch has great potential and offers many possibilities for the development of flipped learning methodology [52].

This digital resource also makes it possible to take advantage of new forms of streaming education to promote teamwork in classrooms and outside of them. Twitch is one of the preferred media for young people for their socialization, entertainment, and leisure [53]. Consequently, this digital platform can become a fundamental element to improve the performance of students in various areas of their training [54,55].

In other matters, the use of Twitch as a techno-pedagogical resource has not been sufficiently disseminated in the educational field. The incorporation of new digital resources to the educational field requires a minimum time for their dissemination. Twitch first appeared online in 2014, but its widespread use globally occurred three to four years ago [52]. Some exploratory research has been able to verify the optimal results in the motivation of the students and in the follow-up of the teaching sessions during the period of confinement of COVID-19 [56]. However, these studies are brief approaches to a field of research that requires greater depth in the educational field. Consequently, there is currently little research looking at the effects of Twitch use in education in general, and no research addresses its combination with flipped learning. Some recent studies have investigated the use of Twitch in other fields, such as medical education for surgical learning [57], learning for the computer programmer community [58], and the effects of live streaming video for video game programming [54]. In general, the available scientific literature reflects that Twitch is not used for educational purposes.

Justification and Research Objective

As has been possible to verify after the state of the matter presented, current teaching is conditioned by the incidence of technology [59] that has reported a set of potentialities, from which the teaching and learning processes have benefited [60]. Until relatively recently, teachers did not fully support the use of ICT in instructional actions, since to some extent, the level of digital competence of teachers determines how they will be able [61] to apply with ease and confidence a training process mediated by educational technology [62] and that allows the implementation of active and emerging methodologies [63]. This is with the purpose of fostering the motivation of students in their daily work as an active agent of the training process [64], a motivation that is currently conditioned by the innovative actions developed in the learning spaces [65]. Along these lines, the pandemic caused by COVID-19 has caused a great transformation in the field of education [66]. This has led to the forced implementation of virtual learning environments [67,68] in which teachers have had to adapt their teachings to meet the training needs of students [69].

The pandemic that the world is currently suffering from has marked the educational landscape. Classrooms have been modified and the number and placement of students have been restructured. Carrying out this work is based on providing the scientific community with the benefits of new digital environments that have been enhanced as a result of COVID-19. In this case, Twitch is used, a social platform initially conceived as a playful, interactive, and entertainment digital space. Given the uncertain panorama of current education, the reach of Twitch in the adolescent population has been used to carry out an educational plan that allows revealing significant and interesting findings for science. As a result, this will allow for greater sustainability in the educational system, in a time of constant renewal and adaptation to new regulations and demands set by the evolution of the pandemic.

The objective of this research is to analyze the motivation of students after carrying out a training contrast from two learning experiences after returning to the classrooms after a period of home confinement. The first training experience was of a traditional nature, without the use of innovative means or techniques. The second experience consisted of carrying out a training program through the social network Twitch. The following hypotheses arise from this general objective:

Hypothesis 1 (H1).

The use of training methodologies that integrate digital resources such as Twitch does not promote increased motivation in students.

Hypothesis 2 (H2).

The use of training methodologies that integrate digital resources such as Twitch encourages increased motivation in students.

2. Materials and Methods

2.1. Research Design and Data Analysis

A quantitative methodology has supported the research design developed. This is based on a case study using a quasi-experimental design of a pre–post nature in a single group. For a correct development, the considerations of experts were followed [70]. In addition, the analytical model was taken from previous studies, in order to use an investigative process validated by the scientific community [71,72].

In the study, the type of instructional action (traditional and innovative) carried out for the development of the didactic contents is established as an independent variable. As a dependent variable, the level of motivation achieved after completing the training processes is postulated. In total, two measurements were made corresponding to the completion of each instructional action.

The data analysis process was carried out with the Statistical Package for the Social Sciences (SPSS) program, version 25. For descriptive analytics, statistics such as the mean (M) and standard deviation (SD) were established. For the comparison of means between pretest and post-test, the Student’s t-test was used, taking the independence value according to the significance level of the p-value. Cohen’s d tests and biserial correlation (r) were used to determine the size of the effect caused. All statistical analysis was based on a p < 0.05 as a statistically significant difference.

2.2. Participants

A total of 52 students from the fourth level of secondary education of the Spanish educational system participated in the study. Of the total students, 42.3% were men and the rest women. The participants were a mean age of 16 years (SD = 1.13). The literature reflects that the sample volume does not condition the results in this type of research design [73,74]. Therefore, the number of subjects is adequate to carry out this research. These students were chosen using a convenience sampling technique, since the educational center in question is immersed in an innovation project that aims to integrate technology and digital resources in the different learning spaces.

2.3. Instrument

For data collection, the adaptation to the Spanish context of the instrument Motivated Strategies for Learning Questionnaire (MSLQ) [75] was used. This tool is composed of two dimensions (1-Intrinsic Motivation (MotInt) and 2-Extrinsic Motivation (MotExt)) that articulate a total of eight items (Appendix A). The questionnaire is configured with a five-point Likert-type response format, on an increasing scale, from lowest to highest degree of agreement. The instrument has adequate psychometric properties. The validation process began with an exploratory factor analysis with varimax rotation. The result obtained in the Kaiser–Meyer–Olkin test was relevant (KMO = (28) = 0.73; p = 0.000). Subsequently, a confirmatory factor analysis was performed. The goodness of fit indices used yielded adequate evaluations (NFI = 0.926; NNFI/TLI = 0.946; CFI = 0.663; IFI = 0.964; RMSEA = 0.053). Finally, the reliability of the questionnaire was calculated using various statistics that revealed robust internal consistency (Cronbach’s alpha = 0.701; CRMotInt = 0.718; CRMotExt = 0.700; MVEMotInt = 0.111; MVEMotExt = 0.111). The application of this instrument was accompanied by several socio-demographic items, referring to sex and age.

2.4. Procedure

The research was carried out after the reopening of educational centers in Spain. Specifically, the study began in November 2020. Access to the sample of participants did not pose any problem. Those responsible for the educational center in question were aware of the purpose of the study. Likewise, informed consent was obtained from the participants. At the didactic level, the experimentation was carried out in the subject of Language and Literature. Contents related to the syntactic analysis and morphological analysis of compound sentences were covered: coordinated sentences, substantive subordinate sentences, and juxtaposed sentences.

The field work was structured at different times (Figure 1). First, the training process was carried out using a traditional methodology (first intervention). The teacher became the only source of knowledge transmission. The students played a passive role. They only limited themselves to receiving explanations from the teacher and carrying out the proposed activities. Next, the first measurement (pretest) took place, in order to collect information on this first training action. Next, the teacher applied a new methodology. This time, it was based on an innovative approach through the Twitch app (second intervention). With this tool, the teacher developed the contents in a virtual environment that allowed interaction with the students. In this case, the students assumed a greater role, being able to interact both with each other and with the teacher. In addition, another potentiality was that the sessions were registered on the platform for later consultation by the students. At the end of this novel approach, the second measurement (post-test) took place.

Once the teaching of the contents was finished through this methodological contrast and all the information was collected, the data were prepared and entered into the statistical program to analyze the data in depth and establish conclusions that would allow reaching the formulated objective and demonstrating the defined hypotheses.

In order to carry out the implementation of the methodology with Twitch (Figure 2) and the use of this digital platform by the students, several legal aspects were taken into account. In the first place, the approval of the management team of the educational center was requested, and the necessary documentation was delivered to duly inform them of the procedure of the investigation methodology. Subsequently, legal authorization was requested from the parents of the students for the use of Twitch, and an attached report was sent that explained the investigation procedure. Twitch declares in its “Terms of Service” and “Use of Twitch by minors and blocked persons” website that this digital platform cannot be used by persons under 13 years of age. All students who participated in this research are over that age and have received authorization from their parents or guardians. The reception of the research by the teaching staff of the center, the students, and the parents and tutors were in general very satisfactory.

3. Results

The results reported in the parametric study of the results generated during the application of the pretest and post-test are analyzed in detail. In this way, an evaluation of the values obtained in the mean in each of the variables of the questionnaire and of the totalized values for intrinsic and extrinsic motivation is carried out, finally establishing an analytical approach to the value of independence between the pretest and the post-test.

Starting with the parametric analysis of the results reported in the pretest and post-test (Table 1), the results reflect variable scores in both cases. Regarding intrinsic motivation, the variable related to student satisfaction in understanding the content effectively represents the highest average in the pretest. However, in the post-test, the center of interest seems to shift toward a preference for material that poses a challenge for learning new content and preference for materials that generate curiosity, regardless of the degree of difficulty. This confirms that in the pre-test, students were more emotional about the compression of the contents than they were regarding the use of new and challenging materials in the case of the post-test. Regarding extrinsic motivation, in the case of the pretest, the average score relative to obtaining a good grade in class by the student stands out. On the other hand, in the case of the post-test, the average score stands out, alluding to the students’ preference for participating adequately in the classroom to show their ability in their environment (family, friends, and teachers).

At the opposite extreme, the lowest average score in the intrinsic motivation of the pretest group was consigned in the variable related to the preference by the students toward the material that represents a challenge for learning new content. In the case of the post-test group, the variable with the lowest average score recorded is that related to student satisfaction after understanding the contents in an optimal way. Regarding extrinsic motivation, in the case of the pretest, the variable of the students with the lowest score is that of showing skills in the immediate environment, that is, among their family, friends, and teachers. On the other hand, in the post-test, it was the variable related to the students’ preference in obtaining a good class grade that obtained a lower score.

Continuing the comparative analysis, Figure 3 shows the average scores obtained by the intrinsic motivation and extrinsic motivation dimension during the application of both tests (pretest–post-test). Except for the intrinsic motivation variable related to the students’ satisfaction with understanding the contents in the best possible way, the rest of the variables show great differences between both tests. If we look at the value of the totalized mean, we observe a clear advantage of intrinsic motivation in flipped learning complemented with Twitch (M = 3.5) compared to the methodological implementation of flipped learning without the techno-pedagogical presence of this transmission platform (M = 2.84). In the case of extrinsic motivation, the totalized mean obtained with the complementation of Twitch within the flipped learning methodology is only slightly higher (M = 3.22) than the implementation of this methodology without Twitch as a techno-pedagogical complement (M = 3.02). This confirms much greater statistical differences in the variables analyzed in extrinsic motivation than in intrinsic motivation.

Finally, Table 2 reflects the results of the Student’s t-test for the analysis of the independence value between the pretest and the post-test, in order to determine the effective incidence of Twitch as a techno-pedagogical complement in the flipped learning methodology. Along these lines, this study has determined the standardized value of p < 0.05 as a statistically significant difference and three levels of correlation strength (d) and corrective value (r): high (d > −0.8; r > −0.5), medium (d = [−0.3, −0.8], r = [−0.2, −0.5]), and low (d < −0.3; r < −0.2).

In the variable related to obtaining a positive score as the main student satisfaction, the implementation of flipped learning without the use of Twitch as a techno-pedagogical complement generates a significantly higher level of extrinsic motivation (d = −0.79; r = 0.371). In the rest of the statistically significant variables, the implementation of flipped learning complemented with Twitch generates higher levels of intrinsic motivation in the variables related to the preference of material that poses a challenge for learning (d = −0.76; r = 0.353) and the preference for material that generates curiosity despite being difficult to understand (d = −0.73; r = 0.344), as well as in extrinsic motivation in the variables related to the preference for obtaining better grades than the rest of the students (d = −0.72; r = 0.337) and the preference for showing their skills in the close environment (d = −1.14; r = 0.494). In all the cases reported as statistically significant, the resulting correlation strength is medium, especially highlighting the values reached by the incidence of Twitch in the students’ preference for showing their skills in the family, teaching, and social environment. This confirms that the use of Twitch generates greater benefits in learning perceived by the students, except in the students’ motivation for obtaining better grades. We observe that the use of dynamic pedagogical approaches can generate a greater role in the learning process itself than in the final results or grades.

4. Discussion and Conclusions

The changes that technological development has brought about in many facets of society have also had a direct impact on the educational field. For this reason, the implementation of emerging, active methodologies based on new technologies should be considered something fundamental for the development of quality teaching [2], always with the goal of achieving a better adaptation of teaching to students [6,7].

The application of new methods, coinciding with this study, improves the quality of student learning [8], as well as producing an improvement in teaching production and in the development of the didactic act [15,16]. Furthermore, the results of our study reveal an improvement with the application of these emerging and innovative methodologies, in coincidence with other studies of a similar nature [17,18].

In the application of flipped learning, and in line with the results of other research [22,23], its application has a positive effect on the development of students by improving important aspects such as motivation [33,34,35], focusing interest on materials and on the socialization of the individual over other educational aspects [36].

In short, the implementation of Twitch in the flipped learning methodology supposes an optimization of this pedagogical methodology, increasing the extrinsic and intrinsic motivation of the students and modifying the centers of interest of the students from quantitative aspects to other qualitative ones. Regarding intrinsic motivation, students in the flipped methodology prioritize their satisfaction for understanding the content effectively, while by complementing learning with Twitch, their interests are focused on materials that pose a challenge for learning new content and content that generates curiosity, regardless of the degree of difficulty.

Regarding extrinsic motivation, students in the flipped learning methodology prioritize obtaining a good grade in class, while by complementing learning with Twitch, their interests turn toward appropriate participation in the classroom to show their ability in their social environment. The totalized results reflect the higher levels of motivation in the implementation of Twitch with respect to its materialization without using said techno-pedagogical complement. This difference is especially significant in intrinsic motivation, since in the case of extrinsic motivation, the difference between both pedagogical proposals is smaller. Only in the field of intrinsic motivation related to student satisfaction in understanding the content in the best possible way have lower results been found in the inverted learning binomial supplemented with Twitch.

One of the main limitations of this study is the type of sampling selected, not probabilistic. The selection of the center and the students’ course was intentional, which is justified by the researchers’ knowledge of the innovative programs that the educational center in question executes. This selection facilitated the study and collaboration with the management team of the institution. In relation to the sample used, the results obtained are not generalizable to the general population, but they initiate research in an area necessary for educational innovation. Another limitation of the research is related to the connectivity problems of some students, as well as the availability of electronic resources to access the social network Twitch. For this, the collaboration of the educational center was requested to provide mobile devices with an Internet connection to develop the experimentation. Regarding the discussion of the study, Twitch is a recently created digital platform, and its implementation in instructional contexts has not been addressed, except for some exploratory studies in other areas of knowledge (medicine, programming, computing). These investigations are exploratory studies that do not analyze the incident factors in the learning process. Additionally, there is no research that analyzes the effects of using Twitch and flipped learning as a pedagogical methodology. Therefore, it is not possible to discuss our results with other investigations. Our research is an exploratory study that begins the path to further research that substantiates the importance of educational innovation.

As a future line of study, we aim to cover a greater number of educational centers in which to replicate the present work. Thus, we aim to bring together centers from different regions of the Spanish state with the purpose of obtaining a representative sample and, in such a way, generalizing the findings that are obtained to the whole population. For this, a first approximation of institutions will be carried out in which innovative plans are carried out and that have the necessary equipment to be able to deploy an investigation of such characteristics.

The prospective of this study focuses on the integration of new platforms of a social nature to the field of education. With the completion of this research, we can see how the use of Twitch has been relevant in increasing the motivation of students. For this reason, as a theoretical implication of this work, the appearance of new resources and innovative proposals such as Twitch complements the wide repertoire currently present in the literature on the topic in question. In addition, this study can serve as a knowledge base for future work that investigates or deepens the use of social networks at an educational level. In addition, this will allow discussing the different findings reported at different times and contexts. On a practical level, this research contributes to the promotion of innovative instructive actions in learning spaces generated through technology. In this sense, the findings presented here can support the development of teacher training plans for the implementation of new teaching and learning approaches to train the teaching community in the use of new training tools in a time conditioned both by the proliferation of mobile applications and digital resources and by the limitations that COVID-19 has placed on the field of education.

Author Contributions

Conceptualization, A.F.-C. and J.-A.L.-N.; methodology, J.L.-B. and S.P.-S.; software, S.P.-S.; formal analysis, S.P.-S.; investigation, S.P.-S., J.L.-B., A.F.-C., and J.-A.L.-N.; data curation, S.P.-S.; writing—Original draft preparation, S.P.-S., J.L.-B., A.F.-C., and J.-A.L.-N.; writing—Review and editing, S.P.-S., J.L.-B., A.F.-C., and J.-A.L.-N.; visualization, S.P.-S.; supervision, J.L.-B.; project administration, A.F.-C. and J.-A.L.-N.; funding acquisition, J.-A.L.-N. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by I+D+i OTRI-Universidad de Granada CNT-4315. Metodologías activas para el aprendizaje mediante recursos tecnológicos para el desarrollo de la sociedad.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of CBDS (protocol code 1174 and 26 October 2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data are contained within the article.

Acknowledgments

We appreciate the support provided by the AREA Research Group (HUM-672).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Motivación Intrínseca

• Prefiero material de clase que realmente sea un reto para poder aprender cosas nuevas (MotInt1).

• Prefiero material de clase que despierte mi curiosidad, aunque sea difícil de aprender (MotInt2).

• Lo más satisfactorio para mí es entender los contenidos de la mejor forma posible (MotInt3).

• Cuando tengo oportunidad de elegir, elijo hacer tareas en las que puedo aprender aunque no garanticen una buena nota (MotInt4).

Motivación Extrínseca

• Obtener una buena nota en clase es lo más satisfactorio para mí en este momento (MotExt1).

• Lo más importante para mí en este momento es mejorar la media de mis notas, así que mi principal preocupación es conseguir una buena nota (MotExt2).

• Si puedo, quiero obtener mejores notas que la mayoría de los otros estudiantes (MotExt3).

• Quiero hacerlo bien en clase porque es importante mostrar mi habilidad a mi entorno (familia, amigos, profesores u otras personas) (MotExt4).

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. Didactic process of experimentation.

Figure 2. Innovative didactic intervention through Twitch.

Figure 3. Comparative analysis between the means obtained in both tests.

Table 1

Parametric analysis of the results reported (pretest–post-test).

		Likert Scale n (%)					Parameters
	Variable	1	2	3	4	5	M	SD	Skew	Kurt
Pretest	MotInt1	13 (25)	14 (26.9)	10 (19.2)	9 (17.3)	6 (11.5)	2.63	1.33	1.23	−1.07
	MotInt2	9 (17.3)	15 (28.8)	15 (28.8)	7 (13.5)	6 (11.5)	2.73	1.23	1.41	−0.77
	MotInt3	6 (11.5)	7 (13.5)	15 (28.8)	16 (30.8)	8 (15.4)	3.25	1.21	1.86	−0.73
	MotInt4	12 (23.1)	13 (25)	10 (19.2)	10 (19.2)	7 (13.5)	2.75	1.36	1.29	−1.18
	Total-MI	40 (19.2)	49 (23.6)	50 (24)	42 (20.2)	27 (13)	2.84	1.3	1.41	−1.09
	MotExt1	4 (7.8)	6 (11.8)	12 (23.5)	17 (32.7)	13 (25.5)	3.56	1.2	2.13	−0.54
	MotExt2	7 (13.5)	7 (13.5)	16 (30.8)	15 (28.8)	7 (13.5)	3.15	1.22	1.77	−0.78
	MotExt3	11 (21.2)	14 (26.9)	10 (19.2)	12 (23.1)	5 (9.6)	2.73	1.29	1.34	−1.14
	MotExt4	8 (15.4)	17 (32.7)	16 (30.8)	8 (15.4)	3 (5.8)	2.63	1.09	1.5	−0.54
	Total-ME	30 (14.4)	44 (21.2)	54 (26)	52 (25)	28 (13.5)	3.02	1.26	1.61	−1.02
Post-test	MotInt1	4 (7.7)	7 (13.5)	12 (23.1)	11 (21.2)	18 (34.6)	3.62	1.29	2.03	−0.89
	MotInt2	4 (7.7)	6 (11.5)	10 (19.2)	17 (32.7)	15 (28.8)	3.63	1.23	2.15	−0.53
	MotInt3	2 (3.8)	10 (19.2)	18 (34.6)	16 (30.8)	6 (11.5)	3.27	1.02	2.22	−0.57
	MotInt4	3 (5.8)	8 (15.4)	16 (30.8)	12 (23.1)	13 (25)	3.46	1.18	2.08	−0.84
	Total-MI	13 (6.3)	31 (14.9)	56 (26.9)	56 (26.9)	52 (25)	3.5	1.19	2.09	−0.79
	MotExt1	8 (15.4)	18 (34.6)	15 (28.8)	7 (13.5)	4 (7.7)	2.63	1.13	1.45	−0.5
	MotExt2	6 (11.5)	17 (32.7)	16 (30.8)	11 (21.2)	2 (3.8)	2.73	1.04	1.66	−0.69
	MotExt3	5 (9.6)	6 (11.5)	7 (13.5)	18 (34.6)	16 (30.8)	3.65	1.28	2.07	−0.55
	MotExt4	0 (0)	9 (17.3)	8 (15.4)	16 (30.8)	19 (36.5)	3.87	1.09	2.62	−1.04
	Total-ME	19 (9.1)	50 (24)	46 (22.1)	52 (25)	41 (19.7)	3.22	1.26	1.76	−1.1

Table 2

Intertest independence value analysis (pretest–post-test).

Variable	Test, M (SD)		M2−M1	Student’s t		d	r
	Pretest	Post-Test		t (df)	p-Value
MotInt1	2.63 (1.33)	3.62 (1.29)	0.99	3.78 (102)	0.000 *	−0.76	−0.353
MotInt2	2.73 (1.23)	3.63 (1.23)	0.9	3.72 (102)	0.000 *	–0.73	–0.344
MotInt3	3.25 (1.21)	3.27 (1.02)	0.02	0.09 (102)	0.931	–	–
MotInt4	2.75 (1.36)	3.46 (1.18)	0.71	2.82 (102)	0.006	–	–
MotExt1	3.56 (1.2)	2.63 (1.13)	−0.93	4.01 (102)	0.000 *	−0.79	−0.371
MotExt2	3.15 (1.22)	2.73 (1.04)	−0.42	1.89 (102)	0.062	−	−
MotExt3	2.73 (1.29)	3.65 (1.28)	0.92	2.92 (102)	0.004	–0.72	–0.337
MotExt4	2.63 (1.09)	3.87 (1.09)	1.24	5.69 (102)	0.000 *	–1.14	–0.494

Note: * value less than one thousandth.