RESUMEN
La adquisición de conceptos es un aspecto fundamental en la formación del profesorado, pero sigue siendo un reto, especialmente en contextos grupales en los que las estrategias de enseñanza tradicionales a menudo no logran transmitir las nociones complejas de forma eficaz. En este estudio se examina el potencial de la analítica del aprendizaje basada en minería de textos (MT) como herramienta didáctica para mejorar el aprendizaje conceptual del profesorado en formación. El objetivo fue analizar el efecto de la analítica del aprendizaje basada en MT en la adquisición de conceptos educativos complejos y abstractos, en comparación con otras estrategias de enseñanza tradicionales, como la elaboración de proyectos individuales o asistir a clases magistrales. Se llevó a cabo un estudio cuasiexperimental pre y postest con 81 estudiantes de máster de un programa de formación a distancia en una universidad española. El estudio se centró en analizar los corpus textuales relacionados con la definición de conceptos educativos no tangibles de tres grupos no equivalentes (Grupos A, B y C, respectivamente). Mediante técnicas de MT, se analizaron 1017 tokens pretest y 1133 postest del Grupo A, 1127 tokens pretest y 1111 postest del Grupo B, y 1101 tokens pretest y 1173 postest del Grupo C. Los resultados revelaron que la analítica de aprendizaje basada en MT mejoró significativamente la adquisición de conceptos de los estudiantes en cuanto a la selección de palabras clave (tYuen = -6.37, p < .001, ?R AKP = -1.03, IC95% = -2.10, -.74) y la asociación de términos relevantes (valores de Jaccard postest de .217 a .917) en sus definiciones, comparado con otros enfoques de enseñanza. Este estudio ofrece pruebas empíricas de que la analítica del aprendizaje basada en MT es una herramienta pedagógica eficaz, que contribuye a mejorar el aprendizaje de conceptos abstractos en la formación del profesorado. Los resultados subrayan el valor de la tecnología educativa basada en MT para optimizar el Educación XX1, 28 (2), 17-43 19 aprendizaje conceptual y la eficiencia de los recursos en entornos grupales de educación superior
Palabras clave: textual analysis, content analysis, concept formaton, social learning, visual learning, educatonal technology, higher educaton, teacher educaton
ABSTRACT
La adquisición de conceptos es un aspecto fundamental en la formación del profesorado, pero sigue siendo un reto, especialmente en contextos grupales en los que las estrategias de enseñanza tradicionales a menudo no logran transmitr las nociones complejas de forma efcaz. En este estudio se examina el potencial de la analítca del aprendizaje basada en minería de textos (MT) como herramienta didáctca para mejorar el aprendizaje conceptual del profesorado en formación. El objetvo fue analizar el efecto de la analítca del aprendizaje basada en MT en la adquisición de conceptos educatvos complejos y abstractos, en comparación con otras estrategias de enseñanza tradicionales, como la elaboración de proyectos individuales o asistr a clases magistrales. Se llevó a cabo un estudio cuasiexperimental pre y postest con 81 estudiantes de máster de un programa de formación a distancia en una universidad española. El estudio se centró en analizar los corpus textuales relacionados con la defnición de conceptos educatvos no tangibles de tres grupos no equivalentes (Grupos A, B y C, respectvamente). Mediante técnicas de MT, se analizaron 1017 tokens pretest y 1133 postest del Grupo A, 1127 tokens pretest y 1111 postest del Grupo B, y 1101 tokens pretest y 1173 postest del Grupo C. Los resultados revelaron que la analítca de aprendizaje basada en MT mejoró signifcatvamente la adquisición de conceptos de los estudiantes en cuanto a la selección de palabras clave (tYuen = -6.37, p < .001, ?RAKP = -1.03, IC95% = -2.10, -.74) y la asociación de términos relevantes (valores de Jaccard postest de .217 a .917) en sus defniciones, comparado con otros enfoques de enseñanza. Este estudio ofrece pruebas empíricas de que la analítca del aprendizaje basada en MT es una herramienta pedagógica efcaz, que contribuye a mejorar el aprendizaje de conceptos abstractos en la formación del profesorado. Los resultados subrayan el valor de la tecnología educatva basada en MT para optmizar el aprendizaje conceptual y la eficiencia de los recursos en entornos grupales de educacion superior.
Keywords: analisis de texto, analisis de contenido, formacion del concepto, aprendizaje social, aprendizaje visual, tecnologia de la educacion, ensefianza superior, formacion de profesores
INTRODUCTION
Concept acquisition occurs as learners actively categorize and label information, connecting keywords and related ideas in cohesive mental models. Conceptualization exercises, such as sorting tasks and identifying attributes, can reinforce this process by encouraging learners to define and refine their understanding (Bruner et al. 1956) as the basis for developing further knowledge and skills and achieving meaningful learning. Following a long tradition of research on this subject, and particularly in the wake of the COVID-19 pandemic (Gaglo et al., 2022), an increasing number of studies have investigated how applications of educational technology can have a role in enhancing concept acquisition. Indeed, data analytics and artificial intelligence resources such as virtual agents, natural language processing, pattern recognition, data mining and data visualization tools have provided new means and opportunities for technology-enhanced learning in higher education. In particular, text mining (TM) is a data mining technique that leverages quantitative content analysis to help visualize concepts and better understand them (Inada, 2018), and has opened the door to a new research line.
This study evaluates the potential of TM-based learning analytics as a teaching tool to enhance the acquisition of abstract educational concepts, by comparing results of its application with those of other teaching strategies, precisely the elaboration of an individual project, and the attendance of a conventional master class, in three non-equivalents groups of pre-service student teachers. Following a literature review, in order to test the hypothesis that TM-based learning analytics indeed had the potential to be an effective tool, two specific research questions were formulated before applying the strategies and collecting the data. Implications for educational practice were then extrapolated from the findings, and considered in the context of the limitations of the study and emerging research.
Literature review
Since the last century, the contributions of Bruner et al. (1956) have led to an expansion in the scope of studies of concept acquisition and improvements to their accuracy. Teacher educators have taken interest in this field, with a view to supporting learning processes, enhancing transfer value, and helping pre-service teachers learn about the consistency of educational materials and procedures and how to respond adaptively to different educational scenarios, considering that, without adequate conceptual knowledge, teachers may lack guidance and a clear awareness of what teaching means and involves. Specialized journals have continued to publish studies on the subject (Azadi et al., 2018; Freeman, 2018; Turner, 1975), with a particular focus on abstract educational concepts in teacher education, such as critical thinking, school culture, and curriculum design. Such fundamental yet abstract concepts can be detached from physical reality and are qualitatively different from concrete concepts that are more connected to perceptual and motor experience (Borghi et al., 2019; Gagne, 1985). This means that abstract concepts are particularly difficult to acquire and to relate to practical applications, and their teaching remains a demanding challenge. Although the current study focuses on student teachers in particular, the existing body of knowledge on how concepts are learned in higher education in general is especially relevant, and implications may well extend beyond the immediate context of teacher education.
Concept acquisition in higher education
Engaging students as active participants in their learning process has been shown to stimulate cognitive aspects such as attention, memory, and comprehension (Hernandez-de-Menendez et al., 2019; Nguyen et al., 2021). Furthermore, recent studies have evidenced improvements in the concept acquisition on the integration of active techniques into the learning process, such as flipped classes (Atkinson et al., 2020), asynchronous online discussions (Breivik, 2020), the gamification of lessons (Kortemeyer et al., 2019), and virtual reality simulations (Liao, 2022).
Active learning has been shown to promote concept acquisition if applied as an integral part of the overall strategic planning of teaching activities, and game-based learning has been observed to yield good results (Casanoves et al., 2022), while pre-training videos and real-time cues have, on the other hand, have been shown to have limited effects on conceptual learning (Tsai et al., 2022). Moreover, timed reading, video enactments and writing assignments have been seen to be helpful in certain studies (Guerrettaz et al., 2020; Reynolds et al., 2020), and making students explore, contrast and compare different meanings has appeared to particularly useful in enhancing the potential of writing activities (Wittek, 2018).
Active learning seems to be more effective the more it engages students and is adapted to their specific learning needs, for example, by setting learning goals according to appropriate difficulty levels. Furthermore, the agile creation of reading-only materials such as digital fanzines (Redondo Lopez, 2021) and the provision of specific feedback from lecturers (Gao & Lloyd, 2020) have been observed to contribute to concept acquisition in university courses, especially when tactile and kinesthetics inputs complement visual information in virtual learning environments (Magana et al., 2019).
However, beyond the design of tasks and materials, some research appears to not fully incorporate the theories of comprehensive learning in the vein of Ausubel et al. (1968) and Novak (1977). For instance, student participation in creative exercises and the use of concept maps has been shown to not have a significant impact on students' abilities in explaining concepts (Ye et al., 2020), likely due to a lack of in-depth contextualization around the subject matter, which has been shown to improve the effectiveness of interventions in other studies (Cortes et al., 2019). Contextualization refers to the conditions in which a concept makes sense rather than merely considering its practical function and is particularly relevant for acquiring abstract concepts that entail no physical functionality.
Accordingly, based on previous evidence, concept acquisition seems to be modulated not only by active or passive learning, but also by the arrangement of materials in task designs, the monitoring of student progress, and the contextualization of concepts. In this general context, TM-based learning analytics may have the potential to be a worthy ally in enhancing concept acquisition.
TM-based learning analytics for concept acquisition
Exchanges between peers have been observed on several occasions to facilitate concept acquisition, mostly when students are organized into small groups (Atkinson et al., 2020; Rodriguez & Potvin, 2021). Interactive systems and peer teaching and assessment have also been widely identified as helpful for concept acquisition (Babaahmadi et al., 2021; Koong et al., 2021), particularly in peer-reviewing of writing assignments focused on developing content knowledge (Finkenstaedt-Quinn et al., 2021). This suggests that group discussions can complement individual analytical descriptions of a concept and both can contribute to positive results (Reyes-Santias et al., 2021; Volkwyn et al., 2020).
Thus far, we have considered that elements such as writing and discussion tasks, lecturers' feedback and visual information supplements can all help concept acquisition. These are, in fact, elements that TM-based learning analytics can offer, although evidence of the effectiveness of its application is currently scarce. Papers published on the applications of TM have concerned, for example, the assessment of learning outcomes after a certain educational intervention (Kong et al., 2021), or the automation of the annotation and categorization of exam queries according to concepts to be assessed (Begusic et al., 2018). Furthermore, studies that have directly addressed conceptual learning with the application of TM have looked at concept acquisition in relation to exam repositories (Pintar et al., 2018), semantic relationships between concepts (Shwartz, 2021) and the identification of students' conceptions and misconceptions (De Lin et al., 2021; Taga et al., 2018) by asking students for written definitions and then applying TM to those definitions.
Surprisingly, although educationalists have applied TM-based learning analytics to understand student concept acquisition, identify learning styles (Aguilar et al., 2022) and analyse outcomes of discussion threads (Hernandez-Lara et al., 2021; Pillutla et al., 2020), it has rarely been evaluated as a didactic tool relevant to the design of teaching tasks regarding concept acquisition. Until now, research has mainly been focused on know-how regarding TM as a university teaching evaluation tool. Therefore, in this paper, we aim to present one of the first empirical studies to give evidence on the use of TM-based learning analytics as a teaching tool for promoting the acquisition of abstract educational concepts by student teachers, and applying some of the above discussed learning aspects, including the joint analysis of definitions by peers, and the visual representation of results. The motivation for the research includes a consideration that TM-based learning analytics may potentially offer new mechanisms for providing targeted instruction and data-driven feedback, allowing teachers to more effectively address misconceptions and gaps in concept acquisition. The current study therefore contributes to efforts to optimize classroom management and the use of resources in fostering concept acquisition, through the application of TM-based learning analytics. Overall, this study paves the way for innovative pedagogical strategies that leverage technology to improve teaching and learning outcomes, particularly in teacher education.
Research questions derived from the literature review
Our literature review, as detailed above, led to the hypothesis that TM-based learning analytics can be an effective teaching tool that facilitates the acquisition of abstract concepts. To test this hypothesis in relation to student teachers, we decided to compare TM-based learning analytics to two other teaching strategies, namely asking students to complete individual project work, and to attend a conventional master class. Furthermore, to guide the evaluation of TM-based learning analytics as a teaching tool, we decided to answer two specific research questions (RQs), aimed at collecting evidence on the students' selection of keywords and on associations made between relevant terms in the definition of a concept.
o RQ1: How does TM-based learning analytics promote keyword selection in student teachers compared to other teaching strategies?
22
Education XXl, 28(2), 17-43
o RQ2: How does TM-based learning analytics promote associations between keywords compared to other teaching strategies?
METHODS
This study compares effects on the acquisition of abstract educational concepts relating to the application of three teaching strategies: TM-based learning analytics; the carrying out of individual project work; and the attendance of a conventional master class. The construct of concept acquisition was operationalized using two key indicators relating to the definition of concepts: keyword selection, and the association of relevant terms.
Study design and participants
Quasi-experimental pre- and post-tests were administered to three non-equivalent groups (A, B, and C), composed of a total of 81 students (62.96% female) following a teacher training master's programme at a Spanish university. Group A consisted of 26 students (65.38% female) with a mean age of 32.65 years (standard deviation or SD=5.78), Group B consisted of 28 students (64.29% female) with a mean age of 32.50 years (SD=6.43), and Group C consisted of 27 students (59.26% female) with a mean age of 32.19 years (SD=6.43). All were following the programme through distance education and used the Blackboard platform for online learning.
Learning environment
As a theoretical frameworkforthe online learning interventions, the Data-Driven Decision-Making model (Khong et al., 2023) was selected. This model outlines a cyclical process in which data are systematically collected, analysed, interpreted and thus transformed into information and, ultimately, into actionable knowledge that can be used to guide and inform educational practice and strategy. It can therefore provide valuable insights into the learning strengths and weaknesses of students and offer guidance for structuring more effective teaching strategies.
In the current study, the teaching strategies were administered by the researchers acting as instructors, and were carried out simultaneously for the three separate groups of students. The researchers therefore set up an environment on the Blackboard platform so that students could define concepts with a 280-character limit, considering that the optimal structure for understanding a concept consists
of assembling and relating a set of fundamental propositions, in accordance with the learning theories of Bruner et al. (1956) and Greco & Piaget (1959), which agree that students who have a comprehensive overview of a concept are better prepared to understand its details, constituent elements and applications. The definitions allowed us to collect evidence on and analyse the students' overview or general representation of each concept.
Procedure and teaching strategies
At first, the students of each group had to define a concept and submit their definitions in an individual assignment on the Blackboard platform. They had 15 minutes to complete the task. These first definitions provided the pre-test data. Then, each group carried out the work corresponding to each of the three teaching strategies. Finally, the students submitted a new definition of the concept based on learning derived from the teaching strategies to which they were exposed. These final submissions provided post-test data to compare with the pre-test data, in order to evaluate the effects of the teaching strategies on concept acquisition.
Group A
The students of Group A were asked to define the concept of "inclusive education". Their submissions were collected, pre-processed and analysed using TM in the software KH Coder 3 to obtain term frequency and two co-occurrence networks. The first network included modularity analysis to detect clusters of words, and the second included betweenness centrality to reveal the influence of each word over the flow of information. The network graphs incorporated words with frequencies (fa) > 4, and the distances between them were measured using the Jaccard coefficient (Jc). Figure 1 illustrates the co-occurrence networks that the students were then shown.
Based on the networks, students identified which keywords appeared most often in their definitions and which were missing. For example, they had emphasized more the "needs" of "students" in schools than "looking for" their "capabilities" or providing "responses" to their special educational needs. Moreover, although they thought about "capabilities" and "difficulties" and therefore connected many other words to these terms, they did not consider "reducing" difficulties, either as a keyword (e.g., reduce, overcome, eliminate) or in connection with other terms (e.g., capabilities, barriers, needs).
Using TM learning analytics as both a visual and verbal complement, the instructor tried to make students aware of their misconceptions by comparing relevant connections between terms, and incorporating new ideas into the class's general notion of inclusive education. Subsequently, students had another 15 minutes to reformulate their first definitions and submit them once again to the Blackboard platform.
GroupB
The students in Group B were asked to define the concept of an "open and flexible curriculum". They carried out an individual project in which they examined the curricular documents of a real school, and elaborated proposals to make the school's curriculum more open and flexible. This exercise took them a couple of weeks, and, afterwards, they briefly discussed their progress with classmates during class time. The instructor then reviewed the projects and gave examples of suggested proposals, and subsequently gave the students 15 more minutes to reformulate their definitions of the concept.
Group C
The students of Group C were asked to define the concept of "meaningful curricular adaptation". After completing and submitting their definitions, they received an expositive master class on the concept, and, at the end of the lecture, they had 15 minutes to reformulate and resubmit their definitions to the Blackboard platform.
Collecting and processing the analytical sample
All the data was collected during regular class hours. The procedure respected participant rights to informed consent, personal data protection, confidentiality, and non-discrimination, and the participants did not receive any compensation.
The researchers downloaded the concept definitions in text format directly from the Blackboard platform, and performed manual pre-processing of the text. This involved the correction of typos or misspellings, the changing of acronyms to their expanded meanings, and the removal of double spaces, special characters, and inclusive language expressions. The cleaned text constituted the dataset for the analysis without including any stop words. The resulting corpora consisted of an analytical sample for Group A of 1017 pre-test and 1133 post-test tokens, for Group B of 1127 pre-test and 1111 post-test tokens, and for Group C of 1101 pre-test and 1173 post-test tokens.
A set of 10 keywords and 10 associations between terms relevant to each concept were identified as priority elements to be included in the definitions. These keywords supported the evaluation of students' comprehensive understanding of the concepts, while the associations served to compare the relationship patterns of terms with word co-occurrences in the pre- and post-tests regarding each group. Appendices 1 and 2 present the criteria used to evaluate the definitions of the concepts.
The test statistic of Yuen's test of robust paired samples on trimmed means for dependent samples was calculated to obtain an effect size, with associated 95% confidence intervals (CI) around the estimates. Yuen's paired sample trimmed mean test is one of the most robust methods for comparing paired samples with non-normal distributions to obtain more robust results than traditional non-parametric tests. Values of f = .10, .30, and .50 were thus taken to correspond respectively to small, medium, and large effect sizes.
All analyses were carried out with "ggstatsplot" package, an extension of the "ggplot2" package, for R.
RESULTS
RQ1: Keyword selection
Table 1 shows the results corresponding to RQ1, while Figure 2 contains box plots representing the pre- and post-tests for each group (A, B and C). The results show that the only significant within-subjects difference was observed in Group A
of the students exposed to TM-based learning analytics as the teaching strategy, while a marginal statistical significance was observed in Group B. Group A therefore saw a larger pre- to post-test effect from TM-based learning analytics (tYuen = -6.37, p < .001, 6RAKP = -1.03, \C95% = -2.10, -.74) compared to the effect from the individual project task (tYuen = -1.78, p = .09, = -.50, IC95% = -.86, -.10).
For Group A, the Jaccard index for word associations in the pre-test ranged from .000 to .375 (see Table 2). After the session in which students were made aware of their misconceptions using the TM-based visual plot provided by instructor, the post-test Jaccard values increased to a range of .217 to .917, revealing statistically significant differences between the pre-test and the post-test in seven of the top ten words associations (1 to 7 all with py and pt < .05) (see Table 2). For certain associations, the Jaccard index increased substantially from pre-test to post-test after applying the TM-based teaching strategy, such as in the case of term associations 2 and 5 (i.e., accessible - contexts, and removing - barriers, respectively).
Garcia-Garcia et al. (2025)
Note 1. Jc tests = Jaccard tests; Pre-test = baseline measurement before application of the teaching strategy; /. = frequency of the first keyword selection in the pre-test for each association;/, = frequency of the second keyword selection in the pre-test for each association;/, = frequency of the first keyword selection in the post-test for each association;/z = frequency of the second keyword selection in the post-test for each association; pe = exact p-value (pre-test), pb: bootstrapped p-value (pretest); p}= exact p-value (post-test); pt= p-value with bootstrapping (post-test; * p < .05, ** p < .01, *** p < .001 with bootstrap and exact methods. Note 2. Appendix 2 shows the term associations for each group. Note 3. aone input vector contained only zeroes.
30
EducationXXl, 28(2), 17-43
The students in Group B who carried out individual projects established two significant associations in the pre-test (term associations 1 [attention -individualized] and 2 [equality - opportunities], with pe and pb < .05, respectively). After the teacher conceptualized, reviewed the projects and suggested proposals, post-test conceptualization results showed up to five significant associations (term associations 1, 2, 4, 6, and 9, all with pyand pt < .05; see Appendix 2 for the term associations). Group B, with the individual project as a teaching strategy, produced pre- to post-test Jaccard coefficient differences less favourable than Group A, but nonetheless produced an evident improvement (tYuen = -1.78, p = .09, 6RAKP = -.50, IC95% = -.86, -.10) (see Table 1 and Figure 2).
Group C, who attended an expositive master class, did not produce significant differences between the pre- and post-tests (tYuen = -.15, p = .88, 6RAKP = -.03, IC95% = -.52, .25). nor did they significantly make relevant associations.
DISCUSSION
The present study provides empirical evidence on the use of TM-based learning analytics as a pedagogical tool, compared to individual project work and attendance of a masterclass, for facilitating the acquisition of abstract educational concepts, specifically within the university context and in relation to student teachers. Previous research has suggested that active learning involving writing and discussions (Breivik, 2020), detailed feedback (Gao & Lloyd, 2020) and visual information (Magana et al., 2019) can facilitate concept acquisition in university courses, especially when applied in small groups (Atkinson et al., 2020; Rodriguez & Potvin, 2021). Furthermore, it has suggested that the technique of TM can be used to make the assessment of student learning outcomes more efficient (Kong et al., 2021). However, until now, there has been a lack of evidence on the effects of TM techniques used as a teaching strategy for conceptual learning at the higher education level. In this study, TM-based learning analytics supported students in selecting keywords and identifying missing aspects in the definition of an abstract concept, and helped them establish more relevant associations between terms.
Consistent with the findings of this study, previous studies at elementary school level have examined the effects of the Sobek topic modelling tool on scientific concept assessments. These studies found that students using TM-based tools performed significantly better on exams (Costa et al., 2017; Reategui et al., 2019). Similarly, research with 54 high school students showed that those using TM-based tools in essay writing incorporated a greater number of relevant concepts into their work (Erkens et al., 2016). These findings are in line with our own, reinforcing the view that TM-based learning analytics can be a valuable tool for enhancing conceptual understanding and retention.
Implications for educational practice
It is no coincidence that TM-based learning analytics can be a powerful ally for university students to acquire abstract concepts, given that the field of neuroscience has evidenced visual attention and its relation to information processing for several decades (Hutmacher, 2019; Kanwisher & Wojciulik, 2000). Indeed, the relevant literature has emphasized that aspects such as colour contrasts and intensity, among others, can stimulate visual attention, and lead to benefits in terms of working memory performance (Itti & Koch, 2001). In fact, in the present study, TM-based visual representations were made to present words in different colours, sizes and contrasts, because we knew that, in the classroom, students tend to have problems maintaining attention, especially when sessions are too long (Ghanizadeh et al., 2024). According the theories of comprehensive learning that are sometimes ignored in research, we know that the quality of the performance of a task, such as defining a concept, depends on a combination of sensory attention and access to prior knowledge records in memory (i.e., executive control) (Nobre & Kastner, 2014).
Although we have seen that TM-based learning analytics help students select keywords and establish relevant associations between terms in an overview of a concept, this does not necessarily guarantee that students will effectively manipulate and handle those conceptions later. It only implies that they acquired the basic notions to the extent of considering their most critical parts, and thus obtaining a general idea of them. We know from seminal works in the literature of the field (Bruner et al., 1956) that having a thorough general idea contributes to successful concept acquisition, but in no case does it guarantee the correct use of concepts in other more complex activities. Some studies have shown how peer review, for instance, did not always promote improvements in higher-order reasoning skills following concept acquisition (Turner et al., 2018).
Just as other tasks are deliberately aimed at developing higher-order thinking (e.g., asynchronous online discussions) (Jeong & Chiu, 2020), there is currently a lack of evidence of any effects of the use of TM-based learning analytics on the this type of learning. Therefore, we cannot yet design reliable tasks incorporating TM-based learning analytics for purposes such as longer-term assessments taking a concept as a criterion or comparing it with other concepts. For now, we know that TM-based learning analytics can certainly be used as a teaching resource at the beginning of a course to help students acquire fundamental abstract concepts. Subsequently, it would be necessary to design other different assignments to learn how to develop these concepts in certain given contexts (Cortes et al., 2019). For example, in the case of Group A in our study, we would need to teach the students to practically assess whether a certain school or teaching strategy was inclusive through other activities such assembling a rubric.
One of the benefits of TM as a tool is that it generates a detailed description of a group's level of acquisition of a given concept, and presents a more precise and generalized explanation for the whole class, detailing missing aspects and misconceptions in definitions of the concept. With the use of TM, lecturers can save time and avoid a situation in which students arrive one by one at resolving doubts about the concept, or, worse, have unresolved doubts and do not achieve desired learning outcomes regarding a subject due to a lack of understanding of fundamental notions.
Nevertheless, TM-based learning analytics involves procedures that certainly not all university instructors will be familiar with, and therefore adequate training programmes will need to be developed presentingTM as a teaching tool, for lecturers to conduct those procedures successfully. Moreover, such training is relevant at this time in which higher education institutions are increasingly investing in educational technology, and their teaching staff need to know how to get the most out of it in order to make investments profitable. All in all, our findings support the investment of resources in TM-based learning analytics as a tool for university teaching and not just for assessment, as has been more common according to the studies we have reviewed (e.g., Begusic et al., 2018; Hernandez-Lara et al., 2021).
Limitations and emerging research
Although the present study provides evidence of the potential of TM-based learning analytics as a teaching tool for concept acquisition at the university level, the design of the study has limitations that impact both the internal and external validity of the results. One specific limitation is the lack of a formal validation process in the design and implementation of the teaching strategy before its execution. A formal validation process should consider aspects such as feasibility, acceptability, and validity, and potentially provide preliminary evidence of the teaching strategy's impact. An expert panel might have further contributed by discussing aspects such as the content being studied, various teaching strategies, the intervention duration and frequency, and administrator criteria, among others. Future studies should include a teaching strategy protocol and prior validation process to ensure the consistency of the findings. Additionally, they should also consider experimental designs with a TM-based intervention control and homogeneous measures between groups (e.g., by analysing the same concept).
Another limitation of the study is its limited statistical power to detect significant differences between conditions, due to the small sample sizes. Additionally, these limited sample sizes may have introduced bias, restricting the generalizability of our results. Therefore, future studies with larger sample trials are needed to confirm these preliminary findings.
Furthermore, while we know that acquiring a concept overview facilitates the meaningful learning of that concept by promoting fundamental inclusiveness in Ausubel's terms, our study does not provide information about the further development of the concept that the students defined. It is likely that TM facilitates only the acquisition of a general overview of an abstract concept, and other teaching strategies are necessary to learn how to develop its contents and apply it successfully in professional practice. More research is needed to further explore how the benefits of TM extend to the acquisition of abstract concepts in groups of undergraduate students, particularly in teacher education.
ACKNOWLEDGMENTS
This work was supported by the Government of Valencia, Spain (under grant number CIGE/2023/53), and the Spanish Ministry of Science, Innovation and Universities (under grant number PID2022-141403NB-I00, with MCIN/ AEI/10.13039/501100011033/FEDER-EU funding).
REFERENCES
Aguilar, J., Buendia, O., Pinto, A., & Gutierrez, J. (2022). Social learning analytics for determining learning styles in a smart classroom. Interactive Learning
Environments, 30(2), 245-261. https://doi.org/10.1080/10494820.2019.1651 745 Atkinson, M. B., Krishnan, S., McNeil, L. A., Luft, J. A., & Pienta, N. J. (2020).
Constructing Explanations in an Active Learning Preparatory Chemistry Course.
Journal of Chemical Education, 97(3), 626-634. https://doi.org/10.1021/acs. jchemed.9b00901 Ausubel, D. P., Novak, J. D., & Hanesian, H. (1968). Educational psychology: a cognitive view. Holt, Rinehart and Winston. Azadi, G., Biria, R., & Nasri, M. (2018). Operationalising the Concept of Mediation in L2 Teacher Education. Journal of Language Teaching and Research, 9(2), 132-140. https://doi.org/10.17507/jltr.0901.17 Babaahmadi, A., Maraghi, E., Moradi, S., & Younespour, S. (2021). Comparison
Between Peer Learning and Conventional Methods in Biostatistics Course Among Postgraduate Nursing Students' Final Score, Statistics and Test Anxiety: A Quasi-experimental Study with a Control Group. Shiraz E-Medical Journal, 22(11), 1-8. https://doi.org/10.5812/semj.111984
Begusic, D., Pintar, D., Skopljanac-Macina, R, & Vranic, M. (2018). Annotating Exam Questions Through Automatic Learning Concept Classification. 2018 26th International Conference on Software, Telecommunications and Computer Networks, SoftCOM 2018, 176-180. https://doi.org/10.23919/ SOFTCOM.2018.8555784
Borghi, A. M., Barca, L, Binkofski, F., Castelfranchi, C, Pezzulo, G., & Tummolini, L. (2019). Words as social tools: Language, sociality and inner grounding in abstract concepts. Physics of Life Reviews, 29, 120-153. https://doi.0rg/lO.lOl6/j. plrev.2018.12.001
Breivik, J. (2020). Argumentative patterns in students' online discussions in an introductory philosophy course: Micro-and macrostructures of argumentation as analytic tools. Nordic Journal of Digital Literacy, 15(1), 8-23. https://doi. org/10.18261/ISSN.1891-943X-2020-01-02
Bruner, J. S., Goodnow, J. J., & Austin, G. A. (1956). A study of thinking. Wiley.
Casanoves, M., Sole-Llussa, A., Haro, J., Gericke, N., y Vails, C. (2022). Assessment of the ability of game-based science learning to enhance genetic understanding. Research in Science & Technological Education, 1-23. https://doi.org/10.1080/ 02635143.2022.2044301
Cortes, D. M. G., Rodriguez, C. M. 0., & Alejo, V. V. (2019). Learning object for contextualization of matrix operations in digital image processing through programming. Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality, 92-98. https://doi. org/10.1145/3362789.3362876
Costa, A. P. M., Reategui, E. B., Epstein, D., Meyer, D. D., Lima, E. G., & Silva, K. H. da. (2017). Emprego de um software baseado em mineracao detexto e apresentacao grafica multirrepresentacional como apoio a aprendizagem de conceitos cientfficos a partir de textos no Ensino Fundamental. Ciencia & Educagao (Bauru), 23(1), 91-109. https://doi.org/10.1590/1516-731320170010006
De Lin, O., Gottipati, S., Ling, L. S., & Shankararaman, V. (2021). Mining Informal & Short Student Self-Reflections for Detecting Challenging Topics - A Learning Outcomes Insight Dashboard. 2021 IEEE Frontiers in Education Conference (FIE), Oct 2021, 1-9. https://doi.org/10.1109/FIE49875.2021.9637181
Erkens, M., Bodemer, D., & Hoppe, H. U. (2016). Improving collaborative learning in the classroom: Text mining based grouping and representing. International Journal of Computer-Supported Collaborative Learning, 11(4), 387-415. https:// doi.org/10.1007/sll412-016-9243-5
Finkenstaedt-Quinn, S. A., Polakowski, N., Gunderson, B., Shultz, G. V., & Gere, A. R. (2021). Utilizing Peer Review and Revision in STEM to Support the Development of Conceptual Knowledge Through Writing. Written Communication, 38(3), 351-379. https://doi.org/10.1177/07410883211006038
Freeman, D. (2018). Arguing for a knowledge-base in language teacher education, then (1998) and now (2018). Language Teaching Research, 24(1), 5-16. https://doi.org/10.1177/1362168818777534
Gaglo, K., Degboe, B.M.,Kossingou,G.M.,& Ouya,S. (2022). Proposal of conversational chatbots for educational remediation in the context of covid-19. 2022 24th International Conference on Advanced Communication Technology (ICACT), Feb 2022, 354-358. https://doi.org/10.23919/ICACT53585.2022.9728860
Gagne, R. M. (1985). The conditions of learning and theory of instruction (4th ed.). Holt, Rinehart and Winston. Gao, R., & Lloyd, J. (2020). Precision and Accuracy: Knowledge Transformation through Conceptual Learning and Inquiry-Based Practices in Introductory and Advanced Chemistry Laboratories. Journal of Chemical Education, 97(2), 368-373. https://doi.org/10.1021/acs.jchemed.9b00563
Ghanizadeh, A., Tabeie, M., & Pourtousi, Z. (2024). The role of university instructor's narrative in students' sustained attention, emotional involvement and cognitive learning. Journal of Applied Research in Higher Education, 16(1), 195-207. https://doi.org/10.1108/JARHE-09-2022-0278
Greco, P., & Piaget, J. (1959). Apprentissage etconnaissance. P.U.F. Guerrettaz, A. M., Zahler, T, Sotirovska, V., & Boyd, A. S. (2020). 'We acted like ELLs': A pedagogy of embodiment in preservice teacher education. Language Teaching Research, 1-25. https://doi.org/10.1177/1362168820909980
Hernandez-de-Menendez, M., Vallejo Guevara, A., Tudon Martinez, J. C, Hernandez Alcantara, D., & Morales-Menendez, R. (2019). Active learning in engineering education. A review of fundamentals, best practices and experiences. International Journal on Interactive Design and Manufacturing, 13(3), 909-922. https://doi.Org/10.1007/S12008-019-00557-8/FIGURES/2
Hernandez-Lara, A. B., Perera-Lluna, A., & Serradell-Lopez, E. (2021). Game learning analytics of instant messaging and online discussion forums in higher education. Education and Training, 63(9), 1288-1308. https://doi.org/10.1108/ET-ll-2020-0334
Hutmacher, F. (2019). Why Is There So Much More Research on Vision Than on Any Other Sensory Modality? Frontiers in Psychology, 10, 2246. https://doi. org/10.3389/fpsyg.2019.02246 Inada, Y. (2018). Collaborative learning in entrepreneurship education in a Japanese business school. Proceedings of the European Conference on Innovation and Entrepreneurship, ECIE, Sep 2018, 319-327.
Itti, L, & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2(3), 194-203. https://doi.org/10.1038/35058500
Jeong, A., & Chiu, M. M. (2020). Production blocking in brainstorming arguments in online group debates and asynchronous threaded discussions. Educational Technology Research and Development, 68, 3097-3114. https://doi. org/10.1007/sll423-020-09845-7
Kanwisher, N., & Wojciulik, E. (2000). Visual attention: Insights from brain imaging. Nature Reviews Neuroscience, 1(2), 91-100. https://doi.org/10.1038/35039043
Khong, I., Aprila Yusuf, N., Nuriman, A., & Bayu Yadila, A. (2023). Exploring the Impact of Data Quality on Decision-Making Processes in Information Intensive Organizations. APTISI Transactions on Management (ATM), 7(3), 253-260. https://doi.org/10.33050/atm.v7i3.2138
Kong, S.-C, Kwok, W.-Y, & Poon, C.-W. (2021). Evaluating a learning trail for academic integrity development in higher education using bilingual text mining.
Technology, Pedagogy and Education, 30(2), 305-322. https://doi.org/10.1080 /1475939X.2021.1899041 Koong, C.-S., Lin, H.-C, Wu, C.-C, Chen, C.-H., Lee, P.-H., & Wang, H.-C. (2021). Design and Implementation of an iOS APP: Multimedia Interactive System and Items for Woodworking Teaching. En M. M. T. Rodrigo, S. Iyer, A. Mitrovic, H. N. H. Cheng, D. Kohen-Vacs, C. Matuk, A. Palalas, R. Rajenran, K. Seta, y J. Wang (Eds.), 29th International Conference on Computers in Education Conference, ICCE 2021 - Proceedings (Vol. 2, pp. 310-316). Asia-Pacific Society for Computers in Education. Kortemeyer, G., Anderson, D., Desrochers, A. M., Hackbardt, A., Hoekstra, K., Holt, A., Iftekhar, A., Kabaker, T., Keller, N., Korzecke, Z., Gogonis, A., Manson, Q., McNeill, G., Mookerjee, D., Nguyen, S., Person, B., Stafford, M., Takamoribraganca, L., Yu, Z., ... Ratan, R. (2019). Using a computer game to teach circuit concepts. European Journal of Physics, 40(5), 1-16. https://doi.org/10.1088/1361-6404/ ab2ald Liao, A. Y. H. (2022). An APP-Based E-Learning Platform for Artificial Intelligence
Cross-Domain Application Practices. En L. Barolli, K. Yim, y H. C. Chen (Eds.), Innovative Mobile and Internet Services in Ubiquitous Computing. IMIS 2021.
Lecture Notes in Networks and Systems (Vol. 279, pp. 341-351). Springer.
https://doi.org/10.1007/978-3-030-79728-7_34 Magana, A. J., Serrano, M. I., & Rebello, N. S. (2019). A sequenced multimodal learning approach to support students' development of conceptual learning.
Journal of Computer Assisted Learning, 35(4), 516-528. https://doi.org/10.llll/ jcal.12356 Nguyen, K. A., Borrego, M., Finelli, C. J., DeMonbrun, M., Crockett, C, Tharayil,
S., Shekhar, P., Waters, C, & Rosenberg, R. (2021). Instructor strategies to aid
implementation of active learning: a systematic literature review. International Journal of STEM Education, 8(1), 1-18. https://doi.org/10.1186/S40594-021-00270-7/TABLES/2
Nobre, A. C. (Kia), & Kastner, S. (Eds.). (2014). The Oxford Handbook of Attention (Vol. 1). Oxford University Press, https://doi.org/10.1093/ oxfordhb/9780199675111.001.0001
Pardo, A., & San Martin, R. (2010). Metodologfa de las Ciencias del Comportamiento y de la Salud II. Sintesis.
Pillutla, V. S., Tawfik, A. A., & Giabbanelli, P. J. (2020). Detecting the Depth and Progression of Learning in Massive Open Online Courses by Mining Discussion Data. Technology, Knowledge and Learning, 25(4), 881-898. https://doi. org/10.1007/sl0758-020-09434-w
Pintar, D., Begusic, D., Skopljanac-Macina, F., & Vranic, M. (2018). Automatic extraction of learning concepts from exam query repositories. Journal of Communications Software and Systems, 14(4), 312-319. https://doi. org/10.24138/jcomss.vl4i4.605
Reategui, E., Costa, A. P. M., Epstein, D., & Carniato, M. (2019). Learning Scientific Concepts with Text Mining Support (pp. 97-105). https://doi.org/10.1007/978-3-319-98872-6J.2
Redondo Lopez, J. M. (2021). Improving Concept Learning Through Specialized Digital Fanzines. 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET), 134-143. https://doi.org/10.1109/ICSE-SEET52601.2021.00023
Reyes-Santfas, F., Rivo-Lopez, E., Villanueva-Villar, M., & Miguez-Alvarez, C. (2021). Movie clips for teaching business management: Step by step. Journal of Education for Business, 1-12. https://doi.org/10.1080/08832323.2021.1991258
Reynolds, J. A., Cai, V., Choi, J., Faller, S., Hu, M., Kozhumam, A., Schwartzman, J., & Vohra, A. (2020). Teaching during a pandemic: Using high-impact writing assignments to balance rigor, engagement, flexibility, and workload. Ecology and Evolution, 10(22), 12573-12580. https://doi.org/10.1002/ece3.6776
Rodriguez, M., & Potvin, G. (2021). Frequent small group interactions improve student learning gains in physics: Results from a nationally representative pre-post study of four-year colleges. Physical Review Physics Education Research, 17(2), 1-11. https://doi.org/10.1103/PhysRevPhysEducRes.17.020131
Shwartz, V. (2021). Dissertation Abstract: Learning High Precision Lexical Inferences. Kl - Kunstliche Intelligenz, 35(3-4), 377-383. https://doi.org/10.1007/sl3218-021-00709-7
Taga, M., Onishi, T, & Hirokawa, S. (2018). Automated Evaluation of Students Comments Regarding Correct Concepts and Misconceptions of Convex Lenses. Proceedings - 2018 7th International Congress on Advanced Applied Informatics, IIAI-AAI2018, 273-277. https://doi.org/10.1109/IIAI-AAI.2018.00059
Tsai, M.-J., Wu, A.-H., & Wang, C.-Y. (2022). Pre-training and cueing effects on students' visual behavior and task outcomes in game-based learning. Computers in Human Behavior Reports, 6,1-9. https://doi.Org/10.1016/j.chbr.2022.100188
Turner, R. L. (1975). An Overview of Research in Teacher Education. Teachers College Record: The Voice of Scholarship in Education, 76(6), 87-110. https:// doi.org/10.1177/016146817507600605
Turner, S. A., Perez-Quinones, M. A., & Edwards, S. H. (2018). Peer Review in CS2: Conceptual Learning and High-Level Thinking. ACM Transactions on Computing Education, 18(3), 1-37. https://doi.org/10.1145/3152715
Volkwyn, T S., Gregorcic, B., Airey, J., & Under, C. (2020). Learning to use Cartesian coordinate systems to solve physics problems: the case of 'movability.' European Journal of Physics, 41(4), 1-15. https://doi.org/10.1088/1361-6404/ab8b54
Wittek, A. L. (2018). Processes of Writing as Mediational Tool in Higher Education. Scandinavian Journal of Educational Research, 62(3), 444-460. https://doi.org/ 10.1080/00313831.2016.1258664
Ye, L, Eichler, J. F., Gilewski, A., Talbert, L. E., Mallory, E., Litvak, M., M. Rigsby, E., Henbest, G., Mortezaei, K., & Guregyan, C. (2020). The impact of coupling assessments on conceptual understanding and connection-making in chemical equilibrium and acid-base chemistry. Chemistry Education Research and Practice, 21(3), 1000-1012. https://doi.org/10.1039/d0rp00038h
APPENDICES
The token words in Appendices 1 and 2 came from student data contained in the submitted definitions. The researchers examined the texts considering multiple synonyms and word combinations to obtain the following tables that made it possible to evaluate the data. The parts of speech identified are indicated at the end of both appendices.
40
Education XXl, 28(2), 17-43
Educación XX1
ISSN: 1139-613X o e-ISSN: 2174-5374
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2025. This work is published under https://creativecommons.org/licenses/by-nc/4.0/ (the "License"). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
Concept acquisiton is a critcal aspect in the educaton of teachers yet is especially challenging in group contexts in which traditonal teaching strategies ofen fail to convey complex notons efectvely. This study investgates the potental of text mining (TM) based learning analytcs as a teaching tool to enhance conceptual learning in pre-service teachers. To do so, it analyses how the learning of complex and abstract educatonal concepts was afected by a TM-based learning analytcs interventon, in comparison with traditonal teaching strategies, including the elaboraton of an individual project, and the atendance of a master class. Quasi-experimental pre- and post-tests were thus administered to three non-equivalent groups (A, B, and C, respectvely) of a total of 81 master's students enrolled in a distance educaton teacher training programme at a Spanish university, and token corpora were analysed using TM techniques in collected defnitons of abstract educatonal concepts (1017 pre-test and 1133 post-test tokens from Group A; 1127 pre-test and 1111 post-test tokens from Group B; and 1101 pretest and 1173 post-test tokens from Group C). It was found that the TM-based learning analytcs interventon signifcantly enhanced the students' keyword selecton in submited defnitons (tYuen = -6.37, p < .001, ?RAKP = -1.03, IC95% = -2.10, -.74) and the associaton of relevant terms (with post-test Jaccard values ranging from .217 to .917) compared to the other teaching approaches. This study therefore ofers empirical evidence that TM-based learning analytcs can be an efectve pedagogical tool that promotes an enhanced learning of abstract concepts in the educaton of teachers. The results underscore the value of TM-based educatonal technology in optmizing conceptual learning and resource efciency in higher educaton setngs.