Evaluating the effectiveness of the integrated research subject research-based learning online (IRS-RBLO) model: retrospective record review study

Abstract

Background

The Integrated research subject-research-based learning online (IRS-RBLO) model is developed based on research-based learning, the components of the research process, and elements of online learning design. We aimed to evaluate the effectiveness of the IRS-RBLO in enhancing the learning outcome and research knowledge among community public health students at Walailak University, Thailand.

Methods

This study applied a retrospective record review of 67 complete records out of 80 fourth-year community public health students of Walailak University to study the effectiveness of the IRS-RBLO model through research knowledge and learning outcomes. Research knowledge and learning outcomes were assessed using 40 multiple choice questions and 40 Likert scale questioners, respectively. The T-test was applied to analyze research knowledge before (1st week) and during learning (7th week), and ANOVA statistics to analyze students’ learning outcomes before (1st week), during (7th week), and after (16th week) learning with the IRS-RBLO model.

Results

Students exhibited a significant difference in research knowledge (p =.00), with 35.88% percentage increase in aggregate knowledge assessment score before and during; and learning outcomes before, during, and after learning with the IRS-RBLO model (P =.00), with 17.44% increase in overall learning outcome increase between the scores of before and after. However, it showed variable results while comparing before and during (P <.001), before and after (P =.001), and during and after (P =.045) learning with the IRS-RBLO model.

Conclusions

The IRS-RBLO model has shown its significant difference in research knowledge and learning outcomes. We recommend the broader implementation of the model, and future research may explore instructor opinions, and investigate educational policy implications to further validate the model’s effectiveness.

Clinical trial number

Not applicable.

Background

Overview of online learning method due to the COVID-19 pandemic

Currently, education is affected by the coronavirus disease 2019 (COVID-19) pandemic [1], as are social, economic, and lifestyle factors for preventing disease [2]. As a response to the COVID-19 pandemic, most educational institutions worldwide have quickly transitioned from conventional learning to online learning through various resources, including radio and television education, online instructional resources, and instructional packages [3]. In Thailand, COVID-19 has been experienced a continuous outbreak [4], which has changed normal life [2]. For this reason, online education evolved as a solution, and an innovation that enhances the outcome of learning [5].

The online course design uses several strategies for teaching, such as web-based methods, video demonstrations, instructor notes, mini-projects, and online discussion forums, which are effectively applied in other courses [6]. Thus, online teaching is an important method for active learning. There are several active learning methods for enhancing learner competency in the 21st century, which focus on the tutor as a person who empowers, supports, creates a good atmosphere, sets an environment, and designs a relationship model for learning [7].

Research-based learning (RBL)

Is a learning modality, that incorporates the research process into the learning experience. Students are motivated to engage in the research process, acquiring skills, such as critical thinking, problem-solving, self-directed learning, and effective communication. In addition, while working with their peers and supervisors, they formulate questions, conduct research, and reflect on their findings [8, 9].

RBL aims to promote and develop student competencies related to research and to benefit from research. It has six levels for designing learning: (1) drawing on personal research in designing and teaching courses; (2) involving students in departmental research projects; (3) teaching research methods, techniques and skills explicitly within programs; (4) build small-scale research activities into undergraduate assignments; (5) encouraging students to feel part of the research culture of departments; and (6) infusing teaching with the values of researchers [10,11,12]. The RBL model highlights the relationship between research-based teaching and research for undergraduate students and focuses on RBL, which is different from traditional teaching methods. The teacher must therefore be motivated to build connections with local stakeholders to create a bridge between research and practice to increase the learning level of students, and the institution needs stakeholder participation to solve problems based on research studies [13, 14]. Research should be conducted on student experiences, motivations for participation, and perceptions of the relative strengths and weaknesses of various aspects of online education [15].

A previous interventional study of engineering students at Technologico de Monterrey, Mexico City, revealed the capacity of undergraduate students in computational skills in RBL [16]. However, this study involved a small group, and only the pre and post-intervention groups took place at 14 weeks in the pilot teaching class. Another study compared standard teaching and research-based learning for diploma students in a semester, and the results revealed high satisfaction with teachers, a high score of research knowledge, a completed research report of 79%, and 6 publications [17].

Integrated research subject research based learning online (IRS-RBLO) model development

It is a model that is developed based on the three concepts. 1st, Six levels of research-based learning (RBL) [9], 2nd, five components of the research process as follows: (1) introduction of the research problem and reason, (2) literature review, (3) research methodology, (4) results and discussion, and (5) conclusion and suggestions [18,19,20], and 3rd, five elements of online learning design [12]. The design of online learning subject needs preparation, online supporting instructors, assessing student experiences, motivators for participation, and perceptions on various aspects of online education [15]. The design of the model followed the standard of UKPSF [21], TQF of HEd, student’s reflections before learning [22], and following Boettcher and Conrad’s fourteen best practices for teaching online [23].

Thai Qualifications Framework for Higher Education (TQF: HEd.) is a comprehensive strategy to maintain the quality and standards of higher education in Thailand. The strategies include defining clear levels of qualifications, credit points, domains of learning, descriptions of learning outcomes and academic and professional components. It encourages sustained and lifelong learning [24]. The Thai education institution needs to prepare and develop a 21st century strategy for student learners following the Thai Qualifications Framework for Higher Education (TQF: HEd.) and United Kingdom professional standards framework (UKPSF) domains of learning outcomes, which includes 5 skills: (1) ethics, (2) knowledge, (3) cognition, (4) interpersonal skills and responsibility, and (5) analytical and communication skill [25, 26] during COVID-19 outbreak. Therefore, developing teaching methodology and research project courses based on research-based learning online (RBLO) was an area of interest in the COVID-19 crisis in the school.

The IRS-RBLO model was developed in 3 phases: 1) Assessment phase. Includes assessments of all the factors that contribute to integrated design development (see Additional file 1.1), such as:

1. A.

Students’ reflection and experience before learning with IRS-RBLO: includes the problems that arise during IRS-RBLO, solutions to the problems, and support needed from their advisors, the school, and the University. In addition, students’ experience before learning with RBLO is based on the findings from the qualitative study done to assess students’ reflection on RBLO [22].

2. B.

Environment scanning to support the design IRS-RBLO: The environment supports the integrated subject following the TQF3 template and UKPSF, preparing fundamental skills for the student’s research study, equipment, lecturers for advisory roles, handouts and the main learning manual, and a research process book [18].

IRS-RBLO model design phase

In the model design phase the IRS-RBLO model was developed by integrating research methodology and research project subject; that helped students to conduct research in 16 weeks. This is similar with the research-based learning model of students on the Electric power system protection which took place in 15 weeks, but the model was nine steps of research-based learning [27]. The content of the research methodology subject was designed in 12 h. in the online class and 36 h. for practice with 18 advisors who integrated the steps of the research project subject into 5 phases of the research process. The models Content validity was checked by five experts, of whom three were experts of research methodology and public health research project and two were experts of active learning. The total CVI of all activities in 16 weeks is 0.80, and suggestions for the improvement of the model were considered. For more information on the model, this document has supplementary data (see Additional file 1.2).

Teaching and evaluating phase

Recap and monitoring were done every week. The teachers and students have been discussing the progress and raising problems to conduct research. 18 research advisors have been giving feedback to overall and individual students of 27 groups. The effectiveness of the integrated design subject was evaluated by assessing and providing feedback to the students in formative evaluation (60%) and summative evaluation (40%), through assessment of research knowledge at before (1st week) and during (7th week) and learning outcome at before (1st week), during (7th week) and after (16th week).

In this design, 80 students were divided into 27 groups for conducting research, and used 18 instructors for advisory roles in 16 weeks. This is consistent with the result from a mixed method study and shows students want smaller groups, short times not more than 45 min, and online teaching for easy learning such as asking, well communication with teacher and peers in digital class, and more motivation to attend class [28].

To our knowledge, there were no research-based online learning model which can be applied in higher education in our context. Therefore, we developed the research question considering this: what is the effectiveness of the Integrated Research Subject Research Based Learning Online Model (IRS-RBLO) in enhancing student’s learning outcomes and research knowledge? This study aimed to evaluate the effectiveness of the Integrated Research-Based Online Learning Model (IRS-RBLO) in enhancing the learning outcome and research knowledge among community public health students at Walailak University.

Methods

Study design

This study applied a retrospective record review study (RRR) [29] to analyze students’ knowledge before and during learning, and learning outcomes before, during, and after learning with the IRS-RBLO model in an online research class through a Micro Soft Team and Google form, a hard copy, and an electronic file in 16 weeks. IRS-RBLO model was designed and the students taught the research processes in 16 weeks and developed a full report of research in 27 groups.

Study setting and period

The study took place at Walalilak University, School of Public Health among fourth-year community public health students starting on July 1, 2021, for 16 weeks.

Study samples

The study samples were all learning records such as academic evaluation forms of the course learning process, assessment forms of learning outcomes, and student teaching and learning assessment results from eighty-fourth-year community public health students. After cleaning and excluding incomplete records, the remaining 67 records of students were analyzed to study the model’s effectiveness through research knowledge and learning outcomes. The review records were categorized into two sections: (1) research knowledge testing before and during learning; and (2) learning outcomes among students before, during, and after learning.

Inclusion and exclusion criteria for review

Inclusion criteria

The records of students who were fourth-year community public health program and learned with IRS-RBLO model at study period, and found to be complete were included in the study.

Exclusion criteria

Student records that were found incomplete to study research knowledge and learning outcomes were excluded.

Sample size and sampling technique for review

The records of eighty students of the fourth-year community public health program of School of Public Health at Walailak University were selected following purposive sampling technique. Among them, the records of 67 students were found to be complete and used for analysis of research knowledge and learning outcomes based on inclusion and exclusion criteria.

Variables of the study

Research knowledge before and during learning with the IRS-RBLO model.

Learning outcome before, during, and after learning with the IRS-RBLO model.

Measurement

Effectiveness of integrated research subject research-based learning online (IRS-RBLO) model

The effectiveness of the IRS-RBLO model is assessed, both by the difference in research’s knowledge before and during, and learning outcome before, during, and after implementing the model.

Research’s knowledge

Assessed before and during learning with 40 research knowledge-related multiple-choice questionnaires. The questionnaires consist of items related to research methodology such as research concept (1 item), title(1item), aim (1 item), question (1 item), Research hypothesis (2 items), Study design (9 items), Variable (2 items), Sample size and Sample sampling technique (6 items), Measurement (3 items), Data collection (3 items), Quality of tool (4 items), Data analysis (4 items), Research report 3 items, total of 40 items.

Learning outcome

Measured by five point Likert scale questionnaires developed based on the concept of the Thai Qualifications Framework for Higher Education (TQF: HEd.) and United Kingdom professional standards framework (UKPSF) [21, 25] for each aspects of learning outcome, such as (1) “Ethics” consists of 7 items (35 points), (2) “Knowledge” consists 11 items (55 points),3) “Cognitive skills” consists of 6 items (30 points), 4) “Interpersonal skills and responsibility” consists of 8 items (40 points), and 5) “analytical, communication and information technology skills” consists of 8 items (40 points) (Additional file 3 Questionnaire) The overall “learning outcome” consists of 40 items questionnaire (200 points).

Data collection

The records of 80 students in community public health were collected based on their academic records. After excluding incomplete records, 67 students’ records (95% complete) were used to study research knowledge and learning outcomes. Participants’ consent was not taken since it is not required for a noninvasive retrospective record review study per the regulation of the Ethics Committee in Human Research at Walailak University. The data included academic evaluation forms of the course learning process, assessment forms of learning outcomes, and student teaching and learning assessment results of the academic year from July 1, 2021, to 16 weeks among sixty-seven 4th-year students. Separate questionnaires were developed to assess research knowledge and learning outcomes (Additional file 3). Part of this data was used to analyze students’ reflections on the IRS-RBLO model [22]. The data for research knowledge was collected before (the 1st week) and during (the 7th week) and learning outcomes before (the 1st week), during (the 7th week), and after (the 16th week) learning with the IRS-RBLO model. Both questionnaire’s content validity was assessed by three experts and found content validity index (CVI) = 0.92 and reliability was tested among 70 students and found alpha Cronbach coefficient = 0.93.

Data analysis

The percentage differences in research knowledge before and after and learning outcomes before and after, before and during, and during and after were calculated. The data from the records were analyzed by comparing the differences in mean scores of knowledge of research among students before (the 1st week) and during (the 7th week) of learning by paired T-test statistics. The cutoff point is set on mean and standard deviation before learning with the model for learning outcome. The score is categorized as a good level if the score point is ≥ 80%, and a poor level if the score point is < 80% for each aspects of learning outcome, and a comparative analysis of differences in mean scores of learning outcomes in all 5 aspects at before (1st week), during (7th week) and after (16th week) learning via a statistical ANOVA test, and pairwise testing if there were differences before, during and after learning via post hoc multiple comparison test. Statistical significance was set at a P-value of less than 0.05.

Results

A. Learning outcome before learning with IRS-RBLO

The results showed interpersonal skills and responsibility aspects (Mean = 33.1; SD.=4.6), and ethical aspects (Mean = 28.7; SD.=3.3) were at a good level, whereas, knowledge (Mean = 37.2; SD.=5.4), cognitive skills (Mean = 20.5, SD.= 3.1) and analytical and communication skills (Mean = 27.3; SD.=4.0) were at a poor level.

Research knowledge before and during learning with the IRS-RBLO model

Sixty-seven students’ records out of 80 students were found to be complete and valid to analyze Research knowledge and learning outcomes. Research knowledge was assessed at 1st week (before application) and 7th week (during application) of the model. The mean score of knowledge assessment before (1st week) and 7th week (during application) of the model are 25/40 and 33.97/40 respectively, resulting in 35.88% score difference between before and during learning with IRS-RBLO model. The aggregate knowledge showed a statistically significant difference (P <.001). Among forty items twenty-one showed statistically significant difference with (P <.001), six items with (P <.01), and six items with (P <.05). However, eight items showed no statistically significant difference (Table 1).

[IMAGE OMITTED: SEE PDF]

Learning outcomes before, during, and after learning with the IRS-RBLO model

The percentage score difference of the learning outcome while comparing before and after learning with the model is 17.44% (Fig. 1).

[IMAGE OMITTED: SEE PDF]

with the IRS-RBLO model.

All items and overall scores in learning outcomes before, during, and after learning with the IRS-RBLO model showed statistically significant differences (P <.001) (Table 3).

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

While comparing learning outcomes before and during, and before and after learning with the IRS-RBLO model showed statistically significant differences (P <.001) in aspects of ethics, knowledge, cognitive skills, interpersonal skills and responsibility, and analytical and communication skills. The comparison of learning outcomes during and after learning with the IRS-RBLO model showed statistically significant differences (P <.05) in aspects of knowledge, cognitive skills, analytical and communication skills, and the overall score. However, there was no statistically significant difference in aspects of ethics, and interpersonal skills and responsibility (Table 2).

Discussion

Teaching and evaluating phase: research knowledge before and during learning with the IRS-RBLO model

In the evaluation of the effectiveness of IRS-RBLO model 67 students’ knowledge and cognitive abilities of research methodology were measured by 40 items before (1st ) and during (7th ) week of learning shown statistically significant difference (P =.00). This finding is consistent with the result from other studies on research-based learning activities [27, 30]. Similarly, the study done to compare the project based learning with traditional teaching during COVID-19 showed consistent finding with the current study [11]. This might be due to research based online learnings encourage students’ full engagement in the course and motivates them to engage in each steps of course development and evaluation.

Learning outcomes before, during, and after learning with the IRS-RBLO model

In another way, the learning outcomes among students all items and overall scores in learning outcomes before, during, and after learning with the IRS-RBLO model showed statistically significant differences (P =.00). But it showed variable results while comparing before and during (P <.001), before and after (P =.001), and during and after (P =.045) learning with the IRS-RBLO model. These indicated that the design promoted learning outcomes because the design used strategies based on the Thai Qualifications Framework for Higher Education (TQF: HEd.) which encourages students to actively engage in education and foster lifelong learning [24]. The application of the IRS-RBLO model showed statistically significant differences in learning outcomes by integrating research methodology and research project subjects that made students translate their theoretical knowledge to practical knowledge, application of RBL which nurtures critical thinking and problem-solving skills, using online platforms to enhance accessibility, cooperation and resource sharing, and performing structured assessment and providing timely feedback [14, 22]. This result is consistent with an experimental study that divided students into small groups (3 students/group) in 14 weeks showed that there was a significant difference between the student’s learning achievement before and after learning with 6 components including ground theories, objectives, instruction processes, social system, principles in responses and supportive system, and learning management [31]. In addition, it showed similar test results against research-based learning on problem-solving skills, looking at students’ self-identification issues, presentation, election strategy, implementation, and evaluation [32]. Moreover, the qualitative study on the reflection of students on the IRS-RBLO model shows that it improved their active engagement, ability to take responsibility, enhanced confidence, and improved instructor support in research and in the consumption of online learning materials [22].

While comparing for each components of learning outcome during and after learning, it showed significant differences for knowledge (P =.037), cognitive skill (P =.011), and analytical and communication skill (P =.015. This result is consistent with multiple studies [32,33,34]. This might be attributed to the active engagements of students, the increased collaboration among themselves and instructors, and multimedia usage. However, it doesn’t show significant difference while comparing for aspects of ethics, and Interpersonal skills and responsibility. Possible explanation for absence of significance difference in in ethics, and interpersonal skills and responsibility could be, due to online learning modality and these aspects require extended time and follow up to ingrain and bring measurable change. Furthermore, this might be due to learning effect is weakened after concluding the intervention since the follow up and encouragement decreases, or the effect of post hoc test. This explanation is consistent with the study done on ethical issue and interpersonal skill of online education and the effect of online learning interactive skills [35, 36]. Another explanation, may go with the result. The aspects showed significant difference while comparing before and during (P =.001), and this might cause ceiling effect making no difference. This highlights the necessity of continuous support, mixture of multiple modality of teaching, follow up and feedback provision to students to gain long term retention.

Limitations and future research

This study assessed the research knowledge, and learning outcomes based on a retrospective record review of the students after learning with IRS-RBLO model among 4th year community public health students. This might result in limited scope and incomplete data which may fail to understand the broader perspectives. In addition, we used purposive sampling technique, and this may affect generalizability. Further study may explore the reflection of instructors, students’ parents, the sustainability of learning outcomes, and prospective studies on related subjects with a larger sample size.

Conclusion and recommendation

The IRS-RBLO model improved students’ research knowledge and learning outcomes in 16 weeks. The research knowledge before and during learning, and learning outcomes before, during, and after learning showed statistically significant differences in aspects of knowledge, cognitive skills, analytical and communication skills, and overall aspects. It is recommended that the model be applied in other departments when there is a need for online RBL and further study be conducted to explore the reflection of instructors and students’ parents and the sustainability of learning outcomes with prospective studies on related subjects incorporating a larger sample size and a mixture of different learning modalities.

Data availability

The dataset used and/or analyzed during the current study is available from the corresponding author upon reasonable request.

Abbreviations

IRS-RBLO:

Integrated research subject research based learning online

RBL:

Research-based learning

TQF:HEd:

Thai qualifications framework for higher education

UKPSF:

United kingdom professional standards framework

References

Alghamdi AA. Impact of the COVID-19 pandemic on the social and educational aspects of Saudi university students’ lives. PLoS ONE. 2021;16(4):e0250026.

Organization WH. Communicating and managing uncertainty in the COVID-19 pandemic: a quick guide. World Health Organization: Geneva, Switzerland. 2020.

Reimers F, Schleicher A, Saavedra J, Tuominen S. Supporting the continuation of teaching and learning during the COVID-19 pandemic. Oecd. 2020;1(1):1–38.

Kaewchandee C, Hnuthong U, Thinkan S, Rahman MS, Sangpoom S, Suwanbamrung C. The experiences of district public health officers during the COVID-19 crisis and its management in the upper Southern region of Thailand: A mixed methods approach. Heliyon. 2023;9(1).

Abuhassna H, Al-Rahmi WM, Yahya N, Zakaria MAZM, Kosnin ABM, Darwish M. Development of a new model on utilizing online learning platforms to improve students’ academic achievements and satisfaction. Int J Educational Technol High Educ. 2020;17:1–23.

Pedroso JEP, Herbuela KME, Prospects, Appropriateness, and Adoption of Online Learning.: The Case of Social Studies Education in A Public University in the Philippines. Journal homepage: www.ijrpr.com ISSN. 2022;2582:7421.

Malik RS. Educational challenges in 21st century and sustainable development. J Sustainable Dev Educ Res. 2018;2(1):9–20.

Yanti FA, Ruyani A, Yunitasari Y, Thohir MA, Kinasih A. Implementation Process of Research Based Learning (RBL) for Doctoral Education Program Students: An Exploratory Case Study. Formatif: Jurnal Ilmiah Pendidikan MIPA. 2024;14(1).

Ratnawati N, Idris I. Improving student capabilities through research-based learning innovation on e-learning system. Int J Emerg Technol Learn (iJET). 2020;15(4):195–205.

Brew A, Saunders C. Making sense of research-based learning in teacher education. Teach Teacher Educ. 2020;87:102935.

Reyk JV, Leasa M, Talakua M, Batlolona JR. Research based learning: added value in students’ science critical thinking skills. Jurnal Penelitian Pendidikan IPA. 2022;8(1):230–8.

Martin F, Bolliger DU. Designing online learning in higher education. Handbook of open, distance and digital education. 2023:1217-36.

Jenkins A, Healey M. Undergraduate research and international initiatives to link teaching and research. Council Undergrad Res Q. 2010;30(3).

Lockman AS, Schirmer BR. Online instruction in higher education: promising, research-based, and evidence-based practices. J Educ e-Learning Res. 2020;7(2):130–52.

Crawford-Ferre HG, Wiest LR. Effective online instruction in higher education. Q Rev Distance Educ. 2012;13(1):11.

Noguez J, Neri L. Based learning: a case study for engineering students. Int J Interact Des Manuf (IJIDeM). 2019;13(4):1283–95.

Jiraro P, Phothisuk A, Suwanaruji N, Klosawasdi P, Jiraro S. Research and development of a research base learning model (RBL) in teaching students of the faculty of education St. Theresa international college. St Theresa J Humanit Social Sci. 2020;6(2):194–209.

Sharma S. Nursing research and Statistics-E-Book. Elsevier Health Sciences. 2022.

Kandi V. Research process, study variables, statistical validations, and sampling methods in public health related research: an update. Am J Biomedical Res. 2022;10(1):1–8.

Polit D, Beck C. Essentials of nursing research: appraising evidence for nursing practice. Lippincott Williams & Wilkins; 2020.

Ihr D. The UK Professional Standards Framework for teaching and supporting learning in higher education. 2021.

Jaroenpool J, Suwanbamrung C, Sukkaew N, Dadras M. Students’ reflections on an integrated research subject using a research-based learning online model during the COVID-19 pandemic: a qualitative study in Thailand. Humanit Arts Social Sci Stud. 2023;23(3):695–707.

Harrington J, Beale J, Fancourt A, Lutz C. The’BrainCanDo’Handbook of teaching and learning: practical strategies to bring psychology and neuroscience. Into the Classroom: Routledge. 2020.

Poonputta A. Characteristics to Thai Qualifications Framework for Higher Education (TQF) of Undergraduate Students Learning in General Education Mahasarakham University. 2019;15:101 – 12.

Commission OotHE. Thai qualifications framework for higher education. Bangkok: Office of the Higher Education Commission. 2009.

Dam-O P, Sirisathitkul Y, Eadkhong T, Srivaro S, Sirisathitkul C, Danworaphong S. Online physics laboratory course: united Kingdom professional standards framework perspective from Walailak university. Thail Distance Educ. 2024;45(1):122–40.

Jalinus N, editor. The application of Research Based Learning Model in electric power system protection course. Journal of Physics: Conference Series 2019: IOP Publishing.

Almendingen K, Morseth MS, Gjølstad E, Brevik A, Tørris C. Student’s experiences with online teaching following COVID-19 lockdown: A mixed methods explorative study. PLoS ONE. 2021;16(8):e0250378.

Buxton PS. The retrospective record review: A methodological option for educational research. J Res Educ. 2021;30(2):17–34.

Yang D. Instructional strategies and course design for teaching statistics online: perspectives from online students. Int J STEM Educ. 2017;4:1–15.

Worapun W. The development of Research-Based learning management in the curriculum design and development course for teacher students. J Educ Learn. 2021;10(6):62–7.

Susiani K, Suastra IW, Arnyana IBP. Study of improving the quality of learning in an effort to improve the quality of elementary school education. Jurnal EDUCATIO: Jurnal Pendidikan Indonesia. 2022;8(1):37–44.

Tutal Ö, Yazar T. Active learning improves academic achievement and learning retention in K-12 settings: A Meta-Analysis. J School Educational Technol (JSCH). 2023;18(3).

Villegas-Ch W, García-Ortiz J, Sánchez-Viteri S. Personalization of learning: machine learning models for adapting educational content to individual learning styles. IEEE Access. 2024;12.

Milliken M, Winter R, Or J. A qualitative analysis of ethicality and interpersonal skills in online higher education. J Educators Online. 2023;20(4):n4.

Yusof R, Yin KY, Norwani NM, Ahmad NL, Ismail Z. Investigating the role of digital learning in enhancing educational values: online socialization and its effect on peer learning, collaborative skills and knowledge construction. Int J Learn Teach Educational Res. 2022;21(9):441–59.