Effects of COVID-19 on Japanese medical students’ knowledge and attitudes toward e-learning in relation to performance on achievement tests

Abstract

The COVID-19 pandemic forced many educational institutions to turn to electronic learning to allow education to continue under the stay-at-home orders/requests that were commonly instituted in early 2020. In this cross-sectional study, we evaluated the effects of the COVID-19 pandemic on medical education in terms of students’ attitudes toward online classes and their online accessibility; additionally, we examined the impacts of any disruption caused by the pandemic on achievement test performance based on the test results. The participants were 674 students (412 in pre-clinical, 262 in clinical) at Juntendo University Faculty of Medicine; descriptive analysis was used to examine the respondents’ characteristics and responses. The majority of respondents (54.2%) preferred asynchronous classes. Mann–Whitney U tests revealed that while pre-clinical students preferred asynchronous classes significantly more than clinical students (39.6%, p < .001), students who preferred face-to-face classes had significantly higher total achievement test scores (U = 1082, p = .021, r = .22). To examine the impacts of pandemic-induced changes in learning, we conducted Kruskal–Wallis tests and found that the 2020 and 2021 scores were significantly higher than those over the last three years. These results suggest that while medical students may have experienced challenges adapting to electronic learning, the impact of this means of study on their performance on achievement tests was relatively low. Our study found that if possible, face-to-face classes are preferable in an electronic learning environment. However, the benefit of asynchronous classes, such as those that allow multiple viewings, should continue to be recognized even after the pandemic.

About the Authors:

Miwa Sekine

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Division of Medical Education, Juntendo University Faculty of Medicine, Tokyo, Japan, Medical Technology Innovation Center, Juntendo University Faculty of Medicine, Tokyo, Japan

ORCID logo https://orcid.org/0000-0002-5201-3175

Makino Watanabe

Roles Data curation, Investigation, Methodology, Writing – review & editing

Affiliations Division of Medical Education, Juntendo University Faculty of Medicine, Tokyo, Japan, Department of Physiology, Juntendo University Faculty of Medicine, Tokyo, Japan

Shuko Nojiri

Roles Formal analysis, Investigation, Writing – review & editing

Affiliation: Medical Technology Innovation Center, Juntendo University Faculty of Medicine, Tokyo, Japan

Tsutomu Suzuki

Roles Data curation, Investigation, Methodology, Writing – review & editing

Affiliations Division of Medical Education, Juntendo University Faculty of Medicine, Tokyo, Japan, Department of Respiratory Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan

Yuji Nishizaki

Roles Formal analysis, Supervision, Writing – review & editing

Affiliations Division of Medical Education, Juntendo University Faculty of Medicine, Tokyo, Japan, Medical Technology Innovation Center, Juntendo University Faculty of Medicine, Tokyo, Japan

Yuichi Tomiki

Roles Investigation, Supervision, Writing – review & editing

Affiliations Division of Medical Education, Juntendo University Faculty of Medicine, Tokyo, Japan, Department of Coloproctological Surgery, Juntendo University Faculty of Medicine, Tokyo, Japan

Takao Okada

Roles Conceptualization, Data curation, Investigation, Methodology, Project administration, Supervision, Writing – review & editing

Affiliation: Division of Medical Education, Juntendo University Faculty of Medicine, Tokyo, Japan

Introduction

The pandemic arising from the COVID-19 disease, which emerged in November 2019 [1] in Wuhan, China, forced many educational institutions to halt their activities, and stay-at-home orders/requests were issued in many cities worldwide. As of May 1, 2021, more than 151 million cases had been documented all over the world, causing approximately 3.1 million deaths [2–5], with 592,709 positive cases and more than 10,293 deaths in Japan [6]. This highly contagious virus required social distancing to be imposed, and the operations of medical schools were affected by these rules. However, the same circumstances inspired creative innovation and ingenuity in the responses developed to this unprecedented crisis [7, 8].

Since the onset of the pandemic, numerous educational institutions have rapidly adapted to online or other forms of instruction to continue education [9], taking account of the risk of infection to students, staff, faculty, and teachers [10, 11]. Not only have institutions improvised entire curricula [12], constructed novel class forms [7, 13, 14], often they were required to select and install online education systems [15, 16]. Additionally, educators and administrative personnel needed to install appropriate systems and learn how to use these systems, often via faculty development [17]. It cannot be denied that there are both advantages and disadvantages to doing online classes using conventional systems, such as YouTube, learning management systems [18, 19], Zoom (Zoom Video Communications, Inc., San Jose, CA, USA), and other electronic means. However, the attitudes of medical students toward e-learning as well as accessibility may vary greatly by geographic region [20–22].

Clinical clerkships in Japanese medical schools incorporate a residency for fourth-year medical students, wherein they join a medical team that performs actual medical procedures and conducts clinical care. Before this step, students undergo a performance assessment, the Objective Structured Clinical Examination, and a knowledge assessment administered through computer-based testing using item response theory [23] are conducted at the end of pre-clerkship courses (in the fourth year), which includes basic clinical knowledge, introduction to clinical medicine, clinical skills, and clinical reasoning. These tests, developed by the Common Achievement Tests Organization [http://www.cato.umin.jp/], were implemented in 2005 as standardized tests [24, 25].

Japan has 82 medical schools, 31 of which are private. Juntendo University is one of these private medical schools. We prepared portable Wi-Fi for students who needed it, as well as a hybrid online/face-to-face system that allowed a small number of students to come to school and attend classes in a location with better internet connectivity, which those who could commute to the university could take advantage of. Many educational institutions turned to online lectures during the pandemic, and students attended them from home. Medical students, even those undergoing clinical training, were required to take online courses to avoid contact with high-risk patients (including pre-clinical students). Numerous studies have explored students’ mental health during the pandemic as well as the impacts of pandemic-induced changes on the medical education system. However, the attitude of medical students toward e-learning and academic performance, especially related to their performance in computer-based achievement tests, remain unknown. This study aimed to understand the impact of the COVID-19 pandemic on medical students and to identify any negative effects on their education using questionnaire surveys and the results of achievement tests. In particular, the study attempted to determine whether the pandemic had any negative impact on the academic performance based on the results of the computer-based achievement test. The findings of this study will help to provide a better understanding of effective educational responses and will be beneficial to educators and educational institutions across the world that may face similar challenges even after the end of the COVID-19 pandemic.

Methods

Study design

A cross-sectional study was conducted at Juntendo University Faculty of Medicine, Tokyo, Japan, using a self-administered questionnaire survey of second- to sixth-year medical students that was distributed from June to July 2020. The completed questionnaires were collected by one of the authors to ensure confidentiality and prevent bias. Official examination scores of computer-based testing from 2017 to 2021 were collected with the written consent of students and the approval of the university ethical review board (No. 2020187). All study procedures were conducted according to the principles of World Medical Association Declaration of Helsinki.

Participants

Paper-based questionnaires were provided to all 686 students (both male and female) in their second to sixth year at Juntendo University Faculty of Medicine from June to July 2020. Respondents who did not provide written informed consent documentation and freshman, who had no previous experience with Juntendo University classes, were excluded. Subsequently, 674 completed, valid questionnaires were collected (a response rate of 98%), 132 of which, those from fourth-year students, were used to conduct a comparison analysis with their test scores. Second- to fourth-year students were categorized as pre-clinical students; fifth- and sixth-year students were categorized as clinical students because they have passed the requisite examinations to become student doctors in their fourth year. Because the fourth-year students had not yet taken the requisite examinations when this study was conducted, they were considered pre-clinical students. The participants were mostly male. A summary of the distribution of responses is given in Table 1.

[Figure omitted. See PDF.]

Table 1. Basic characteristics of participants (n = 674).

https://doi.org/10.1371/journal.pone.0265356.t001

This questionnaire was designed to survey and assess internet access and attitudes toward online classes. Students were informed of this study’s purpose, guaranteed anonymity, and informed that their answers would have no effect on their grades; further, they were instructed that they could opt out of the project anytime if they no longer wished to participate.

Testing

The computer-based achievement (CBT) test for the fourth-year students consisted of 320 test questions (multiple-choice and extended matching items) selected randomly from a pool of validated test questions used by medical schools in Japan. Students were evaluated on a total of 240 out of those 320, with 80 being new trial questions under evaluation for future use. The questions were designed to evaluate basic clinical knowledge.

Statistical analysis

The collected data were managed using Microsoft Excel. IBM SPSS Statistics for Windows, Version 27.0. (Armonk, NY: IBM Corp) was used for statistical analysis. Descriptive analyses were used to examine respondents’ characteristics and responses. Frequencies and percentages were used for categorical variables, and means and standard deviations were used for continuous variables. The Pearson chi-square test was used to determine the correlation of variables based on gender or clinical/pre-clinical student groups. The correlation was further investigated using z test for the population proportion. The p-value threshold for significance was adjusted for multiple comparisons using the Bonferroni correction. Fourth-year students’ questionnaires were used to conduct comparison analysis with their test scores in 2020. The Shapiro–Wilk test and box plots were used to assess the normality of test scores. Using the test results for normality, the Mann–Whitney U tests was used to determine the significance of the differences in test scores based on questionnaire answers, and Kruskal–Wallis tests were used to examine the mean rank differences in CBT scores by year. The scores were further explored through the use of Dunn–Bonferroni tests. r and ε2 were used to determine the effect sizes for the Mann–Whitney U tests and the Kruskal–Wallis tests, respectively. The threshold for statistical significance was set at p < .05 for each analysis.

Results

Technology usage and availability

A large portion of students (84%) had connections to the internet at their residences even before the pandemic. A small fraction (6.4%) of students had unsatisfactory internet access. There were no significant differences in device usage between male and female; however, tablet usage was higher among clinical students than pre-clinical students (p < .001).

Medical students’ attitude toward online classes

The majority of respondents preferred asynchronous classes (54.2%). Female students rated the advantage of “being able to review” materials (p = .008) and “being able to watch multiple times” (p = .013) significantly more often than male students. A large proportion of medical students preferred asynchronous classes. Female students considered the “ease of keeping daily routine” (20.3%), “being able to meet friends” (27%), and “ease of asking questions” (7.1%) to be the main advantages of face-to-face classes. Male students found face-to-face classes’ “strict attendance requirement” to be a disadvantage (28.6%).

Clinical students reported that face-to-face classes had the advantage that they “give sense of presence/involvement” (p = .016), while pre-clinical students found face-to-face classes caused “difficulty staying focused” (p < .001).

Asynchronous classes were preferred by many students, especially pre-clinical students (X2 (4, N = 674) = 91.293, p < .001) with the reason being “convenience in scheduling class,” “being able to rewind and review,” “being able to watch multiple times,” and “being able to plan study schedule,” with the main disadvantage being “difficulty in keeping daily routine.” Synchronous classes such as Zoom were even less popular than face-to-face classes, and pre-clinical students reported that they found it difficult that synchronous classes made them “bound by class schedule” while clinical students found synchronous classes to be preferable, as they found that they created an “ease of asking questions” that was less recognized among pre-clinical students.

The characteristics of and differences between male and female participants are summarized in Table 2A, and the differences between pre-clinical students and clinical students are presented in Table 2B.

[Figure omitted. See PDF.]

Table 2. Internet accessibility, device usage, and attitude toward online classes during the COVID-19 pandemic.

https://doi.org/10.1371/journal.pone.0265356.t002

Score correlation with technology availability and attitude toward online classes

We investigated the correlations between students’ attitudes toward online classes, the availability of technology, and the corresponding total score (correct answer rate) for computer-based achievement test (CBT) (Table 3). It was found that the scores had a significant correlation with the questionnaire answers. Students who preferred face-to-face classes had significantly higher scores overall (p = .021) relative to students who preferred other methods. Students who found “ease of keeping daily routine” total score (p = .046) to be advantages in face-to-face classes had significantly higher scores.

[Figure omitted. See PDF.]

Table 3. Results of comparison of CBT score between students who selected answers listed below and students who selected otherwise.

https://doi.org/10.1371/journal.pone.0265356.t003

Comparison of test scores between 2020 and 2021 and the previous three years (2017–2019)

We used the Kruskal–Wallis test to analyze item response theory (IRT) standard scores and total scores (correct answer rate) in relation to other years wherein the examination format was the same as 2020 and 2021 to evaluate whether the pandemic had any effect on basic knowledge or test-taking ability. There were significant differences at the p < .05 level for the IRT standard score, H (4) = 42.74, p < .001, ε2 = .064; total score, H (4) = 27.87, p < .001, ε2 = .042, between the years. Post-hoc analyses using a Dunn–Bonferroni test indicated that test scores from 2020 and 2021 were significantly higher compared to the previous three years in the IRT standard scores: 2017 (p = .005), 2018 (p < .001), and 2019 (p = .004) against 2020; 2017 (p = .033), 2018 (p < .001), and 2019 (p = .027) against 2021; total score, 2017 (p = .016), 2018 (p < .001), 2019 (p = .014) against 2020; 2018 (p < .011) against 2021 (Figs 1 and 2). The results of the Kruskal–Wallis tests and a post-hoc analysis of the test scores are shown in Tables 4 and 5.

[Figure omitted. See PDF.]

Fig 1. A comparison of IRT standard test scores between 2020 and 2021 and 2017 through 2019.

Upper whisker represents maximum observation, lower whisker represents minimum observation, circle symbols represent outliers, upper line of box represents third quartile, middle line of box represents median, lower line of box represents first quartile, and x represents mean. Significance level for * < .05, ** < .01, and *** < .001.

https://doi.org/10.1371/journal.pone.0265356.g001

[Figure omitted. See PDF.]

Fig 2. Total scores between 2020 and 2021 and 2017 through 2019.

Upper whisker represents maximum observation, lower whisker represents minimum observation, circle symbols represent outliers, upper line of box represents third quartile, middle line of box represents median, lower line of box represents first quartile, and x represents mean. Significance level for * < .05, ** < .01, and *** < .001.

https://doi.org/10.1371/journal.pone.0265356.g002

[Figure omitted. See PDF.]

Table 4. Summary of Kruskal–Wallis test score comparison, 2017–2021.

https://doi.org/10.1371/journal.pone.0265356.t004

[Figure omitted. See PDF.]

Table 5. Dunn–Bonferroni test comparison of CBT sections between 2020 and 2021.

https://doi.org/10.1371/journal.pone.0265356.t005

Discussion

This study explored medical students’ attitudes toward online classes and evaluated the effects of the pandemic on fourth-year students’ performance based on the CBT achievement test.

It was found that a large portion of students use computers rather than tablets or smart phones to take classes. Internet access was relatively high, but some students were not satisfied. Additionally, a significant difference was found in tablet use between pre-clinical students (33.4% of all pre-clinical students) and clinical students (47.1% of all clinical students), perhaps because students more commonly use tablets in clinical clerkship duties. Interestingly, when asked about their preference for class delivery, clinical students reported preferring asynchronous lectures less often than pre-clinical students, while pre-clinical students were more likely to prefer asynchronous lectures, such as on-demand classes. In asynchronous learning, students receive a lecture form through e-mail or a learning management system at times that can be convenient for students; additionally, the costs are relatively low, and the learning methods and schedules are flexible. On the other hand, this method may isolate students, and for many, a flexible schedule is less necessary [26]. New challenges have also emerged on the teaching side: some students do not turn on their cameras during synchronous classes [27], and teachers report an increase in the difficulty of evaluating students online [28], especially in competency-based medical education [29].

Synchronous online lectures, such as those conducted over Zoom, are bound by lecture times and their delivery is affected by students’ financial situation as well, such as in the internet speed, computer specifications, and in some cases, packet communication fees. This may be a reason why students do not turn on their cameras during classes [27], and it may also be a reason why many students prefer asynchronous classes, as well as not being satisfied with the online environment.

During the pandemic, many educational institutions have struggled to maintain their teaching quality. Both synchronous and asynchronous online classes may be a burden on both students and faculty, both financially and technically, and several sociological, financial, and technological barriers also affect students’ motivation [30–32]. Prior learning experiences have a positive effect on students’ evaluation and satisfaction with current online education [33]. If utilized properly, e-leaning can foster a range of abilities [34], such as creativity in children [35].

The correlation of CBT score with technology availability and attitude toward online classes shows that students who preferred face-to-face classes scored significantly higher than other students. This supports a report that indicated that students felt they were able to learn better in physical classrooms [22]. Further, students who reported that face-to-face classes made it easy to maintain their daily routines scored significantly higher on total score than students who answered otherwise. This shows that, if possible, face-to face classes that allow students to maintain their daily routines would be preferable to maximize education.

Although COVID-19 disrupted many lectures and training sessions, a comparison of the CBT results suggests that students were able to maintain their motivation at a relatively high level. Further exploration of Kruskal–Wallis test in each CBT sections (S1 Table) showed that five out of six sections were significantly higher in 2020 or 2021 (S2 Table). From the end of July to early August every year, Japanese medical schools hold the All-Japan Medical Students Athletic Meet, in which the majority of medical students participate. However, in 2020 and 2021, this tournament was canceled due to the pandemic, which led us to speculate that it is possible that scores were higher than in previous years because students were able to study more in the absence of preparation for the tournament. A comparison between the test results for participants and non-participants of this athletic meet over the past three years indicates that non-participants scored higher in 2019 (S3 Fig). This suggests that we cannot deny the possibility that students simply had more time to study due to the pandemic. There have been reports on the effects on mental health of the stay-home situation as well as isolation effects due to the spread of COVID-19 [36–38], there have also been reports that medical students’ motivations, attitudes, other mental health indicators, and even academic performance have remained relatively high [11, 15]. Further, unexpected positive effects of the pandemic, including innovative educational methods and increased motivation to learn, have also been reported [39, 40].

Since the pandemic began, numerous studies have been conducted on the difficulties that students and educators have encountered with e-learning as a result of the sudden shift to online learning. This study provides contributions to educators and to educational institutions to better understand students’ academic performance during COVID-19 as well as to provide better preparation for possible future educational crises similar to this pandemic. Consequently, we believe that further study of the long-term effects of the pandemic on students’ performance will be beneficial to the educational field.

Conclusions

During the early phase of the pandemic, students and faculty alike experienced confusion and dread regarding online classes. Their dissatisfaction with the internet learning environment may have been due to sociological or financial problems during the pandemic, with the possibility of severe, disruptive financial change. A small number of dissatisfied students is never a small amount of casualties for an educational institution but can rather entail grave damage that signifies the need for prompt remedies. This study provides a glimpse of a way in which medical students are more resilient than was thought. This study concludes that while the pandemic caused disruptive effects on medical education, it is also the catalyst for unexpected innovation and inspiration, as well as the motivation to adapt and improve. Be that as it may, now that the situation has subsided and e-learning is the new standard style of education, we propose that educational facilities consider making recorded classes available for students to review multiple times, as well as providing additional assistance in cases of inadequate environments, such as by creating spaces for students to attend/watch classes if the internet environment is insufficient for learning.

In this study, we observed that most medical students had access to medical education via e-learning and were able to use electronic devices to attend classes in Juntendo University. However, our study was conducted in a single private medical school in a particular setting in a single country. Each country has exhibited its own characteristic response to the spread of COVID-19 in terms of allowance to the flow and movement of people and of course vaccination accessibility; therefore, individual countries’ regulations and technological infrastructure, economic situations, and so on are undeniably different. Thus, the results presented here might not be generalizable to other settings or countries and require further study. As of this writing (September 2021), the COVID-19 pandemic is ongoing, and for that reason, the study setting and its design were somewhat limited. As we face long-term restrictions of movement, coupled with social distancing, either voluntary or otherwise, we must take maximal measures to minimize the negative effects on education. We believe that our findings shed light on students’ educational performance and attitudes toward learning during the pandemic and could bear fruit used in further studies to investigate optimal pre-clinical and clinical education and facilitate students’ growth.

Supporting information

S1 Fig. Internet accessibility, device usage, and attitude toward online classes during the COVID-19 pandemic.

(a). Comparisons Based on Gender. Percentage is of valid N (Male, Female).

https://doi.org/10.1371/journal.pone.0265356.s001

(TIF)

S2 Fig. Internet accessibility, device usage, and attitude toward online classes during the COVID-19 pandemic.

(b). Comparisons Based on Students’ Education Level. Percentage is of valid N (clinical, pre-clinical).

https://doi.org/10.1371/journal.pone.0265356.s002

(TIF)

S3 Fig. Comparison of CBT score for the last 5 years between All-Japan students athletic meet participants and non-participants.

Significance level for *< .05, **< .01. Error bars represent standard deviation.

https://doi.org/10.1371/journal.pone.0265356.s003

(TIF)

S1 Table. Test sections.

https://doi.org/10.1371/journal.pone.0265356.s004

(DOCX)

S2 Table. Summary of the Kruskal–Wallis test of CBT sections test score comparison, 2017–2021.

https://doi.org/10.1371/journal.pone.0265356.s005

(DOCX)

S3 Table. Dunn–Bonferroni test comparisons of CBT sections between 2020 and 2021.

https://doi.org/10.1371/journal.pone.0265356.s006

(DOCX)

S1 File. Questionnaire raw data.

https://doi.org/10.1371/journal.pone.0265356.s007

(SAV)

S2 File. CBT raw data.

https://doi.org/10.1371/journal.pone.0265356.s008

(SAV)

Acknowledgments

The authors would like to thank Enago (www.enago.jp) for the English language review. We thank the students who participated in this study. We also thank David Aune for his insight and English editing.

Citation: Sekine M, Watanabe M, Nojiri S, Suzuki T, Nishizaki Y, Tomiki Y, et al. (2022) Effects of COVID-19 on Japanese medical students’ knowledge and attitudes toward e-learning in relation to performance on achievement tests. PLoS ONE 17(3): e0265356. https://doi.org/10.1371/journal.pone.0265356

References

1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A Novel Coronavirus from Patients with Pneumonia in China, 2019. The New England journal of medicine. 2020;382(8):727–33. pmid:31978945

2. COVID Live Update: 152,899,371 Cases and 3,208,499 Deaths from the Coronavirus—Worldometer. 2021. [Cited 2021 May 1].

3. Coronavirus COVID-19 (2019-nCoV) 2021[Cited 2021 May 1]. Available from: https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6.

4. @sprinklr. WHO Coronavirus (COVID-19) Dashboard 2021[Cited 2021 May 1]. Available from: https://covid19.who.int.

5. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020;20(5):533–4. pmid:32087114

6. Situation report 2021[Cited 2021 May 1]. Available from: https://www.mhlw.go.jp/stf/covid-19/kokunainohasseijoukyou_00006.html.

7. Cooke PC, Hajamohideen N, Gooneratne H. Developing a blended learning postgraduate teaching programme in anaesthesia: pandemic and beyond. Postgraduate medical journal. 2021. pmid:33931560

8. Woolliscroft JO. Innovation in Response to the COVID-19 Pandemic Crisis. Acad Med. 2020;95(8):1140–2. pmid:32282372

9. Kaul V, Gallo De Moraes A, Khateeb D, Greenstein Y, Winter G, Chae J, et al. Medical Education During the COVID-19 Pandemic. Chest. 2021;159(5):1949–60. pmid:33385380

10. Farooq F, Rathore FA, Mansoor SN. Challenges of Online Medical Education in Pakistan During COVID-19 Pandemic. J Coll Physicians Surg Pak. 2020;30(6):67–9. pmid:32723456

11. Rajab MH, Gazal AM, Alkattan K. Challenges to Online Medical Education During the COVID-19 Pandemic. Cureus. 2020;12(7):e8966. pmid:32766008

12. Ross DA. Creating a "Quarantine Curriculum" to Enhance Teaching and Learning During the COVID-19 Pandemic. Acad Med. 2020;95(8):1125–6. pmid:32744816

13. Li Y, Zhang X, Dai DY, Hu W. Curriculum Innovation in Times of the COVID-19 Pandemic: The Thinking-Based Instruction Theory and Its Application. Front Psychol. 2021;12:601607. pmid:33912100

14. Gordon M, Patricio M, Horne L, Muston A, Alston SR, Pammi M, et al. Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Medical Teacher. 2020;42(11):1202–15. pmid:32847456

15. T IN, Goodwin RL, Fowler L, McPhail B, Black AC Jr., An adaptive blended learning model for the implementation of an integrated medical neuroscience course during the Covid-19 pandemic. Anat Sci Educ. 2021. pmid:33915035

16. Yeo SC, Lai CKY, Tan J, Gooley JJ. A targeted e-learning approach for keeping universities open during the COVID-19 pandemic while reducing student physical interactions. PLoS One. 2021;16(4):e0249839. pmid:33831082

17. Swaminathan N, Govindharaj P, Jagadeesh NS, Ravichandran L. Evaluating the effectiveness of an online faculty development programme for nurse educators about remote teaching during COVID-19. J Taibah Univ Med Sci. 2021;16(2):268–73. pmid:33897333

18. Ruiz JG, Mintzer MJ, Leipzig RM. The Impact of E-Learning in Medical Education. Academic Medicine. 2006;81(3):207–12. pmid:16501260

19. Kasim NNM, Khalid F. Choosing the Right Learning Management System (LMS) for the Higher Education Institution Context: A Systematic Review. International Journal of Emerging Technologies in Learning. 2016;11(6).

20. Alsoufi A, Alsuyihili A, Msherghi A, Elhadi A, Atiyah H, Ashini A, et al. Impact of the COVID-19 pandemic on medical education: Medical students’ knowledge, attitudes, and practices regarding electronic learning. PLoS One. 2020;15(11):e0242905. pmid:33237962

21. Olum R, Atulinda L, Kigozi E, Nassozi DR, Mulekwa A, Bongomin F, et al. Medical Education and E-Learning During COVID-19 Pandemic: Awareness, Attitudes, Preferences, and Barriers Among Undergraduate Medicine and Nursing Students at Makerere University, Uganda. Journal of Medical Education and Curricular Development. 2020;7:238212052097321.

22. Chakraborty P, Mittal P, Gupta MS, Yadav S, Arora A. Opinion of students on online education during the COVID ‐19 pandemic. Human Behavior and Emerging Technologies. 2021;3(3):357–65.

23. Nitta Y, Maekawa S, Yanagimoto T, Maeda T, Yoshida M, Nara N, et al. Statistical Analysis with the Item-Response Theory of the First Trial of the Computer-Based Nationwide Common Achievement Test in Medicine. Medical Education. 2005;36(1):3–9.

24. Horita S, Park YS, Son D, Eto M. Computer-based test (CBT) and OSCE scores predict residency matching and National Board assessment results in Japan. BMC Med Educ. 2021;21(1):85. pmid:33531010

25. Nakaya H. 3. Medical Education in Japan and National Medical Practitioner Examination. Igaku Kyoiku / Medical Education (Japan). 2015;46(1):14–7.

26. Advantages Srivastava P. & disadvantages of e-education & e-learning. Journal of Retail Marketing & Distribution Management. 2019;2(3):22–7. pmid:31154955

27. Castelli FR, Sarvary MA. Why students do not turn on their video cameras during online classes and an equitable and inclusive plan to encourage them to do so. Ecol Evol. 2021;11(8):3565–76. pmid:33898009

28. Prigoff J, Hunter M, Nowygrod R. Medical Student Assessment in the Time of COVID-19. J Surg Educ. 2021;78(2):370–4. pmid:32819868

29. Nimavat N, Singh S, Fichadiya N, Sharma P, Patel N, Kumar M, et al. Online Medical Education in India–Different Challenges and Probable Solutions in the Age of COVID-19. Advances in Medical Education and Practice. 2021;12:237–43. pmid:33692645

30. Arima M, Takamiya Y, Furuta A, Siriratsivawong K, Tsuchiya S, Izumi M. Factors associated with the mental health status of medical students during the COVID-19 pandemic: a cross-sectional study in Japan. BMJ Open. 2020;10(12):e043728. pmid:33303472

31. Motte-Signoret E, Labbé A, Benoist G, Linglart A, Gajdos V, Lapillonne A. Perception of medical education by learners and teachers during the COVID-19 pandemic: a cross-sectional survey of online teaching. Med Educ Online. 2021;26(1):1919042. pmid:33871308

32. Tang YM, Chen PC, Law KMY, Wu CH, Lau YY, Guan J, et al. Comparative analysis of Student’s live online learning readiness during the coronavirus (COVID-19) pandemic in the higher education sector. Comput Educ. 2021;168:104211. pmid:33879955

33. Wang C, Xie A, Wang W, Wu H. Association between medical students’ prior experiences and perceptions of formal online education developed in response to COVID-19: a cross-sectional study in China. BMJ Open. 2020;10(10):e041886. pmid:33122327

34. Mittal P, Raghuvaran S. Entrepreneurship education and employability skills: the mediating role of e-learning courses. Entrepreneurship Education. 2021;4(2):153–67.

35. Yadav S, Chakraborty P, Mittal P. Designing drawing apps for children: Artistic and technological factors. International Journal of Human–Computer Interaction. 2022;38(2):103–17.

36. Blake H, Knight H, Jia R, Corner J, Morling JR, Denning C, et al. Students’ Views towards Sars-Cov-2 Mass Asymptomatic Testing, Social Distancing and Self-Isolation in a University Setting during the COVID-19 Pandemic: A Qualitative Study. Int J Environ Res Public Health. 2021;18(8). pmid:35010280

37. Dodd RH, Dadaczynski K, Okan O, McCaffery KJ, Pickles K. Psychological Wellbeing and Academic Experience of University Students in Australia during COVID-19. Int J Environ Res Public Health. 2021;18(3). pmid:35010280

38. Arora A, Chakraborty P, Bhatia MPS, Mittal P. Role of Emotion in Excessive Use of Twitter During COVID-19 Imposed Lockdown in India. Journal of Technology in Behavioral Science. 2021;6(2):370–7. pmid:33102690

39. Gonzalez T, de la Rubia MA, Hincz KP, Comas-Lopez M, Subirats L, Fort S, et al. Influence of COVID-19 confinement on students’ performance in higher education. PLoS One. 2020;15(10):e0239490. pmid:33035228

40. Sani I, Hamza Y, Chedid Y, Amalendran J, Hamza N. Understanding the consequence of COVID-19 on undergraduate medical education: Medical students’ perspective. Ann Med Surg (Lond). 2020;58:117–9. pmid:32983429