The Factors Affecting Students’ Online Learning Outcomes during the COVID-19 Pandemic: A Bayesian Exploratory Factor Analysis

Abstract

The COVID-19 pandemic, a public health crisis of worldwide importance, announced by the World Health Organization (WHO) in January 2020 as an outbreak, has made distance education through the E-learning system an urgent and irreplaceable requirement. The study assessed factors affecting students’ online learning outcomes during the COVID-19 pandemic through interviews with 404 students who were subjects of the survey using the convenience sampling method via questionnaires. The study utilized the reliability analysis through Cronbach’s Alpha and the Bayesian Exploratory Factor Analysis (BEFA). The evaluation results of the research scales showed that 28 observed variables were used to measure 7 research concepts. Test results of the hypotheses showed that students’ online learning outcomes are affected by 6 factors in the descending order, respectively, learner characteristics, perceived usefulness, course content, course design, ease of use, and faculty capacity.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Recently, advances in modern computer and network technology have driven the development of distance education [1]. In addition, the COVID-19 pandemic, a public health crisis of worldwide importance, announced by the World Health Organization (WHO) in January 2020 as an outbreak, has made distance education through the E-learning system an urgent and irreplaceable requirement. Despite the current pandemic that is hindering education worldwide, online learning based on Internet services has become available and universal, facilitating the learning system. Colleges and universities use online resources to continue their educational journey through software applications such as Zoom and Microsoft Teams.

As a result, the effectiveness of E-learning and students’ online learning outcomes become a matter of concern for universities in particular and the society in general. In fact, there has been a significant increase in research on factors affecting students’ online learning outcomes. According to [2], improved communication technologies enable easy learning systems since access to social media is a beneficial source of information and communication. Online technology is seen as an active element of both students’ and lecturers’ learning systems. During the pandemic era, several nations used television broadcasts and online sources to promote distance education. Prioritizing distance education primarily through online systems is a “model change in education.” The jammed education wheel causes certain instabilities regarding learners’ future, emphasizing the importance of technology in our lives. Online learning is a useful tool to overcome the challenges of the pandemic crisis in particular and other difficulties in general [3]. However, many argued that online learning is an education crisis today. Most learners are not interested in online learning due to limited interactions, unstable sound and visual quality due to dependence on Internet quality, and technological equipment not meeting demand. Therefore, this study aimed to explore factors that affect students’ outcomes during the online learning process.

Previous studies on the factors affecting students’ online learning outcomes used the traditional exploratory factor analysis (EFA) method to identify the representative factors. This study will contribute to the existing empirical literature by integrating the Bayesian approach to traditional EFA that simultaneously selects the dimension of the factor model, the allocation of manifest variables to factors, and the factor loadings. Theoretically, traditional EFA is divided into four steps: (i) choosing the dimension of the factor model; (ii) allocating manifest variables to factors; (iii) estimating factor loadings; and (iv) discarding measurements that load on multiple factors. There are several methods for selecting the dimension of the latent factors to extract and rotate factors [4–6]. However, each of the dimensions selected by analysts at each stage of a traditional EFA has substantial consequences on the estimated factor structure [7]. To overcome this problem, Conti et al. [7] proposed not to choose the number of factors in the first step but to choose factors together with other parameters by the Bayesian approach. Besides, by this approach, the allocation of manifest variables to factors will base the model with the highest probability. These are the fundamental ideas that prompted us to conduct this study.

The study is structured as follows: A literature review is provided in Section 2, followed by research methodology in Section 3, while empirical results are described in Section 4. Finally, the conclusion and policy implications are reported in Section 5.

2. Literature Review

The theory of factors affecting online learning outcomes of students in particular and the effectiveness of using technology, in general, is derived from the technology acceptance model (TAM) proposed in [8]. Davis proposed TAM to explain people’s attitudes and behaviors in adopting technology in the presence of other external variables. This model is often applied in the study of technology use behavior to understand the reasons for accepting or rejecting information systems. Information technology plays a prominent role in teaching as it can encourage innovation, provide new learning spaces, and transform teaching activities [9, 10], all associated with the ease of IT operations. Ease of operation, user experience convenience, and proficiency in information technologies directly affect users’ perception and motivation to learn [11]. Studies have proven that factors in TAM such as perceived ease of use and perceived usefulness positively impact student learning outcomes.

2.1. Perceived Ease of Use

Online learning platforms are designed for the purpose of knowledge sharing and learning. Today, as we live in a globalized world, using technology to obtain knowledge, acquiring information, and learning has become a daily need [12]. These sources are easy to use and accessible, facilitating knowledge-sharing processes. Many studies have shown that ease of use, accessibility, and transmission speed of online media and mobile devices are an important part of the learning process. Increased online learning adaptability is due to easing access, thus resulting in positive outcomes [13, 14]. Based on these rationales, the following hypothesis is designed for this study.

H₁: perceived ease of use has a positive effect on students’ online learning outcomes.

2.2. Perceived Usefulness

Perceived usefulness is the degree to which learners believe that the use of online learning will help improve their performance [8]. The usefulness of online learning is demonstrated by helping learners save travel time and travel costs and access a variety of methods [15]. Many studies have shown that perceived usefulness positively impacts learners’ attitudes and motivation, thereby improving learning outcomes [2, 13]. Based on these rationales, the following hypothesis is designed for this study.

H₂: perceived usefulness has a positive effect on students’ online learning outcomes.

2.3. Faculty Capacity

The approach in the online learning process is learner-centered rather than teacher-centered as in traditional education [16]. Pedagogical methods, professional competence, science and technology application level, the ability to form and combine different ideas, and practices in developing online course contents in higher education help students achieve better learning outcomes [17–20]. Based on these rationales, the following hypothesis is designed for this study.

H₃: faculty capacity has a positive effect on students’ online learning outcomes.

2.4. Course Content

Engaging course content attracts lots of participation and proactiveness among students, thereby influencing learning outcomes [21, 22]. The E-learning content includes the structure and content of chapters of learning materials. Besides, the E-learning content also includes additional materials to help students understand more clearly and deeply about the knowledge [23]. This factor facilitates the improvement of student’s analytical and critical thinking and problem-solving skills [24]. Based on these rationales, the following hypothesis is designed for this study.

H₄: course content has a positive effect on students’ online learning outcomes.

2.5. Course Design

E-learning course design includes structure, course design interface, testing and evaluation methods, and exchange forums between lecturers and learners. A good course design will attract and facilitate students to learn through online classes [25]. The course design interface is used to introduce course content, designed according to student’s competence and level of understanding, and appropriate in terms of time and space to promote and support the self-study process [26–28]. Based on these rationales, the following hypothesis is designed for this study.

H₅: course design has a positive effect on students’ online learning outcomes.

2.6. Learner Characteristics

Social interaction with lecturers and with co-learners is imperative to achieve better online learning quality. Through strong interaction and consistent practice, the effectiveness of online learning can be achieved [29–31]. In addition, proactiveness, self-study ability, and sense of compliance are important requirements for achieving better learning outcomes since regulations and requirements of online learning are more comfortable. The process is more difficult to control than traditional methods. Based on these rationales, the following hypothesis is designed for this study.

H₆: learner characteristics have an effect on students’ online learning outcomes.

3. Research Methodology

3.1. Research Model

The theoretical framework denoting the study hypotheses as presented in Figure 1 was derived based on the literature discussed above.

[figure omitted; refer to PDF]

Hence, BEFA extracted 6 factors and the observed variables in each factor had a factor loading coefficient greater than 0.5. The specific factors are as follows:

(i) The first factor includes observed variables EOU1, EOU2, EOU3, and EOU4 representing ease of use. Name this factor as EOU, and calculate it as the mean of the component observed variables.

(ii) The second factor includes observed variables PU1, PU2, PU3, and PU4 representing perceived usefulness. Name this factor as PU, and calculate it as the mean of the component observed variables.

(iii) The third factor includes observed variables FC1, FC2, FC3, and FC4 representing faculty capacity. Name this factor as FC, and calculate it as the mean of the component observed variables.

(iv) The fourth factor includes observed variables CC1, CC2, CC3, and CC4 representing course content. Name this factor as CC, and calculate it as the mean of the component observed variables.

(v) The fifth factor includes observed variables CD1, CD2, CD3, and CD4 representing course design. Name this factor as CD, and calculate it as the mean of the component observed variables.

(vi) The sixth factor includes observed variables LC1, LC2, LC3, and LC4 representing learner characteristics. Name this factor as LC, and calculate it as the mean of the component observed variables.

Second, we use the traditional EFA method for observed variables representing students’ online learning outcomes. This method is used because these observed variables only measure one factor, students’ online learning outcomes. The results are explained in Figure 6 and Table 5.

[figure omitted; refer to PDF]

Table 5

EFA results with factor students’ online learning outcomes.

	Factor	Cronbach’s Alpha
SP1	0.878	0.840
SP3	0.842
SP2	0.817
SP4	0.750
Eigenvalues	2.710	KMO = 0.802
Extraction sum of squared loadings	67.75%	Bartlett’s test
		Sig. = 0.001

The scree plot in Figure 6 shows that the number of factors is 1, with an eigenvalue of 2.710, greater than 1. Besides, Table 5 shows that the KMO coefficient of 0.802 is greater than 0.5 and less than 1, indicating that the EFA method is in agreement with the actual data. Bartlett’s test shows that observed variables correlate with the factor. This factor includes observed variables SP1, SP2, SP3, and SP4 representing students’ online learning outcomes. Name this factor as SP, and calculate it as the mean of the component observed variables.

4.4. Multivariate Regression Analysis (OLS)

We used multivariate regression analysis based on the least squares method (OLS) to evaluate the factors affecting students’ online learning outcomes and test the hypotheses. The results are shown in Table 6.

Table 6

Multivariate regression analysis results.

Variable	Unstandardized coefficients		Standardized coefficients	t	Sig.	Collinearity statistics
	Beta	Std. error				Tolerance	VIF
(Constant)	−0.618	0.192	—	−3.221	0.001	—	—
EOU	0.150	0.039	0.137	3.822	0.001	0.789	1.267
PU	0.290	0.045	0.242	6.480	0.001	0.729	1.372
FC	0.098	0.037	0.098	2.640	0.009	0.739	1.353
CC	0.215	0.044	0.182	4.943	0.001	0.751	1.331
CD	0.150	0.034	0.146	4.398	0.001	0.920	1.087
LC	0.296	0.039	0.335	7.530	0.001	0.514	1.945

R² = 59.7%, F-test: $p$ value = 0.001, Durbin–Watson d = 2.020, Breusch–Pagan/Cook–Weisberg test for heteroskedasticity: $p$ value = 0.1709.

Table 6 shows that the model does not have multicollinearity because the corresponding VIF values for the independent variables in the model are less than 5 [38]. Besides, the Durbin–Watson d has a value of 2.020, which is close to 2, so the model does not have autocorrelation. Finally, the Breusch–Pagan/Cook–Weisberg test has a $p$-value of 0.1709, which is greater than the 5% significance level, so the model does not have heteroskedasticity.

Table 6 also shows that the regression coefficients of the variables EOU, PU, FC, CC, CD, and LC all have $p$ values greater than the 5% significance level. Thus, the variables EOU, PU, FC, CC, CD, and LC all have an impact on the dependent variable SP. In other words, ease of use, perceived usefulness, faculty capacity, course content, course design, and learner characteristics affect students’ online learning outcomes. In addition, the regression coefficients of these variables are all positive. These results show that hypotheses H1, H2, H3, H4, H5, and H6 are correct.

Finally, the standardized coefficients in Table 6 show that the order of impact of these factors on students’ online learning outcomes from strong to weak is as follows: learner characteristics, perceived usefulness, course content, course design, ease of use, and faculty capacity. The impact of each factor on students’ online learning outcomes is shown in Figure 7.

[figure omitted; refer to PDF]

5. Conclusion and Policy Implications

The official research was also conducted using quantitative research methods with 404 respondents who are students in Ho Chi Minh City using the convenience sampling method with detailed questionnaires. The study utilized the reliability analysis through Cronbach’s Alpha and BEFA methods. Our empirical results proved that students’ outcomes during the online learning process are affected by 6 factors in the descending order, respectively, learner characteristics, perceived usefulness, course content, course design, ease of use, and faculty capacity (see Table 7). This result is also similar to that of studies in [2, 13, 14, 28, 31].

Table 7

Factors affecting students’ online learning outcomes during the COVID-19 pandemic.

Code	Explanation	Factor	Coefficient	Hypothesis
EOU1	Online learning software is easy to use	Ease of use	0.150	H1 accepted
EOU2	Online learning software is easy to understand
EOU3	Online learning software is flexible
EOU4	Online learning software comes with a support team when needed
PU1	Using E-learning makes learning easier	Perceived usefulness	0.290	H2 accepted
PU2	Using E-learning saves time
PU3	Using E-learning saves costs
PU4	Using E-learning creates more learning excitement
FC1	Appropriate teaching methods	Faculty capacity	0.098	H3 accepted
FC2	Ability to apply science and technology
FC3	Ability to form and combine different ideas and practices
FC4	Professional competence
CC1	Course content at the suitability level	Course content	0.215	H4 accepted
CC2	Diverse learning and supporting materials
CC3	Innovative and updated subject content
CC4	Practical and comprehensive subject content and structure
CD1	Appropriate course design structure and interface	Course design	0.150	H5 accepted
CD2	Flexible time schedule
CD3	Appropriate testing and evaluation methods
CD4	Convenient exchange forums
LC1	Social interaction with lecturers and collaborative interaction with co-learners	Learner characteristics	0.296	H6 accepted
LC2	Quick adaptability to changes
LC3	Proactiveness and self-study ability
LC4	Sense of regulatory compliance

The study helped educators, lecturers, and students understand the importance of factors affecting students’ outcomes during the online learning process, thereby forming policies that focus on organizing, designing, and conducting online courses in particular and higher education in general. First, for students’ online learning to be successful, the university must hold training sessions to improve students’ initiative, encourage students to actively interact with lecturers and classmates, and improve students’ self-study ability. Besides, through training sessions, schools need to help students realize the usefulness of online learning, especially in the context of the COVID-19 pandemic. The online learning system should be built with a friendly and easy-to-use interface and diverse learning programs through the E-learning system, should improve system accessibility, should allow students to actively register, and should be flexible about the time to use.

Although this study accomplished its original goal, it does have some limitations. To begin with, because the new study was conducted on a small scale, generalizability may be limited. Second, the study focuses primarily on factors related to the online learning system, but it does not assess factors outside the system, such as the school’s incentive policy, communication quality, student support, and family circumstances. These are the limitations that should be addressed in future research.

Authors’ Contributions

Dr. Hoang Anh Le conceived the idea and wrote Introduction, Literature Review, Research Methodology sections. Thi Tinh Thuong Pham MSc. wrote Empirical Results and Conclusion and Policy Implications. Dr. Doan Trang Do wrote Research Methodology.

References

[1] L. Li, "Analysis of psychological factors affecting the quality of online learning," Advanced Materials Research, vol. 926–930, pp. 4461-4464, DOI: 10.4028/www.scientific.net/amr.926-930.4461, 2014.

[2] M. Habes, S. A. Salloum, M. Alghizzawi, M. S. Alshibly, "The role of modern media technology in improving collaborative learning of students in Jordanian universities," International Journal of Information Technology and Language Studies, vol. 2 no. 3, 2019. https://journals.sfu.ca/ijitls/index.php/ijitls/article/view/51

[3] O. Almuhaisen, M. Habes, M. Alghizzaw, "An empirical investigation the use of information, communication technologies to English language acquisition: a case study from the Jordan," International Journal of Innovations in Engineering Research and Technology, vol. 7 no. 5, pp. 261-269, 2020.

[4] A. Costello, J. Osborne, "Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis," PARE, vol. 10, 2005. https://doi.org/10.7275/JYJ1-4868

[5] R. L. Gorsuch, "Factor analysis," Handbook of Psychology: Research Methods in Psychology, vol. 2, pp. 143-164, DOI: 10.1002/0471264385.wei0206, 2003.

[6] R. I. Jennrich, "3-rotation algorithms: from beginning to end," Handbook of Latent Variable and Related Models, pp. 45-63, DOI: 10.1016/b978-044452044-9/50006-9, 2007.

[7] G. Conti, S. Frühwirth-Schnatter, J. J. Heckman, R. Piatek, "Bayesian exploratory factor analysis," Journal of Econometrics, vol. 183 no. 1, pp. 31-57, DOI: 10.1016/j.jeconom.2014.06.008, 2014.

[8] F. D. Davis, "Perceived usefulness, perceived ease of use, and user acceptance of information technology," MIS Quarterly, vol. 13 no. 3, pp. 319-340, DOI: 10.2307/249008, 1989.

[9] F. A. A. Eksail, E. Afari, "Factors affecting trainee teachers’ intention to use technology: a structural equation modeling approach," Education and Information Technologies, vol. 25 no. 4, pp. 2681-2697, DOI: 10.1007/s10639-019-10086-2, 2020.

[10] Y.-W. Liao, Y.-M. Huang, S.-H. Huang, H.-C. Chen, C.-W. Wei, "Exploring the switching intention of learners on social network-based learning platforms: a perspective of the push-pull-mooring model," Eurasia Journal of Mathematics, Science and Technology Education, vol. 15 no. 9, 2019. https://doi.org/10.29333/ejmste/108483

[11] J. G. Chaka, I. Govender, "Students’ perceptions and readiness towards mobile learning in colleges of education: a Nigerian perspective," South African Journal of Education, vol. 37 no. 1,DOI: 10.15700/saje.v37n1a1282, 2017.

[12] N. Bakhuisen, "Knowledge sharing using social media in the workplace," 2012. https://www.semanticscholar.org/paper/Knowledge-Sharing-using-Social-Media-in-the-Bakhuisen/448bd43d5db562592313d0d5735dc8a70066bba4

[13] S. A. Khadija Alhumaid, A. Waheed, E. Zahid, M. Habes, "COVID-19 & elearning: perceptions & attitudes of teachers towards E-learning acceptance in the developing countries," Multicultural Education, vol. 6 no. 2, 2020. https://doi.org/10.5281/zenodo.4060121

[14] S. A. Salloum, M. Al-Emran, M. Habes, M. Alghizzawi, M. A.. Ghani, K. Shaalan, "Understanding the impact of social media practices on E-learning systems acceptance," Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2019, pp. 360-369, DOI: 10.1007/978-3-030-31129-2_33, 2020.

[15] I. Erkan, C. Evans, "The influence of eWOM in social media on consumers’ purchase intentions: an extended approach to information adoption," Computers in Human Behavior, vol. 61, pp. 47-55, DOI: 10.1016/j.chb.2016.03.003, 2016.

[16] M. Debattista, "A comprehensive rubric for instructional design in e-learning," The International Journal of Information and Learning Technology, vol. 35 no. 2, pp. 93-104, DOI: 10.1108/ijilt-09-2017-0092, 2018.

[17] Z. Alrefaie, M. Hassanien, A. Al-Hayani, "Monitoring online learning during COVID-19 pandemic; suggested online learning portfolio (COVID-19 OLP)," MedEdPublish, vol. 9,DOI: 10.15694/mep.2020.000110.1, 2020.

[18] M. Kebritchi, A. Lipschuetz, L. Santiague, "Issues and challenges for teaching successful online courses in higher education: a literature review," Journal of Educational Technology Systems, vol. 46 no. 1,DOI: 10.1177/0047239516661713, 2017.

[19] D. H. A. M. Malik, F. Abid, R. Kalaicelvi, Z. Bhatti, "Challenges of computer science and IT in teaching-learning in Saudi Arabia," Sukkur IBA Journal of Computing and Mathematical Sciences, vol. 2 no. 1, pp. 29-35, DOI: 10.30537/sjcms.v2i1.71, 2018.

[20] M. H. Taha, M. E. Abdalla, M. Wadi, H. Khalafalla, "Curriculum delivery in Medical Education during an emergency: a guide based on the responses to the COVID-19 pandemic," MedEdPublish, vol. 9,DOI: 10.15694/mep.2020.000069.1, 2020.

[21] P. Ashwin, D. McVitty, "The meanings of student engagement: implications for policies and practices," The European Higher Education Area: Between Critical Reflections and Future Policies, pp. 343-359, DOI: 10.1007/978-3-319-20877-0_23, 2015.

[22] B. Little, L. Knihova, "Modern trends in learning architecture," Industrial and Commercial Training, vol. 46 no. 1, pp. 34-38, DOI: 10.1108/ict-07-2013-0046, 2014.

[23] A. Khamparia, B. Pandey, "Impact of interactive multimedia in E-learning technologies," Enhancing Academic Research with Knowledge Management Principles,DOI: 10.4018/978-1-5225-2489-2.ch007, 2017.

[24] H. İ. Akyüz, S. Samsa, "The effects of blended learning environment on the critical thinking skills of students," Procedia - Social and Behavioral Sciences, vol. 1 no. 1, pp. 1744-1748, DOI: 10.1016/j.sbspro.2009.01.308, 2009.

[25] E. G. Oh, Y. Chang, S. W. Park, "Design review of MOOCs: application of E-learning design principles," Journal of Computing in Higher Education, vol. 32 no. 3, pp. 455-475, DOI: 10.1007/s12528-019-09243-w, 2020.

[26] N. Ahmad, N. Quadri, M. Qureshi, M. Alam, "Relationship modeling of critical success factors for enhancing sustainability and performance in E-learning," Sustainability, vol. 10 no. 12,DOI: 10.3390/su10124776, 2018.

[27] F. Y. Ong, J. Manimekalai, "Critical success factors of E-learning implementation at educational institutions," Journal of Interdisciplinary Research in Education, vol. 5 no. 1, pp. 17-24, 2015.

[28] S. Ricart, R. A. Villar-Navascués, S. Gil-Guirado, M. Hernández-Hernández, A. M. Rico-Amorós, J. Olcina-Cantos, "Could MOOC-takers’ behavior discuss the meaning of success-dropout rate? players, auditors, and spectators in a geographical analysis course about natural risks," Sustainability, vol. 12 no. 12,DOI: 10.3390/su12124878, 2020.

[29] I. Jung, S. Choi, C. Lim, J. Leem, "Effects of different types of interaction on learning achievement, satisfaction and participation in web-based instruction," Innovations in Education and Teaching International, vol. 39 no. 2, pp. 153-162, DOI: 10.1080/14703290252934603, 2002.

[30] S. S. Noesgaard, R. Ørngreen, "The effectiveness of E-learning: an explorative and integrative review of the definitions, methodologies and factors that promote e-learning effectiveness," Electronic Journal of E-Learning, vol. 13 no. 4, pp. 278-290, 2015.

[31] T. K. Shih, W. K. T. M. Gunarathne, A. Ochirbat, H.-M. Su, "Grouping peers based on complementary degree and social relationship using genetic algorithm," ACM Transactions on Internet Technology, vol. 19 no. 1, 2018. https://doi.org/10.1145/3193180

[32] J. C. Nunnally, I. H. Bernstein, Psychometric Theory, 1994.

[33] W. Ledermann, "On the rank of the reduced correlational matrix in multiple-factor analysis," Psychometrika, vol. 2 no. 2, pp. 85-93, DOI: 10.1007/bf02288062, 1937.

[34] J. F. J. Hair, W. Black, B. Babin, R. Anderson, R. L. Tatham, "Multivariate data analysis," Technometrics, vol. 31, 2006.

[35] K. A. Bollen, Structural Equations with Latent Variables, 1989.

[36] J. C. Anderson, D. W. Gerbing, "The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis," Psychometrika, vol. 49 no. 2, pp. 155-173, DOI: 10.1007/bf02294170, 1984.

[37] W. G. Cochran, Sampling Techniques, 1963.

[38] D. G. Kleinbaum, L. L. Kupper, K. E. Muller, Applied Regression Analysis and Other Multivariable Methods, 1988.