The Recommendation System of Innovation and Entrepreneurship Education Resources in Universities Based on Improved Collaborative Filtering Model

Abstract

In the huge number of online university education resources, it is difficult for learners to quickly locate the resources they need, which leads to “information trek.” Traditional information recommendation methods tend to ignore the characteristics of learners, who are the main subjects of education. In order to improve the recommendation accuracy, a recommendation algorithm based on improved collaborative filtering model is proposed in this paper. Firstly, according to the student behavior data, consider the behavior order to create the behavior graph and behavior route. Then, the path of text type is vectorized by the Keras Tokenizer method. Finally, the similarity between multidimensional behavior path vectors is calculated, and path collaborative filtering recommendations are performed for each dimension separately. The MOOC data of a university in China are introduced to experimentally compare the algorithm of the article as well as the control group algorithm. The results show that the proposed algorithm takes better values in evaluation indexes, thus verifying that this algorithm can improve the effectiveness of innovation and entrepreneurship education resources recommendation in universities.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

In recent years, the use of online education platform is more and more accepted by learners [1]. Especially, after the outbreak of New Coronavirus, online learning is the preferred learning mode when offline learning cannot be carried out normally. Faced with the huge demand for online learning, online education institutions are providing free online courses and sharing online teaching resources, and the online education industry is showing an explosive growth trend. With the widespread use of online learning platforms, the number of educational resources for innovation and entrepreneurship in universities has also increased dramatically.

In the huge number of online college education resources, learners are difficult to quickly locate the resources they need, resulting in “information trek” [2]. Information recommendation is a method for students to quickly and effectively screen out the objects that meet their preference characteristics from the mass objects. At present, information recommendation technology is widely used in many fields [3]. It is an effective way to solve the problem of “information trek” to apply information recommendation technology to online education and realize personalized recommendation of innovation and entrepreneurship education resources in universities in the process of online education.

With the upgrading of Internet technology, the scale of online education services, represented by Massive Open Online Courses (MOOC), Net Ease Cloud Classroom, and innovation and entrepreneurship education resource platforms built by universities, continues to expand. Meanwhile, teaching data through social media such as Weibo, forum, and post bar also constitute a massive amount of online education resources [4]. These abundant online education service resources not only expand students’ knowledge scope and supplement the deficiency of offline learning, but also bring a lot of inconvenience to students. First, complicated online education resources cause “information trek,” and the quality of online education services is uneven, which makes it difficult for students to identify and compare, resulting in poor learning experience and effects. Second, faced with the same high-quality educational resources, students with high information literacy can easily obtain relevant information, while students with low information literacy can hardly access such resources, resulting in new educational inequities in the information environment. Third, all kinds of teaching resources are not pushed to students with the same probability. Mainstream education resources are pushed to students more frequently due to their high frequency of use, thus covering the minority and high-quality education resources. As a result, it is difficult for students to obtain such resources, resulting in “information cocoon.”

How to help learners acquire the knowledge they are interested in quickly and accurately is an urgent problem to be solved by online education service platforms of universities [5]. Educational recommendation services are based on learners’ personalized network information and learning behavior. The recommendation algorithm is used to quickly recommend precise college innovation and entrepreneurship education resources that meet students’ potential learning needs from the massive educational information resources. It can enhance the experience and effectiveness of online learning [6].

The development of the Internet has broadened the channels for students to obtain information, but not all online information is valuable. Therefore, in the era of information overload, helping students efficiently and accurately screen out the resources they are interested in has become the key to the development of the Internet [7]. Collaborative filtering algorithm is the main technology to solve this problem [8]. As a widely recognized recommendation technology, collaborative filtering can effectively process structured information without studying the content and attributes of the recommendation information. At the same time, it can combine some concepts that are not easy to embody to complete information filtering, and the recommendation has a high degree of intelligence, which fully meets the requirements of personalized recommendation.

Relevant scholars further improve the collaborative filtering recommendation effect by using different methods. A personalized system filtering recommendation method based on coverage minimalism is proposed. According to the method of coverage reduction and student reduction in coverage rough set, the matching of redundant elements and redundant students is realized. Redundant students are removed from the target student set by reduction algorithm to ensure the effectiveness of collaborative filtering. Under the open data set, personalized recommendation services are provided for target students. The literature [9] proposed a method based on the clustering of student interest degree. Combining with the frequency of the target students scoring keywords, the degree of students’ preference for keywords is obtained, and the student-keyword preference matrix is constructed, through which the clustering is realized. Logistic function is used to obtain students’ interest in the project, determine students’ preferences, find out students’ similar students in the cluster, and collect some information about neighbours’ hobbies to realize the recommendation of target students. However, the above recommendation method has some limitations, such as the serious problem of cold start, and it is difficult to accurately reflect the relationship between students and education resources.

In the era of big data, information overload is a huge problem we are faced with. It is extremely difficult for students to acquire the content they are interested in in time and accurately from massive data [10]. Recommendation system can alleviate the problem of information overload well and has been widely used in e-commerce, online news, and social networking sites.

Personalized recommendation system mainly uses the historical interaction information between students and projects to establish a model to predict students’ evaluation of the project. In the traditional recommendation system, the recommendation based on collaborative filtering is the most classic [11]. An intuitive interpretation of collaborative filtering recommendations is that a student’s rating of a project can be predicted by the rating of other students who are similar to him. Matrix decomposition is a common collaborative filtering method [12]. The idea of recommendation is to map the student vector and the project vector to a common vector representation space, in which the implicit vector is used to represent the student and the project, and then the score of the project can be directly obtained by calculating the similarity between the implicit vector of the student and the project. The successful application of matrix decomposition model in the recommendation field has led to the emergence of many improved methods based on matrix decomposition in the research field of recommendation system, such as the integration of neighbourhood model in matrix decomposition in the literature [13]. Factorization machine is used for feature interaction to generate more features and add them to model training. The literature [14] improves the effect of matrix decomposition by introducing auxiliary information such as project content. In recent years, the application of deep learning in image, natural language processing, and other fields has achieved good results, and many studies have begun to introduce deep learning into the field of the recommendation system [15]. Deep learning is usually introduced from two perspectives: (1) neural network is used to learn the representation vectors of students and projects, such as the deep matrix factorization (DMF) model proposed in the literature [16]. Dual-path neural network is used to replace the traditional matrix decomposition to learn the representation vectors of students and projects, and then score prediction is made. (2) The neural network is used as the matching function of the recommendation model, and the student vector and the project vector are fed into the matching function to directly obtain the scores of the students on the project, such as the neural collaborative filtering (NCF) model proposed in reference [17]. NeuMF in NCF simply concatenates the student vector and the project vector into a multilayer perceptron (MLP) to calculate the student’s rating of the project. The literature [18] comprehensively considered the above two aspects and proposed a deep collaborative filtering model (DeepCF) combining representation learning and function learning. The above collaborative filtering recommendation model and deep learning recommendation model only extract the interaction features between students and projects from a single perspective, and the feature extraction is not comprehensive enough, which will limit the model’s recommendation ability.

The essence of recommendation is a continuous decision-making process, and students’ interest degree is not constant, so resource recommendation cannot be carried out from a static perspective. Therefore, this paper proposes a recommendation system of innovation and entrepreneurship education resources in universities based on the improved collaborative filtering model.

The innovations and contributions of this paper are listed as follows:

(1) According to the student behavior data, consider the behavior order to create the behavior graph and behavior route.

(2) Use Keras Tokenizer to vectorize the path of text type.

(3) The similarity between multidimensional behavior path vectors is calculated, and path collaborative filtering recommendations are performed for each dimension separately.

This paper consists of five main parts: the first part is the introduction, the second part is state of the art, the third part is methodology, the fourth part is result analysis and discussion, and the fifth part is the conclusion.

2. State of the Art

In order to improve the expansibility of the recommendation system, a scholar proposed Spark Hierarchical CF, which uses the student preference model and clustering algorithm to divide students into different student clusters according to different preference characteristics. And then it makes collaborative filtering recommendations for different student clusters. Compared with the algorithm based on MapReduce, the recommendation accuracy is improved and the operation time is saved, but the centre point of clustering is difficult to determine.

At present, there are four kinds of research on student behavior path, including methods based on reinforcement learning, methods based on statistical analysis, methods based on activity trajectory clustering, and methods based on the ant colony algorithm.

In order to improve the problem that the collaborative filtering algorithm ignores item attributes, a dynamic individual recommendation method based on reinforcement learning was proposed in the literature [19]. Firstly, students’ attribute labels are mined from the operation behaviors, and then the reward and punishment models of attribute labels are adjusted according to the operate path and recall path to achieve dynamic adjustment of label weight. Finally, reinforcement learning is used to achieve label recommendation. The disadvantage of this method is that only recent behaviors are considered. Student preference information implied by behavioural pathways is not fully utilized.

The literature [20] proposed that students’ experience perception can improve the recommendation effect. According to the behavioural data, analyse the student behavior path and create the student experience perception model. The behavioural path is defined as continuous indicators such as attracting attention, expressing interest, clicking intention, and watching video, which can better show the development trend of students’ interest. However, this method has the problems of cold start and sparse data.

Considering the similarity between student visit and ant foraging, the ant colony algorithm was introduced in the literature [21] to realize recommendation, and the transition probability was calculated according to students’ browsing behavior and preferred browsing path, so as to dynamically make recommendation. However, there are cold start problem and Matthew effect in this method, and there is occasional error in recommendation based on transfer probability. The literature [22] imitates the concept of pheromone in the ant colony algorithm and proposes pheromone-based algorithm to combine students’ browsing behavior with reading behavior through commodity pheromone, and then recommend students according to their browsing trajectory. However, the properties of resource pheromones lead to the Matthew effect.

3. Methodology

This paper proposes a behavior path collaborative filtering recommendation algorithm based on knowledge graph. First, t behaviors in user behavior data are initialized to t nodes = $g_1,g_2,\dots ,g_t$, each node $g$ stores the attribute of this behavior. And these nodes are constructed into a behavior knowledge map in Neo4j, which is referred to as the behavior map. Second, define the behavior path for the user against the same target entity in long weeks $\mathrm{Rout}{\mathrm{e}}_{\mathrm{uid},\mathrm{iid}}=h_1,h_2,\dots ,h_t$, and construct it according to the behavior sequence in the behavior graph. Uid and iid indicate the student ID and education resource ID, respectively. Then, the behavior path is derived for vectorization. Finally, the similarity between multidimensional behavior paths is calculated. The flow of this algorithm is shown in Figure 1. This section describes the specific steps of the algorithm.

[figure(s) omitted; refer to PDF]

3.1. Construct Behavior Map

Taking each behavioural datum as a node, $w$ genera of each behavior $f_1,f_2,\dots ,f_w$ is sealed as nodes and stored in the Neo4j data library. The construction algorithm is shown in$$\begin{array}{c}\text{(1)}& g=\text{Graph}\text{host,http\_port,user,password},\text{node}=\text{Node}{}^{\prime }f1{}^{\prime }=f1,{}^{\prime }f2{}^{\prime }=f2,\dots ,{}^{\prime }fm{}^{\prime }=fm,g.\text{create}\text{node},\\ g.\text{create}\text{node},\end{array}$$where host, http_ port, user, and password represent host address, port address, database user name, and password, respectively, which are used to connect to the neo4j database. Node encapsulates each behavior datum into a node, and the create (node) command stores each node in the neo4j database. The preliminarily constructed behavior map is shown in Figure 2.

[figure(s) omitted; refer to PDF]

3.2. Creating a Behavior Path

First, prioritize your actions. According to t behavior types contained in the behavior map, the behavior priority is defined $h_1,h_2,h_3,\dots ,h_t$, h represents a single behavior, h₁ has the lowest priority, and h_t has the highest priority.

Then, define the behavior path. This paper defines the behavior path as an ordered set of all behaviors generated by students for the same educational resource in a long period. $\mathrm{Rout}{\mathrm{e}}_{\mathrm{uid},\text{id}}=h_1,h_2,\dots ,h_t$. Uid and iid represent the student ID and education resource ID, respectively.

Finally, create a behavior path. According to the behavior sequence, traverse the behavior map, create a “continuous action” relationship for the continuous behavior made by the same student about the same educational resources, skip the single node that does not constitute a path, and get the Route_{uid, iid} of all students about different educational resources after completing the traversal. The specific steps are shown in Figure 3. The created behavior path is shown in Figure 4.

[figure(s) omitted; refer to PDF]

3.3. Path Vectorization

Because the data type of the behavior path created in the previous step is text, and the similarity calculation requires digital data. The next is the vectorization and alignment of paths.

Firstly, classify the behavior paths by behavior priority, as listed in Table 1.

Table 1

Path classification.

Classification	Classification basis
Class h1	The highest behavior priority in the path is h1
Class h2	The highest behavior priority in the path is h2
…	…
Class ht	The highest behavior priority in the path is ht

Secondly, vectorize each path in each class from text format to digital format. In this paper, Keras’s Tokenizer class is used for vector transformation.

Finally, vector alignment is required. Because the path length is different, the length of each vector is also different, which means that the spatial dimension of the vector is not unified, and the similarity cannot be calculated. Calculate the maximum path length of all behavior paths and use Keras’s kps.pad_sequences method to make the spatial dimension of each path the same as the maximum length, with the principle of complement 0. The data change process is listed in Table 2.

Table 2

Example of data change process.

Path status	Numerical value
Initial behavior path set	{“h1–h2–h4–h1–h1” “h1–h1–h1”}
	{h1:[“h1–h1–h1”]
After path classification	h2:[ ],
	h3:[ ],
	h4:[“h1–h2–h4–h1–h1”]}
	{h1:[[1, 1, 1]]
After path steering	h2:[ ],
	h3:[ ],
	h4:[[1, 2, 4, 1, 1]]}
	{h1:[[1, 1, 1, 0, 0]],
After path alignment	h2:[ ],
	h3:[ ],
	h4:[[1, 2, 4, 1, 1]]}

3.4. Calculation of Path Similarity

Here, the path similarity is measured by the sum of the distances between a path in class G and each path in class H. As shown in equation (3), the smaller the total distance, the greater the similarity.

As shown in Table 2, the array of class h₂ and class h₃ is empty, it can be seen that there may be empty classes in the path classification. Direct calculation of similarity will lead to empty objects. Therefore, it is necessary to distinguish which classes are empty in the path classification and finally decide which behaviors can be recommended.

According to the permutation and combination principle, there are 2^T − 1 types of classification for T types of behavior. For example, there are 15 types of classification for 4 types, as listed in Table 3, where combination (h₁) means that all classes except fot h₁ are empty, and combination (h₂, h₃, h₄) means that h₂, h₃, and h₄ are not empty. The h₁ class is empty, and so on.

Table 3

Path category combination.

Possible combination of route categories
1	(h1)
2	(h2)
3	(h3)
4	(h4)
5	(h1, h2)
6	(h1, h3)
7	(h1, h4)
8	(h2, h3)
9	(h2, h4)
10	(h3, h4)
11	(h1, h2, h3)
12	(h1, h2, h4)
13	(h1, h3, h4)
14	(h2, h3, h4)
15	(h1, h2, h3, h4)

After the path combination is determined, the path similarity is calculated. In this paper, Euclidean distance is used to calculate the similarity of paths. The calculation formula of Euclidean distance in t-dimensional space is shown in$$\begin{array}{}\text{(2)}& di,j=\sqrt{{\displaystyle \sum _{x=1}^t{i_x-j_x}^2}}.\end{array}$$

Secondly, the sum of the distance between each path in class₁ and the path in class₂ is calculated, as shown in (3), where class₁ and class₂ represent two classes in the path subclass. The classification of other paths is similar:$$\begin{array}{}\text{(3)}& \text{distances}={\displaystyle \sum _{i=1}^{t}di,j_x},\hspace{1em}i\in {\text{class}}_1,j\in {\text{class}}_2.\end{array}$$

Finally, the corresponding total distance list can be obtained for h_t class path, as shown in$$\begin{array}{}\text{(4)}& \text{List\_distances}\_h_t={\text{distances}}_1,{\text{distances}}_2,\dots ,\hspace{1em}t\in 1,t.\end{array}$$

If the combination is (h₂, h₃, h₄), the total distance list can be obtained by calculation, as shown in$$\begin{array}{c}\text{(5)}& \text{List\_distances}\_h_2={\text{distances}}_1,{\text{distances}}_2,\dots ,{\text{distances}}_w,\text{List\_distances}\_h_3={\text{distances}}_1,{\text{distances}}_2,\dots ,{\text{distances}}_t,\end{array}$$where $w$ and t are the number of paths in h₂ and h₃, respectively. List_distances_h₂ represents the total distance list between each path in h₂ and all paths in h₃. And List_distances_h₃ represents the total distance list between each path in h₃ and all paths in h₄.

3.5. Multidimensional Recommendation

In this paper, the definition is derived from the multidimension $d_1,d_2,d_3,\dots ,d_{t-1}$. D represents a set of recommendations that may be generated in line h. The d₁ pair should be able to generate the h₂ recommendation set. D_t −1 corresponds to the possible recommendation set of h_t, which can be divided into possible_h_t table. D is the set of target entities corresponding to min (List_distances), that is, the set of target entities corresponding to the path with the minimum total distance. The results of recommendation obtained are shown in$$\begin{array}{c}\text{(6)}& d_1,d_2,\dots ,d_{n-1}=\text{possible}\_h_2{}^{″}:\cdots \cdots ,{}^{″}\text{possible}\_h_3{}^{″}:\cdots \cdots ,\\ \cdots \cdots \\ {}^{″}\text{possible}\_h_t{}^{″}:\cdots \cdots .\end{array}$$

According to the example results in equation (5), there are two recommendation dimensions d₂ and d₃. The returned recommendation results are shown in equation (7), which, respectively, represents the set of educational resources where the user recommended by this algorithm may have behaviors h₃ and h₄, and the value is the ID or other unique identification of the educational resource:$$\begin{array}{c}\text{(7)}& d_3,d_4={}^{″}\text{possible}\_h_3{}^{″}\begin{array}{c}\mathrm{2733371,332733}\end{array},{}^{″}\text{possible}\_h_4{}^{″}:124451.\end{array}$$

The item collaborative filtering recommendation algorithm, referred to as item CF, is used to recommend educational resources similar to those they liked before to college students.

In order to further analyse the recommendation performance of BR-CF, item CF and BR-CF are cascaded and mixed, respectively, in this section, and an improved algorithm based on the combination of item CF and BR-CF is proposed.

The cascade recommendation algorithm of item CF and BR-CF is referred to as Combine item CF and BR-CF. In the recommendation result of resource collaboration, the algorithm further calculates the set of target entities with behavior paths and the minimum total distance from the behavior paths of all target entities in the verification set. The flowchart is shown in Figure 5.

[figure(s) omitted; refer to PDF]

4. Result Analysis and Discussion

The experimental data in this paper are from a MOOC of a university in China bound by QQ account, and the crawler tool is used to directly obtain student information and innovation and entrepreneurship education resource data of universities. A total of 36,967 pieces of online learner information were obtained, including major, educational background, geographical location, friend relationship, learning time, course name, and evaluation content. After data screening, the dormant accounts and abandoned accounts whose monthly login times are less than 3 are excluded. Finally, the effective data set obtained contains 2400 students. The students were divided into two data sets in the ratio of 8 : 2. Among them, 1920 students’ information constituted the training data set, and the other 480 students’ information constituted the test data set.

In order to test the quality of the proposed algorithm, the proposed algorithm based on the improved collaborative filtering model is compared with the literature [23], literature [24], and literature [25]. The number of adjacent recommendation students is set as 5, and the value is increased by 15 until it is 50. The Precision, Recall, and RMSE values of the four recommendation models under different number of recommendation students are listed, as shown in Table 4.

Table 4

Comparison of experimental results of four recommended algorithms with different TOP-N.

TOP-N	Index	Literature [23]	Literature [24]	Literature [25]	Proposed
5	Precision (%)	30.61	28.42	32.05	31.21
	Recall (%)	41.24	39.94	43.21	42.92
	RMSE	0.88	0.93	0.85	0.86
20	Precision (%)	28.16	26.96	28.08	29.98
	Recall (%)	44.37	43.65	46.78	47.58
	RMSE	0.79	0.91	0.78	0.72
35	Precision (%)	26.12	24.94	24.06	28.73
	Recall (%)	46.82	45.42	48.88	49.11
	RMSE	0.75	0.81	0.72	0.65
50	Precision (%)	23.24	22.25	21.78	25.97
	Recall (%)	50.16	48.9	51.34	51.79
	RMSE	0.68	0.72	0.61	0.57

As shown in Figures 6–8, when the number of recommended students is greater than 20, the accuracy and root mean square error of the proposed algorithm in experimental data are better than those of the other three recommendation algorithms. In terms of recall rate, when the number of recommended students is greater than 35, there is not much difference between the proposed algorithm and the knowledge-based recommendation algorithm, but it is significantly better than the other two algorithms. Compared with the currently commonly used recommendation algorithm, the proposed algorithm takes into account student information, social relationship, and online learning behavior, so the recommendation quality is higher.

[figure(s) omitted; refer to PDF]

5. Conclusion

At present, there are a lot of learning resources in online education learning platform, but the personalized service is not high. The target of online education services is learners. The commonly used personalized recommendation algorithm of education services only considers static information or dynamic characteristics of users, resulting in low recommendation quality. The collaborative filtering algorithm is the mainstream of information recommendation technology at present. However, the traditional recommendation system based on collaborative filtering algorithm has some limitations, such as low accuracy and cold start. And it is difficult to accurately reflect the relationship between students and educational resources. In order to improve the recommendation accuracy, this paper proposes a recommendation algorithm based on the improved collaborative filtering model for innovation and entrepreneurship education resources in universities. This paper uses 20,059 real behavior data and successfully creates 2,266 behavior paths according to the sequence of students’ behaviors, which enriches the semantic information of the data. In addition, students’ behavior data are fully used to make recommendations from three dimensions, which improves the data utilization rate and the diversity of recommendations. In particular, this paper puts forward the concept of behavior path, which can well represent the evolution and development process of students’ behavior and better predict students’ learning preferences by creating associations between behaviors. In order to test the quality of the recommendation algorithm, the proposed algorithm and other three recommendation algorithms were used as the control group in the experiment, and the Precision, Recall, and RMSE of these four recommendation algorithms were calculated under different recommended users. It shows the recommendation the proposed algorithm has more advantages in three evaluation indicators, and can be applied to online education services to better provide students with high-quality recommendation of college innovation and entrepreneurship education resources. Since the experimental platform and data in this paper are relatively single, in order to further verify and improve the personalized recommendation effect of the algorithm in this paper, other online education service platforms will be introduced in the future to verify the scalability and accuracy of this method.

Acknowledgments

This study was sponsored by Xi’an Physical Education University.

References

[1] L. P. Dinh, T. T. Nguyen, "Pandemic, social distancing, and social work education: students’ satisfaction with online education in Vietnam," Social Work Education, vol. 39 no. 8, pp. 1074-1083, DOI: 10.1080/02615479.2020.1823365, 2020.

[2] Q. Yang, "Research on active service model with personalized education resources," Asian Journal of Contemporary Education, vol. 3 no. 1, pp. 95-104, DOI: 10.18488/journal.137.2019.31.95.104, 2019.

[3] M. C. Urdaneta-Ponte, A. Mendez-Zorrilla, I. Oleagordia-Ruiz, "Recommendation systems for education: systematic review," Electronics, vol. 10 no. 14,DOI: 10.3390/electronics10141611, 2021.

[4] N. A. Albelbisi, F. D. Yusop, "Factors influencing learners’ self -regulated learning skills in a massive open online course (MOOC) environment," The Turkish Online Journal of Distance Education, vol. 20 no. 3,DOI: 10.17718/tojde.598191, 2019.

[5] W. Zhang, Y. Wang, L. Yang, C. Wang, "Suspending classes without stopping learning: China’s education emergency management policy in the COVID-19 outbreak," Journal of Risk and Financial Management, vol. 13 no. 3,DOI: 10.3390/jrfm13030055, 2020.

[6] S. Ali, Y. Hafeez, M. Humayun, N. S. M. Jamail, M. Aqib, A. Nawaz, "Enabling recommendation system architecture in virtualized environment for e-learning," Egyptian Informatics Journal, vol. 23 no. 1, pp. 33-45, DOI: 10.1016/j.eij.2021.05.003, 2022.

[7] S. McGrew, J. Breakstone, T. Ortega, M. Smith, S. Wineburg, "Can students evaluate online sources? Learning from assessments of civic online reasoning," Theory & Research in Social Education, vol. 46 no. 2, pp. 165-193, DOI: 10.1080/00933104.2017.1416320, 2018.

[8] Q. Wang, C. Zeng, W. Zhou, "Online interactive collaborative filtering using multi-armed bandit with dependent arms," IEEE Transactions on Knowledge and Data Engineering, vol. 31 no. 8, pp. 1569-1580, 2018.

[9] S. Wan, Z. Niu, "A hybrid e-learning recommendation approach based on learners’ influence propagation," IEEE Transactions on Knowledge and Data Engineering, vol. 32 no. 5, pp. 827-840, 2019.

[10] P. G. Roetzel, "Information overload in the information age: a review of the literature from business administration, business psychology, and related disciplines with a bibliometric approach and framework development," Business research, vol. 12 no. 2, pp. 479-522, DOI: 10.1007/s40685-018-0069-z, 2019.

[11] Z. Cui, X. Xu, F. Xue, X. Cai, Y. Cao, W. Zhang, J. Chen, "Personalized recommendation system based on collaborative filtering for IoT scenarios," IEEE Transactions on Services Computing, vol. 13 no. 4, pp. 685-695, DOI: 10.1109/tsc.2020.2964552, 2020.

[12] R. M. Sallam, M. Hussein, H. M. Mousa, "An enhanced collaborative filtering-based approach for recommender systems," International Journal of Computer Application, vol. 176 no. 41,DOI: 10.5120/ijca2020920531, 2020.

[13] X. Ma, D. Dong, Q. Wang, "Community detection in multi-layer networks using joint nonnegative matrix factorization," IEEE Transactions on Knowledge and Data Engineering, vol. 31 no. 2, pp. 273-286, 2018.

[14] Y. Pan, F. He, H. Yu, "Learning social representations with deep autoencoder for recommender system," World Wide Web, vol. 23 no. 4, pp. 2259-2279, DOI: 10.1007/s11280-020-00793-z, 2020.

[15] Z. Batmaz, A. Yurekli, A. Bilge, C. Kaleli, "A review on deep learning for recommender systems: challenges and remedies," Artificial Intelligence Review, vol. 52 no. 1,DOI: 10.1007/s10462-018-9654-y, 2019.

[16] B. Yi, X. Shen, H. Liu, Z. Zhang, W. Zhang, S. Liu, N. Xiong, "Deep matrix factorization with implicit feedback embedding for recommendation system," IEEE Transactions on Industrial Informatics, vol. 15 no. 8, pp. 4591-4601, DOI: 10.1109/tii.2019.2893714, 2019.

[17] W. Chen, F. Cai, H. Chen, M. D. Rijke, "Joint neural collaborative filtering for recommender systems," ACM Transactions on Information Systems, vol. 37 no. 4,DOI: 10.1145/3343117, 2019.

[18] Y. Zhang, C. Yin, Q. Wu, "Location-aware deep collaborative filtering for service recommendation," IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51 no. 6, pp. 3796-3807, 2019.

[19] T. T. Nguyen, N. D. Nguyen, S. Nahavandi, "Deep reinforcement learning for multiagent systems: a review of challenges, solutions, and applications," IEEE Transactions on Cybernetics, vol. 50 no. 9, pp. 3826-3839, DOI: 10.1109/tcyb.2020.2977374, 2020.

[20] M. Mailizar, D. Burg, S. Maulina, "Examining university students’ behavioural intention to use e-learning during the COVID-19 pandemic: an extended TAM model," Education and Information Technologies, vol. 26 no. 6, pp. 7057-7077, DOI: 10.1007/s10639-021-10557-5, 2021.

[21] H. Liu, X. Li, "Learning path combination recommendation based on the learning networks," Soft Computing, vol. 24 no. 6, pp. 4427-4439, DOI: 10.1007/s00500-019-04205-x, 2020.

[22] S. Shamshirband, M. Babanezhad, A. Mosavi, N. Nabipour, E. Hajnal, L. Nadai, K.-W. Chau, "Prediction of flow characteristics in the bubble column reactor by the artificial pheromone-based communication of biological ants," Engineering Applications of Computational Fluid Mechanics, vol. 14 no. 1, pp. 367-378, DOI: 10.1080/19942060.2020.1715842, 2020.

[23] J. Shu, X. Shen, H. Liu, B. Yi, Z. Zhang, "A content-based recommendation algorithm for learning resources," Multimedia Systems, vol. 24 no. 2, pp. 163-173, DOI: 10.1007/s00530-017-0539-8, 2018.

[24] H. Jia, L. Yang, B. Cui, "Collaborative filtering recommendation of online learning resources based on knowledge association model," International Journal of Emerging Technologies in Learning (iJET), vol. 17 no. 2, pp. 158-172, DOI: 10.3991/ijet.v17i02.29013, 2022.

[25] J. K. Tarus, Z. Niu, G. Mustafa, "Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning," Artificial Intelligence Review, vol. 50 no. 1, pp. 21-48, DOI: 10.1007/s10462-017-9539-5, 2018.