CET-4 Listening Test Effect on Listening Learning Based on Machine Learning

Abstract

CET-4 is a large-scale test used to test the English proficiency of Chinese non-English majors. However, the number of empirical studies on CET-4 is not large. This paper conducted an empirical study on the effect of the machine learning-based CET-4 listening test on listening learning. This paper first introduces the related concepts and influencing factors of listening and then summarizes the college English test and listening research, and then proposes machine learning related algorithms, including support vector machines, principal component analysis, statistical learning methods, and BP neural network. Finally, the method of questionnaire survey is used to conduct the research. In this study, 520 non-English major students were selected by random sampling to conduct a questionnaire survey, of which 400 were valid questionnaires. The experimental results showed that students’ listening learning strategies (Sig.=. 000<.05), learning behaviors (Sig.=. 000<.05), and learning attitudes (Sig.=. 00 3 <.05) were significantly different and students with higher listening scores paid more attention to the use of learning strategies.

This work is licensed under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.

1. Introduction

CET-4 is an important indicator to measure students’ language ability. It was originally started in 1987 and later became the most commonly used large-scale test. It is suitable for undergraduate non-English majors and has a great impact on college English teaching. With the rapid development of English demand, CET-4 has been reformed by the Ministry of Education in March 2004. Generally speaking, most university teachers and students strive to improve students’ comprehensive practical ability in English to meet the requirements of CET-4, especially listening ability. Therefore, the listening part in CET-4 has become more and more important. It has been greatly improved and reflected especially in college English teaching. However, there are many differences in the current English teaching and teaching status; therefore, how to strengthen English listening teaching has become an important research.

Machine learning can solve large-scale data problems that cannot be solved by human beings at present and has strong self-learning ability, self-optimization ability, and strong generalization ability. English is used as a Foreign Language Test Language (TOEFL), International English Language Testing System (IELTS), Test English Professional Level 4 (TEM-4), and College English Level 4 (CET-4). The findings of these studies have not only contributed to the rapid development of testing theory but have also provided a clear direction for language teaching and learning to a certain extent. They lead to a better understanding of machine learning and make reforms in testing and teaching more important and possible.

The innovation of this paper is as follows: (1) the application of machine learning to the research on the effect of CET-4 listening test on listening learning is innovative and practical; (2) from a theoretical perspective, this research hopes to be able to improve and clarify the theoretical basis of the role of machine learning, and provide relevant data and case support for future related research. (3) from a practical point of view, there is a more in-depth understanding of the relationship between the CET-4 listening test and students’ learning, which has certain practical significance to the reform of the CET-4 test and the preparation of students for the test.

2. Related Work

With the development of science and technology, machine learning technology has penetrated into all aspects of people’s lives, and more and more scholars are studying it. Buczak A described a focused literature review of machine learning (ML) and data mining (DM) methods supporting network analysis for intrusion detection, providing a brief tutorial description of each ML/DM method [1]. Predicting the output power of a solar system is necessary for a well-functioning grid or for optimal management of the energy flux into a solar system. In this context, Voyant C used machine learning methods to predict solar radiation [2]. Zhou L introduced the ML on Big Data (MLBiD) framework to guide the discussion of its opportunities and challenges. The framework was ML-centric and followed the preprocessing, learning, and evaluation phases [3]. The purpose of the Kavakiotis I study was to systematically review the application of machine learning, data mining techniques, and tools in the field of diabetes research, including prediction and diagnosis, diabetes complications, genetic background and environment, and the first category that appeared to be the most popular health care and management [4]. Huge advances in big data analysis have led to a wide variety of interactive model analysis tasks. Liu S made a comprehensive analysis and explanation of this rapidly developing area [5]. Coley CW reported a model framework for predicting response outcomes that combined the traditional use of response templates with the pattern recognition flexibility provided by neural networks [6]. Weinan discussed the idea of using continuous dynamical systems to model general high-dimensional nonlinear functions used in machine learning [7]. Zheng T proposed a data-based framework to identify subjects with and without T2DM from EHR through feature engineering and machine learning [8]. The downside of these studies, however, is that the considerations are not comprehensive enough to adapt to more complex situations, and precision needs to be improved.

3. Machine Learning and Related Methods

3.1. Concept of Listening Comprehension

3.1.1. The Essence of Listening Comprehension

Before the 1950s, people thought that listening comprehension was a process of passively receiving information. With the development of information theory, people have a new understanding of the nature of listening comprehension. In this paper, listening comprehension is a process in which the listener actively uses the existing knowledge to analyze and process [9].

3.1.2. Factors Affecting Listening Comprehension

As shown in Figure 1, the analysis is as follows:

(1) Voice

[figure(s) omitted; refer to PDF]

Phonetic knowledge comes first in the process of listening comprehension. The listener first receives phonetic information, and the brain accepts words, phrases, and sentences composed of factors such as phonemes, syllables, and mid-tones before making them appear in their own language map. However, many students do not understand the basic knowledge of phonetics, and have little knowledge of repeated reading, weak reading, blasting, continuous reading, assimilation, and sound saving, resulting in poor listening effects. Consolidating the knowledge of pronunciation is the basis for improving listening ability.

(2) Vocabulary

There is a misconception that listening ability is proportional to the amount of listening input, but this is not the case. Increasing the frequency of listening is indeed a way to strengthen the listening ability. It forms a habit through constant stimulation, which is imprinted in the brain and becomes a long-term memory. But the listening level is not completely determined by the number of listening; it is a reflection of the comprehensive ability of English, of which vocabulary is the foundation. The learning of a language starts with the accumulation of vocabulary, and gradually expands the vocabulary. And words form sentences, sentences form articles, and articles form articles. The role of vocabulary in English learning is very important. If people do not understand the meaning of words, they may misunderstand the meaning of the original text. According to the regulations, the general requirement for college English vocabulary mastery is 4794 words, the higher degree is 1601 words, the much more higher degree is 1281 words, and the total number of words is 7676. To achieve a certain level of listening, students must memorize these 7676 vocabulary. Therefore, theoretically speaking, the number of vocabulary is one of the important factors for listening comprehension. Strengthening the memory of words and expanding vocabulary are the key to improving listening ability [10].

(3) Grammar

For English listening, the higher the sentence complexity, the harder it is to understand. People can fully understand simple sentences by memorizing a large number of words, while complex sentences must map the relevant phonetic input to grammar rules; otherwise, it will be difficult to understand or even be mistaken. Grammar can help learners to correctly analyze the text, clarify the key points, and eliminate interference, which is the core of improving listening ability.

(4) Read

Reading is a must for language learning and a means to increase the reserve of language knowledge. The more sufficient the reserve of language knowledge, the higher the comprehension ability. The reading in this article includes reading text and listening and reading voice. For listening learners, reading can not only expand the knowledge of vocabulary, grammar, culture, etc. but also help enrich the learners’ thinking. Listening comprehension examines the learner’s judgment of semantics. The process of judgment involves previously accumulated knowledge and flexible thinking and analysis ability. Therefore, extensive knowledge accumulation is one of the fundamental ways to achieve effective listening goals. In the process of listening learning, it is necessary to fully improve the learners’ reading capacity, vigorously strengthen the training of listening, enrich the thinking of listening, and let reading run through the listening learning. Reading is the guarantee of improving listening ability.

(5) Skills

Listening skills refer to how to use listening strategies to optimize listening performance. English listening strategies are divided into metacognitive, cognitive, and affective strategies. Metacognitive strategies refer to the activities of organization, supervision, regulation, and other activities that learners conduct to promote learning, including planning, monitoring, and evaluation. Cognitive strategies are the use of the strategies of input materials, involving the processing and application of language information, such as retelling, summarizing, and reasoning. Emotional strategies are related to the adjustment and control of learners’ emotions in the learning process, such as seeking help, cooperating with others, self-encouraging, and eliminating anxiety. People can master the listening skills and make up for mistakes in the listening process by overcoming the operations that affect the listening, so as to achieve the purpose of optimizing the listening effect [11].

The above five aspects are the analysis of the main factors affecting college English listening. The lack of any one may lead to incomplete listening and affect the listening effect. Therefore, these five aspects must be added to the listening training [12]. At the same time, in order to meet the teaching requirements and improve the listening level of learners, a test platform also needs to be provided for learners to examine their ability to comprehensively apply knowledge, and finally determine the actual listening level of learners. Simply speaking, it is a measure of the learner’s listening accuracy. Accuracy is the basis for measuring the listening effect. In order to be able to understand, the learner must integrate the previously accumulated knowledge of phonics, vocabulary, grammar, reading, and listening skills, and fully analyze the connection between the content of the heard utterance and the purpose of the speaker, in order to make an appropriate response or correct choice.

To sum up, the training of listening ability requires a complete system, which not only requires a solid accumulation of language knowledge but also requires skilled strategic guidance, and is inseparable from a lot of practice and feedback. Pronunciation, vocabulary, grammar, reading, and skills are the five basic elements to improve listening ability. Therefore, in the design of listening learning, whether the services provided to students are complete and how to achieve the requirements specified by the teaching objectives must be fully considered [13].

3.2. CET-4 and Listening Test

3.2.1. Introduction to CET-4

In mainland China, CET-4 is a national examination designed by the National English Language Council and administered by the Ministry of Education. It is for non-English majors at university who have completed a two-year basic English course. CET4 is used to measure the English proficiency of non-English majors, including listening, reading, writing, and speaking. This test is also used to check whether students have met the standards set out in the syllabus. It is usually held in June and December. Students can participate multiple times. Students who fail the first time have a second chance. There is also a second chance for students who pass the test and wish to get a higher score.

CET-4 consists of two parts: written test and oral test [14]. The written test consists of four parts: listening comprehension, reading comprehension, writing, and translation. All non-English majors who have completed two years of English study in this program are entitled to this section. The CET-4 was first implemented in 1987, and it was considered a means of improving English proficiency. The assessment on the implementation of the national syllabus began to test the effect of college English teaching and then provide a clear direction and learning method for English teaching, so that the implementation of the syllabus is smoother. In order to reflect the requirements according to the syllabus, the CET-4 has been reformed in terms of score reporting, measurement, and assessment. The system, content, and format were released in 2006, but have not been put into practice in every university and college, only some selected universities were required to enter. The new CET-4 has been implemented in colleges and universities. Of all the reform projects between 2007, the listening section underwent the greatest changes.

3.2.2. College English Level 4 Listening Test

CET-4 is one of the important tests to test the English level of non-English major students in universities. It underwent a reform in 2007. Afterwards, therefore, the listening test has a larger proportion in the current CET-4. Since then, college English teachers and students have come to realize this and both of them have begun to pay more attention to listening to English. Table 1 demonstrates the percentages, scores, and time allocations for each test format in the new version of CET-4 [15].

Table 1

The distribution of the four listening questions in 2021.

Test content		Test question type	Score proportion	Examination time
Listening dialogue	Short dialogue	Multiple choice	8%	30 minutes
	Long dialogue	Multiple choice	7%
Listening passages	Essay comprehension	Multiple choice	10%
	Passage dictation	Dictation of words and phrases	10%

3.3. Machine Learning Algorithms

Machine learning algorithms aim to find patterns autonomously from a class of unknown data and then use this pattern to classify the remaining data or predict the next incoming data in advance. Therefore, the purpose of machine learning is to design algorithms that allow computers to learn autonomously, thereby realizing the application of artificial intelligence [16].

In recent years, artificial intelligence has become more and more popular, and many companies are more or less related to artificial intelligence; otherwise, they will be regarded as not keeping up with the trend of the times. And machine learning based on big data has become very popular, because it can realize the prediction of data and provide guarantee and basis for decision-making through the calculation of big data and the mining of hidden data.

This chapter mainly summarizes principal component analysis, support vector machine, statistical learning method, and BP neural network. Using the mathematical model of these parameters can generate the data required for the experiment by computer simulation.

3.3.1. Principal Component Analysis

(1) Introduction to principal component analysis

The idea of principal component analysis (PCA) was proposed in 1901. But its main idea is shown in Figure 2, which reduces the language and numerical ability to the result of a few psychological factors through a method [17].

[figure(s) omitted; refer to PDF]

As a data dimensionality reduction algorithm, PCA can greatly improve the speed of unsupervised feature learning. The principal component analysis algorithm can reduce the dimension and overcome the problem that a single indicator cannot reflect data information. Although a large number of indicators can reflect sufficient original information, it is easy to cause information redundancy. The principal component analysis algorithm is to convert complex original information into a small amount of data information, and these small amounts of data information can also reflect the data information of the original data to the greatest extent.

(2) Principal component analysis model

Assuming $A$ is a data matrix, in which there are $q$ variables $A_1,A_2,\cdots ,A_q$, then the linear combination of these variables is shown in: $$\begin{array}{}\text{(1)}& \begin{array}{c}{B}_1=x_{11}{A}_1+x_{12}{A}_2+\cdots +x_{1q}{A}_q\hfill \\ B_2=x_{21}{A}_1+x_{22}{A}_2+\cdots +x_{2q}{A}_q\hfill \\ \cdots \cdots \hfill \\ B_3=x_{q1}{A}_1+x_{q2}{A}_2+\cdots +x_{qq}{A}_q\hfill \end{array}\end{array}$$

Among them, $q$ comprehensive index $B_1,B_2,\cdots ,B_q$ are obtained, and the above matrix transformation needs to meet the following conditions:

Unit vector is all coefficient vectors $$\begin{array}{}\text{(2)}& x_u^{\prime }=x_{u1},x_{u2},\cdots ,x_{uq}\end{array}$$

Which is $$\begin{array}{}\text{(3)}& x_u^{\prime }{x}_u=1\end{array}$$

Satisfying $$\begin{array}{}\text{(4)}& x_{u1}^2+x_{u2}^2+\cdots +x_{uq}^2,u=1,2,\cdots ,q\end{array}$$

(3) Basic steps of principal component analysis

PCA generally adopts the following steps:

Step 1: Calculating the covariance matrix of the original data $\sum$

Step 2: Calculating the variance contribution rate of each principal component: $$\begin{array}{}\text{(5)}& {\varphi }_u=\frac{{\lambda }_u}{{\sum }_{v=1}^q{\lambda }_v},u=1,2,\cdots ,q\text{(6)}& {\phi }_u=\frac{{\sum }_{u=1}^i{\lambda }_u}{{\sum }_{v=1}^q{\lambda }_v},i<q\end{array}$$

Step 3: Calculating the score on the $i$ principal components of the observed sample: $$\begin{array}{}\text{(7)}& B_u=x_{11}{A}_1+x_{12}{A}_2+\cdots +x_{1q}{A}_q,u=1,2,\cdots ,i\end{array}$$

If the above three steps are satisfied, the original $q$-dimensional data can be transformed into the $i$-dimensional data after dimensionality reduction through principal component analysis [18]. Its flow chart is shown in Figure 3:

[figure(s) omitted; refer to PDF]

3.3.2. Statistical Learning Methods

Statistical learning, also known as statistical machine learning, is a cross-section of computer building models based on training sample data to predict unknown data through models and analysis.

(1) VC dimension

An important concept in statistical learning theory is the VC dimension (Vapnik-Chervonenkis dimension). The VC dimension reflects the learning ability of the hypothesis space, which is a set of mappings from the input space to the output space (Figure 4).

(2) Loss function and empirical risk

[figure(s) omitted; refer to PDF]

0-1 loss function $$\begin{array}{}\text{(8)}& PB,gA=\begin{array}{c}1,B\ne gA\hfill \\ 0,B=gA\hfill \end{array}\end{array}$$

Quadratic loss function $$\begin{array}{}\text{(9)}& PB,gA={B-gA}^2\end{array}$$

Absolute loss function $$\begin{array}{}\text{(10)}& PB,gA=B-gA\end{array}$$

Logarithmic loss function $$\begin{array}{}\text{(11)}& PB,gA=-\log DBA\end{array}$$

Given $i$ training samples $$\begin{array}{}\text{(12)}& A_1,B_1,A_2,B_2,\cdots ,A_i,B_i\text{(13)}& S{g}_{\exp }=EPB,gA={\int }_{a,b}Pb,gADa,bdadb\end{array}$$

The goal of learning is to find an optimal function $B=gA,{\theta }_0$ in the hypothesis space $B=gA,\theta$ to minimize the expected risk according to the given training sample data. Expected risk is the expectation of the loss function, denoted as $S{g}_{\exp }$:

where $Pb,ga$ is the loss function.

Empirical risk is the average loss on the training dataset, denoted as $S{g}_{emp}$: $$\begin{array}{}\text{(14)}& S{g}_{emp}=\frac{1}{i}\sum _{u=1}^{i}P{b}_u,g{a}_u\end{array}$$

From the theorem of large numbers, only when the number of samples in the sample space tends to infinity, there is the probability of $S{g}_{emp}$ be close to $S{g}_{\exp }$. However, the number of samples in the limited sample space cannot represent all patterns, and it is impossible to ensure that the empirical risk tends to the expected risk, which also leads to problems such as “overfitting” in many learning methods. Therefore, using empirical risk minimization to estimate expected risk minimization is not ideal.

3.3.3. Support Vector Machine

(1) Support vector machine concept

BP neural network is good at solving the problem of pattern classification and nonlinear mapping. Support vector machine (SVM) is a general-purpose feedforward neural network that can also be used for tasks such as pattern classification and nonlinear regression. As far as the standard linearly separable classification is concerned, the idea of the SVM algorithm is to create an optimal classification hyperplane and maximize the distance between the two classes of samples located at the farthest close to both sides. Different from BP neural network based on some biological principle, the theoretical basis of SVM is statistical learning theory [19].

Support vector machines have many advantages, such as versatility and simple calculation. The key to building the SVM learning algorithm is to establish the conceptual entry region of the inner product kernel between the “support vector” $xa$ and the $x$ vector extracted from the sample.

(2) Support vector machine model

The model diagram of the support vector machine is shown in Figure 5:

[figure(s) omitted; refer to PDF]

Standard SVM are nonprobabilistic binary linear classifiers. For each given input, it predicts that the input is one of two classes. Since SVM is classifier, given a set of training sets, and each training sample is marked as belonging to one of two classes, the SVM algorithm is trained to predict whether a sample falls into one class or the other class of model. Intuitively, a support vector machine model is a representation of multiple discrete points in space that is mapped to high dimensions so that the gaps between classes are magnified as much as possible. Then, a new test set is mapped into the same space and is predicted to belong to that class based on which side of the gap they fall on.

The basic principle of support vector machine is as follows:

Setting the training set samples as $$\begin{array}{}\text{(15)}& A_u,B_u{A}_{\text{u}}\in R^{\text{z}},B_{\text{u}}\in -1,1,\text{among}\text{u}=1,2,\cdots ,\text{I}\end{array}$$

For a linearly separable data sample set, there exists a generalized optimal classification hyperplane: $$\begin{array}{}\text{(16)}& K•A+C=0\text{(17)}& \begin{array}{c}\min \frac{1}{2}{K}^2\\ s.t.B_u{K}^T{A}_u+C\ge 1,u=1,2,\cdots ,j\end{array}\end{array}$$

For many practical problems, the data is not necessarily completely linearly separable, so on the basis of Formula (17), slack variables and penalty coefficients are introduced, which can be transformed into the following optimization problems after Lagrange transformation: $$\begin{array}{}\text{(18)}& \begin{array}{c}\underset{\alpha }{\max }\sum _{u=1}^j{\alpha }_u-\frac{1}{2}\sum _{u,v=1}^j{\alpha }_u{\alpha }_v{B}_u{B}_v{A}_u^T{A}_v\\ s.t.0\le \alpha \le G\\ \sum _{u=1}^i{\alpha }_u{B}_u=0u=1,2,\cdots ,i\end{array}\end{array}$$

By solving the above formula to get ${\alpha }_u$, there is $$\begin{array}{}\text{(19)}& K=\sum _{u=1}^i{\alpha }_u{B}_u{A}_u\end{array}$$

$K$ is obtained. $$\begin{array}{}\text{(20)}& hA=\mathrm{sgn}K•A+C=\mathrm{sgn}\sum _{u=1}^i{\alpha }_u{B}_u{A}_u•A+C\text{(21)}& L{A}_u,A=\phi A_u,\phi A\end{array}$$

Substituting into Formula (18), that is, the decision function of the nonlinear support vector machine, and the decision function is as: $$\begin{array}{}\text{(22)}& hA=\mathrm{sgn}\sum _{u=1}^i{\alpha }_u{B}_{u}L{A}_u•A+C\end{array}$$

3.3.4. Artificial Neural Network

(1) Artificial neural network model

Artificial neural network (ANN) is a kind of information processing system inspired by biological neuron structure and function. Neural network is a complex network composed of many connected neurons. After adjusting the weights between neurons, the input sample data is modeled and simulated to finally have the ability to solve and deal with problems [20]. Figure 6 is a flow chart of neural network learning.

(2) Artificial neural network classification

[figure(s) omitted; refer to PDF]

Artificial neural networks are extremely malleable, and any combination of network structure and activation function can build artificial neural networks of different forms.

The artificial neural network classification diagram is shown in Figure 7.

[figure(s) omitted; refer to PDF]

Based on the difference in composition, the network can be divided into a forward network and a feedback network; based on the difference in learning methods, the network can be divided into tutored learning networks and unsupervised learning networks.

(a) Forward network

The neurons in the forward network are arranged hierarchically, and each neuron is only connected to the previous layer of neurons. The forward network structure is shown in Figure 7(a) [21].

The training data flows from the input layer through the hidden layer to the output layer and propagates in hierarchical order. Each layer of neurons will only receive input signal of the neuron from the previous layer; the subsequent layer will not bring signal feedback to the previous layer. The perceptron network and BP network that people are familiar with are both types of forward network.

(b) Feedback network

A feedback network refers to a neural network that contains one or more feedback loops. The feedback network structure is shown in Figure 7(b). There is a feedback signal between the output layer and the input layer in the feedback network, and the output value of the network will become the input value of the network again, so that the data information in the whole network circulates until the training results meet the requirements of the network.

(c) Tutored learning network

Learning with a tutor means that in network learning, a given target output is required for network training. The neural network does not know the external environment, and the artificially given target output samples are equivalent to “mentors,” which represent the best corresponding effect of neural network training. The network parameters are adjusted according to the target output, so that the network output approaches the termination condition.

(d) Tutor-free learning network

Contrary to tutored learning networks, tutor-free learning does not have external tutors or evaluations to monitor the learning process, and its learning methods are divided into reinforcement learning and self-organizing learning. There is a mapping relationship between the output and the output of the network in reinforcement learning, and the learning and training of the network are based on the performance scalar index that minimizes its continuous effect with the external environment. In self-organizing learning, a measurement standard about the quality of the network learning representation method is provided, and the network parameters are adjusted according to the provided standard to achieve the optimal result.

(3) BP neural network and its algorithm

BP (back propagation) neural network is a multi-layer forward network with tutor learning based on error back propagation algorithm training. It is composed of multiple neurons connected to each other according to certain criteria. The BP neural network model is shown in Figure 8. The essence of BPNN is to solve the problem of the minimum value of the error function, and it is an important and classic algorithm [22].

[figure(s) omitted; refer to PDF]

BP neural network is generally a three-layer neural network composed of a single input layer, a single output layer, and a single hidden layer. Each layer is composed of many neurons with processing power, and the neurons in each layer are connected to each other, and there is no connection between the neurons in one layer. The core idea is that the input samples are forwarded from the input layer and then passed to the output layer after being mapped by the hidden layer function. If the output of the output layer is inconsistent with the expected value, the gradient descent method is used to convert the output error from the hidden layer to the input in a certain form. At the same time, the network weights between each neuron and the threshold of each neuron are adjusted layer by layer. The above network learning process is repeated iteratively until the gap between the network output and the expectation is reduced to an acceptable range or the number of iterations reaches a predetermined number of times before ending the network training.

4. Experiment on the Effect of CET-4 Listening Test on Listening Learning

4.1. Questionnaire on the Washback Effect of the CET-4 Listening Test on Students’ Listening Learning

This study mainly selected non-English majors who participated in the New College English CET-4 as the survey objects and investigated the washback effect of the CET-4 listening test on students’ listening learning through questionnaires and interviews [23, 24].

This research used random sampling to select 520 non-English majors for a questionnaire survey, of which 400 were valid questionnaires. From Table 2, the basic information distribution of the respondents can be seen. From the perspective of gender distribution, it can be seen that the school has a majority of females. From the perspective of grade distribution, most of the students participating in this survey were sophomores and juniors. Because most majors require sophomores to take the CET-4. From the perspective of major distribution, most of the students who participated in the survey were in literature and history; from the distribution of the total score of CET-4, most of the students passed the CET-4 English test. Judging from the distribution of the four-level listening scores, the listening scores of the students participating in this survey were mostly in the 121 to 180 score range, indicating that most of the students had average listening skills [25].

Table 2

Individual distribution characteristics of survey respondents ($N$ =1200).

		Number of people	Percentage
Gender	Male	234	18.5
	Female	966	80.5
Grade	Freshman	156	13
	Sophomore	528	44
	Junior	375	31.25
	Senior	78	6.5
	Senior and above	63	5.25
Major	Science and engineering	486	40.5
	Literature and history	714	59.5
Total score of level 4	424 points and below	96	8
	425 to 475	258	21.5
	476 to 526	375	31.25
	527to 577	243	20.25
	578to 628	156	13
	Above 628 points	72	6
CET4 listening score	120 points and below	108	9
	121to 150	351	29.25
	151to 180	387	32.25
	181 to 210	258	21.5
	211 points and above	96	8

The above are the gender, major, grade, the basic information of the four-level total score, and the four-level listening score of the respondents. These five items constitute the independent variables of this research [26].

As can be seen from Figure 9, the passing rate of CET-4 among college students nationwide accounts for 40%, and the pass rate of the questionnaire in this paper is 92%, indicating that most people can pass CET-4; 28% of college students in the country have a score of 500 or above, and the questionnaire in this paper is 500%. Scores above 54.88% indicate that most people’s English proficiency is not low, and it is not too difficult to score above 500.

[figure(s) omitted; refer to PDF]

4.2. Washback Effect of the Four-Level Listening Test on Students’ Listening Learning Attitude

The list of questions from the questionnaire is shown in Table 3.

Table 3

Questionnaire.

Serial number	Problem
A	CET-4 listening test makes me more motivated to learn English listening
B	CET-4 listening test makes me more interested in English listening learning
C	CET-4 listening test will make me pay more attention to English listening learning
D	CET-4 listening test prompted me to spend more time training my English listening
E	CET-4 listening test prompted me to increase the amount of extracurricular English listening practice
F	CET-4 listening test prompted me to carry out targeted English listening training

In Figure 10(a), it can be seen from the data results of question A that most students do not think that the CET-4 listening test makes them more motivated to learn English listening. According to the interview results, some students expressed concern about the CET-4 English listening test, but some students said that the CET-4 can motivate and promote listening learning [27].

[figure(s) omitted; refer to PDF]

From the data results of question B, it can be seen that more than half of the students believe that CET-4 has not made them more interested in English listening learning. According to the interview results, some students believed that the CET-4 English listening test did not make them interested in English listening learning, and some students said that in order to improve their listening scores, they would use more listening learning materials in the preparation stage, and they did become interested in English listening in the process.

From the data of question C, it can be seen that most students believe that the college English listening test will make them pay more attention to English listening. According to the interview results, most of the students said that the listening score was very important. In order to ensure that they could pass the Level 4 exam, they would pay more attention to listening learning and spend more time practicing listening.

From the above data, it can be seen that the CET-4 test has a positive impact on students’ learning attitude more than a negative impact. The positive impact is that most students pay more attention to English listening learning, usually spend more time learning listening, and have a positive attitude towards listening learning. Negative effects are manifested in anxiety and pressure on the generation of new question types, and negative learning attitudes. According to the interview results, students’ anxiety mainly comes from worrying about failing the exam due to the increased difficulty of listening. There are also some students having too much pressure and anxiety due to the increased difficulty of listening, resulting in a lack of self-confidence, showing a negative tendency in listening learning attitude [28].

In Figure 10(b), it can be seen from the data of question D that most students believe that the CET-4 listening test encourages them to spend more time training their English listening; from the data of question E, it can be seen that most students believe that the CET-4 listening test will increase the amount of extracurricular English listening practice; it can be seen from the data of question F that most students believe that the CET-4 listening test prompts them to carry out targeted English listening training.

It can be seen from the above data that the CET-4 listening test has an obvious positive washback effect on students’ learning behavior. Most of the students will spend more time training English listening and will do targeted English listening training according to new question types.

4.3. Differences in the Washback Effect of the CET-4 Listening Test on the Listening Learning of Students with DifferentCET-4 Listening Scores

In the following, “students’ CET-4 listening score” is selected as the independent variable, and the three dimensions of listening learning attitude, learning strategy and learning behavior, and the washback effect are selected as the dependent variables for ANOVA [29].

It can be seen from Table 4 that students’ listening learning strategy (Sig.=. 001<.05), learning behavior (Sig.=. 001<.05), and learning attitude (Sig.=. 003 <.05) are three dimensions with very significant difference. To further understand the specific differences between the listening scores and these three dimensions, the following is a table of the mean values of the listening scores on each dimension [30].

Table 4

ANOVA of different listening scores on each dimension.

		Sum of squares	df	Mean square	$F$	Significance
Learning strategy	Between groups	8.674	5	1.721	6.012	.001
	Within group	99.523	355	.287
	Total	108.125	361
Learning behavior	Between groups	9.345	5	1.901	4.789	.001
	Within group	137.152	355	.368
	Total	146.658	361
Learning attitude	Between groups	5.074	5	1.012	3.542	.003
	Within group	101.667	355	.286
	Total	106.742	361

It can be seen from Figure 11 that the higher the listening score, the higher the average value of students’ listening learning strategy, and the students with scores above 211 have the highest average value in the learning strategy dimension. Students with scores from 151 to 210 are slightly higher in listening learning behavior than students in other scores, and students with scores from 151 to 180 are slightly higher in learning attitude than students in other scores. It can be seen that students with higher listening scores can use learning strategies more actively in listening learning, which is consistent with the data results that students with higher total scores in CET-4 are more active in learning strategies. In the listening learning attitude and learning behavior, the students in the middle score segment showed the most positive performance. This is also consistent with the CET-4 score dimension, that is, the students on the middle-level CET-4 score have the most positive performance in listening learning attitude and learning behavior.

[figure(s) omitted; refer to PDF]

From the above data, it can be seen that students’ listening scores are different, and the washback effect of the CET-4 listening test is also different. Students with higher listening scores pay more attention to the use of learning strategies, because their listening learning goals are not only to pass the CET-4 test, but really improve the English level. So they will not change the attitude and learning behavior of English listening learning because of the reform of the listening test, and only adjust the learning strategy appropriately according to the new question type in the test preparation stage. Students with intermediate English proficiency will pay more attention to listening learning because of the reform, and usually spend more time learning listening. Students with low listening level have low self-confidence due to the reform of the four-level listening test, and their performance in listening learning attitude, learning strategy, and learning behavior is not obvious.

5. Discussion

Inspirations and suggestions for students’ English listening learning:

First, students should clarify the test objectives. The purpose of the CET-4 listening test is to provide certain feedback on teaching and learning by testing students’ listening and speaking ability, in order to improve students’ listening level. Because students cannot clarify the learning goals of listening, in order to pass the test or get high scores, they adopt wrong learning strategies and focus on mastering listening skills instead of improving their basic skills and comprehensive language ability. Therefore, students should clarify the test objectives and adopt correct learning strategies in order to truly improve their basic skills and comprehensive language ability;

Second, students should carry out a lot of targeted training in various aspects to improve their comprehensive English ability. The CET-4 listening test is more difficult due to the high frequency of professional terms and terms. If there are many words, phrases, and sentence patterns that students do not know or have not mastered in the listening materials, then it is naturally impossible for students to understand the text content. Therefore, learners should strengthen their proficiency in vocabulary and grammar, and improve their comprehensive English ability, so that the listening level can be gradually improved;

Third, the cultural background in listening also has a certain impact on students’ listening. The increased knowledge of news listening involves foreign politics, current affairs, science, etc. If students do not understand foreign cultural backgrounds, it will lead to students’ different understanding of the same thing and concepts, and even lead to misunderstandings. Due to the lack of background knowledge, students will naturally be ignorant of the material;

Fourth, students’ pronunciation skills hinder the improvement of students’ listening ability to a large extent. Accents tend to be diversified in the CET-4 listening test. Therefore, most non-English major students need to strengthen training and practice in this area when learning English listening and speaking, and correct their own pronunciation mistakes through professional pronunciation training, then the pronunciation will be more standard and authentic.

6. Conclusions

This research adopted the research method of questionnaire survey. Through the research, it was found that the CET-4 listening test had a strong washback effect on students’ English listening achievement. It is mainly reflected in the following aspects: On the one hand, CET-4 listening test helps students understand the importance of English listening learning, and the student test preparation process helps to improve students’ overall English ability level. The CET-4 listening test provides students with a channel to assess their English listening level, so that students can understand the advantages and disadvantages of changing their learning strategies. On the other hand, the CET-4 listening test is becoming more and more difficult, which brings a lot of mental pressure to students and hinders their energy and passion for learning English listening.

References

[1] A. Buczak, E. Guven, "A survey of data mining and machine learning methods for cyber security intrusion detection," IEEE Communications Surveys & Tutorials, vol. 18 no. 2, pp. 1153-1176, DOI: 10.1109/COMST.2015.2494502, 2016.

[2] C. Voyant, G. Notton, S. Kalogirou, M. Nivet, C. Paoli, F. Motte, A. Fouilloy, "Machine learning methods for solar radiation forecasting: a review," Renewable Energy, vol. 105 no. MAY, pp. 569-582, DOI: 10.1016/j.renene.2016.12.095, 2017.

[3] L. Zhou, S. Pan, J. Wang, A. V. Vasilakos, "Machine learning on big data: opportunities and challenges," Neurocomputing, vol. 237 no. MAY10, pp. 350-361, DOI: 10.1016/j.neucom.2017.01.026, 2017.

[4] I. Kavakiotis, O. Tsave, A. Salifoglou, N. Maglaveras, I. Chouvarda, "Machine learning and data mining methods in diabetes research," Biotechnology Journal, vol. 15 no. C, pp. 104-116, DOI: 10.1016/j.csbj.2016.12.005, 2017.

[5] S. Liu, X. Wang, M. Liu, J. Zhu, "Towards better analysis of machine learning models: a visual analytics perspective," Visual Informatics, vol. 1 no. 1, pp. 48-56, DOI: 10.1016/j.visinf.2017.01.006, 2017.

[6] C. W. Coley, R. Barzilay, T. S. Jaakkola, W. H. Green, K. F. Jensen, "Prediction of organic reaction outcomes using machine learning," ACS Central Science, vol. 3 no. 5, pp. 434-443, DOI: 10.1021/acscentsci.7b00064, 2017.

[7] A. Weinan, "A proposal on machine learning via dynamical systems," Communications in Mathematics & Statistics, vol. 5 no. 1,DOI: 10.1007/s40304-017-0103-z, 2017.

[8] T. Zheng, W. Xie, L. Xu, X. He, Y. Zhang, M. You, G. Yang, Y. Chen, "A machine learning-based framework to identify type 2 diabetes through electronic health records," International Journal of Medical Informatics, vol. 97 no. jan., pp. 120-127, DOI: 10.1016/j.ijmedinf.2016.09.014, 2017.

[9] J. X. Wang, J. L. Wu, H. Xiao, "Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data," Physical Review Fluids, vol. 2 no. 3,DOI: 10.1103/PhysRevFluids.2.034603, 2017.

[10] J. H. Chen, S. M. Asch, "Machine learning and prediction in medicine — beyond the peak of inflated expectations," New England Journal of Medicine, vol. 376 no. 26, pp. 2507-2509, DOI: 10.1056/NEJMp1702071, 2017.

[11] N. Poret, R. R. Twilley, R. M. Coronado-Molina, "Object-based correction of LiDAR DEMs using RTK-GPS data and machine learning modeling in the coastal Everglades," Environmental Modelling and Software, vol. 112 no. 3, pp. 491-496, 2018.

[12] J. F. Hernández, Z. Díaz, M. J. Segovia, E. M. D. Pozo, "Machine learning and statistical techniques. An application to the prediction of insolvency in Spanish non-life insurance companies," The International Journal of Digital Accounting Research, vol. 5 no. 9, 2020.

[13] J. Moon, S. Jung, S. Park, E. Hwang, "Machine learning-based two-stage data selection scheme for long-term influenza forecasting," Computers, Materials and Continua, vol. 68 no. 3, pp. 2945-2959, DOI: 10.32604/cmc.2021.017435, 2021.

[14] Y. Jiang, B. Li, "Exploration on the teaching reform measure for machine learning course system of artificial intelligence specialty," Scientific Programming, vol. 2021 no. 5,DOI: 10.1155/2021/8971588, 2021.

[15] A. Sharma, N. Kumar, A. Kumar, K. Dikshit, B. Singh, "Comparative investigation of machine learning algorithms for detection of epileptic seizures," Intelligent Decision Technologies, vol. 15 no. 2, pp. 269-279, DOI: 10.3233/IDT-200091, 2021.

[16] R. Kumar, S. K. Khatri, M. J. Diván, "Power usage efficiency (PUE) optimization with counterpointing machine learning techniques for data center temperatures," International Journal of Mathematical, Engineering and Management Sciences, vol. 6 no. 6, pp. 1594-1611, DOI: 10.33889/IJMEMS.2021.6.6.095, 2021.

[17] M. Gastegger, J. Behler, P. Marquetand, "Machine learning molecular dynamics for the simulation of infrared spectra," Chemical Science, vol. 8 no. 10, pp. 6924-6935, DOI: 10.1039/C7SC02267K, 2017.

[18] I. Goodfellow, P. Mcdaniel, N. Papernot, "Making machine learning robust against adversarial inputs," Communications of the ACM, vol. 61 no. 7, pp. 56-66, DOI: 10.1145/3134599, 2018.

[19] D. S. Char, N. H. Shah, D. Magnus, "Implementing machine learning in health care — addressing ethical challenges," New England Journal of Medicine, vol. 378 no. 11, pp. 981-983, DOI: 10.1056/NEJMp1714229, 2018.

[20] D. Assouline, N. Mohajeri, J. L. Scartezini, "Quantifying rooftop photovoltaic solar energy potential: a machine learning approach," Solar Energy, vol. 141 no. JAN., pp. 278-296, DOI: 10.1016/j.solener.2016.11.045, 2017.

[21] D. Sacha, M. Sedlmair, L. Zhang, J. A. Lee, J. Peltonen, D. Weiskopf, S. C. North, D. A. Keim, "What you see is what you can change: human-centered machine learning by interactive visualization," Neurocomputing, vol. 268 no. dec. 11, pp. 164-175, DOI: 10.1016/j.neucom.2017.01.105, 2017.

[22] S. Mullainathan, Z. Obermeyer, "Does machine learning automate moral hazard and error?," The American Economic Review, vol. 107 no. 5, pp. 476-480, DOI: 10.1257/aer.p20171084, 2017.

[23] J. Huang, "A case probe into emotional experiences of Chinese English majors in L2 listening learning process: a positive psychology perspective," SAGE Open, vol. 12 no. 1, article 215824402210798,DOI: 10.1177/21582440221079815, 2022.

[24] S. Sumarni, S. Suwandi, H. Waluyo, A. Andayani, "Listening learning of drama employing contextual approach for students of vocational high schools in Central Java," Jurnal Pendidikan Bahasa dan Sastra, vol. 19 no. 1, pp. 100-110, DOI: 10.17509/bs_jpbsp.v19i1.20762, 2020.

[25] P. Zhang, S. Graham, "Vocabulary learning through listening: comparing L2 explanations, teacher codeswitching, contrastive focus-on-form and incidental learning," Language Teaching Research, vol. 24 no. 6, pp. 765-784, DOI: 10.1177/1362168819829022, 2020.

[26] M. Kusumaningrum, D. R. Pertiwi, "An intelligent tutoring system for learning listening TOEIC," Jurnal Berdaya Mandiri, vol. 3 no. 1, pp. 456-467, DOI: 10.31316/jbm.v3i1.1336, 2021.

[27] N. Sault, "Avian voices, avian silences: learning by listening to birds," Ethnobiology Letters, vol. 11 no. 2,DOI: 10.14237/ebl.11.2.2020.1730, 2020.

[28] D. Cunandar, "Gender-based students’ listening ability of auditory learning modality," Acuity Journal of English Language Pedagogy Literature and Culture, vol. 5 no. 2, pp. 17-23, DOI: 10.35974/acuity.v5i2.2237, 2020.

[29] M. Melani, "The effect of Google Classroom assisted extensive listening on listening comprehension accross learning autonomy," Lingua Cultura, vol. 14 no. 1, pp. 129-142, DOI: 10.21512/lc.v14i1.6493, 2020.

[30] N. N. Koneva, D. A. Zhigulskaia, "YouTube as a means of developing Chinese students’ listening skills when learning Russian language for business communication," Science and School, vol. 2020 no. 4, 2020, pp. 144-151, DOI: 10.31862/1819-463X-2020-4-144-151, 2020.