An Empirical Study of EFL Learners' Dictionary Use in Chinese–English Translation

Abstract

This article reports on the results of a study which investigated English as Foreign Language (EFL) learners' use of an electronic dictionary in a L1-L2 translation task. Forty-seven university graduate students from a Chinese university were asked to translate a Chinese passage into English on computers with the support of an embedded dictionary. Screen recorders were used to record their dictionary use behavior and a follow-up interview was conducted to tap into the thinking processes behind their behavior. The results of the study show that when translating, EFL learners demonstrate preferences for L2 equivalents and content words in their lookups, and reveal specific problems such as a preoccupation with L2 equivalents and lack of awareness of other lexical information, which may hinder correct application of dictionary information. This study suggests that dictionary use behavior may affect the development of students' ability to translate and requires attention from both EFL learners and teachers. It is further suggested that translation teachers should alert learners to the importance of checking other lexical information in a dictionary in their translation practice.

Alternative Kurzfassung:

In hierdie artikel word verslag gelewer oor die resultate van 'n studie waarin leerders van Engels as Vreemde Taal (EVT) se gebruik van 'n elektroniese woordeboek in 'n L1-L2-vertalingsopdrag ondersoek is. Sewe en veertig nagraadse studente van 'n Chinese universiteit is versoek om op die rekenaar 'n Chinese stuk in Engels te vertaal met behulp van 'n ingeboude woordeboek. Skermopnemers is gebruik om hul gedrag rakende woordeboekgebruik vas te le, en 'n opvolgonderhoud is gevoer om die denkprosesse wat hul gedrag rig, te probeer bepaal. Die resultate van die studie dui daarop dat EVT-leerders in die naslaanproses 'n voorkeur vir L2-ekwivalente en inhoudswoorde toon, en dit le spesifieke probleme soos 'n beheptheid met L2-ekwivalente en 'n onkunde oor ander leksikale inligting bloot, wat kan verhinder dat die woordeboekinligting korrek toegepas word. Hierdie studie suggereer dat woordeboekgebruiksgedrag die ontwikkeling van studente se vertaalvermoěns mag affekteer en dat sowel EVT-leerders as -onderwysers aandag hieraan moet skenk. Daar word ook voorgestel dat vertaalonderwysers leer- ders se aandag moet vestig op hoe belangrik die kontrolering van ander leksikale inligting in 'n woordeboek in hul vertaalpraktyk is.

Headnote

Abstract: This article reports on the results of a study which investigated English as Foreign Language (EFL) learners' use of an electronic dictionary in a L1-L2 translation task. Forty-seven university graduate students from a Chinese university were asked to translate a Chinese passage into English on computers with the support of an embedded dictionary. Screen recorders were used to record their dictionary use behavior and a follow-up interview was conducted to tap into the thinking processes behind their behavior. The results of the study show that when translating, EFL learners demonstrate preferences for L2 equivalents and content words in their lookups, and reveal specific problems such as a preoccupation with L2 equivalents and lack of awareness of other lexical information, which may hinder correct application of dictionary information. This study suggests that dictionary use behavior may affect the development of students' ability to translate and requires attention from both EFL learners and teachers. It is further suggested that translation teachers should alert learners to the importance of checking other lexical information in a dictionary in their translation practice.

Keywords: dictionary use preferences, dictionary use processes, efl LEARNERS, TRANSLATION TASK, INTERVIEW, SCREEN RECORDING, LOG FILES

Opsomming: In hierdie artikel word verslag gelewer oor die resultate van 'n studie waarin leerders van Engels as Vreemde Taal (EVT) se gebruik van 'n elektroniese woordeboek in 'n L1-L2-vertalingsopdrag ondersoek is. Sewe en veertig nagraadse studente van 'n Chinese universiteit is versoek om op die rekenaar 'n Chinese stuk in Engels te vertaal met behulp van 'n ingeboude woordeboek. Skermopnemers is gebruik om hul gedrag rakende woordeboekgebruik vas te le, en 'n opvolgonderhoud is gevoer om die denkprosesse wat hul gedrag rig, te probeer bepaal. Die resultate van die studie dui daarop dat EVT-leerders in die naslaanproses 'n voorkeur vir L2-ekwivalente en inhoudswoorde toon, en dit le spesifieke probleme soos 'n beheptheid met L2-ekwivalente en 'n onkunde oor ander leksikale inligting bloot, wat kan verhinder dat die woordeboekinligting korrek toegepas word. Hierdie studie suggereer dat woordeboekgebruiksgedrag die ontwikkeling van studente se vertaalvermoěns mag affekteer en dat sowel EVT-leerders as -onderwysers aandag hieraan moet skenk. Daar word ook voorgestel dat vertaalonderwysers leer- ders se aandag moet vestig op hoe belangrik die kontrolering van ander leksikale inligting in 'n woordeboek in hul vertaalpraktyk is.

Sleutelwoorde: woordeboekgebruikersvoorkeure, woordeboekgebruiksprosesse, evt-leerders, vertalingsopdrag, onderhoud, skermopname, logleers

(ProQuest: ... denotes non-US-ASCII text omitted.)

1.Introduction

Traditionally, the dictionary is considered an important tool in language learning. A number of studies have demonstrated that dictionary use contributes to the acquisition of a foreign language (Lew and Doroszewska 2009; Chen 2011; Dziemianko 2014; Chen 2017; Liang and Xu 2017). However, the importance of applying dictionary information correctly has not been thoroughly examined. Researchers often fail to acknowledge that for most language learners, the purpose of looking up words in a dictionary is not to memorize vocabulary or acquire language, but to solve problems in various language tasks such as reading, writing and translation. in other words, vocabulary acquisition is incidental in dictionary use, while the availability, accessibility and application of lexical information are the immediate needs of most dictionary users. In addition, it is the correct use of retrieved information that forms the basis of vocabulary (language) acquisition. In this sense, studies of dictionary use should not only focus on the incidental acquisition of words but also the application of lexical information. After all, the incidental acquisition of vocabulary in dictionary use depends on the correct application of the target words on repeated occasions. On the other hand, the majority of research on dictionary use has employed elicitation tasks to collect data, either through various forms of production questionnaires (Barnhart 1962; Tomaszczyk 1979; Hartmann 1983; Atkins and Varantola 1997; Sánchez Ramos 2005) or log files (Laufer and Hill 2000; Lew and Doroszewska 2009; Chen 2011; Liang and Xu 2017). Research employing more naturalistic data is needed to explore how learners use dictionaries in real situations, particularly in language learning contexts such as reading, writing and translation. Tarp (2009: 293) argues that various methods should be combined to obtain more knowledge about real user needs. Lew (2011b) believes that there is room for engaging both positivistic and naturalistic approaches, as in fact they do not exclude, but rather complement one another. Possibly due to the difficulty of collecting naturalistic data, and the relative recency of electronic dictionaries, these kind of studies of dictionary use seem to be under-represented in the field of lexicography.

The present exploratory study aims to contribute to the literature on user research in lexicography by employing mixed research methods (observation, a test and interviews) to collect naturalistic data, exploring what EFL learners look up in electronic dictionaries and how they use the lexical information in L1-L2 translation.

The article is structured as follows: first, we provide a brief summary of research on dictionary use by English as a foreign language (EFL) learners and research on the application of dictionary information. In the second part of the article, we present our study, starting with the research questions, participants and a description of the experimental dictionary. We then describe the methods used to observe user behavior and collect data, present and discuss the results obtained, and conclude the article with a summary and suggestions for future studies.

2.Literature review

The literature review consists of two parts. The first part is concerned with studies of dictionary use in general and the second part focuses on studies of the application of dictionary information.

2.1Studies of dictionary use

Studies of dictionary use have a long history as lexicographers have learned to recognize the importance of this research field. According to Welker (2010: 531), about 70 empirical studies were published from 1962 to 1989 and there have been more than 250 investigations since 1990. Some studies have focused on assessing the dictionary skills of learners (Frankenberg-Garcia 2011; Chan 2012), discovering where students look up multi-word expressions (Tono 1989; Bogaards 1998, 2003; Frankenberg-Garcia 2011; Gromann and Schnitzer 2016), as well as which type of dictionary - bilingual, monolingual or semi-bilingual - is easiest to use and gives students the most reliable results (Laufer and Melamed 1994; Laufer and Hadar 1997; Kaneta 2011; Chen 2011; Chan 2014). According to Nesi (2014), dictionary use research covers five themes: learners' preference and attitudes, the influence of dictionaries on text comprehension, the influence of dictionaries on text production, the role of dictionaries as an aid to English language learning and English language learners' dictionary consultation behavior. Lew (2011a: 1) notes that interest in the empirical study of dictionary use is on the rise.

As electronic dictionaries replace print dictionaries (Lew 2012: 243), research into dictionary use is increasingly focusing on the former. In the digital age, the status of the dictionary is changing, and so are the patterns of user behavior. As such, we need to know more about user behavior in the digital environment (Lew and De Schryver 2014). Carolin Müller-Spitzer (2014: 46) found that a majority of studies had been concerned with bilingual dictionaries and the comparison in students' use of bilingual and monolingual dictionaries. This is connected to the fact that some of the studies concentrate in particular on vocabulary learning (Laufer and Hill 2000; Lew and Doroszewska 2009; Chen 2010; Dziemianko 2010; Chen 2017). Laufer and Hill (2000) and Chen (2010), for example, investigated the relationship between which low-frequency words students looked up while reading and how well those words were remembered. In their studies, the relevant lexical information was incorporated into a CALL program comprised of a text, highlighted low-frequency words, and access to different lexical information about these words (with explanations in English, translations into the L1, sound and "extra" information). These studies reveal the role of electronic dictionaries in vocabulary learning but they are not without problems. One concern is that they do not reflect actual dictionary use, because users looked up both low-frequency words and high-frequency words in reading and translation, especially when high-frequency words have many different senses (Bogaards 1998; Frankenberg-Garcia 2011; Koplenig, Meyer and Müller-Spitzer 2014). Another problem is that most users consulted a dictionary to solve the problems arising during the linguistic activities rather than to memorize words. When learners have difficulty understanding a word or expressing an idea in linguistic activities, they turn to a dictionary for help. Then they try to understand and use the word. In addition, these studies only investigated incidental vocabulary acquisition while reading. As we know, receptive tasks such as reading are less demanding than productive activities such as writing and L1-L2 translation because they do not require learners to know lexical information in great depth. In decoding tasks, users "will be 'blind' to the grammatical contexts in which a target word appears"(Chan 2012: 134).

2.2Information application study

At present, only a few studies have focused on the application of dictionary information, but they have not provided a complete picture of dictionary use, likely because they fail to combine positivistic and naturalistic research methods. For instance, Atkins and Varantola (1992, 1993, 1997) carried out a series of studies monitoring dictionary use in translation. They performed a detailed examination of the words looked up by users and the motivations for their look-ups. They aimed to monitor the dictionary look-up process in as natural a situation as possible. To that end, the researchers asked students to note down what their partners looked up, using forms designed for the research. This method was unobtrusive but it only recorded the information that the form required. In addition, participants in their study were not from the same language background, so the user group was not representative of any particular language community. Also, the researchers did not rate how successful the look-ups were. Harvey and Yuill (1997) also studied the use of monolingual dictionaries by EFL learners while writing. They asked students to recall what they had looked up. This method may well help to produce easily quantifiable results from natural settings but sacrifices the crucial criterion of reliability in data collection by relying solely on students' memory. Bogaards (2003: 26-33) concludes that 'uses and users of dictionaries remain for the moment relatively unknown'.

In recent years, researchers (Dziemianko 2010, 2014; Chen 2011, 2012, 2017; Chan 2012, 2014; Hu and Zhang 2013; Frankenberg-Garcia 2015) have focused on dictionary use and language acquisition. However, they tend to report on factors that affect learners' use of information while not providing a comprehensive description of the application of this information in natural settings. In addition, some studies only investigated the effects of dictionary features without gathering information from the users about their behavior. For instance, Frankenberg-Garcia (2015) investigated the effect of the type and number of examples in dictionary entries by asking 161 students to perform partial sentence translations. Students were made aware that their production might be problematic and they were encouraged to make revisions. She found that the number of examples did affect users' production. Although experiments of this kind are indispensable, we need to know both the effects and the causes underlying the performance of users by combining positivistic and naturalistic research methods.

These studies highlight the need for an in-depth analysis of dictionary use in a more natural setting. In response, this study offers a detailed account of how students applied dictionary information in a production assignment, analyzes the possible relationship between application and behavior and explores underlying causes, and identifies some implications for the presentation of information in electronic dictionaries.

3. Research design

3.1 Research questions

Our specific research questions were as follows:

1. What is the status quo of EFL learners' use of the electronic dictionary during a translation task?

(1) What do users look up in an electronic dictionary during a production activity?

(2) Are there any look-up preferences? If yes, what are they?

(3) What are the underlying causes of users' different lookup preferences?

2. How well do EFL learners apply retrieved information in the translation task?

(1) What types of errors did learners make in the application of dictionary information? And what are the causes?

(2) What contributed to the learners' successful application of dictionary information?

3.2 Participants

The study included 50 students from two translation classes in a course that was taught in a CALL classroom, wherein students listened to the teacher and practised translation on computers. Participants' ages ranged from 22 to 25. 28 of the students were female and 22 were male. One class of students (27) were majoring in computer science at a Chinese College, and the other class of students were psychology majors. All of the study's participants had passed College English Test Band 4 (CET4 ?5 in IELTS) and 30 of them had passed College English Test Band 6 (CET6 5.5 in IELTS).

3.3 Instruments

The research instruments included a self-designed CALL program with an embedded dictionary which was used to present the translation task, provide dictionary help and record students translation products, AntConc 3.2.1 (Laurence 2007), ICTCLAS 2014 (Zhang 2014), a screen recorder PMLX (Pan 2012), and an outline of our interview questions.

ICTCLAS (Institute of Computing Technology, Chinese Lexical Analysis System) was used to compute how many words a Chinese text (in our study, the translation task) contains and to tag parts of speech onto the words. The accuracy of lexical analysis is 97.58%. AntConc 3.2.1 was first used to analyze the tokens and types of Chinese words in the translation task. The result is a factor we considered when deciding how many entries the embedded dictionary should include. AntConc 3.2.1 was also used to analyze the application of dictionary information (spelling, collocational and syntactical features) in students' translation products. PMLX, a screen recorder software, was used to record students' choice of lexical information category and retrieval behavior.

The researchers designed a CALL program similar to that used in the studies of Laufer and Hill (2000) and Chen (2013). The program in our study consisted of a task box, an embedded dictionary with a search box and a display box, a click counter, and a writing box. The translation task consisted of a text of 328 Chinese characters. It contains about 198 word tokens (according to an analysis using ICTCLAS 2014) and 134 word types. Similar to the programs in Laufer and Hill (2000) and Chen (2013), this program also had a task box, an embedded dictionary with information category labels, and a dictionary information presentation box. Different from their programs, this program had a search box and a writing box because in productive tasks (such as translation and writing) learners usually have more lexical needs than they would in receptive tasks (such as reading and listening). The search box gave users more freedom to look up words (both low-frequency and high-frequency) when needed than previous studies (Laufer and Hill 2000; Chen 2013). In Laufer and Hill's study (2000), the target words included 12 low-frequency words and in Chen's study (2013), the target words were 10 unknown words. In addition, since the translation activity involved the change of word forms and different ways of expressing meanings, we also added three information categories to help students. They were derivative, collocation and phrase, synonyms and antonyms.

The interface of the CALL program is shown in Figure 1. The top box in the left colored yellow is the translation task. The bottom box in the left colored blue is the writing box. Between them lies the display box colored green. The top box in the right colored green is the search box. The six buttons under this search box are the labels of information categories. When users input a word into the search box and clicked on one information category label, the corresponding information for the word appeared in the display box. Like other electronic dictionaries, this search box also carried a function of association, that is, when users input letter "a" into the search box, a group of words beginning with "a" appeared in the pull-down list. This helped users locate the target word entries immediately. When the translation task was finished, users clicked on the save button below the writing box and the program saved the work.

Several factors were taken into consideration for the selection of entries in the dictionary. First, in classroom practice of translation, we found that students did not look up all the words of the translation task in the dictionary. In tasks similar to the one in the test, they usually looked up about one-third of the words in the texts. In the pilot study, the number of words one student looked up in the dictionary was 35 and the other student looked up 30 words. Second, as all the students passed CET4 (CET4 ?5 in IELTS), the two researchers in this study decided that there were about 20 words unfamiliar to students. Third, considering some low-frequency words students might look up, we decided to provide 43 words in the dictionary, which included all the words the two students in the pilot study looked up.

The lexical information of these entries in the dictionary was collected from two print Chinese-English dictionaries (Chinese-English Dictionary, 2010 and New Century Chinese-English Dictionary, 2012) and two electronic dictionaries (Youdao Dictionary and Kingsoft PowerWord). Compiled after the founding of the People's Republic of China, the Chinese-English Dictionary (1978) is the first of its kind and is regarded as the most authoritative. Chinese-English Dictionary (2010) is currently in a third edition. New Century Chinese-English Dictionary (2012) is ranked number one of its kind in terms of sales volume. The two print dictionaries both cover over 100 thousand entries and were published in recent years. Youdao Dictionary (7.0) and Kingsoft PowerWord (2017) are the two most used electronic dictionaries by college students in China (Xie 2014; Yang 2017). Youdao Dictionary (7.0), with over 500 million users, is a digitalized collection of many print dictionaries, like LDOCE (Longman Dictionary of Contemporary English, 5th edition) and Collins Learners' English-Chinese Dictionary (2012). It includes over 37 million entries and 23 million examples. Like Youdao Dictionary, Kingsoft PowerWord (2017) is a digitalized collection of many print dictionaries, such as the Collins COBUILD Advanced Learner's English-Chinese Dictionary (2012). It has 30 million users and is famous for its over 5 million bilingual examples. At present, there is no digitalized form of the two print Chinese-English dictionaries in China. However, the lexical information in the two print dictionaries is too limited for translation learners. Take the word "yu xiang" (which literally means "expect") for example.

⅞⅞ yu xiang <?> anticipate; expect; prefigure; preconceive: ~?? prefigure the future У satisfy sb's preconceptions of sth II o It is more complicated than first thought.

(New Century Chinese English Dictionary 2012)

This entry only provides pronunciation, one word class of the word, four English equivalents, two phrases and one bilingual example. It does not demonstrate the usage of all the equivalents.

⅞⅞ yu xiang I ¾ anticipate; expect; preconceive ~?? prefigure(or envisage) the future / unexpected consequences / #i 1~0¾?⅞⅞ obtain the anticipated results II ^ preconception

(Chinese English Dictionary 2010)

This entry only provides pronunciation, two classes of the word, three English equivalents, and three phrases. It does not demonstrate the usage of all the equivalents and it does not even provide one sentence example.

In the pilot study, students reported that they wanted to read more phrases and examples in the dictionary, so we decided to take some information (chiefly bilingual examples and phrases) from these two electronic English-Chinese dictionaries because they support Chinese-English translation and have more phrases and examples. As a result, the information categories of the embedded dictionary included the part of speech (POS), English equivalents, derivatives of the English equivalents, bilingual examples, collocations and phrases, and synonyms and antonyms. Information about the part of speech for the 43 word entries was taken from Chinese-English Dictionary (2010), collocations and phrases from New Century Chinese-English dictionary (2012), and equivalents from Chinese-English Dictionary (2010). Derivatives, Synonyms and antonyms were taken from Youdao Dictionary (7.0), and bilingual examples were taken from Kingsoft PowerWord (2017). For example, the entry of "yu xiang" (again which literally means "expect") in the dictionary is as follows:

... (yu xiang)

Part of speech: Verb;

English equivalents: anticipate; expect; speculate;

Derivatives: (N.) anticipation; expectation; (ADJ) anticipated; expected

Bilingual examples:

1. ...

It is more complicated than first thought.

2. ...

The costs of the vacation surpassed my expectation.

3. ...

The beauties of the West Lake in spring were beyond his expectation.

Collocations and phrases: ⅞⅞?? Prefigure the future;

... satisfy one 's preconceptions of sth.

Synonyms and antonyms:

... (synonym): foresee; expect; hope;

... (antonym): recall; review; recollect

3.4Research methods

The data collection methods employed for this research were observation by means of a screen recorder, a translation task, and follow-up interviews used to explore the underlying reasons for users' behavior.

Before the experiment, the researchers conducted a pilot study. Participants in the pilot study were two students from another class in the same grade as participants from two classes in our study. Based on their feedback, some information in the dictionary was revised. After completing the translation task with the embedded dictionary, both students suggested that the bilingual examples in the dictionary should be numbered in the display box to make them more readable. We numbered all the examples in the dictionary. One stu- dent indicated that some examples were too long. We replaced the examples with shorter ones. They also reported some spelling mistakes in the entry. We corrected them. The pilot study found that both students preferred to click on English equivalents. Li (1998) found that Chinese students often choose the first equivalents in bilingual dictionaries when translating from English into Chinese. This is one reason why they often cannot produce correct translations. Therefore, we decided to investigate their preference in choosing these equivalents when translating from Chinese into English. In designing the embedded dictionary, we chose ten words which have at least three English equivalents and placed the most familiar equivalent in second position. The familiarity with the words was rated by the two students in the pilot study. It needs to be explained that "unremitting" and "vanquish" appeared in the essay If I Rest, I Rust written by Orison Marden, the first unit in the textbook students used. That is why the two students rated them as the most familiar words among the equivalents.

The experiment was performed in a computer center where the learners took translation classes. At the beginning, participants were shown a demo about how to use this program without being told the purpose of the study, although they were told that it was part of an innovation program about computer-aided translation training. The experiment was carried out over two classes (90 minutes). Since two students used online dictionaries and one failed to finish the translation task, the data of their performance was excluded. Therefore, the effective number of participants was 47. The experiment produced the following data: video records (entries retrieved and information category click counts), and the products of the translation task.

Based on a preliminary observation of video recordings, interviews with five participants were carried out the second day of the study to learn about the reasons for their consulting behavior. The interviews were guided by the following topical questions: (1) Does the dictionary provide sufficient help in translation? (2) Why did they click on the equivalents most often or click only on the equivalents? (3) Why did they click on the examples, or why not? (4) What are the criteria for their choice of an equivalent?

To minimize the difficulty of expressing their ideas, interviews with students were conducted in Chinese. Interviews were recorded and then transcribed.

4.Results and Discussion

In the pilot study, informants indicated that the embedded dictionary provided sufficient information for the translation task. Video recordings confirmed this. By comparing the number of entries participants input into the search box and the results they obtained, we concluded that the embedded dictionary helped them address most of their lexicographic needs. In the experiment, the consultation could not solve all the problems of students who lacked dictionary use skills, nor could the dictionary solve all of the problems users met in the translation process. Therefore, we believe that this experiment can represent actual use of a dictionary for translation in a natural setting.

Table 1 shows that the average retrieval success was 84%. This means that 84% of the words users searched were available in the embedded dictionary. We believe this result reflects the authentic situation of dictionary use in linguistic activities. Firstly, as new words or new usages of existing words emerge almost every day, the available Chinese-English dictionaries cannot immediately include all the words in use. Secondly, since a Chinese character could be part of a word, a word or a phrase, some users do not know the lexical unit they should look up in the dictionary. This is evidenced by some students failing to find the target words in the dictionary because they looked up phrases, clauses or even sentences rather than words. Thirdly, some students lacked the instrumental ability of translation competence. They read only the English equivalents while ignoring other information categories which might be helpful to their translation.

4.1 Translation learners' dictionary use

4.1.1 The words looked up

Drawing on ICTCLAS2014, the original text was found to contain 198 Chinese running words, and our results showed that users consulted high-frequency words most often. This finding is consistent with that of Varantola (1998) and Koplenig, Meyer and Muller-Spitzer (2014). The reason could be that users consult dictionaries not only to learn about new words but also to check whether their understanding or use of high-frequency words is accurate. The learners looked up a small number of function words. This is possibly because translation learners felt they were more familiar with these words than other highfrequency words and function words usually were not the barrier to understanding. With regard to the word classes users retrieved, Table 2 reveals that the users consulted content words most often; 89% of the words looked up were verbs, nouns, adjectives and adverbs. This is quite understandable. Firstly, content words are related to both meaning comprehension and pro- duction. Secondly, the number of content words was high in the original text, about 134. Thirdly, many content words are in wide use and carry multiple meanings which usually cause trouble for students' meaning understanding and production, while function words, users assumed, were relatively familiar to them. The average number of words looked up by each user was 45 while the average number of content words looked up by users was 40. The high proportion of content words in the words that were looked up reflects the fact that users relied heavily on dictionaries to express their meaning.

In addition, the majority of words looked up were basic words such as "effort" (looked up 31 times), "progress" (looked up 35 times), "road" (looked up 31 times), "ability" (looked up 26 times). Some students even looked for the equivalents of such words as "easy" (looked up twice), "important" (looked up 4 times) and "now" (looked up twice). Interviews revealed that some students looked up these words to check whether what they remembered about them was correct.

We also found that users treated multi-word expressions as retrieval units. Most of the items that were looked up were actually phrases and expressions. For instance, zhi li jin qü, (i.e. make great efforts) was looked up 9 times; que ding mu biāo (i.e. set the target) was looked up 4 times and yousuo jin bū (i.e. make some progress) was looked up 5 times. This revealed that translation students understand texts in terms of semantic units rather than lexical units. Therefore, to help users' retrieval efficiency, we believe more phrases should be included in the dictionary for translation learners. This would be easy to tackle in electronic dictionaries. To improve learners' understanding of these expressions, dictionaries should provide more contextual information within entries for this group of users.

4.1.2The information categories users clicked on

Since a small number of the words users searched for were not found in the dictionary, the relevant clicks were not included in the results of this study. Repeated clicks were included, however, as this reflected the users' actual dictionary use behavior and needs.

Figure 2 shows that there are big differences between the click counts for different information categories. Equivalents are ranked first and account for 65% of the total click counts whereas Examples account for 17%, Derivatives 9% and Collocation and Part-of-Speech only 4%. English equivalents were the most consulted parts of the dictionary entries while synonyms and antonyms were the least looked-up elements. This might be because equivalents are usually the first step towards transferring an idea into English, but it could also be attributed to the learners' lack of translation skills. Frankenberg-Garcia (2011) found that users do not know which information to look up beyond L1-L2 equivalents. This finding is consistent with that of Atkins and Varantola (1997), who found that in L1-L2 translation, consulting and checking foreign language equivalents accounted for 77% of dictionary look-ups. Our follow-up interviews revealed that the users regarded the provision of English equivalents as a basic requirement for a dictionary and some even went so far as to claim that the provision of English equivalent was sufficient for translation most of the time. If necessary, the learners also browsed other information categories. For example, when they did not know the usage of the equivalent, they looked at other information such as "examples". Interviewees responded that examples could enhance their understanding of the equivalents and could serve as models in translation. When asked about their preference, those who did not browse examples said that examples could be very useful. They did not look at the examples just because they forgot to do so, or because they thought that the equivalents were sufficient for their purposes. If they had looked at examples, their expression would have been more natural and idiomatic. This finding is different from that of Chan (2014: 34), who found that in determining the meaning of words and making sentences, Chinese monolingual English dictionary users relied most on examples (90%), and then definitions (63.6%). Examples could help users learn about the detailed usage of words because they demonstrate the specific use of words in context, but some translation learners in our study lacked the skills to use dictionaries in translation. As translation learners, they missed the opportunity to learn new words and expressions that could have been used in their translations later.

4.1.3Selection of equivalents

To understand users' preference in choosing equivalents, we chose ten words with three equivalents and placed the most familiar ones in the second positions in the dictionary. The frequency of the users' choice among the three positions is as follows:

Table 3 shows that selection of the first equivalent was most frequent (48.7%). Selection of the second equivalent did not fall far behind (41%). Selection of the third equivalent was the smallest (10.3%). These results confirm the finding of other researchers (Tono 1984; Li 1998) about users' strategy in using a dictionary: they tended to utilize the beginning of an entry. Tono (1984) found that dictionary users tended to choose the first definition unless clear information to reject it was indicated. Li (1998) found that one factor responsible for mistranslation was that dictionary users tended to choose the first equivalent in the dictionary entry. We also wondered why the second equivalents were nearly as popular as the first equivalents. In follow-up interviews, respondents indicated that they preferred to choose the equivalents they were most familiar with. In the first place, these choices could give them assurance. In the second place, these choices could facilitate their expression because they are more familiar with the usage of these words than other words. For those words they encountered before, this strategy offered learners opportunities to use them again and ultimately could contribute to the acquisition of these words.

4.2 Application of retrieved information

Previous studies (Varantola 1994, Atkins and Varantola 1997) highlight the need for a more in-depth analysis of dictionary use during a translation task. We hold that analysis of dictionary information application would be a step toward that end. Varantola (1998) has argued that it is difficult to evaluate the use of words in the translation product because translators use different standards for their choices. However, we believe that this analysis is significant and feasible. Although there are different ways to evaluate a translation product, we can judge whether the use of words is grammatically correct or not. The analysis of the content can inform us of the application ability of translator trainees, that is, whether a user can adapt the information from a dictionary to the context of a translation text. In other words, this analysis can reveal the particular linguistic and transfer needs of users in translation. To be more effective, bilingual dictionaries should gear their information toward the needs of translation learners as most of them claim that translators and translation learners are their target users. For example, if users have difficulty in choosing the correct part-of-speech form of a word, dictionaries can provide more instruction in presentation of definition, senses or examples.

As for the operation of this analysis, we focus on how well users applied the dictionary information to the translation task by conducting errors analysis, correctness analysis, and by examining possible causes for errors.

4.2.1 Error analysis

From the products of students, we determined that students' errors in using the dictionary information fall into two categories: parts of speech and collocation. We offer a focused case study that illustrates our wider findings in these categories.

For errors in tense, we take the verb "Ц yu xiang" (which literally means "expect").

The original sentence:

...

Suggested translation: To make some progress, discoveries or creations, we must make more efforts than expected.

In the original text, "И, yu xiăng" (which literally means "expect") is used as a noun. In the translation, it could be used as a noun. If students want to express it as a verb in English, they must shift the word class of the equivalents.

As "⅞⅞ yu xiăng" (which literally means "expect") is labeled as a verb in the four dictionaries, the dictionary in the study provides three verbs in the Equivalents (anticipate; expect; speculate) and noun forms (anticipation; expectation; speculation) in the Derivative category. In the categories such as Collocation and Examples, it also provides the verb form.

Among the forty-seven students, thirty-three students used "expect". Video recordings informed us that only twenty-two students looked up "yu xiăng" in the dictionary and eighteen users chose "expect" in the equivalents. That is, most of students chose the most familiar equivalent. A closer observation found that in the use of the word "expect", seven instances of incorrect usage were found. The following sentence fragments were taken from students' products of translation and students' IDs are in the parentheses.

We should often pay out much more effort than expecting if ... (Student 1091)

..., we will make more effort to make it than expecting before. (Student 1128)

We need to devote more than we had expected if ... (Student 1006)

..., you must pay more than you have expected. (Student 1002)

We should pay much more efforts than our expect so that ... (Student 1133)

..., we usually need to pay out more effort than expect. (Student 1140)

So we should make great efforts which beyond our expectation ... (Student 1132)

From the translation products of students, we can see that two learners (Student 1091 and Student 1128) used it as "expecting" in the context. They used the gerund form of "expect" incorrectly, as there is no objective. The video informed us that these students just read the information category of English equivalents. If they had clicked on the other information categories such as Derivative, Collocations or Bilingual examples, they would probably have known more about this word and chosen its form appropriate for this context. Two learners (Student 1006 and Student 1002) used "expect" in the perfect tense. One (Student 1006) used it in the past perfect tense and the other (Student 1002) used it in the present perfect tense. Four students who did not consult the dictionary made similar mistakes. They used it in the past tense as "we expected ..." One user (Student 1133) took it as a noun. If the student had clicked on the Past of Speech or Derivatives, he or she would have found the noun form of "expect". One learner (Student 1132) transferred the phrase "beyond one's expectation" from the bilingual example "The beauties of the West Lake in spring were beyond his expectation." to his or her translation but the phrase was not used correctly. Such errors indicate that these learners lacked knowledge of the general grammatical rules. It would be an advantage if the description and explanation of some general rules could be incorporated into the dictionary as a separate section and individual dictionary articles could refer to them (Tarp 2008: 234). Study Pages in the OALD8 (Oxford Advanced Learner's Dictionary, 8th Edition) could serve as a good example.

When we asked about errors in tense in follow-up interviews, some students responded that they forgot to find information about the different forms of the equivalents in the dictionary. When they were engaged in translation, they focused on the meaning transfer rather than on the form of the words they used. Others said that they failed to find enough tense information about verbs in the examples. This has implications for both teachers and dictionary compilers. Since the Chinese language does not have as many tense markers as the English language, it would be helpful to Chinese translation learners if the general rules of tense could be incorporated into Chinese-English dictionaries as a separate section. At the same time, teachers should draw learners' attention to this difference between two languages in their instruction.

With regard to collocation, we take "mu biāo" (which literally means "goal") as an example.

The original clause: АЯйЁВЁЮШ/А...

Suggested translation: After setting a goal, ...

In the original Chinese text, mu biāo (which literally means "goal") collocated with the verb que ding (which literally means "define"). For the verb que ding, the dictionary provided three equivalents, namely, determine, define, fix. In the category of Collocation, the dictionary provides two phrases (que ding ri qi) fix a date; (que ding mu biāo) set a goal/an aim. Video recordings showed that twenty-one students did not look up this word. Twelve students looked up que ding while six students looked up mu biāo. Eight students looked up both que ding and mu biāo. The following sentence fragments were taken from students' products of translation and students' IDs are in the parentheses.

..., after people fixing on the goals, (Student 1091)

Once people determine their goals, ... (Student 1096)

After people determine their goals, ... (Student 5010)

Once you determine a goal, ... (Student 1003)

Once we set up our goal, ... (Student 1009)

When you set an ambitious goal, ... (Student 1010)

Once we make a clear goal, ... (Student 1132)

Once you have defined your goal, ... (Student 5002)

In the translation products of students, we found that collocate words used with goal were as follows: determine 9 times, set 6 times, set up 1 time, make 3 times, define 1 times, fix on 1 time. We checked these collocation choices with goal in BNC (British National Corpus) and found that in the first 100 collocates; set is ranked the second, make 13rd, fix 93rd. We regard these collocations as acceptable. However, for the collocations with determine, set up, define, and fix on, which are not found in the corpus, we regard these collocations as unacceptable.

In follow-up interviews, some students responded that they did not give much attention to collocation. They just picked the first equivalent or the one they were familiar with and then applied it in the translation. Sometimes, even when they looked for the information, they could not find it in the dictionary. In the translation task, the verb que ding collocates with mu biāo. So most students who translated word by word felt it unnecessary to think about collocation. That is to say, these translation learners were not aware that the collocation of a word in two languages might be different. This also has implications for Chinese-English dictionary compilers. For example, when they provide equivalents for a word, they should also give more information such as definition, style and collocation, which can help users to identify the distinctions between equivalents and then make informed choices. This problem exists in almost all Chinese-English dictionaries available in China and had already been pointed out by researchers (Wei 2000; Hu and Zhang 2011; Xu 2012).

4.2.2 Correctness analysis

The success of word application in learners' translations was decided by the negotiation between the two researchers. When students looked up the same word but chose different expressions, their application of dictionary information would be regarded as successful if the words or expressions were used correctly. For instance, twenty-two students looked up "yu xiang" (which literally means "expect") in the dictionary. Eighteen of them chose "expect" and eleven of them used it correctly. At the same time, one student chose "speculate" and used it as "make more efforts than we speculate", and it was coded as correct. Another student who did not choose any of the equivalents but read the bilingual examples produced the following translation, "make more efforts than our first thought". It was also judged as successful application of dictionary information. Correctness ratio refers to the comparison between the number of words or expressions a student looked up and the words she or he applied correctly in their translation. For example, a student looked up twenty-nine words and found the equivalents of twenty-four words. If fifteen equivalents were used correctly in the translation, his or her correctness ratio would be 63%. The following is an overview of the correctness ratio.

Li (1998) found that 73% of the lookups in English-Chinese translation is successful. Table 4 shows that in our study, 82% of consultation was successful and dictionary use contributed to Chinese-English (L1-L2) translation.

To investigate whether consultation preference has an impact on the correctness of their dictionary information application in translation, we divided students into three groups on the basis of their consulting preferences. Group one consisted of students who only looked up Equivalents; Group two consisted of students who consulted both Equivalents and Examples; Group three consisted of students who consulted Equivalents, Examples and Collocation. A oneway ANOVA test was carried out to explore whether there was a significant difference between the groups. The statistical results indicated that there is a significant difference between these groups (F=6.968, P=0.002<0.01). The following table shows the result.

For more detailed information about the difference, we made a further analysis.

The results show that there is a significant difference between group one and group two (Mean Difference=0.13281, p=0.001<0.01). A significant difference also exists between Group two and Group three (Mean Difference=0.10193, p=0.03<0.05). This implies that when student translators know more information about a word, the correctness of translation also increases. This is consis- tent with Laufer (1993) who found that the combination of definition and examples contributes more to translation than definition or examples alone. The latter has no significant influence on translation. This could be explained by the fact that examples and collocation provide more detailed usage of words. The information can either demonstrate the usage of words in context or provide exemplary use of the word in the task. As no student only looked up the information category Examples, we cannot find out the relationship between examples and correctness of word use. The findings suggest that dictionaries should provide more information for learners and more importantly, translation trainers should encourage students to read more information in the dictionary.

In previous studies, researchers (Peters 2007; Lew and Doroszewska 2009; Chen 2011) also investigated the relationship between click counts and vocabulary retention, with various conclusions. Our study showed that there is no correlation between correctness and click counts. As learners cannot use them correctly in the first place, it can be predicted that there is no correlation between click counts and vocabulary retention. This could be attributed to the fact that the majority of clicks were on the Equivalents and this information category did not provide detailed information about the usage of words. In addition, the number of clicks does not necessarily equate to a deepening of understanding. Therefore, it can be concluded that information category rather than click counts has more influence on correctness of lexical information application.

5.Conclusion

This study investigated the use of an electronic dictionary (digitalized print dictionaries) by students in a natural setting. It provides a more complete picture of dictionary use by EFL learners as it utilized both positivistic and naturalistic research methods. It contributes to the literature of dictionary use study by providing a detailed description and analysis of users' dictionary information application during a translation task. The study has five findings: 1) EFL learners' consulting preferences include Equivalents and Examples; 2) EFL learners preferred to choose the most familiar equivalents; 3) EFL learners looked up content words and phrases more than other words; 4) EFL learners' errors in dictionary information application lie in collocation and parts of speech; 5) EFL learners' correctness of dictionary information application increases as students consult, or click on, additional information categories. These findings have implications for Chinese-English dictionary compilers, who are tasked with providing high-quality equivalents and examples as users relied heavily on them. For example, dictionary compilers could provide more information about equivalents so that users know the difference between them and make the informed choices. In the bilingual examples, compilers could demonstrate the usage of the equivalents so that users could learn how to use these words in context. To enhance users' retrieval success, dictionaries could provide more content words and phrases. Results also confirm that translation teachers should encourage students to read more information categories in dictionary use.

This study focused on the looking up preferences of translation trainees and their application of dictionary information. It has some limitations: the number of participants is not very large, the number of dictionary entries is small and findings are based chiefly on an observation. To improve its reliability, further studies with mixed research methods should be conducted.

Acknowledgement

The research was funded by the College Philosophy and Social Science Foundation of Jiangsu Provincial Department of Education (Grant No. 2016SJB740007) and by The Third Phase of the Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD: Phase iii) (Project No. 20180101). We are especially grateful to two anonymous reviewers and Dr Wei Xiangqing for their insightful comments and constructive suggestions. Our thanks also go to Roy Stamper from NC State University for proofreading the manuscript and all the participants in the study.

References

References

A. Dictionaries

Hornby, A.S. (Ed.). 2010. Oxford Advanced Learner's Dictionary (8th edition). Oxford: Oxford University Press.

Hui, Y. (Ed.). 2012. New Century Chinese-English Dictionary. Beijing: Foreign Language Teaching and Research Press.

Kingsoft PowerWord 2017. Accessed on March 10, 2017. http: //www.iciba.com.

Yao, X.P. (Ed.). 2010. A Chinese-English Dictionary. Third edition. Beijing: Foreign Language Teaching and Research Press.

Youdao Dictionaries 7.0. Accessed on March 10, 2017. http: //dict.youdao.com.

B. Other literature

Anthony, L. 2010. AntConc (Version 3.2.1) [Computer Software]. Tokyo, Japan: Waseda University. Available from http: / /www.laurenceanthony.net/software.

Atkins, B.T.S. and K. Varantola. 1997. Monitoring Dictionary Use. International Journal of Lexicography 10(1): 1-45.

Barnhart, C.L. 1962. Problems in Editing Commercial Monolingual Dictionaries. Householder, Fred W. and Sol Saporta (Eds.). 1962. Problems in Lexicography: 161-181. Bloomington: Indiana University/The Hague: Mouton.

Bogaards, P. 1998. What Type of Words do Language Learners Look Up? Atkins, B.T. Sue (Ed.). 1998. Using Dictionaries: Studies of Dictionary Use by Language Learners and Translators: 151-158. Tübingen: Max Niemeyer.

Bogaards, P. 2003. Uses and Users of Dictionaries. Van Sterkenburg, P. (Ed.). 2003. A Practical Guide to Lexicography: 26-33. Amsterdam/Philadelphia: John Benjamins.

Chan, A.Y.W. 2012. The Use of a Monolingual Dictionary for Meaning Determination by Advanced Cantonese ESL Learners in Hong Kong. Applied Linguistics 33(2): 115-140.

Chan, A.Y.W. 2014. Using LDOCE5 and COBUILD6 for Meaning Determination and Sentence Construction: What Do Learners Prefer? International Journal of Lexicography: 27(1): 25-53.

Chen, Y. 2010. Dictionary Use and EFL Learning. A Contrastive Study of Pocket Electronic Dictionaries and Paper Dictionaries. International Journal of Lexicography 23(3): 275-306.

Chen, Y. 2013. A Correlational Study between Dictionary Lookup Behavior and Vocabulary Acquisition under CALL Context. Foreign Languages and Their Teaching 5: 46-51.

Chen, Y. 2017. Dictionary Use for Collocation Production and Retention: A CALL-based Study. International Journal of Lexicography 30(2): 225-251.

Chen, Y.Z. 2011. The Use of Bilingualized English-Chinese Learner's Dictionaries: A Survey and An Experiment. Lexicographical Studies 2: 141-158.

Dziemianko, A. 2010. Paper or Electronic? The Role of Dictionary Form in Language Reception, Production and the Retention of Meaning and Collocations. International Journal of Lexicography 23(3): 257-273.

Dziemianko, A. 2014. On the Presentation and Placement of Collocations in Monolingual English Learners' Dictionaries: Insights into Encoding and Retention. International Journal of Lexicography 27(3): 259-279.

Frankenberg-Garcia, A. 2011. Beyond L1-L2 Equivalents: Where Do Users of English as a Foreign Language Turn for Help? International Journal of Lexicography 24(1): 97-123.

Frankenberg-Garcia, A. 2015. Dictionaries and Encoding Examples to Support Language Production. International Journal of Lexicography 28(4): 490-512.

Gromann, D. and J. Schnitzer. 2016. Where Do Business Students Turn for Help? An Empirical Study on Dictionary Use in Foreign-language Learning. International Journal of Lexicography 29(1): 55-99.

Hartmann, R.R.K. 1983. The Bilingual Learner's Dictionary and Its Uses. Multilingua 2(4): 195-201.

Harvey, K. and D. Yuill. 1997. A Study of the Use of a Monolingual Pedagogical Dictionary by Learners of English Engaged in Writing. Applied Linguistics 18(3): 253-278.

Hu, W.F. and Y.H. Zhang. 2011. A Survey of Sense Representation in Chinese-English Dictionaries from the Perspective of Users. Foreign Languages Research 3: 78-84.

Hu, W.F. and Y.H. Zhang. 2013. The Effect of Definition Model in C-E Dictionaries on Chinese EFL Learner's English Productive Ability. Journal of Foreign Languages 34(5): 54-62.

Kaneta, T. 2011. Folded or Unfolded: Eye-tracking Analysis of L2 Learners' Reference Behavior with Different Types of Dictionary Interfaces. Akasu, K. and U. Satoru (Eds.). 2011. ASIALEX 2011 Proceedings. Lexicography: Theoretical and Practical Perspectives, 22-24 August 2011: 219224. Kyoto: Asian Association for Lexicography.

Koplenig, A., P. Meyer and C. Müller-Spitzer. 2014. Dictionary Users Do Look Up Frequent Words. A Log File Analysis. Müller-Spitzer, Carolin (Ed.). 2014. Using Online Dictionaries: 229-250. Berlin/Boston: De Gruyter.

Laufer, B. 1993. The Effect of Dictionary Definitions and Examples on the Use and Comprehension of New L2 Words. Cahiers de Lexicologie 63(2): 131-142.

Laufer, B. and L. Hadar. 1997. Assessing the Effectiveness of Monolingual, Bilingual, and "Bilingualised" Dictionaries in the Comprehension and Production of New Words. The Modern Language Journal 81(2): 189-196.

Laufer, B. and M. Hill. 2000. What Lexical Information Do L2 Learners Select in a CALL Dictionary and How Does It Affect Word Retention? Language Learning & Technology 3(2): 58-76.

Laufer, B. and L. Melamed. 1994. Monolingual, Bilingual and "Bilingualised" Dictionaries: Which are More Effective, for What and for Whom? Martin, W. et al. (Eds.). 1994. Euralex 1994 Proceedings, Papers submitted to the 6th EURALEX International Congress on Lexicography in Amsterdam, The Netherlands: 565-576. Amsterdam: Vrije Universiteit.

Lew, R. 2011a. Studies in Dictionary Use: Recent Developments. International Journal of Lexicography 24(1): 1-4.

Lew, R. 2011b. User Studies: Opportunities and Limitations. Akasu, K. and U. Satoru (Eds.). 2011. ASIALEX 2011 Proceedings. Lexicography: Theoretical and Practical Perspectives, 22-24 August 2011: 7-16. Kyoto: Asian Association for Lexicography.

Lew, R. 2012. How Can We Make Electronic Dictionaries More Effective? Granger, S. and M. Paquot (Eds.). 2012. Electronic Lexicography: 343-361. Oxford: Oxford University Press.

Lew, R. and G.-M. de Schryver. 2014. Dictionary Users in the Digital Revolution. International Journal of Lexicography 27(4): 341-359.

Lew, R. and J. Doroszewska. 2009. Electronic Dictionary Entries with Animated Pictures: Lookup Preferences and Word Retention. International Journal of Lexicography 22(3): 239-257.

Li, L. 1998. A Study of Dictionary Use by Chinese University Learners of English for Specific Purposes. Ph.D. dissertation. Exeter: University of Exeter.

Liang, P. and D. Xu. 2017. The Contribution of Dictionary Use to the Production and Retention of the Middle Construction for Chinese EFL Learners. International Journal of Lexicography 30(1): 85-107.

Nesi, H. 2014. Dictionary Use by English Language Learners. Language Teaching 47(1): 38-55.

Pan, X.S. 2012. PMLX (Version V2012) [Computer Software]. Wenzhou, China: Tianlangxing Software Studio. Available from: http: //www.tlxsoft.com.

Peters, E. 2007. Manipulating L2 Learners' Online Dictionary Use and Its Effect on L2 Word Retention. Language learning & Technology 11(2): 36-58.

Sánchez Ramos, M.M. 2005. Research on Dictionary Use by Trainee Translators. Translation Journal 12(10): 25-35.

Tarp, S. 2009. Reflections on Lexicographical User Research. Lexikos 19: 275-296.

Tomaszczyk, J. 1979. Dictionaries: Users and Uses. Glottodidactica 12: 103-119.

Tono, Y. 1984. On the Dictionary User's Reference Skills. Unpublished B.Ed. thesis. Tokyo: Gakugei University.

Tono, Y. 1989. Can a Dictionary Help One Read Better? On the Relationship Between EFL Learners' Dictionary Reference Skills and Reading Comprehension. James, G. (Ed.). 1989. Lexicographers and Their Works: 192-200. Exeter: University of Exeter Press.

Varantola, K. 1994. The Dictionary User as Decision Maker. Martin, Willy et al. (Eds.). 1994. EURALEX 1994 Proceedings: 606-611. Amsterdam: Vrije Universiteit.

Varantola, K. 1998. Translators and Their Use of Dictionaries. Atkins, B.T. Sue (Ed.). 1998. Using Dictionaries: Studies of Dictionary Use by Language Learners and Translators: 179-192. Tübingen: Max Niemeyer.

Wei, X.Q. 2000. Multilevel Exemplification in Active Bilingual Learner's Dictionaries. Lexicographical Studies 6: 68-75.

Welker, H.A. 2010. Dictionary Use: A General Survey of Empirical Studies. Brasilia: Eigenverlag.

Xie, X.Y. 2014. The Evaluation of Online English-Chinese Dictionaries. Unpublished M.A. thesis. Shanghai, China: Fudan University.

Xu, H. 2012. Decoding and Encoding Functions of Examples in English Learners' Dictionaries: A Case Study of the Exemplification of the Word "Monopoly". Lexicographical Studies 2: 33-39.

Yang, S.M. 2017. Effects on English Learning of Online Dictionaries-Taking Youdao Dictionary and Kingsoft Powerword for Example. Journal of Hainan Radio and TV University 2017(1): 114-117.

Zhang, H.P. 2014. ICTCLAS2014 (Institute of Computing Technology, Chinese Lexical Analysis System) [Computer Software]. Beijing, China: Chinese Academy of Science. Available from http: / /ictclas.nlpir.org/downloads.