Analysis of IELTS and TOEFL reading and listening tests in terms of Revised Bloom’s Taxonomy

Abstract

The main purpose of this quantitative-qualitative content analysis study was to compare IELTS and TOEFL listening and reading tests based on the representation of the learning objectives of Revised Bloom’s taxonomy. To this end, 12 Academic IELTS listening and reading tests and 12 TOEFL iBT listening and reading tests were analyzed qualitatively and quantitatively. The contents of these tests were codified using a coding scheme developed by the researcher. To ensure the reliability of the coding, the test items were coded twice by the main researcher with a two-week time interval. Three coders (the researchers in this study) also independently codified the test items. The inter-coder and intra-coder reliability of the coding turned out to be .68 and .83, respectively. The Fisher-Freeman-Halton and Monte Carlo tests were used to analyze the data. Our results revealed that IELTS listening test items mainly revolved around Understanding and Remembering Factual Knowledge, respectively, while TOEFL listening test items drew on Understanding, Analyzing, and Remembering Factual Knowledge, respectively. Furthermore, the results demonstrated that lower-order thinking skills featured more prominently in IELTS listening tests than TOEFL listening tests. A significant difference was also found between IELTS and TOEFL reading tests in terms of learning objectives. That is, IELTS and TOEFL reading tests covered three and seven learning levels, respectively with the former overwhelmingly assessing lower-order thinking skills and the latter drawing on both lower-order and higher-order thinking skills. The implications of the findings are also delineated.