THE EFFECTS OF TEST CHARACTERISTICS ON THE HIERARCHICAL ORDER OF READING SKILLS

Abstract

Purpose – The study sought to determine the hierarchical nature of reading skills. Whether reading is a ‘unitary’ or ‘multi-divisible’ skill is still a contentious issue. So is the hierarchical order of reading skills. Determining the hierarchy of reading skills is challenging as item difficulty is greatly influenced by factors related to test characteristics. To examine the interaction between these factors and item difficulty, and determine the possibility of such a hierarchy, this study used the multifaceted Rasch approach. Methodology – In this descriptive study, a 42-MCQ reading test was administered to 944 ESL lower secondary students, randomly selected from eleven Malaysian national-type schools in the Federal Territory of Kuala Lumpur and the state of Selangor. These student populations were selected as the development of reading ability was considered critical at this stage of schooling. The reading test items were identified according to the following aspects: Reading Skill Areas (Interpreting information, making Inference, Understanding figurative language, Drawing conclusions, Scanning for details and Finding word meanings), Context Type (Linear and Non-Linear), and Text Type (Ads, Notice, Chart, Story extract, Short message, Poem, Short news report, Brochure, Formal letter, Conversation, Long passage and weather forecast). Applying the Many-Facet Rasch model of measurement, the study analyzed student responses to the test items with the help of FACETS, version 3.7.1.4. Findings – The findings showed that context types, skill areas, and text types differed in difficulty (p<.01), with those items that required understanding and interpretation being more demanding. Test items based on linear contexts were more difficult than those based on non-linear contexts. Understanding figurative language was found to be the most difficult skill followed by Making inference and Interpreting information. The easiest reading skill was Scanning for details, followed by Finding word meanings. The reading skill, Drawing conclusions, was close to the average difficulty level. The findings also indicated that texts that were longer and had more information tended to be more difficult. Significance – This study has also shed new light on the theory and practice of reading. The findings support the hierarchical nature of reading skills. Different reading skills were found to exert differential cognitive demands, and those which required higher cognitive ability were more difficult for learners to acquire and perform. Understanding the hierarchy of reading skills will help language teachers to target their teaching more effectively; course designers to produce more appropriate teaching and learning materials; and test writers to develop test items that better meet students’ reading competencies.