Content area
Abstract
The large samples and item pools required for accurate parameter estimation under the three-parameter Item Response Theory model potentially limit its viability for many practitioners and researchers. The one-parameter logistic or Rasch model requires fewer subjects and items; however, this model may not be appropriate for multiple choice data. Studies of the Rasch model show it is inappropriate when guessing is present in the data, but that it is generally robust with respect to violations of homogeneity of item discrimination. Therefore, it is reasonable to propose that the addition of a guessing parameter to the Rasch model should result in improved estimation.
This study used simulated and actual small sample test data in comparing Rasch ability and item estimates with estimates obtained from the three-parameter model (3-PAR) and with those obtained from two modified Rasch models that incorporated a constant guessing parameter. Simulated data were generated under the assumptions of the three-parameter model. For both the simulated and actual test data, sample size was varied and in the simulation, test length was also varied. LOGIST 5 (Wingersky, Barton, and Lord, 1982) was used to obtain item and ability estimates for the four estimation models.
Results of a number of dependent measures showed that a modified Rasch model produced the most accurate estimates of item difficulty based on both simulated and actual test data, and the most accurate estimation of item charcteristic curves. The Rasch model was the least accurate of the models in estimating item difficulty and in recovering true item characteristic curves. The Rasch model was the better model for estimating ability at extremely low ranges; however, when extreme outliers were eliminated, the four models produced comparable results across the ability range. The ability estimation errors tended to be systematic in that the Rasch model overestimated ability, whereas the guessing models underestimated ability. The three-parameter model provided neither the least nor the most accurate results based on any of the dependent measures. Analyses of item information functions showed that the Rasch model achieved its maximum item information at substantially lower theta values than did the other models.
