Evaluating the Equal-Interval Hypothesis with Test Score Scales

Abstract

In psychometrics, it is difficult to verify that measurement instruments can be used to produce numeric values with the desirable property that differences between units are equal-interval because the attributes being measured are latent. The theory of additive conjoint measurement (e.g., Krantz, Luce, Suppes, & Tversky, 1971, ACM) guarantees that interval scales are possible--for latent and manifest variables alike--if certain axioms hold. However, ACM was initially developed under the assumption that data could be gathered to test the axioms that was free from measurement error. It wasn't until Karabatsos (2001) that the methodology allowed for measurement error. In this dissertation, an improved version of Karabatsos's methodology is applied to simulated and empirical data to test whether such data are consistent with the axioms. It is first shown that the methodology behaves reasonably using data simulated to meet the cancellation axioms of ACM. It is then shown that the methodology is capable of distinguishing data simulated to meet the axioms from data that is not. In particular, it is demonstrated that the methodology is sensitive to item-side violations of the axioms. Empirical examples are then used to illustrate the fact that test score data may or may not conform to the ACM axioms. Empirical demonstration shows that an existing test scale thought to satisfy the ACM axioms using the Karabatsos (2001) approach does not do so using the modified approach here. Since not all data may meet the ACM axioms (and hence not warrant interval interpretations), this dissertation also examines whether scale distortions can lead to erroneous conclusions. At the score-level, an approach was developed using "difference matrices" to highlight the fact that when the Rasch (1960) model is applied to certain non-Rasch data, the estimates will be more compressed at lower abilities. This same phenomena was noted in two simulations meant to capture how educational assessment data is used--with respect to schools and educational interventions--although the effects of the distortions were small. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone [phone]. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]