Revised Test Fairness Framework (RTFF): modeling and structure

Abstract

The last two decades have witnessed a growing interest in the theoretical understanding of test fairness and the intricacies of the concept have led to a heated discussion about its main characteristics, on the one hand, and have called for its reconceptualization on the other hand. As a partial address of the call, this study intended to advance a measurement and structural model of the test fairness concept in the Iranian educational context. A 118-item questionnaire comprising 12 subscales was developed, pilot-tested, and then administered to 600 participants. The reliability indices of the subscales ranged from .87 to .97, and as for the full scale, Cronbach’s Alpha measure of internal consistency turned out to be .98. A set of Exploratory and Confirmatory Factorial Analyses were used to explore the interrelationships among the set of variables and to gauge the construct validity of the scales, with convergent and discriminant validity being ensured. A resulting model with a general higher-order factor (i.e., test fairness) and twelve lower-order factors of validity, construction and structure, administration, scoring, reporting, decision-making, consequences, security, explicitness, accountability, equality, and rights demonstrated the best fit, with the final solution explaining a total of 73.3 percent of the variance. The verified interplay between test fairness and the identified related factors confirmed fairness as a broad concept inclusive of test use validity. This finding entails that fairness as an all-encompassing and broad multi-disciplinary concept (Sen, A. (2009). The idea of justice. Harvard University Press.) might not be confined only to the testing context as is the case with validity. This idea in turn strengthens the idea that validity might possibly be incapable of justifying all intricacies of fairness in general and test fairness in particular.