A unified evaluation of global and local fit performance under differing test construction conditions and model misspecifications

Abstract

Social scientists and researchers frequently use latent variable models to analyze the relationships between observed variables and latent variables representing the hypothesized constructs. The population, or true, model is not always known, resulting in a degree of misspecification in the relationships between variables in the model. Therefore, model- and item-fit statistics have been developed in order to provide evidence for the validity of a specific latent variable model.

Conditions for mathematical equivalence between two popular latent variable modeling methods, confirmatory factors analysis (CFA) and item response theory (IRT), have been established, availing the researcher and practitioner of a variety of model- and item-fit indices. This dissertation employs a simulation design to examine the behavior of three model-fit indices (χ²/df, RMSEA, and GDDM) and three item-fit indices (S-χ ², Modification Index, Wald Test) under various conditions of model misspecification and test design conditions. The results of this study show the empirically-derived cut points to out-perform the theoretical and suggested cut points when true models are estimated; these cut points are employed in subsequent analysis of misspecified models. In addition to examining the statistical power of each fit index to correctly reject the misspecified models, recommendations are made for the use of each fit statistic according to the model misspecification and test design conditions manipulated in the simulation study. Analysis of a real data set is provided as an illustration.