Creating Short Forms of Early Childhood Development Measures: A Framework for Quantifying Statistical, Conceptual, and Practical Tradeoffs in Direct Assessment. EdWorkingPaper No. 25-1143

Abstract

Direct assessments of early childhood development (ECD) are a cornerstone of research in developmental psychology and are increasingly used to evaluate programs and policies in lower- and middle-income countries. Despite strong psychometric properties, these assessments are too expensive and time consuming for use in large-scale monitoring or national-level surveys. Short forms of direct assessments can provide some benefits of direct assessment at substantially lower cost and complexity. However, selecting the best items for inclusion on shorter forms is not a straightforward task. Traditional approaches to creating short forms, which rely on statistical properties of items, can neglect important non-statistical considerations and result in narrowed construct coverage that does not maximize improvements in usability. Automated Test Assembly (ATA) is an ideal approach to generate optimal forms given numerical constrains, but can be difficult to operationalize. This paper proposes a theoretical framework for an empirically driven, human-centered process to create short forms of ECD direct assessments. It builds on the goals of ATA in an accessible manner by evaluating items across three dimensions: "statistical" (how reliably items distinguish between children with higher and lower development), "conceptual" (how representative items are of the constructs being assessed), and "practical" (how time-consuming and difficult items are to administer). Having defined this framework, the paper then applies it to the International Development and Early Learning Assessment, a popular direct assessment of ECD, to suggest a general-purpose short form selected after considering these three dimensions.