Factors Affecting Regression-Discontinuity

Abstract

The regression-discontinuity approach to evaluating educational programs is reviewed, and regression-discontinuity post-program mean differences under various conditions are discussed. The regression-discontinuity design is used to determine whether post-program differences exist between an experimental program and a control group. The difference between program and comparison group regression lines is tested at the cutoff score point. The difference between regression lines at the cutoff score point is tested for significance against the null hypothesis. Factors that affect regression-discontinuity program outcome/effect interpretations relate to: (1) measurement error and reliability of assignment measures; (2) selection bias and a valid statistical model; and (3) program outcome and cutoff score placement. Specific issues related to the cutoff score--selection, placement, and adherence to the cutoff criterion--combined with the reliability of the pretest variable and selection bias must be considered when conducting regression-discontinuity analysis. Five tables and one figure present simulated data from examples. An appendix contains the simulation program used, and a 25-item list of references is included. (SLD)