School Type and Mathematics Achievement: A Comparison of Magnet and Public Secondary Schools Using the Educational Longitudinal Study of 2002 Data Set

Abstract

Proponents of school choice and school reform often claim that different school "types" will produce better academic outcomes for students than does the traditional public school model. Unstated but implicit in these views is a causal assumption that certain school types are "better" and produce improved student academic achievement, a very difficult hypothesis to test. Not only is large-scale randomized treatment assignment of students to schools infeasible, but it is essentially impossible to create school settings that are identical in every aspect but the "treatment." Because of these differences in subject assignment and setting equivalency, as well as the nested nature of education, when there are differences in academic performances between schools it is extremely difficult to determine whether it is due to school differences, student differences, or some combination of the two.

This study uses student and school-level observational data from the Educational Longitudinal Study (ELS) of 2002, a nationally representative sample of high school sophomores and seniors with vertically scored mathematics tests from the 10^th and 12^th grades. To determine if there are statistically significant (p<05) differences in performance based on school type, I employ propensity score models to identify and match similar treatment and control students, as well as similar treatment and control schools, more closely approximating a randomized experiment. I estimate the magnet school effect on individual student mathematics achievement two ways: calculating the within and overall stratum average treatment effects on the stratified matched students and schools using analysis of variance, and with a hierarchical linear regression model that accounts for the clustering of students within schools. A final analysis examines selected post-secondary indicators to see if there are differences between magnet and public school students on these outcomes.

The results question the current broad and imprecise policy of wholesale school change to accomplish school improvement, and suggest a more precise focus on specific school practices. A policy that focused on school practices and targeting and reforming those practices in need of change might be more productive, less disruptive, and less expensive than changing school types or closing or restructuring entire schools.