Content area
Full text
The fixed effects regression model is commonly used to reduce selection bias in the estimation of causal effects in observational data by eliminating large portions of variation thought to contain confounding factors. For example, when units in a panel data set are thought to differ systematically from one another in unobserved ways that affect the outcome of interest, unit fixed effects are often used since they eliminate all between-unit variation, producing an estimate of a variable’s average effect within units over time (Allison 2009; Wooldridge 2010). Despite this feature, social scientists often fail to acknowledge the large reduction in variation imposed by fixed effects. This article discusses two important implications of this omission: imprecision in descriptions of the variation being studied and the use of implausible counterfactuals—discussions of the effect of shifts in an independent variable that are rarely or never observed within units—to characterize substantive effects.
By using fixed effects, researchers make not only a methodological choice but a substantive one (Bell and Jones 2015), narrowing the analysis to particular dimensions of the data, such as within-country (i.e., over time) variation. By prioritizing within-unit variation, researchers forgo the opportunity to explain between-unit variation since it is rarely the case that between-unit variation will yield plausible estimates of a causal effect. When using fixed effects, researchers should emphasize the variation that is being used in descriptions of their results. Identifying which units actually vary over time (in the case of unit fixed effects) is also crucial for gauging the generalizability of results, since units without variation provide no information during one-way unit-fixed effects estimation (Plümper and Troeger 2007). 1
In addition, because the within-unit variation is always smaller (or at least, no larger) than the overall variation in the independent variable, researchers should use within-unit variation to motivate counterfactuals when discussing the substantive impact of a treatment. For example, if only within-country variation in some treatment X is used to estimate a causal effect, researchers should avoid discussing the effect of changes in X that are larger than any changes observed within units in the data. Besides being unlikely to ever occur in the real world, such “extreme counterfactuals,” which extrapolate to regions of sparse (or nonexistent) data rest on strong modeling assumptions, and...