Don't let spurious accusations of pseudoreplication limit our ability to learn from natural experiments (and other messy kinds of ecological monitoring)

Abstract

Pseudoreplication is defined as the use of inferential statistics to test for treatment effects where treatments are not replicated and/or replicates are not statistically independent. It is a genuine but controversial issue in ecology particularly in the case of costly landscape-scale manipulations, behavioral studies where ethics or other concerns may limit sample sizes, ad hoc monitoring data, and the analysis of natural experiments where chance events occur at a single site. Here key publications on the topic are reviewed to illustrate the debate that exists about the conceptual validity of pseudoreplication. A survey of ecologists and case studies of experimental design and publication issues are used to explore the extent of the problem, ecologists’ solutions, reviewers’ attitudes, and the fate of submitted manuscripts. Scientists working across a range of ecological disciplines regularly come across the problem of pseudoreplication and build solutions into their designs and analyses. These include carefully defining hypotheses and the population of interest, acknowledging the limits of statistical inference and using statistical approaches including nesting and random effects. Many ecologists face considerable challenges getting their work published if accusations of pseudoreplication are made – even if the problem has been dealt with. Many reviewers reject papers for pseudoreplication, and this occurs more often if they haven't experienced the issue themselves. The concept of pseudoreplication is being applied too dogmatically and often leads to rejection during review. There is insufficient consideration of the associated philosophical issues and potential statistical solutions. By stopping the publication of ecological studies, reviewers are slowing the pace of ecological research and limiting the scope of management case studies, natural events studies, and valuable data available to form evidence-based solutions. Recommendations for fair and consistent treatment of pseudoreplication during writing and review are given for authors, reviewers, and editors.

References Anderson, M. J. 2001. A new method for non-parametric multivariate analysis of variance. Austral Ecol. 26:32–46. Bates, D., M. Maechler, B. Bolker, and S. Walker. 2014. lme4: Linear mixed-effects models using Eigen and S4. R package version 1.1-7. Available from: http://CRAN.R-project.org/package=lme4 [Accessed 29= May 2015]. Cottenie, K., and L. De Meester. 2003. Comment to Oksanen (2001): reconciling Oksanen (2001) and Hurlbert (1984). Oikos 100:394–396. Davies, G. M., and J. Bodart. 2015. Changes in vegetation composition and diversity following livestock removal along an upland elevational gradient. iForest 8:582–589. Davies, G. M., R. O'Hara, A. A. Smith, A. MacDonald, and C. J. Legg. 2010. Winter desiccation and rapid changes in the live fuel moisture content of Calluna vulgaris. Plant Ecol. Divers 3:289–299. Ellison, A. M., N. J. Gotelli, B. D. Inouye, and D. R. Strong. 2014. P values, hypothesis testing, and model selection: it's déjà vu all over again. Ecology 95:609-610. Feeberg, T. M., and J. R. Lucas. 2009. Pseudoreplication is (still) a problem. J. Comp. Psychol. 123:450–451. Friedman, J. M., W. R. Osterkamp, and W. M. Jr Lewis. 1996. Channel narrowing and vegetation development following a Great Plains flood. Ecology 77:2167–2181. Gurevitch, J., P. S. Curtis, and M. H. Jones. 2001. Meta-analysis in ecology. Adv. Ecol. Res. 32:199–247. Hargrove, W. W., and J. Pickering. 1992. Pseudoreplication: a sine qua non for regional ecology. Landscape Ecol. 6:251–258. Harrison, F. 2011. Getting started with meta-analysis. Methods Ecol. Evol. 2:1–10. Hurlbert, S. H. 1984. Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54:187–211. Hurlbert, S. H. H. 2004. On misinterpretations of pseudoreplication and related matters: a reply to Oksanen. Oikos 104:591–597. Johnstone, J. F., and E. S. Kasischke. 2005. Stand-level effects of soil burn severity on postfire regeneration in a recently burned black spruce forest. Can. J. For. Res. 35:2151–2163. Koricheva, J., J. Gurevitch, and K. Mengersen. 2013. Handbook of meta-analysis in ecology and evolution. Princeton Univ. Press, Princeton, NJ. Kozlov, M. V., and S. H. Hurlbert. 2006. Pseudoreplication, chatter, and the international nature of science: a response to D V Tatarnikov. Journal of Fundamental Biology (Moscow) 67:145–152. Legendre, P. 1993. Spatial autocorrelation: trouble or new paradigm? Ecology 74:1659–1673. Maltby, E., C. J. Legg, and M. C. F. Proctor. 1990. The ecology of severe moorland fire on the North York Moors: effects of the 1976 fires, and subsequent surface and vegetation development. J. Ecol. 78:490–518. Miller, D. A., E. B. Arnett, and M. J. Lacki. 2003. Habitat management for forest-roosting bats of North America: a critical review of habitat studies. Wildl. Soc. Bull. 31:30–44. del Moral, R., and I. L. Lacher. 2005. Vegetation patterns 25 years after the eruption of Mount St. Helens, Washington, USA. Am. J. Bot. 92:1948–1956. Nagel, T. A., M. Svoboda, and J. Diaci. 2006. Regeneration patterns after intermediate wind disturbance in an old-growth Fagus-Abies forest in southeastern Slovenia. For. Ecol. Manage. 226:268–278. Oksanen, L. 2001. Logic of experiments in ecology: is pseudoreplication a pseudoissue? Oikos 94:27–38. Oksanen, L. 2004. The devil lies in details: reply to Stuart Hurlbert. Oikos 104:598–605. Olsen, R. 2008. Self-selection bias Pp. 809–811 in P. J. Lavrakas, ed. Encyclopedia of survey research methods. Sage Publications, Thousand Oaks, CA. R Core Team. 2014. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria URL http://www.R-project.org [last accessed 01 Jun 2015]. Ramage, B. S., D. Sheil, H. M. W. Salim, C. Fletcher, N. Z. A. Mustafa, J. C. Luruthusamay, et al. 2013. Pseudoreplication in tropical forests and the resulting effects on biodiversity conservation. Conserv. Biol. 27:364–372. Rosenthal, R. 1979. The "file drawer problem" and tolerance for null results. Psychological Bulletin 86:638-641. Schank, J. C., and T. J. Koehnle. 2009. Pseudoreplication is a pseudoproblem. J. Comp. Psychol. 123:421–433. Waller, B. M., L. Warmelink, K. Liebal, J. Micheletta, and K. E. Slocombe. 2013. Pseudoreplication: a widespread problem in primate communication research. Anim. Behav. 86:483–488.